New subject: X-Analytics is NULL on part of the data in webrequest

30 Apr 2015

When I was working on related stuff, I found that the value of
x_analytics_map ia null on the wmf.webrequest table in stat1002, when
is_pageview is filtered for true, and agent_type is user. I'm wondering why
that would be.

These are the things I found:

For 28th April 2015, of 741,858,511 requests, 28,827,374 have
x_analytics(is set to '-') and x_analytics_map set to null. It's about 3.9%
of all requests that day.

You can find these counts by doing something like this on hive in the
server. -
SELECT count(*)
FROM webrequest
WHERE x_analytics_map IS NULL
  AND agent_type = 'user'
  AND is_pageview = TRUE
  AND YEAR = 2015
  AND MONTH = 4
  AND DAY = 28;

Does anyone have ideas on why this might be and if something underlying is
broken?

--Madhu :)