I wanted to alert us to an alarming trend in our overall mobile page stats
http://stats.wikimedia.org/EN/TablesPageViewsMonthlyMobile.htm
We've been trending down over the last couple of months. This is not good.
We need these stats to be growing not dropping. If any of us see any reason
about why this may be happening then we need to speak up. I did some
digging and found out that we may have another logging issue.
Asher from our ops team informed me that
*binasher <member:binasher>: *bad… i need to do something like make puppet
monitor and ensure that our customized varnish logging daemon is always
running
*binasher <member:binasher>: *it died on one of the varnish servers, but
not the other
Which would account for the massive dips we are seeing. We can confirm that
the logger died on 10/15 and that we've been under reporting for October
and November. October is likely under reported by up to 25% while November
is under reported by up to 50%.
Now i'm happy that we have a simple operations related reason for this dip
but we should have caught it sooner. With the foundations goal of 2 billion
page views by July we have to stay on top of this.
--tomasz