On Tue, Nov 29, 2011 at 7:08 PM, Tomasz Finc <
tfinc@wikimedia.org> wrote:
> I wanted to alert us to an alarming trend in our overall mobile page stats
>
>
http://stats.wikimedia.org/EN/TablesPageViewsMonthlyMobile.htm
>
> We've been trending down over the last couple of months. This is not good.
> We need these stats to be growing not dropping. If any of us see any reason
> about why this may be happening then we need to speak up. I did some digging
> and found out that we may have another logging issue.
>
> Asher from our ops team informed me that
>
> binasher: bad… i need to do something like make puppet monitor and ensure
> that our customized varnish logging daemon is always running
>
> binasher: it died on one of the varnish servers, but not the other
>
> Which would account for the massive dips we are seeing. We can confirm that
> the logger died on 10/15 and that we've been under reporting for October and
> November. October is likely under reported by up to 25% while November is
> under reported by up to 50%.
>
> Now i'm happy that we have a simple operations related reason for this dip
> but we should have caught it sooner. With the foundations goal of 2 billion
> page views by July we have to stay on top of this.
>
> --tomasz
>