Hi Denny,
On Tue, Jan 14, 2014 at 09:50:24PM +0000, Denny Vrandečić wrote:
And we are sure that the patch did have no other consequences than remove these requests,
I am pretty convinced that the webstatscollector patch does only what it is supposed to do. Andrew and Erik looked over the patch, but please have a look at the patch yourself: https://gerrit.wikimedia.org/r/#/c/98505/1
Just to be sure it that the patch does the right thing, I also added a short test harness and added test cases to make sure the change does not misbehave. See the bottom part of: https://gerrit.wikimedia.org/r/#/c/98524/2/tests/test.sh
The patch has the known short-coming of handling only non-localized versions of the Special:CentralAutoLogin requests. But the localized versions were very, very few in number, and discussion was already ongoing to no longer localize them, as localization was causing problems for Ops as well. In the meanwhile, localizing Special:CentralAutoLogin has been removed: https://gerrit.wikimedia.org/r/#/c/98461/ https://bugzilla.wikimedia.org/show_bug.cgi?id=54195
[...] i.e. a recalculation would lead to the same numbers, I assume.
Erik Z did the patching of the files back then. I did recalculate the data and it looked good back then.
But since we were also concerned that ... maybe we overlooked something ... I recomputed the trends that wikistats exposed once again directly from the cache logs (using the wikistats pageview definition). And it matches the picture wikistats shows for the last months. So I also arrived at the different patterns of ruwiki, ptwiki, eswiki, kowiki, ...
There are many other possible sources of problems (and general room for improvement), but to me, employing the wikistats pageview definition to either the stored cache logs or the stored webstatscollector data, we really arrive changes that wikistats is depicting.
Best regards, Christian