Hi,
since mid-April 2014, logs from SSL terminators did no longer get routed into the machine that is running webstatscollector. As a result webstatscollector output does not contain SSL traffic since mid-April 2014. This issue affects
http://dumps.wikimedia.org/other/pagecounts-raw/
and all consumers of that data, as for example
http://dumps.wikimedia.org/other/pagecounts-ez/ http://stats.grok.se/ http://tools.wmflabs.org/wikiviewstats/ http://stats.wikimedia.org/ (only non-squid part) http://reportcard.wmflabs.org/
.
The bug is getting tracked at https://bugzilla.wikimedia.org/show_bug.cgi?id=67456
Best regards, Christian
Does this hit our granular pageview counts or both our granular counts and our overall PV count? Because if the latter, I've been investigating some lines in the sampled logs caught by our internal IP regex that look a lot like genuine outside traffic coming through SSL.
On 3 July 2014 04:12, Christian Aistleitner christian@quelltextlich.at wrote:
Hi,
since mid-April 2014, logs from SSL terminators did no longer get routed into the machine that is running webstatscollector. As a result webstatscollector output does not contain SSL traffic since mid-April 2014. This issue affects
http://dumps.wikimedia.org/other/pagecounts-raw/
and all consumers of that data, as for example
http://dumps.wikimedia.org/other/pagecounts-ez/ http://stats.grok.se/ http://tools.wmflabs.org/wikiviewstats/ http://stats.wikimedia.org/ (only non-squid part) http://reportcard.wmflabs.org/
.
The bug is getting tracked at https://bugzilla.wikimedia.org/show_bug.cgi?id=67456
Best regards, Christian
-- ---- quelltextlich e.U. ---- \ ---- Christian Aistleitner ---- Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christian@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax: +43 7946 / 20 5 81 Homepage: http://quelltextlich.at/
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Hi Oliver,
On Fri, Jul 04, 2014 at 02:19:39AM -0400, Oliver Keyes wrote:
Does this hit our granular pageview counts or both our granular counts and our overall PV count?
since we're having different pageview counts and not all of them are generated through the same pipeline, it's hard to answer your question without having the concrete urls you are referring to.
If you are referring to wikistats generated files, Erik Zachte would know best for which parts webstatscollector data gets used. But from my limited understanding of wikistats, all non-purely-squid parts consume (directly or indirectly) webstatscollector data, and hence are affected (see original email). So for example, I'd consider
http://stats.wikimedia.org/EN/TablesPageViewsMonthlyOriginalCombined.htm http://stats.wikimedia.org/EN/TablesPageViewsMonthlyCombined.htm http://stats.wikimedia.org/EN/TablesPageViewsMonthlyOriginal.htm http://stats.wikimedia.org/EN/TablesPageViewsMonthly.htm http://stats.wikimedia.org/EN/TablesPageViewsMonthlyOriginalMobile.htm http://stats.wikimedia.org/EN/TablesPageViewsMonthlyMobile.htm
affected by /this/ issue, while I'd consider
http://stats.wikimedia.org/wikimedia/squids/TablesPageViewsMonthlySquidsOrig... http://stats.wikimedia.org/wikimedia/squids/TablesPageViewsMonthlySquidsMobi...
to not be affected by /this/ issue.
@Erik Zachte, please do correct me if the above is wrong.
Because if the latter, I've been investigating some lines in the sampled logs [...]
There are different kinds of sampled logs, and their treatment of ssl differs considerably. However, none of the files underneath
/a/squid/archive/{sampled,mobile,zero,edits}
on stats1002 pass through webstatscollector, and hence they are not affected by /this/ issue.
(Note that although those files they are not affected by /this/ issue, they are affected by other issues like January's https://bugzilla.wikimedia.org/show_bug.cgi?id=60315 )
Have fun, Christian
On Friday, 4 July 2014, Christian Aistleitner christian@quelltextlich.at wrote:
Hi Oliver,
There are different kinds of sampled logs, and their treatment of ssl differs considerably. However, none of the files underneath
/a/squid/archive/{sampled,mobile,zero,edits}
on stats1002 pass through webstatscollector, and hence they are not affected by /this/ issue.
Aha; okay, that clarifies the discrepancy :). Thanks!
(Note that although those files they are not affected by /this/ issue, they are affected by other issues like January's https://bugzilla.wikimedia.org/show_bug.cgi?id=60315 )
Have fun, Christian
-- ---- quelltextlich e.U. ---- \ ---- Christian Aistleitner ---- Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christian@quelltextlich.at javascript:; 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax: +43 7946 / 20 5 81 Homepage: http://quelltextlich.at/