Analytics November 2013

analytics@lists.wikimedia.org

34 participants
20 discussions

Hadoop Administration
by Andrew Otto 20 Nov '13

20 Nov '13

Hi all! I had to add a new JournalNode to a running Hadoop cluster for the first time today. I've started a Hadoop Administration page on wikitech to document the steps for things like this. https://wikitech.wikimedia.org/wiki/Analytics/Kraken/Hadoop/Administration -Ao

2 1

MediaWiki performance analysis
by Yury Katkov 15 Nov '13

15 Nov '13

Hi everyone! I'm searching for some report about MediaWiki performance testing and load testing results. Is such thing available anywhere? ----- Yury Katkov, WikiVote

3 3

Page save instrumentation
by Adam Wight 13 Nov '13

13 Nov '13

Hi, I added some server-side logging during the research hackathon last weekend, and I'd like to solicit feedback. The patch will record all attempts to save through EditPage or ApiEditPage, whether or not they are successful. Would it make sense to log further down, like in doEditContent? Should we also write a log at the beginning of each edit session, to track dropout? https://gerrit.wikimedia.org/r/94512 https://gerrit.wikimedia.org/r/94516 Thanks, Adam

2 1

Broken edit conflict detection
by Adam Wight 13 Nov '13

13 Nov '13

People doing revert analysis and stuff might be interested in this: https://bugzilla.wikimedia.org/show_bug.cgi?id=56849 It turns out, for some time we have been failing to detect edit conflicts if the two page saves occur within one second of each other. The second save will silently overwrite the previous edit, because it derives from an earlier parent... but the parent_id recorded in the revision table will incorrectly point to the conflicting edit. So the edit will look like a reversion plus some editing. G'luck! -Adam

5 8

MediaWiki sites activity analyzier
by Alvaro del Castillo 13 Nov '13

13 Nov '13

Hi guys, There is a new tool in MetricsGrimoire used to analyze the activity in MediaWiki sites: https://github.com/MetricsGrimoire/MediaWikiAnalysis It uses the MediaWiki API so it could be used with any MediaWiki based site (recently enough to have the needed API). You can see it working in the Tech Community Metrics dashboard applied to mediawiki.org: http://korma.wmflabs.org/browser/mediawiki.html Right now it shows: * Total page created and evolution in time of page creation. * The same for editions * The same for editors It has incremental support so in panel is going to be updated daily. The panel does not include bot filtering yet. Cheers -- |\_____/| Alvaro del Castillo San Félix [o] [o] acs(a)bitergia.com - Chief Technical Officer (CTO) | V | http://www.bitergia.com | | "Bridging the gap between developers and stakeholders" -ooo-ooo-

4 11

Queryable public interface for pageview data
by Diederik van Liere 12 Nov '13

12 Nov '13

Heya, We spoke a little bit more about getting a queryable public interface for pageview data up and running and we decided the following: 1) Start importing webstatscollector pageview daily data for 2013 into mysql running on labs (not yet scheduled in a sprint) 2) Make simple datawarehouse schema for mysql db (based on the current webstatscollector datafiles) Page Table ========== page_id page title Fact Table ======== fact_id page_id (FK -> Page Table) pageview date pageview count bytes served 3) Collect more datapoints to determine how high of a priority mobile site article pageview counts are to decide whether we should add this to webstatscollector or not. Best, Diederik

9 24

Presto DB
by Andrew Otto 06 Nov '13

06 Nov '13

https://www.facebook.com/notes/facebook-engineering/presto-interacting-with…

3 2

Analytics slave for s6 not replicating
by Christian Aistleitner 05 Nov '13

05 Nov '13

Hi, just a quick heads up that the analytics slave for s6 (frwiki, jawiki, ruwiki) seems to not be replicating since 2013-10-28 ~07:15. I filed an RT ticket. But if your scripts rely on the slave, expect the numbers to be off until the problem has been fixed. We know that at least the following graphs (and corresponding CSVs) are affected: http://gp.wmflabs.org/graphs/active_editors_total http://gp.wmflabs.org/graphs/frwiki_editor_counts http://gp.wmflabs.org/graphs/jawiki_editor_counts http://gp.wmflabs.org/graphs/ruwiki_editor_counts We'll rerun data aggregation for them after the problem has been fixed. So expect the numbers for >=2013-10-28 to jump after the fix. Best regards, Christian -- ---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ---- Companies' registry: 360296y in Linz Christian Aistleitner Gruendbergstrasze 65a Email: christian(a)quelltextlich.at 4040 Linz, Austria Phone: +43 732 / 26 95 63 Fax: +43 732 / 26 95 63 Homepage: http://quelltextlich.at/ ---------------------------------------------------------------

5 8

Catalan Wikipedia pageviews boost (thanks to the Google box?)
by Quim Gil 04 Nov '13

04 Nov '13

Last January, http://ca.wikipedia.org surpassed the 30M montlhy pageview for the first time. Last month, they surpassed 45M according to http://stats.wikimedia.org/EN/TablesPageViewsMonthlyCombined.htm ! Catalan-speaking editors believe that the Google search box has a lot to do with this. I hope it's not a consequence of some bug somewhere. See the very illustrative 2008-2013 graph at http://www.wikimedia.cat/2013/11/02/la-viquipedia-en-catala-cada-cop-mes-co… This hypothetical increase in pageviews doesn't find a correspondence (yet?) in the increase of number of editors, although they had their 5+ all-time record last January: http://stats.wikimedia.org/EN/SummaryCA.htm Do you see any similarities with Wikipedias in other languages? -- Quim Gil Technical Contributor Coordinator @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

2 1

Fwd: donate.wikimedia.org, ulsfo, and IPV6
by Matthew Walker 01 Nov '13

01 Nov '13

A question came up in Fundraising yesterday that I now have results for -- basically it's how many page requests view the site via IPv6; and how much can we trust ip geolocation on those IPs. tldr + a little extra: We can trust IPv6 geolocation for rough pinpointing of countries. Based on complaints we get from donors, that for those countries with not great connectivity we should take the GeoIP resolution with a large grain of salt. City resolution for the US seems to work OK with the exception of mobile -- I have no anecdotal evidence for anywhere else. -- [snip] Basically it comes down like this: Last month we served 6,166,994,300 banner requests (served to every page in a content namespace -- so not Special pages but pretty much everything else.) - of which 28,739,500 were IPv6 (0.47%) - of which 39,800 failed to resolve to a country, even with IPv4 fallback (0.0006%) -- (in the noise floor we had 300 people resolve to anonymous proxy or 'European Union' IPs; there were no satellite provider or 'Asia Pacific' resolutions recorded) I had Ottomata do a filter on geoiplookup.wikimedia.org (our IPv4 only fallback host). In the period between 22:58 and 23:17 we served 344 requests. In that time period we had roughly 210,000 IPv6 requests. Meaning that our fallback is important for ~0.16% of IPv6 users. I still think we should, in fundraising, do an interstitial page because that's a friendly thing to do. But for sitewide operations; I think the success rate is fairly good. Most of the 'crummyness' that Josh routinely reports is fuzzyness around borders -- e.g. people in India resolving to being in Pakistan. [snip] ~Matt Walker Wikimedia Foundation Fundraising Technology Team

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

Analytics November 2013