January 2013 - Analytics - lists.wikimedia.org

Access to view stats
by Magnus Manske 16 Jun '13

16 Jun '13

Hi all, as you might know, I have a few GLAM-related tools on the toolserver. Some are updated once a month, some can be used live, but all are in high demand by GLAM institutions. Now, the monthly updated stats have always been slow to run, but did almost grind to a halt recently. The on-demand tools have stalled completely. All these tools get their data from stats.grok.se, which works well but not really high-speed; my on-demand tools have apparently been shut out recently because too many people were using them, DDOSing the server :-( I know you are working on page view numbers, and for what I gather it's up-and-running internally already. My requirements are simple: I have a list of pages on many Wikimedia projects; I need view counts for these pages for a specific month, per-page. Now, I know that there is no public API yet, but is there any way I can get to the data, at least for the monthly stats? Cheers, Magnus

11 29

Skin and active editor correlation
by Matthew Flaschen 28 Mar '13

28 Mar '13

There's discussion at https://bugzilla.wikimedia.org/show_bug.cgi?id=44448 about how skin usage correlates with who's an active editor. It would be great to know what percentage of active editor (5+ edits in the main namespace) uses each skin on English Wikipedia. Perhaps for the last three months. Matt Flaschen

6 22

RFC: Tab as field delimiter in logging format of cache servers
by Diederik van Liere 21 Mar '13

21 Mar '13

Apologies for crossposting Heya, The Analytics Team is planning to deploy "tab as field delimiter" to replace the current space as fielddelimiter on the varnish/squid/nginx servers. We would like to do this on February 1st. The reason for this change is that we need to have a consistent number of fields in each webrequest log line. Right now, some fields contain spaces and that require a lot of post-processing cleanup and slows down the generation of reports. What is affected and maintained by Analytics * udp-filter already has support for the tab character * webstatscollector: we compiled a new version of filter to add support for the tab character * wikistats: we will fix the scripts on an ongoing basis. * udp2log: we have a patch ready for inserting sequence numbers separated by tab. In particular, I would like to have feedback to three questions: 1) Are there important reasons not to use tab as field delimiter? 2) Are there important pieces of logging that expect a space instead of a tab and that need to be fixed and that I did not mention in this email? 3) Is February 1st a good date to deploy this change? (Assuming that all preps are finished) Best, Diederik

11 19

wikidata stats
by Ed Summers 01 Feb '13

01 Feb '13

Hi all, There has been some recent discussion [1] on wikidata-l about making sure wikidata.org page views are part of the stats that are being collected at stats.grok.se. I did a quick check to see if page views on www.wikidata.org were showing up, and they don't appear to be. curl --silent http://dumps.wikimedia.org/other/pagecounts-raw/2013/2013-01/pagecounts-201… | zcat - | egrep ' Q\d+ ' de Q100 2 15502 de Q5 2 95286 en Q0 1 7815 en Q1 6 70841 en Q10 2 32918 en Q101 1 7709 en Q102 1 8128 en Q107 1 26148 en Q11 1 387 en Q19 2 14304 en Q2 7 44454 en Q3 1 8624 en Q35 1 22856 en Q4 3 40588 en Q4000 2 28866 en Q5 3 16698 en Q612 1 7152 en Q6600 2 29929 en Q7 2 16604 en Q9450 2 59452 fr Q4 2 24352 fr Q400 1 17339 hu Q10 3 105745 ja Q0 1 8685 ja Q10 36 781763 ja Q4 1 8873 ja Q400 1 25208 ko Q3 1 22350 nl Q65 2 24562 ru Q10 1 31355 ru Q4000 1 11140 sv Q10 1 365 zh Q1 1 10950 zh Q10 2 17738 Does anyone have any idea how to get stats.grok.se updated? I think I've seen from previous conversation here that an alternate source for stats is being created? Is there any information available on how to use that if it's available yet? //Ed [1] http://lists.wikimedia.org/pipermail/wikidata-l/2013-January/001646.html

4 7

Return of Wikihadoop
by Diederik van Liere 28 Jan '13

28 Jan '13

Heya, Last week I resurrected Wikihadoop from the Summer of Research 2011 when we wrote a Hadoop-based input parser for bzipped XML dumps of Wikipedia and the ability to create diffs between revisions. This work was mainly done by Yusuke Matsubara, Aaron Halfaker and Fabian Kaelin. This is working again and if you have input / suggestions on how to expose this data, then please let me know! Best, Diederik

1 0

Issues with Emery
by Diederik van Liere 28 Jan '13

28 Jan '13

Heya, Since yesterday afternoon we have been having intermittent packetloss issues with Emery. Ori and myself wrote yesterday a simple patch to enable sampling on two filters to reduce the workload. That seemed to work however this morning packetloss came back. Paravoid pointed out that it was probably due to the fact that Emery was running for 208 days and that older versions of the kernel start doing weird things. Rebooting Emery seemed to have resolved the issue and we are planning to upgrade to Precise soon. Best, Diederik

2 1

Access to view stats
by Brian Keegan 25 Jan '13

25 Jan '13

I wanted to follow up and second on Magnus' December request<http://lists.wikimedia.org/pipermail/analytics/2012-December/000272.html>for more usable way to access page view stats. Mining these stats is attracting an increasing amount of attention from researchers ( http://www.l3s.de/~kanhabua/papers/ECIR2013-WikiEvents.pdf, http://arxiv.org/pdf/1212.5943v1.pdf, http://arxiv.org/pdf/1211.0970.pdf) even as the current approaches for extracting them from stats.grok.se or the dumps are slow and inhumane (respectively). I'm also interested in looking at bursts of pageview activity on articles and then examining the extent to which this pageview activity diffuses over the local wiki-link network. I suspect this has strong implications for understanding patterns of editing activity; namely, editing activity may be non-trivially coupled with sudden attention to articles that are a few degrees of separation away. I'd be happy to chat with folks inside or outside of WMF about getting access to the relevant view stats and beginning such an analysis. Best, Brian

1 0

The Uniﬁed Logging Infrastructure for Data Analytics at Twitter
by Dario Taraborelli 22 Jan '13

22 Jan '13

http://arxiv.org/pdf/1208.4171.pdf This is a pretty interesting and accessible description of best practices and design decisions driven by practical problems they had to solve at Twitter in the area of client-side event logging, funnel analysis, user modeling. E3: check out section "3.2 Client Events" in particular, which is quite relevant to EventLogging. Dario

4 6

getting Wikisource added to the page-view data logs
by Sumana Harihareswara 22 Jan '13

22 Jan '13

I saw this discussion: https://en.wikipedia.org/wiki/User_talk:Emw#Stats thanks to Andrea Zanni (cc'd), who would like to add Wikisources to this tool: http://toolserver.org/~emw/wikistats/?p1=Coupling_%28computer_programming%2… In order for that to happen, Emw said, we'd need to add an "identifier for that project in the raw page-view data". How feasible is that? Thanks. -- Sumana Harihareswara Engineering Community Manager Wikimedia Foundation

6 5

Pageview counts for wikivoyage domain are available
by Diederik van Liere 17 Jan '13

17 Jan '13

Hey everyone! Pageview counts for the wikivoyage project are availabe on dumps.wikimedia.org as part of the regular hourly article and project counts files. The first file that contains them has the timestamp " 20130116-160001<http://dumps.wikimedia.org/other/pagecounts-raw/2013/2013-01/projectcounts-…> " Example file http://dumps.wikimedia.org/other/pagecounts-raw/2013/2013-01/projectcounts-… and search for ".voy". If you run into any issues please let us know or file a bug on bugzilla (Analytics -> webstatscollector). Best regards, Diederik

2 1

Article stats
by Itzik Edri 16 Jan '13

16 Jan '13

Hi, Someone knows about a tool or someone who can help with getting articles stats from csv file? Next week is the election day in Israel and I'm looking to get articles stats of the parties and the candidate. to do it one by one will take too much time. If there is any tool which can get CSV file with the article names and return CSV file with the data - will be great. Thanks, Itzik

2 2

Most frequently edited articles in a category?
by Tilman Bayer 15 Jan '13

15 Jan '13

Does anyone know how to find or generate a list of the most frequently edit Wikipedia articles in a certain category? We received a question from a large newspaper who is interested in learning which topics from their country received the most editor attention on the English Wikipedia. There are tools like http://www.wikistats.co/ (most edits during the past 24h hours) and Wikirage (http://en.wikipedia.org/wiki/Wikipedia:Wikirage , seems currently defunct though). But they don't offer restricting the list per category. I guess I'm looking for something like http://toolserver.org/~magnus/ts2/treeviews/ with edit counts instead of page views, and capable of handling large category trees like that for a country. Hints about related results are appreciated too (e.g. articles most frequently edited *from* a certain country). -- Tilman Bayer Senior Operations Analyst (Movement Communications) Wikimedia Foundation

2 1

Page views on Meta?
by Bence Damokos 13 Jan '13

13 Jan '13

Hi all, How come there are 11 billion page views counted on Meta in November and December according to http://stats.wikimedia.org/wikispecial/EN/SummaryMETA.htm and http://stats.wikimedia.org/wikispecial/EN/TablesPageViewsMonthly.htm ? Best regards, Bence

2 1

Replication halted on db1047
by Aaron Halfaker 13 Jan '13

13 Jan '13

Hey folks, I can't remember who to contact about this, It looks like DB replication on db1047 halted on Jan. 8th. Specifically, the last event in the recentchanges table is 20130108215551*. * What's the current protocol for giving it a kick? -Aaron

3 2

Data Visualization Summit, April 2013, in SF
by Erik Zachte 11 Jan '13

11 Jan '13

FYI http://analytics.theiegroup.com/data-visualization Erik

3 5

Suspicious increase in mobile pv's
by Diederik van Liere 09 Jan '13

09 Jan '13

Hi all, It has come to our attention that there is quite a big jump in mobile pageviews on http://stats.wikimedia.org/EN/TablesPageViewsMonthlyMobile.htm We are looking into the exact cause of it right now, but suffice to say for now, do not use these numbers for the month of December 2012. Best, Diederik

9 17

Community Metrics - December report
by Quim Gil 07 Jan '13

07 Jan '13

Third issue of the MediaWiki community metrics monthly report! http://www.mediawiki.org/wiki/Community_metrics/December_2012 We have added a section with social media stats. Hopefully next month we will show mediawiki.org data to reflect the documentation work. There is a big increase in the total amount of lines of code and active contributors. The addition of Kiwix and MediaWiki Editor Engagement Experiments has something to do with this. Probably Ohloh crawled more of the hundreds of repositories we have on file. But underneath this there is the main factor that the MediaWiki community is growing year-to-year! As a contrast, MediaWiki core got the lowest amount of commits since June 2010 - probably related with the post 1.20.* releases + holidays? -- Quim Gil Technical Contributor Coordinator @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

1 0

How to get (category X country X article) PV data
by prasenjit mukherjee 07 Jan '13

07 Jan '13

I would like to know the top Wikipedia articles in music category for a country ( lets say India ). Is there a way I can get access to this data. If this data is not readily available can I get the per-article PV data for a given country, so that I can infer my desired outcome. Thanks, Prasenjit

3 3

Page edits from mobile devices
by Jan Ainali 06 Jan '13

06 Jan '13

Hello! Last month saw a huge increase in page views from mobile devices, up 44% overall. http://stats.wikimedia.org/EN/TablesPageViewsMonthlyMobile.htm I think it is time to start publish page edits from mobile devices. It is probably not a lot right now, but with all the tablets around it is probably more than from non-mobile on several smaller wikis. Would that be possible? *Med vänliga hälsningar, Jan Ainali* Ordförande, Wikimedia Sverige <http://se.wikimedia.org/wiki/Huvudsida> 076-2122776

2 1

Frontend Webrequest Log Format Changes
by Andrew Otto 02 Jan '13

02 Jan '13

Hi all! At the request of the Wikipedia Zero folks, we have finally merged and deployed this change: https://gerrit.wikimedia.org/r/#/c/12188/ This adds two new fields to the end of all webrequest log lines. 1) Accept-Language HTTP header. This will support the Internationalization and Language team with creating new metrics. 2) X-Carrier HTTP header. This will simplify detecting whether a request was made through the Wikipedia Zero project Accept-Language comes from the user's browser, so this will be filled in for the 15th field on many of the lines. X-Carrier is manually put in place by the mobile Varnish frontends, so most of the request will not have anything useful here. As usual, if any of the fields are blank, a single '-' will be in the field's position. We've got a few upcoming changes too. We still want to change the field separator in these lines from space to tab. We are also considering geocoding all IP addresses by default, which would add a 17th field for the geolocation. We'll keep you updated if and when those changes get ready to go in place. Thanks everybody! -Ao

1 0

Maintainers of Analytics Projects
by David Schoonover 02 Jan '13

02 Jan '13

Hiya all, I've updated https://www.mediawiki.org/wiki/Developers/Maintainers with info about analytics projects, assigning them owners based on what I recall. The goal is to make it easy for contributors to figure out who to talk to if they want to help out. (So please check this page to make sure it's right.) Also, if I've missed any projects, please add them. Cheers! -- David Schoonover dsc(a)wikimedia.org

1 0

Fwd: [Wmfall] Wikipedia's most-searched articles
by Dario Taraborelli 02 Jan '13

02 Jan '13

forwarding to the list Begin forwarded message: > From: Matthew Roth <mroth(a)wikimedia.org> > Subject: [Wmfall] Wikipedia's most-searched articles > Date: January 2, 2013 10:39:29 AM PST > To: "Staff (All)" <wmfall(a)lists.wikimedia.org> > > Hi all, > I imagine you may have seen some of this coverage already, but in case not, there have been a number of stories about the most searched Wikipedia articles on various language Wikipedias in 2012. A researcher created the lists on toolserver and it has gotten quite a bit of attention. > > The lists: > http://toolserver.org/~johang/2012.html > > > And, I'd point you to a couple of the stories here (I think the Atlantic one is particularly interesting): > > All Things D | The most popular pages on Wikipedia in 2012... (27 December 2012) > http://allthingsd.com/20121227/the-most-popular-pages-on-wikipedia-in-2012-… > > BBC News | Wikipedia's most-searched articles of the year revealed (28 December 2012) > http://www.bbc.co.uk/news/technology-20858333 > > The Atlantic Cities | WIkipedia's Most Popular Pages, in 35 Languages (29 December 2012) > http://www.theatlanticcities.com/technology/2012/12/wikipedias-most-popular… > > > > -- > > Matthew Roth > Global Communications Manager > Wikimedia Foundation > +1.415.839.6885 ext 6635 > www.wikimediafoundation.org > https://donate.wikimedia.org > _______________________________________________ > Wmfall mailing list > Wmfall(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wmfall

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

Analytics January 2013