Hey all,
heads up that a refreshed Wikipedia Clickstream dataset is now available for March 2016, containing 25 million (referer, resource) pairs extracted from about 7 billion webrequests.
https://dx.doi.org/10.6084/m9.figshare.1305770.v16
Ellery (the author of the dataset) is cc'ed if you have any questions, or you can chime in on the talk page of the dataset entry on Meta https://meta.wikimedia.org/wiki/Research:Wikipedia_clickstream.
Show us what you do with this data, if you use it in your research.
Dario
*Dario Taraborelli *Head of Research, Wikimedia Foundation wikimediafoundation.org • nitens.org • @readermeter http://twitter.com/readermeter
Hi Dario,
This data is super interesting! How realistic is it that your team make it available through the Wikimedia REST API [1]? I would then in turn love to add it to Wikipedia Tools [2], just imagine how amazing it would be to be able to ask a spreadsheet for…
=WIKI{OUT|IN}BOUNDTRAFFIC("en:London", TODAY()-2, TODAY()-1)
…(or obviously the API respectively) and get the results back immediately without the need to download a dump first. What do you think?
Cheers, Tom
-- [1] https://wikimedia.org/api/rest_v1/?doc [2] http://bit.ly/wikipedia-tools-add-on
Hey Thomas,
yes, I agree this dataset is really valuable (just looking at the sheer number of downloads [1] and requests for similar data we've received). I can see the value of making it more easily accessible via an API.
Ellery and I have been talking about the idea of – at the very least – scheduling the generation of new dumps, if not exposing the data programmatically. Right now, I am afraid this is not within my team's capacity and Analytics has a number of other high-priority areas to focus on. We were planning to talk to Joseph et al anyway and decide how to move forward (hi Joseph!), we'll report back on the lists as soon as this happens.
Dario
[1] https://figshare.altmetric.com/details/3707715
On Mon, May 2, 2016 at 3:12 AM, Thomas Steiner tomac@google.com wrote:
Hi Dario,
This data is super interesting! How realistic is it that your team make it available through the Wikimedia REST API [1]? I would then in turn love to add it to Wikipedia Tools [2], just imagine how amazing it would be to be able to ask a spreadsheet for…
=WIKI{OUT|IN}BOUNDTRAFFIC("en:London", TODAY()-2, TODAY()-1)
…(or obviously the API respectively) and get the results back immediately without the need to download a dump first. What do you think?
Cheers, Tom
-- [1] https://wikimedia.org/api/rest_v1/?doc [2] http://bit.ly/wikipedia-tools-add-on
-- Dr. Thomas Steiner, Employee (http://blog.tomayac.com, https://twitter.com/tomayac)
Google Germany GmbH, ABC-Str. 19, 20354 Hamburg, Germany Managing Directors: Matthew Scott Sucherman, Paul Terence Manicle Registration office and registration number: Hamburg, HRB 86891
-----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.29 (GNU/Linux)
iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck0fjumBl3DCharaCTersAttH3b0ttom hTtPs://xKcd.cOm/1181/ -----END PGP SIGNATURE-----
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
wiki-research-l@lists.wikimedia.org