Since the beginning of February, English Wikivoyage has seen it's daily
pageviews double:
http://tools.wmflabs.org/siteviews/?platform=all-access&source=pageviews&ag…
This seems to be caused by a sustained spike in desktop-only views to the
Zimbabwe page. Is this maybe a bot that needs to be filtered out?
Hi !
The hadoop cluster maintenance (upgrade to Java 8) was planned to happen
earlier today but is finally happening now.
Il will require a complete shutdown and should not last longer than a
couple of hours (expected less than one).
Thanks !
Joseph on behalf of the Analytics-Team
Hi Analytics folks,
*TL;DR: Hadoop cluster maintenance postponed to Tue 13th February*
We've experienced an issue in getting some data onto the cluster this
month, making some of our monthly datasets (the ones that depend on that
late data) not yet computed.
We have decided to postpone the maintenance of the cluster to next week,
allowing for those jobs to be finished.
We are very sorry about the short notice and will send another email the
day before maintenance.
Best
Joseph Allemandou on behalf of the Analytics-Team
Data Engineer @ Wikimedia Foundation
IRC: joal
*Hey all,We’re thrilled to announce the Wikimedia Research team now has a
simple, navigable, and accessible landing page, making our output,
projects, and resources easy to discover and learn about:
https://research.wikimedia.org <https://research.wikimedia.org/> The
Research team decided to create a single go-to page (T107389
<https://phabricator.wikimedia.org/T107389>) to provide an additional way
to discover information we have on wiki, for the many audiences we would
like to engage with – particularly those who are not already familiar with
how to navigate our projects. On this page, potential academic
collaborators, journalists, funding organizations, and others will find
links to relevant resources, contact information, collaboration and
partnership opportunities, and ways to follow the team's work.There are
many more research resources produced by different teams and departments at
WMF – from Analytics, to Audiences, to Grantmaking, and Programs. If you
see anything that’s missing within the scope of the Research team, please
let us know <https://phabricator.wikimedia.org/T107389>!Dario*
--
*Dario Taraborelli *Director, Head of Research, Wikimedia Foundation
wikimediafoundation.org • nitens.org • @readermeter
<http://twitter.com/readermeter>
Hello,
The pageviews API seems to have been slow to write data for 1/31/2018. It
looks like the data has become available in the past hour, but it's
normally accessible within 3 hours after midnight UTC. Does anybody know
what caused the slowdown, and if we should expect it to continue?
Thank you very much,
-CS
*TL;DR*: The Analytics Hadoop cluster will be completely down for max
2h on *Feb
6th* (EU/CET morning) to upgrade all the daemons to Java 8.
Hi everybody,
we are planning to upgrade the Analytics Hadoop cluster to Java 8 on *Feb
6th* (EU/CET morning) for https://phabricator.wikimedia.org/T166248.
Sadly we can't do a rolling upgrade of all the jvm-based Hadoop daemons
since the distribution that we use (Cloudera) suggests to perform the
upgrade only after a complete cluster shutdown. This means that for a
couple of hours (hopefully a lot less) all the Hadoop based services will
be unavailable (Hive, Oozie, HDFS, etc..).
We have tested the new configuration in labs and all the regular Analytics
jobs seem to work correctly, so we don't expect major issues after the
upgrade, but if you have any question or concern please follow up in the
task.
Thanks!
Luca and Andrew (on behalf of the Analytics team)
Hi Simon,
I copy the analytics mailing list to this message, as this is best way to
get answers to your requests or data or technical aspects of tha analytics
systems.
The dataset you ask for contains data that we don't provide without NDAs.
To be precise, we don't disclose precisely timestamped hits publicly,
trying to prevent easily reconstructible sessions.
Now the easiest way for you to get your hands on that data would be to set
up a formal collaboration with WMF, involving a NDA.
I'm not an expert in how to do that, you might be willing to contact the
research team (wiki-research-l(a)lists.wikimedia.org), and read more here:
https://www.mediawiki.org/wiki/Wikimedia_Research/Formal_collaborations.
Best
Joseph
On Fri, Jan 26, 2018 at 1:55 PM, Jianyun Sun <simonjoylet(a)gmail.com> wrote:
> Hi joal,
>
> I'm a student from Southeast University and now I'm on a research about
> better scheduling of web request.
> For experiment, I need the data of web request of wikimedia, especially
> page request records with timestamps and response size. Only a month-long
> data is enough. Would you please send me a copy or help me get an access
> ticket on Hive so I can get it by myself?
>
> I'm looking forward to your reply. Thank you sincerely!
>
>
> Simon
> 2018.01.26
>
--
*Joseph Allemandou*
Data Engineer @ Wikimedia Foundation
IRC: joal
--
*Joseph Allemandou*
Data Engineer @ Wikimedia Foundation
IRC: joal