Anybody know about stats.grok.se going down? - Analytics

Dan Garry

7 Aug 7 Aug

8:21 a.m.

Hi Vipul, stats.grok.se is pretty much deprecated now. You ran in to one of the reasons why: it's not very reliable. You should use the Pageviews Analysis <https://tools.wmflabs.org/pageviews/> tool instead, which was put together by MusikAnimal and Community Tech. This tool was intended to replace stats.grok.se. There is documentation <https://meta.wikimedia.org/wiki/Community_Tech/Pageview_stats_tool> about the tool that you may wish to read. Thanks, Dan On 7 August 2017 at 06:34, Vipul Naik <vipulnaik1(a)gmail.com> wrote:

...

stats.grok.se (a source of pageview stats for the time before the Wikimedia API became available) has been down for about a week. I tried emailing Henrik Abelsson, whom I've previously contacted when the site had issues, but haven't received a response this time. Any ideas on why it's down and whom to reach out to to help resolve the issue? Vipul _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

-- Dan Garry Senior Product Manager, Editing Wikimedia Foundation

Reply

Vipul Naik

12 Aug 12 Aug

10:59 p.m.

Hi Dan and Dan, Thanks for taking the time to respond. I appreciate it! I'm aware of the APIs and the WMF Labs tool. I am specifically interested in stats.grok.se for accessing data *before* July 2015, for which the only way right now is to process rather large raw dumps. I have built-in integrations that get data from stats.grok.se; processing raw dumps to generate pageview counts is possible but a lot of extra work :). Cheers, Vipul On Mon, Aug 7, 2017 at 4:17 AM, Dan Andreescu <dandreescu(a)wikimedia.org> wrote:

...

And if you need more of an API / raw data download, take a look at: https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews (available at https://wikimedia.org/api/rest_v1/) and: https://dumps.wikimedia.org/other/pagecounts-ez/ On Mon, Aug 7, 2017 at 4:21 AM, Dan Garry <dgarry(a)wikimedia.org> wrote:

Hi Vipul, stats.grok.se is pretty much deprecated now. You ran in to one of the reasons why: it's not very reliable. You should use the Pageviews Analysis <https://tools.wmflabs.org/pageviews/> tool instead, which was put together by MusikAnimal and Community Tech. This tool was intended to replace stats.grok.se. There is documentation <https://meta.wikimedia.org/wiki/Community_Tech/Pageview_stats_tool> about the tool that you may wish to read. Thanks, Dan On 7 August 2017 at 06:34, Vipul Naik <vipulnaik1(a)gmail.com> wrote:

stats.grok.se (a source of pageview stats for the time before the Wikimedia API became available) has been down for about a week. I tried emailing Henrik Abelsson, whom I've previously contacted when the site had issues, but haven't received a response this time. Any ideas on why it's down and whom to reach out to to help resolve the issue? Vipul _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

-- Dan Garry Senior Product Manager, Editing Wikimedia Foundation _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Reply

Vipul Naik

3:12 p.m.

Hi Dan,

...

From the documentation of legacy metrics it looks like the legacy metrics

are only available for sitewide pageviews for each site, rather than for individual pages. Is per-page data also part of your existing or planned legacy metrics? Vipul On Sat, Aug 12, 2017 at 6:17 PM, Dan Andreescu <dandreescu(a)wikimedia.org> wrote:

...

Hi Vipul, actually that's also available via the API now! https://wikitech.wikimedia.org/wiki/Analytics/AQS/Legacy_Pagecounts It's a different path though, to highlight that pre-2015 numbers were counted slightly differently. On Sat, Aug 12, 2017 at 18:59 Vipul Naik <vipulnaik1(a)gmail.com> wrote:

Hi Dan and Dan, Thanks for taking the time to respond. I appreciate it! I'm aware of the APIs and the WMF Labs tool. I am specifically interested in stats.grok.se for accessing data *before* July 2015, for which the only way right now is to process rather large raw dumps. I have built-in integrations that get data from stats.grok.se; processing raw dumps to generate pageview counts is possible but a lot of extra work :). Cheers, Vipul On Mon, Aug 7, 2017 at 4:17 AM, Dan Andreescu <dandreescu(a)wikimedia.org> wrote:

And if you need more of an API / raw data download, take a look at: https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews (available at https://wikimedia.org/api/rest_v1/) and: https://dumps.wikimedia.org/other/pagecounts-ez/ On Mon, Aug 7, 2017 at 4:21 AM, Dan Garry <dgarry(a)wikimedia.org> wrote:

Hi Vipul, stats.grok.se is pretty much deprecated now. You ran in to one of the reasons why: it's not very reliable. You should use the Pageviews Analysis <https://tools.wmflabs.org/pageviews/> tool instead, which was put together by MusikAnimal and Community Tech. This tool was intended to replace stats.grok.se. There is documentation <https://meta.wikimedia.org/wiki/Community_Tech/Pageview_stats_tool> about the tool that you may wish to read. Thanks, Dan On 7 August 2017 at 06:34, Vipul Naik <vipulnaik1(a)gmail.com> wrote: > stats.grok.se (a source of pageview stats for the time before the > Wikimedia API became available) has been down for about a week. I tried > emailing Henrik Abelsson, whom I've previously contacted when the site had > issues, but haven't received a response this time. > > Any ideas on why it's down and whom to reach out to to help resolve > the issue? > > Vipul > > _______________________________________________ > Analytics mailing list > Analytics(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/analytics > > -- Dan Garry Senior Product Manager, Editing Wikimedia Foundation _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Reply

Leon Ziemba

18 Aug 18 Aug

4:35 p.m.

For the record, this is what T149358 <https://phabricator.wikimedia.org/T149358> was originally about <https://phabricator.wikimedia.org/T149358#3106745>. I was under the impression we were going to have pagecounts for all endpoints (per-article, top and aggregate), and it was somewhat disappointing to find out we only added support for aggregate. From my experience per-article data is actually of greatest interest, and I've gotten requests to add it to Pageviews Analysis since its inception. This was also part of one the top wishes in the German Technical Wishlist (I can dig up a link if need be). In addition, some things like the Did You Know <https://en.wikipedia.org/wiki/Wikipedia:Did_you_know> project on enwiki rely on it, where tens of thousands of template <https://en.wikipedia.org/wiki/Template:DYK_talk> transclusions link to stats.grok.se on article talk pages (see the template test cases <https://en.wikipedia.org/w/index.php?title=Template:DYK_talk/testcases&oldid=796118708#Live> for how this works). With stats.grok.se now gone, we have no public-facing web service to get this historical data. So I'd love to see it added to the awesome RESTBase API, but I understand it probably involves a lot of challenges. I can create another phabricator task if Vipul has not already. At any rate, I have endless thanks to give to the Analytics team for everything you've done for us. It seems we're always asking more from you! :) R.I.P. stats.grok.se! 10 years was a good run! ~MA On Sun, Aug 13, 2017 at 1:15 PM, Dan Andreescu <dandreescu(a)wikimedia.org> wrote:

...

Ah, yes, for now we have no plans to add the per-article stats, but do open a task and explain how it would be useful, we'll prioritize it accordingly. And in the meantime, looks like the pagecounts-ez are your best bet (use that instead of pagecounts-raw because the compression is lossless and saves a lot of download time) *From: *Vipul Naik *Sent: *Sunday, August 13, 2017 11:12 *To: *A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. *Reply To: *A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. *Subject: *Re: [Analytics] Anybody know about stats.grok.se going down? Hi Dan, From the documentation of legacy metrics it looks like the legacy metrics are only available for sitewide pageviews for each site, rather than for individual pages. Is per-page data also part of your existing or planned legacy metrics? Vipul On Sat, Aug 12, 2017 at 6:17 PM, Dan Andreescu <dandreescu(a)wikimedia.org> wrote:

Hi Vipul, actually that's also available via the API now! https://wikitech.wikimedia.org/wiki/Analytics/AQS/Legacy_Pagecounts It's a different path though, to highlight that pre-2015 numbers were counted slightly differently. On Sat, Aug 12, 2017 at 18:59 Vipul Naik <vipulnaik1(a)gmail.com> wrote:

Hi Dan and Dan, Thanks for taking the time to respond. I appreciate it! I'm aware of the APIs and the WMF Labs tool. I am specifically interested in stats.grok.se for accessing data *before* July 2015, for which the only way right now is to process rather large raw dumps. I have built-in integrations that get data from stats.grok.se; processing raw dumps to generate pageview counts is possible but a lot of extra work :). Cheers, Vipul On Mon, Aug 7, 2017 at 4:17 AM, Dan Andreescu <dandreescu(a)wikimedia.org> wrote:

And if you need more of an API / raw data download, take a look at: https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews (available at https://wikimedia.org/api/rest_v1/) and: https://dumps.wikimedia.org/other/pagecounts-ez/ On Mon, Aug 7, 2017 at 4:21 AM, Dan Garry <dgarry(a)wikimedia.org> wrote: > Hi Vipul, > > stats.grok.se is pretty much deprecated now. You ran in to one of the > reasons why: it's not very reliable. You should use the Pageviews > Analysis <https://tools.wmflabs.org/pageviews/> tool instead, which > was put together by MusikAnimal and Community Tech. This tool was intended > to replace stats.grok.se. There is documentation > <https://meta.wikimedia.org/wiki/Community_Tech/Pageview_stats_tool> about > the tool that you may wish to read. > > Thanks, > Dan > > On 7 August 2017 at 06:34, Vipul Naik <vipulnaik1(a)gmail.com> wrote: > >> stats.grok.se (a source of pageview stats for the time before the >> Wikimedia API became available) has been down for about a week. I tried >> emailing Henrik Abelsson, whom I've previously contacted when the site had >> issues, but haven't received a response this time. >> >> Any ideas on why it's down and whom to reach out to to help resolve >> the issue? >> >> Vipul >> >> _______________________________________________ >> Analytics mailing list >> Analytics(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/analytics >> >> > > > -- > Dan Garry > Senior Product Manager, Editing > Wikimedia Foundation > > _______________________________________________ > Analytics mailing list > Analytics(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/analytics > > _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Reply