stats.grok.se (a source of pageview stats for the time before the Wikimedia API became available) has been down for about a week. I tried emailing Henrik Abelsson, whom I've previously contacted when the site had issues, but haven't received a response this time.
Any ideas on why it's down and whom to reach out to to help resolve the issue?
Vipul
Hi Vipul,
stats.grok.se is pretty much deprecated now. You ran in to one of the reasons why: it's not very reliable. You should use the Pageviews Analysis https://tools.wmflabs.org/pageviews/ tool instead, which was put together by MusikAnimal and Community Tech. This tool was intended to replace stats.grok.se. There is documentation https://meta.wikimedia.org/wiki/Community_Tech/Pageview_stats_tool about the tool that you may wish to read.
Thanks, Dan
On 7 August 2017 at 06:34, Vipul Naik vipulnaik1@gmail.com wrote:
stats.grok.se (a source of pageview stats for the time before the Wikimedia API became available) has been down for about a week. I tried emailing Henrik Abelsson, whom I've previously contacted when the site had issues, but haven't received a response this time.
Any ideas on why it's down and whom to reach out to to help resolve the issue?
Vipul
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
And if you need more of an API / raw data download, take a look at:
https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews (available at https://wikimedia.org/api/rest_v1/)
and:
https://dumps.wikimedia.org/other/pagecounts-ez/
On Mon, Aug 7, 2017 at 4:21 AM, Dan Garry dgarry@wikimedia.org wrote:
Hi Vipul,
stats.grok.se is pretty much deprecated now. You ran in to one of the reasons why: it's not very reliable. You should use the Pageviews Analysis https://tools.wmflabs.org/pageviews/ tool instead, which was put together by MusikAnimal and Community Tech. This tool was intended to replace stats.grok.se. There is documentation https://meta.wikimedia.org/wiki/Community_Tech/Pageview_stats_tool about the tool that you may wish to read.
Thanks, Dan
On 7 August 2017 at 06:34, Vipul Naik vipulnaik1@gmail.com wrote:
stats.grok.se (a source of pageview stats for the time before the Wikimedia API became available) has been down for about a week. I tried emailing Henrik Abelsson, whom I've previously contacted when the site had issues, but haven't received a response this time.
Any ideas on why it's down and whom to reach out to to help resolve the issue?
Vipul
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Dan Garry Senior Product Manager, Editing Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Hi Dan and Dan,
Thanks for taking the time to respond. I appreciate it!
I'm aware of the APIs and the WMF Labs tool. I am specifically interested in stats.grok.se for accessing data *before* July 2015, for which the only way right now is to process rather large raw dumps. I have built-in integrations that get data from stats.grok.se; processing raw dumps to generate pageview counts is possible but a lot of extra work :).
Cheers,
Vipul
On Mon, Aug 7, 2017 at 4:17 AM, Dan Andreescu dandreescu@wikimedia.org wrote:
And if you need more of an API / raw data download, take a look at:
https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews (available at https://wikimedia.org/api/rest_v1/)
and:
https://dumps.wikimedia.org/other/pagecounts-ez/
On Mon, Aug 7, 2017 at 4:21 AM, Dan Garry dgarry@wikimedia.org wrote:
Hi Vipul,
stats.grok.se is pretty much deprecated now. You ran in to one of the reasons why: it's not very reliable. You should use the Pageviews Analysis https://tools.wmflabs.org/pageviews/ tool instead, which was put together by MusikAnimal and Community Tech. This tool was intended to replace stats.grok.se. There is documentation https://meta.wikimedia.org/wiki/Community_Tech/Pageview_stats_tool about the tool that you may wish to read.
Thanks, Dan
On 7 August 2017 at 06:34, Vipul Naik vipulnaik1@gmail.com wrote:
stats.grok.se (a source of pageview stats for the time before the Wikimedia API became available) has been down for about a week. I tried emailing Henrik Abelsson, whom I've previously contacted when the site had issues, but haven't received a response this time.
Any ideas on why it's down and whom to reach out to to help resolve the issue?
Vipul
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Dan Garry Senior Product Manager, Editing Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Hi Vipul, actually that's also available via the API now! https://wikitech.wikimedia.org/wiki/Analytics/AQS/Legacy_Pagecounts
It's a different path though, to highlight that pre-2015 numbers were counted slightly differently. On Sat, Aug 12, 2017 at 18:59 Vipul Naik vipulnaik1@gmail.com wrote:
Hi Dan and Dan,
Thanks for taking the time to respond. I appreciate it!
I'm aware of the APIs and the WMF Labs tool. I am specifically interested in stats.grok.se for accessing data *before* July 2015, for which the only way right now is to process rather large raw dumps. I have built-in integrations that get data from stats.grok.se; processing raw dumps to generate pageview counts is possible but a lot of extra work :).
Cheers,
Vipul
On Mon, Aug 7, 2017 at 4:17 AM, Dan Andreescu dandreescu@wikimedia.org wrote:
And if you need more of an API / raw data download, take a look at:
https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews (available at https://wikimedia.org/api/rest_v1/)
and:
https://dumps.wikimedia.org/other/pagecounts-ez/
On Mon, Aug 7, 2017 at 4:21 AM, Dan Garry dgarry@wikimedia.org wrote:
Hi Vipul,
stats.grok.se is pretty much deprecated now. You ran in to one of the reasons why: it's not very reliable. You should use the Pageviews Analysis https://tools.wmflabs.org/pageviews/ tool instead, which was put together by MusikAnimal and Community Tech. This tool was intended to replace stats.grok.se. There is documentation https://meta.wikimedia.org/wiki/Community_Tech/Pageview_stats_tool about the tool that you may wish to read.
Thanks, Dan
On 7 August 2017 at 06:34, Vipul Naik vipulnaik1@gmail.com wrote:
stats.grok.se (a source of pageview stats for the time before the Wikimedia API became available) has been down for about a week. I tried emailing Henrik Abelsson, whom I've previously contacted when the site had issues, but haven't received a response this time.
Any ideas on why it's down and whom to reach out to to help resolve the issue?
Vipul
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Dan Garry Senior Product Manager, Editing Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Hi Dan,
From the documentation of legacy metrics it looks like the legacy metrics
are only available for sitewide pageviews for each site, rather than for individual pages. Is per-page data also part of your existing or planned legacy metrics?
Vipul
On Sat, Aug 12, 2017 at 6:17 PM, Dan Andreescu dandreescu@wikimedia.org wrote:
Hi Vipul, actually that's also available via the API now! https://wikitech.wikimedia.org/wiki/Analytics/AQS/Legacy_Pagecounts
It's a different path though, to highlight that pre-2015 numbers were counted slightly differently.
On Sat, Aug 12, 2017 at 18:59 Vipul Naik vipulnaik1@gmail.com wrote:
Hi Dan and Dan,
Thanks for taking the time to respond. I appreciate it!
I'm aware of the APIs and the WMF Labs tool. I am specifically interested in stats.grok.se for accessing data *before* July 2015, for which the only way right now is to process rather large raw dumps. I have built-in integrations that get data from stats.grok.se; processing raw dumps to generate pageview counts is possible but a lot of extra work :).
Cheers,
Vipul
On Mon, Aug 7, 2017 at 4:17 AM, Dan Andreescu dandreescu@wikimedia.org wrote:
And if you need more of an API / raw data download, take a look at:
https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews (available at https://wikimedia.org/api/rest_v1/)
and:
https://dumps.wikimedia.org/other/pagecounts-ez/
On Mon, Aug 7, 2017 at 4:21 AM, Dan Garry dgarry@wikimedia.org wrote:
Hi Vipul,
stats.grok.se is pretty much deprecated now. You ran in to one of the reasons why: it's not very reliable. You should use the Pageviews Analysis https://tools.wmflabs.org/pageviews/ tool instead, which was put together by MusikAnimal and Community Tech. This tool was intended to replace stats.grok.se. There is documentation https://meta.wikimedia.org/wiki/Community_Tech/Pageview_stats_tool about the tool that you may wish to read.
Thanks, Dan
On 7 August 2017 at 06:34, Vipul Naik vipulnaik1@gmail.com wrote:
stats.grok.se (a source of pageview stats for the time before the Wikimedia API became available) has been down for about a week. I tried emailing Henrik Abelsson, whom I've previously contacted when the site had issues, but haven't received a response this time.
Any ideas on why it's down and whom to reach out to to help resolve the issue?
Vipul
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Dan Garry Senior Product Manager, Editing Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
For the record, this is what T149358 https://phabricator.wikimedia.org/T149358 was originally about https://phabricator.wikimedia.org/T149358#3106745. I was under the impression we were going to have pagecounts for all endpoints (per-article, top and aggregate), and it was somewhat disappointing to find out we only added support for aggregate. From my experience per-article data is actually of greatest interest, and I've gotten requests to add it to Pageviews Analysis since its inception. This was also part of one the top wishes in the German Technical Wishlist (I can dig up a link if need be). In addition, some things like the Did You Know https://en.wikipedia.org/wiki/Wikipedia:Did_you_know project on enwiki rely on it, where tens of thousands of template https://en.wikipedia.org/wiki/Template:DYK_talk transclusions link to stats.grok.se on article talk pages (see the template test cases https://en.wikipedia.org/w/index.php?title=Template:DYK_talk/testcases&oldid=796118708#Live for how this works). With stats.grok.se now gone, we have no public-facing web service to get this historical data. So I'd love to see it added to the awesome RESTBase API, but I understand it probably involves a lot of challenges. I can create another phabricator task if Vipul has not already. At any rate, I have endless thanks to give to the Analytics team for everything you've done for us. It seems we're always asking more from you! :)
R.I.P. stats.grok.se! 10 years was a good run!
~MA
On Sun, Aug 13, 2017 at 1:15 PM, Dan Andreescu dandreescu@wikimedia.org wrote:
Ah, yes, for now we have no plans to add the per-article stats, but do open a task and explain how it would be useful, we'll prioritize it accordingly. And in the meantime, looks like the pagecounts-ez are your best bet (use that instead of pagecounts-raw because the compression is lossless and saves a lot of download time)
*From: *Vipul Naik *Sent: *Sunday, August 13, 2017 11:12 *To: *A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. *Reply To: *A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. *Subject: *Re: [Analytics] Anybody know about stats.grok.se going down?
Hi Dan,
From the documentation of legacy metrics it looks like the legacy metrics are only available for sitewide pageviews for each site, rather than for individual pages. Is per-page data also part of your existing or planned legacy metrics?
Vipul
On Sat, Aug 12, 2017 at 6:17 PM, Dan Andreescu dandreescu@wikimedia.org wrote:
Hi Vipul, actually that's also available via the API now! https://wikitech.wikimedia.org/wiki/Analytics/AQS/Legacy_Pagecounts
It's a different path though, to highlight that pre-2015 numbers were counted slightly differently.
On Sat, Aug 12, 2017 at 18:59 Vipul Naik vipulnaik1@gmail.com wrote:
Hi Dan and Dan,
Thanks for taking the time to respond. I appreciate it!
I'm aware of the APIs and the WMF Labs tool. I am specifically interested in stats.grok.se for accessing data *before* July 2015, for which the only way right now is to process rather large raw dumps. I have built-in integrations that get data from stats.grok.se; processing raw dumps to generate pageview counts is possible but a lot of extra work :).
Cheers,
Vipul
On Mon, Aug 7, 2017 at 4:17 AM, Dan Andreescu dandreescu@wikimedia.org wrote:
And if you need more of an API / raw data download, take a look at:
https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews (available at https://wikimedia.org/api/rest_v1/)
and:
https://dumps.wikimedia.org/other/pagecounts-ez/
On Mon, Aug 7, 2017 at 4:21 AM, Dan Garry dgarry@wikimedia.org wrote:
Hi Vipul,
stats.grok.se is pretty much deprecated now. You ran in to one of the reasons why: it's not very reliable. You should use the Pageviews Analysis https://tools.wmflabs.org/pageviews/ tool instead, which was put together by MusikAnimal and Community Tech. This tool was intended to replace stats.grok.se. There is documentation https://meta.wikimedia.org/wiki/Community_Tech/Pageview_stats_tool about the tool that you may wish to read.
Thanks, Dan
On 7 August 2017 at 06:34, Vipul Naik vipulnaik1@gmail.com wrote:
stats.grok.se (a source of pageview stats for the time before the Wikimedia API became available) has been down for about a week. I tried emailing Henrik Abelsson, whom I've previously contacted when the site had issues, but haven't received a response this time.
Any ideas on why it's down and whom to reach out to to help resolve the issue?
Vipul
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Dan Garry Senior Product Manager, Editing Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Thanks Leon,
I agree adding *pagecounts (legacy) per article* and *top articles with most pagecounts (legacy) *to AQS would be awesome. We analytics already knew this should happen at some point. I created the task you suggest: https://phabricator.wikimedia.org/T173720 I think, though, that it will take some time (a couple months) for us to be able to work on this. We'll groom it this week and prioritize it. I added you as a subscriber.
Cheers!
On Fri, Aug 18, 2017 at 6:35 PM, Leon Ziemba musikanimal@wikimedia.org wrote:
For the record, this is what T149358 https://phabricator.wikimedia.org/T149358 was originally about https://phabricator.wikimedia.org/T149358#3106745. I was under the impression we were going to have pagecounts for all endpoints (per-article, top and aggregate), and it was somewhat disappointing to find out we only added support for aggregate. From my experience per-article data is actually of greatest interest, and I've gotten requests to add it to Pageviews Analysis since its inception. This was also part of one the top wishes in the German Technical Wishlist (I can dig up a link if need be). In addition, some things like the Did You Know https://en.wikipedia.org/wiki/Wikipedia:Did_you_know project on enwiki rely on it, where tens of thousands of template https://en.wikipedia.org/wiki/Template:DYK_talk transclusions link to stats.grok.se on article talk pages (see the template test cases https://en.wikipedia.org/w/index.php?title=Template:DYK_talk/testcases&oldid=796118708#Live for how this works). With stats.grok.se now gone, we have no public-facing web service to get this historical data. So I'd love to see it added to the awesome RESTBase API, but I understand it probably involves a lot of challenges. I can create another phabricator task if Vipul has not already. At any rate, I have endless thanks to give to the Analytics team for everything you've done for us. It seems we're always asking more from you! :)
R.I.P. stats.grok.se! 10 years was a good run!
~MA
On Sun, Aug 13, 2017 at 1:15 PM, Dan Andreescu dandreescu@wikimedia.org wrote:
Ah, yes, for now we have no plans to add the per-article stats, but do open a task and explain how it would be useful, we'll prioritize it accordingly. And in the meantime, looks like the pagecounts-ez are your best bet (use that instead of pagecounts-raw because the compression is lossless and saves a lot of download time)
*From: *Vipul Naik *Sent: *Sunday, August 13, 2017 11:12 *To: *A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. *Reply To: *A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. *Subject: *Re: [Analytics] Anybody know about stats.grok.se going down?
Hi Dan,
From the documentation of legacy metrics it looks like the legacy metrics are only available for sitewide pageviews for each site, rather than for individual pages. Is per-page data also part of your existing or planned legacy metrics?
Vipul
On Sat, Aug 12, 2017 at 6:17 PM, Dan Andreescu dandreescu@wikimedia.org wrote:
Hi Vipul, actually that's also available via the API now! https://wikitech.wikimedia.org/wiki/Analytics/AQS/Legacy_Pagecounts
It's a different path though, to highlight that pre-2015 numbers were counted slightly differently.
On Sat, Aug 12, 2017 at 18:59 Vipul Naik vipulnaik1@gmail.com wrote:
Hi Dan and Dan,
Thanks for taking the time to respond. I appreciate it!
I'm aware of the APIs and the WMF Labs tool. I am specifically interested in stats.grok.se for accessing data *before* July 2015, for which the only way right now is to process rather large raw dumps. I have built-in integrations that get data from stats.grok.se; processing raw dumps to generate pageview counts is possible but a lot of extra work :).
Cheers,
Vipul
On Mon, Aug 7, 2017 at 4:17 AM, Dan Andreescu <dandreescu@wikimedia.org
wrote:
And if you need more of an API / raw data download, take a look at:
https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews (available at https://wikimedia.org/api/rest_v1/)
and:
https://dumps.wikimedia.org/other/pagecounts-ez/
On Mon, Aug 7, 2017 at 4:21 AM, Dan Garry dgarry@wikimedia.org wrote:
Hi Vipul,
stats.grok.se is pretty much deprecated now. You ran in to one of the reasons why: it's not very reliable. You should use the Pageviews Analysis https://tools.wmflabs.org/pageviews/ tool instead, which was put together by MusikAnimal and Community Tech. This tool was intended to replace stats.grok.se. There is documentation https://meta.wikimedia.org/wiki/Community_Tech/Pageview_stats_tool about the tool that you may wish to read.
Thanks, Dan
On 7 August 2017 at 06:34, Vipul Naik vipulnaik1@gmail.com wrote:
> stats.grok.se (a source of pageview stats for the time before the > Wikimedia API became available) has been down for about a week. I tried > emailing Henrik Abelsson, whom I've previously contacted when the site had > issues, but haven't received a response this time. > > Any ideas on why it's down and whom to reach out to to help resolve > the issue? > > Vipul > > _______________________________________________ > Analytics mailing list > Analytics@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/analytics > >
-- Dan Garry Senior Product Manager, Editing Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics