Hello All,
I am working on a project the uses page view numbers for wiki articles and I was hoping somebody could help me out. I am using wikipedia redirects to find aliases for query names. Unfortunately there is a lot of noise in the redirects. I was hoping to use the page views as a heuristic to weed out bad redirects. I was looking at the page view files but the ones on stats.grok.se are hourly which is too much to process in a reasonable amount of time. I was wondering if anybody had (or knew where I could access) page view files for a longer amount of time like yearly, monthly, or even daily. I need to able to download the file locally because I will be dealing with a lot of query names. I appreciate any help you can provide.
Thanks,
Manny
Does https://tools.wmflabs.org/pageviews help? בתאריך 9 ביולי 2016 05:23, "Manny Manny" mannya897@gmail.com כתב:
Hello All,
I am working on a project the uses page view numbers for wiki articles and I was hoping somebody could help me out. I am using wikipedia redirects to find aliases for query names. Unfortunately there is a lot of noise in the redirects. I was hoping to use the page views as a heuristic to weed out bad redirects. I was looking at the page view files but the ones on stats.grok.se are hourly which is too much to process in a reasonable amount of time. I was wondering if anybody had (or knew where I could access) page view files for a longer amount of time like yearly, monthly, or even daily. I need to able to download the file locally because I will be dealing with a lot of query names. I appreciate any help you can provide.
Thanks,
Manny
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Hi Manny,
Have a look at http://www.wikipediatrends.com.
Alex
On Sat, Jul 9, 2016 at 4:22 AM, Manny Manny mannya897@gmail.com wrote:
Hello All,
I am working on a project the uses page view numbers for wiki articles and I was hoping somebody could help me out. I am using wikipedia redirects to find aliases for query names. Unfortunately there is a lot of noise in the redirects. I was hoping to use the page views as a heuristic to weed out bad redirects. I was looking at the page view files but the ones on stats.grok.se are hourly which is too much to process in a reasonable amount of time. I was wondering if anybody had (or knew where I could access) page view files for a longer amount of time like yearly, monthly, or even daily. I need to able to download the file locally because I will be dealing with a lot of query names. I appreciate any help you can provide.
Thanks,
Manny
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
If those two don't help there is some raw data available here: https://dumps.wikimedia.org/other/analytics/
Those files are also hourly but you make a good case for us including monthly totals, so we're open to that if you can't use the other resources. (I work on the analytics team)
On Saturday, July 9, 2016, Alex Druk alex.druk@gmail.com wrote:
Hi Manny,
Have a look at http://www.wikipediatrends.com.
Alex
On Sat, Jul 9, 2016 at 4:22 AM, Manny Manny <mannya897@gmail.com javascript:_e(%7B%7D,'cvml','mannya897@gmail.com');> wrote:
Hello All,
I am working on a project the uses page view numbers for wiki articles and I was hoping somebody could help me out. I am using wikipedia redirects to find aliases for query names. Unfortunately there is a lot of noise in the redirects. I was hoping to use the page views as a heuristic to weed out bad redirects. I was looking at the page view files but the ones on stats.grok.se are hourly which is too much to process in a reasonable amount of time. I was wondering if anybody had (or knew where I could access) page view files for a longer amount of time like yearly, monthly, or even daily. I need to able to download the file locally because I will be dealing with a lot of query names. I appreciate any help you can provide.
Thanks,
Manny
Analytics mailing list Analytics@lists.wikimedia.org javascript:_e(%7B%7D,'cvml','Analytics@lists.wikimedia.org'); https://lists.wikimedia.org/mailman/listinfo/analytics
-- Thank you.
Alex Druk, PhD. wikipediatrens.com alex.druk@gmail.com javascript:_e(%7B%7D,'cvml','alex.druk@gmail.com'); (775) 237-8550 Google voice
I collected some non-technical tools at https://meta.wikimedia.org/wiki/Traffic_reporting
On Sat, Jul 9, 2016 at 8:29 AM, Dan Andreescu dandreescu@wikimedia.org wrote:
If those two don't help there is some raw data available here: https://dumps.wikimedia.org/other/analytics/
Those files are also hourly but you make a good case for us including monthly totals, so we're open to that if you can't use the other resources. (I work on the analytics team)
On Saturday, July 9, 2016, Alex Druk alex.druk@gmail.com wrote:
Hi Manny,
Have a look at http://www.wikipediatrends.com.
Alex
On Sat, Jul 9, 2016 at 4:22 AM, Manny Manny mannya897@gmail.com wrote:
Hello All,
I am working on a project the uses page view numbers for wiki articles and I was hoping somebody could help me out. I am using wikipedia redirects to find aliases for query names. Unfortunately there is a lot of noise in the redirects. I was hoping to use the page views as a heuristic to weed out bad redirects. I was looking at the page view files but the ones on stats.grok.se are hourly which is too much to process in a reasonable amount of time. I was wondering if anybody had (or knew where I could access) page view files for a longer amount of time like yearly, monthly, or even daily. I need to able to download the file locally because I will be dealing with a lot of query names. I appreciate any help you can provide.
Thanks,
Manny
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Thank you.
Alex Druk, PhD. wikipediatrens.com alex.druk@gmail.com (775) 237-8550 Google voice
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Manny,
If you're willing to download large files, monthly totals are here
https://dumps.wikimedia.org/other/pagecounts-ez/merged/
pagecounts-2016-06-views-ge-5.bz2 https://dumps.wikimedia.org/other/pagecounts-ez/merged/pagecounts-2016-06-views-ge-5.bz2 has all titles with 5 or more requests per month
It contains monthly totals, plus efficiently packed hourly data, all in one line.
pagecounts-2016-06-views-ge-5-totals.bz2 https://dumps.wikimedia.org/other/pagecounts-ez/merged/pagecounts-2016-06-views-ge-5-totals.bz2 is derived from the previous file, with hourly data stripped.
But monthly totals in the API would also be great (and asked for by several people ; I count Magnus Manske as one, but his audience would rejoice as well ;-)
Cheers
Erik
From: Analytics [mailto:analytics-bounces@lists.wikimedia.org] On Behalf Of Lane Rasberry Sent: Saturday, July 09, 2016 15:01 To: A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. Subject: Re: [Analytics] Wiki Page Views Project
I collected some non-technical tools at https://meta.wikimedia.org/wiki/Traffic_reporting
On Sat, Jul 9, 2016 at 8:29 AM, Dan Andreescu dandreescu@wikimedia.org wrote:
If those two don't help there is some raw data available here: https://dumps.wikimedia.org/other/analytics/
Those files are also hourly but you make a good case for us including monthly totals, so we're open to that if you can't use the other resources. (I work on the analytics team)
On Saturday, July 9, 2016, Alex Druk alex.druk@gmail.com wrote:
Hi Manny,
Have a look at http://www.wikipediatrends.com.
Alex
On Sat, Jul 9, 2016 at 4:22 AM, Manny Manny mannya897@gmail.com wrote:
Hello All,
I am working on a project the uses page view numbers for wiki articles and I was hoping somebody could help me out. I am using wikipedia redirects to find aliases for query names. Unfortunately there is a lot of noise in the redirects. I was hoping to use the page views as a heuristic to weed out bad redirects. I was looking at the page view files but the ones on stats.grok.se are hourly which is too much to process in a reasonable amount of time. I was wondering if anybody had (or knew where I could access) page view files for a longer amount of time like yearly, monthly, or even daily. I need to able to download the file locally because I will be dealing with a lot of query names. I appreciate any help you can provide.
Thanks,
Manny
_______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
But monthly totals in the API would also be great (and asked for by
several people ; I count Magnus Manske as >one, but his audience would rejoice as well ;-) I do not think we have a phab task about it so please do file one. The pageview API has scaling issues as is and until developing any new features that might add more data we need to solve those.
Ticket about scaling work is here: https://phabricator.wikimedia.org/T124314
On Sat, Jul 9, 2016 at 10:42 AM, Erik Zachte ezachte@wikimedia.org wrote:
Manny,
If you're willing to download large files, monthly totals are here
https://dumps.wikimedia.org/other/pagecounts-ez/merged/
pagecounts-2016-06-views-ge-5.bz2 https://dumps.wikimedia.org/other/pagecounts-ez/merged/pagecounts-2016-06-views-ge-5.bz2 has all titles with 5 or more requests per month
It contains monthly totals, plus efficiently packed hourly data, all in one line.
pagecounts-2016-06-views-ge-5-totals.bz2 https://dumps.wikimedia.org/other/pagecounts-ez/merged/pagecounts-2016-06-views-ge-5-totals.bz2 is derived from the previous file, with hourly data stripped.
But monthly totals in the API would also be great (and asked for by several people ; I count Magnus Manske as one, but his audience would rejoice as well ;-)
Cheers
Erik
*From:* Analytics [mailto:analytics-bounces@lists.wikimedia.org] *On Behalf Of *Lane Rasberry *Sent:* Saturday, July 09, 2016 15:01 *To:* A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. *Subject:* Re: [Analytics] Wiki Page Views Project
I collected some non-technical tools at https://meta.wikimedia.org/wiki/Traffic_reporting
On Sat, Jul 9, 2016 at 8:29 AM, Dan Andreescu dandreescu@wikimedia.org wrote:
If those two don't help there is some raw data available here: https://dumps.wikimedia.org/other/analytics/
Those files are also hourly but you make a good case for us including monthly totals, so we're open to that if you can't use the other resources. (I work on the analytics team)
On Saturday, July 9, 2016, Alex Druk alex.druk@gmail.com wrote:
Hi Manny,
Have a look at http://www.wikipediatrends.com.
Alex
On Sat, Jul 9, 2016 at 4:22 AM, Manny Manny mannya897@gmail.com wrote:
Hello All,
I am working on a project the uses page view numbers for wiki articles and I was hoping somebody could help me out. I am using wikipedia redirects to find aliases for query names. Unfortunately there is a lot of noise in the redirects. I was hoping to use the page views as a heuristic to weed out bad redirects. I was looking at the page view files but the ones on stats.grok.se are hourly which is too much to process in a reasonable amount of time. I was wondering if anybody had (or knew where I could access) page view files for a longer amount of time like yearly, monthly, or even daily. I need to able to download the file locally because I will be dealing with a lot of query names. I appreciate any help you can provide.
Thanks,
Manny
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
--
Thank you.
Alex Druk, PhD.
wikipediatrens.com alex.druk@gmail.com (775) 237-8550 Google voice
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
--
Lane Rasberry
user:bluerasberry on Wikipedia
206.801.0814 lane@bluerasberry.com
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Thanks Erik. This is exactly what I'm looking for. Thanks for the help for everybody else as well.
On Sat, Jul 9, 2016 at 1:42 PM, Erik Zachte ezachte@wikimedia.org wrote:
Manny,
If you're willing to download large files, monthly totals are here
https://dumps.wikimedia.org/other/pagecounts-ez/merged/
pagecounts-2016-06-views-ge-5.bz2 https://dumps.wikimedia.org/other/pagecounts-ez/merged/pagecounts-2016-06-views-ge-5.bz2 has all titles with 5 or more requests per month
It contains monthly totals, plus efficiently packed hourly data, all in one line.
pagecounts-2016-06-views-ge-5-totals.bz2 https://dumps.wikimedia.org/other/pagecounts-ez/merged/pagecounts-2016-06-views-ge-5-totals.bz2 is derived from the previous file, with hourly data stripped.
But monthly totals in the API would also be great (and asked for by several people ; I count Magnus Manske as one, but his audience would rejoice as well ;-)
Cheers
Erik
*From:* Analytics [mailto:analytics-bounces@lists.wikimedia.org] *On Behalf Of *Lane Rasberry *Sent:* Saturday, July 09, 2016 15:01 *To:* A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. *Subject:* Re: [Analytics] Wiki Page Views Project
I collected some non-technical tools at https://meta.wikimedia.org/wiki/Traffic_reporting
On Sat, Jul 9, 2016 at 8:29 AM, Dan Andreescu dandreescu@wikimedia.org wrote:
If those two don't help there is some raw data available here: https://dumps.wikimedia.org/other/analytics/
Those files are also hourly but you make a good case for us including monthly totals, so we're open to that if you can't use the other resources. (I work on the analytics team)
On Saturday, July 9, 2016, Alex Druk alex.druk@gmail.com wrote:
Hi Manny,
Have a look at http://www.wikipediatrends.com.
Alex
On Sat, Jul 9, 2016 at 4:22 AM, Manny Manny mannya897@gmail.com wrote:
Hello All,
I am working on a project the uses page view numbers for wiki articles and I was hoping somebody could help me out. I am using wikipedia redirects to find aliases for query names. Unfortunately there is a lot of noise in the redirects. I was hoping to use the page views as a heuristic to weed out bad redirects. I was looking at the page view files but the ones on stats.grok.se are hourly which is too much to process in a reasonable amount of time. I was wondering if anybody had (or knew where I could access) page view files for a longer amount of time like yearly, monthly, or even daily. I need to able to download the file locally because I will be dealing with a lot of query names. I appreciate any help you can provide.
Thanks,
Manny
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
--
Thank you.
Alex Druk, PhD.
wikipediatrens.com alex.druk@gmail.com (775) 237-8550 Google voice
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
--
Lane Rasberry
user:bluerasberry on Wikipedia
206.801.0814 lane@bluerasberry.com
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics