Hi all,
Specifically what I am looking for is page view data for these pages, preferably for all months on http://dumps.wikimedia.org/other/pagecounts-raw/ (appeared as named 4 Dec): Abacarus hystrix Acarus siro Aceria tosichella Acyrthosiphon pisum Ahasverus advena Anthrenus flavipes Aphis craccivora Arhopalus Balaustium medicagoense Bemisia tabaci Brevicoryne brassicae Bruchus Ceratitis capitata Cicadulina Cryptolestes Daktulosphaira vitifoliae Delia Ephestia elutella Ephestia kuehniella Etiella behrii Frankliniella occidentalis Frankliniella Henosepilachna vigintioctopunctata Heteronychus arator Lachesilla quercus Lasioderma serricorne Liposcelis bostrychophila Macrosiphum euphorbiae Marchalina hellenica Myzus persicae Naupactus Nezara viridula Oligonychus ununguis Oryzaephilus surinamensis Panonychus ulmi Penthaleus Pieris rapae Piezodorus Plodia interpunctella Plutella xylostella Rhopalosiphon rhopalosiphum maidis Rhopalosiphum padi Rhyzopertha dominica Sirex noctilio Sitophilus granarius Sitophilus oryzae Sitotroga cerealella Sminthurus viridis Spodoptera exempta Stegobium paniceum Tetranychus Thrips palmi Thrips Tribolium castaneum Tribolium confusum Trogoderma granarium Trogoderma
I then also want a total number of page views to standardise the individual page views.
I have looked at stats.gronk.se and wikitrends and I have two issues: 1. The data is only month by month and I want as many years of data as possible. 2. Some pages have too few page views for wikitrends.
Thanks for your help!
-----Original Message----- From: Analytics [mailto:analytics-bounces@lists.wikimedia.org] On Behalf Of analytics-request@lists.wikimedia.org Sent: Tuesday, 15 December 2015 4:11 AM To: analytics@lists.wikimedia.org Subject: Analytics Digest, Vol 46, Issue 23
Send Analytics mailing list submissions to analytics@lists.wikimedia.org
To subscribe or unsubscribe via the World Wide Web, visit https://lists.wikimedia.org/mailman/listinfo/analytics or, via email, send a message with subject or body 'help' to analytics-request@lists.wikimedia.org
You can reach the person managing the list at analytics-owner@lists.wikimedia.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of Analytics digest..."
Today's Topics:
1. Re: Readership metrics for the fortnight until December 6, 2015 (Federico Leva (Nemo)) 2. Re: Data collection (Erik Zachte) 3. Re: Data collection (Federico Leva (Nemo)) 4. Re: Python client for the new pageview API (Dan Andreescu) 5. Re: mobile and zero legacy tsvs on stat1002 (Oliver Keyes)
----------------------------------------------------------------------
Message: 1 Date: Mon, 14 Dec 2015 13:08:11 +0100 From: "Federico Leva (Nemo)" nemowiki@gmail.com To: A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and "analytics." analytics@lists.wikimedia.org Subject: Re: [Analytics] Readership metrics for the fortnight until December 6, 2015 Message-ID: 566EB12B.5080201@gmail.com Content-Type: text/plain; charset=utf-8; format=flowed
Interesting country breakdown!
Tilman Bayer, 14/12/2015 12:32:
For the top three, I looked at how pageviews developed on a daily basis during the last three month including the week after this large change (until Dec 6):
In Greece, the +21.6% rise was the result of an isolated spike from November 23-25. This can be traced to a single page on the Greek Wiktionary which on most days before and after only saw a single-digit number of pageviews, but on these three days received more than 2.8 million: τάλε κουάλε https://el.wiktionary.org/wiki/%CF%84%CE%AC%CE%BB%CE%B5_%CE%BA%CE%BF%CF%85%CE%AC%CE%BB%CE%B5. It’s about an expression that apparently comes from Latin via Italian (“tale quale”) https://en.wiktionary.org/wiki/tale_e_qualeand means something like “exactly the same” or “spitting image”. From the form of the spike, it was likely not the result of actual human interest, rather an undetected bot trying to learn exactly the same about exactly the same.
In Ireland, the -20.6% drop marked the end of a plateau whose start had actually shown up in the report for the week until November 1 https://lists.wikimedia.org/pipermail/mobile-l/2015-November/009919.h tmlalready, where the country was the top changer with a 40.2% rise.
For South Africa, the -20.6% drop does not form part of a clear pattern.
------------------------------
Message: 2 Date: Mon, 14 Dec 2015 14:14:17 +0100 From: "Erik Zachte" ezachte@wikimedia.org To: "'A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics.'" analytics@lists.wikimedia.org Subject: Re: [Analytics] Data collection Message-ID: 007901d13671$56855cc0$03901640$@wikimedia.org Content-Type: text/plain; charset="utf-8"
Hi Caitlin,
Here is a breakdown of categories within Phytopathology on English wikipedia: http://ow.ly/VQNVL
and the articles within those categories ranked by page view for Oct 2015 : http://ow.ly/VQNCv
I can run similar reports for earlier months.
Cheers,
Erik
From: Analytics [mailto:analytics-bounces@lists.wikimedia.org] On Behalf Of Alex Druk Sent: Monday, December 14, 2015 10:44 To: A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. Subject: Re: [Analytics] Data collection
Hi Caitlin,
If you have a list of relevant articles and understanding what time period you would like to research, contact me of the list and I probably can help you.
Also my advise: have a look at wikipediatrends.com or stats.grok.se and try some of your queries to get a better undestanding of possible results.
Best wishes,
On Mon, Dec 14, 2015 at 12:04 AM, Caitlin.Gardner@csiro.au wrote:
Hi All,
I am a summer research intern with the Commonwealth Scientific and Industrial Research Organisation (CSIRO) in Australia. I am studying a statistics degree and so I don’t really have skills in the type of data collection required to access the Wiki data for my research. I was wondering if someone might be able to give me a hand (by pointing me in the right direction)?
I have a list of pest species that I wish to find the total number of page views via stats.grok.se or https://dumps.wikimedia.org/other/pagecounts-raw/ . There must be a good method to go through and pick out page views by name rather than by hand (which obviously isn’t feasible)? I’d also need to be able to find the total number of page views for each period in order to standardize the response to account for the increase in traffic over the years.
We are in the process of gathering similar data through a Plant Pest database but due to privacy concerns, the organisation is arranging to reconcile the data on our behalf and so I do not have a part in that.
Any help would be really appreciated!
Kind regards,
Caitlin Gardner
_______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
If you want it going back that far, I'm afraid the stats.grok.se style data is all there is :(. The new API only covers the last few months thus far.
On 14 December 2015 at 17:31, Caitlin.Gardner@csiro.au wrote:
Hi all,
Specifically what I am looking for is page view data for these pages, preferably for all months on http://dumps.wikimedia.org/other/pagecounts-raw/ (appeared as named 4 Dec): Abacarus hystrix Acarus siro Aceria tosichella Acyrthosiphon pisum Ahasverus advena Anthrenus flavipes Aphis craccivora Arhopalus Balaustium medicagoense Bemisia tabaci Brevicoryne brassicae Bruchus Ceratitis capitata Cicadulina Cryptolestes Daktulosphaira vitifoliae Delia Ephestia elutella Ephestia kuehniella Etiella behrii Frankliniella occidentalis Frankliniella Henosepilachna vigintioctopunctata Heteronychus arator Lachesilla quercus Lasioderma serricorne Liposcelis bostrychophila Macrosiphum euphorbiae Marchalina hellenica Myzus persicae Naupactus Nezara viridula Oligonychus ununguis Oryzaephilus surinamensis Panonychus ulmi Penthaleus Pieris rapae Piezodorus Plodia interpunctella Plutella xylostella Rhopalosiphon rhopalosiphum maidis Rhopalosiphum padi Rhyzopertha dominica Sirex noctilio Sitophilus granarius Sitophilus oryzae Sitotroga cerealella Sminthurus viridis Spodoptera exempta Stegobium paniceum Tetranychus Thrips palmi Thrips Tribolium castaneum Tribolium confusum Trogoderma granarium Trogoderma
I then also want a total number of page views to standardise the individual page views.
I have looked at stats.gronk.se and wikitrends and I have two issues:
- The data is only month by month and I want as many years of data as possible.
- Some pages have too few page views for wikitrends.
Thanks for your help!
-----Original Message----- From: Analytics [mailto:analytics-bounces@lists.wikimedia.org] On Behalf Of analytics-request@lists.wikimedia.org Sent: Tuesday, 15 December 2015 4:11 AM To: analytics@lists.wikimedia.org Subject: Analytics Digest, Vol 46, Issue 23
Send Analytics mailing list submissions to analytics@lists.wikimedia.org
To subscribe or unsubscribe via the World Wide Web, visit https://lists.wikimedia.org/mailman/listinfo/analytics or, via email, send a message with subject or body 'help' to analytics-request@lists.wikimedia.org
You can reach the person managing the list at analytics-owner@lists.wikimedia.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of Analytics digest..."
Today's Topics:
- Re: Readership metrics for the fortnight until December 6, 2015 (Federico Leva (Nemo))
- Re: Data collection (Erik Zachte)
- Re: Data collection (Federico Leva (Nemo))
- Re: Python client for the new pageview API (Dan Andreescu)
- Re: mobile and zero legacy tsvs on stat1002 (Oliver Keyes)
Message: 1 Date: Mon, 14 Dec 2015 13:08:11 +0100 From: "Federico Leva (Nemo)" nemowiki@gmail.com To: A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and "analytics." analytics@lists.wikimedia.org Subject: Re: [Analytics] Readership metrics for the fortnight until December 6, 2015 Message-ID: 566EB12B.5080201@gmail.com Content-Type: text/plain; charset=utf-8; format=flowed
Interesting country breakdown!
Tilman Bayer, 14/12/2015 12:32:
For the top three, I looked at how pageviews developed on a daily basis during the last three month including the week after this large change (until Dec 6):
In Greece, the +21.6% rise was the result of an isolated spike from November 23-25. This can be traced to a single page on the Greek Wiktionary which on most days before and after only saw a single-digit number of pageviews, but on these three days received more than 2.8 million: τάλε κουάλε https://el.wiktionary.org/wiki/%CF%84%CE%AC%CE%BB%CE%B5_%CE%BA%CE%BF%CF%85%CE%AC%CE%BB%CE%B5. It’s about an expression that apparently comes from Latin via Italian (“tale quale”) https://en.wiktionary.org/wiki/tale_e_qualeand means something like “exactly the same” or “spitting image”. From the form of the spike, it was likely not the result of actual human interest, rather an undetected bot trying to learn exactly the same about exactly the same.
In Ireland, the -20.6% drop marked the end of a plateau whose start had actually shown up in the report for the week until November 1 https://lists.wikimedia.org/pipermail/mobile-l/2015-November/009919.h tmlalready, where the country was the top changer with a 40.2% rise.
For South Africa, the -20.6% drop does not form part of a clear pattern.
Message: 2 Date: Mon, 14 Dec 2015 14:14:17 +0100 From: "Erik Zachte" ezachte@wikimedia.org To: "'A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics.'" analytics@lists.wikimedia.org Subject: Re: [Analytics] Data collection Message-ID: 007901d13671$56855cc0$03901640$@wikimedia.org Content-Type: text/plain; charset="utf-8"
Hi Caitlin,
Here is a breakdown of categories within Phytopathology on English wikipedia: http://ow.ly/VQNVL
and the articles within those categories ranked by page view for Oct 2015 : http://ow.ly/VQNCv
I can run similar reports for earlier months.
Cheers,
Erik
From: Analytics [mailto:analytics-bounces@lists.wikimedia.org] On Behalf Of Alex Druk Sent: Monday, December 14, 2015 10:44 To: A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. Subject: Re: [Analytics] Data collection
Hi Caitlin,
If you have a list of relevant articles and understanding what time period you would like to research, contact me of the list and I probably can help you.
Also my advise: have a look at wikipediatrends.com or stats.grok.se and try some of your queries to get a better undestanding of possible results.
Best wishes,
On Mon, Dec 14, 2015 at 12:04 AM, Caitlin.Gardner@csiro.au wrote:
Hi All,
I am a summer research intern with the Commonwealth Scientific and Industrial Research Organisation (CSIRO) in Australia. I am studying a statistics degree and so I don’t really have skills in the type of data collection required to access the Wiki data for my research. I was wondering if someone might be able to give me a hand (by pointing me in the right direction)?
I have a list of pest species that I wish to find the total number of page views via stats.grok.se or https://dumps.wikimedia.org/other/pagecounts-raw/ . There must be a good method to go through and pick out page views by name rather than by hand (which obviously isn’t feasible)? I’d also need to be able to find the total number of page views for each period in order to standardize the response to account for the increase in traffic over the years.
We are in the process of gathering similar data through a Plant Pest database but due to privacy concerns, the organisation is arranging to reconcile the data on our behalf and so I do not have a part in that.
Any help would be really appreciated!
Kind regards,
Caitlin Gardner
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
--
Thank you.
Alex Druk, PhD
wikipediatrends.com alex.druk@gmail.com (775) 237-8550 Google voice
Hi Caitlin,
Using the python client for the pageview API ( https://github.com/mediawiki-utilities/python-mwviews), you could do:
from mwviews.api import PageviewsClient p = PageviewsClient() p.article_views('en.wikipedia', ['Abacarus_hystrix','Acarus_siro','Aceria_tosichella','Acyrthosiphon_pisum','Ahasverus_advena','Anthrenus_flavipes','Aphis_craccivora','Arhopalus','Balaustium_medicagoense','Bemisia_tabaci','Brevicoryne_brassicae','Bruchus','Ceratitis_capitata','Cicadulina','Cryptolestes','Daktulosphaira_vitifoliae','Delia','Ephestia_elutella','Ephestia_kuehniella','Etiella_behrii','Frankliniella_occidentalis','Frankliniella','Henosepilachna_vigintioctopunctata','Heteronychus_arator','Lachesilla_quercus','Lasioderma_serricorne','Liposcelis_bostrychophila','Macrosiphum_euphorbiae','Marchalina_hellenica','Myzus_persicae','Naupactus','Nezara_viridula','Oligonychus_ununguis','Oryzaephilus_surinamensis','Panonychus_ulmi','Penthaleus','Pieris_rapae','Piezodorus','Plodia_interpunctella','Plutella_xylostella','Rhopalosiphon','rhopalosiphum_maidis','Rhopalosiphum_padi','Rhyzopertha_dominica','Sirex_noctilio','Sitophilus_granarius','Sitophilus_oryzae','Sitotroga_cerealella','Sminthurus_viridis','Spodoptera_exempta','Stegobium_paniceum','Tetranychus','Thrips_palmi','Thrips','Tribolium_castaneum','Tribolium_confusum','Trogoderma_granarium','Trogoderma'], start='20150501')
Some of the articles in your list don't exist on en.wikipedia (like 'Frankliniella') but for what exists this returns the views as far back as we have them. When we're done filling up the API we'll have data back to May 2015, but right now it only goes to August. If you need it back further, you have to parse the dumps as others have said. I'm curious why you need the older data, it's interesting to us as we try to figure out what else to expose through the API. Would monthly pageviews be just as good?
I attached the result of that query here in JSON format