Hi Everyone,
We had our quarterly review with WMF management last week. The minutes[1] are posted up on meta along with the deck we presented. (Thank you to Tilman for taking the minutes and helping post the slides)
Please take a look at the deck and let me know if you have any questions. In particular, I'd like to highlight our reprioritization[2] of our projects. We continue to focus on our Editor Engagement Vital Signs project and have added a couple of new projects, including taking over the Event Logging system from the Platform Team.
I want to call out the Page View API project specifically. Everyone on the team wants to work on this but we have prioritized other projects ahead of it. While this is challenging for everyone, Editor Growth remains the priority for the Foundation and Analytics needs to support this initiative.
In the meantime, we have worked with Henrik, the maintainer of stats.grok.seto help scale out this service. We've purchased a new machine and the initial performance numbers are very encouraging. We'll have more updates on this shortly.
In conclusion, now that the team is fully staffed, I'll have more time to communicate about our projects and how they will interact with the community. I'm looking forward to it :)
Thanks,
-Toby
[1] https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/Quarterly_re... [2] https://www.mediawiki.org/wiki/Analytics/Prioritization_Planning
Hi everyone,
So, quick update: I've received the server installed it and set up an instance of the code on and sent it on my hosting provider, and they've received it and are hooking it up. I'll migrate over stats.grok.se to the new server asap, hopefully over the weekend.
Hopefully it will be much faster for users, and GLAM usage in particular. I'm also feeling pretty motivated to code new features after you guys, and Toby in particular, have been so generous as to provide the stats service with hardware. So, I'm taking suggestions - anything in particular you'd like to see implemented on stats.grok.se first?
Many thanks! -henrik
On 09/04/14 17:45, Toby Negrin wrote:
Hi Everyone,
We had our quarterly review with WMF management last week. The minutes[1] are posted up on meta along with the deck we presented. (Thank you to Tilman for taking the minutes and helping post the slides)
Please take a look at the deck and let me know if you have any questions. In particular, I'd like to highlight our reprioritization[2] of our projects. We continue to focus on our Editor Engagement Vital Signs project and have added a couple of new projects, including taking over the Event Logging system from the Platform Team.
I want to call out the Page View API project specifically. Everyone on the team wants to work on this but we have prioritized other projects ahead of it. While this is challenging for everyone, Editor Growth remains the priority for the Foundation and Analytics needs to support this initiative.
In the meantime, we have worked with Henrik, the maintainer of stats.grok.se http://stats.grok.se to help scale out this service. We've purchased a new machine and the initial performance numbers are very encouraging. We'll have more updates on this shortly.
In conclusion, now that the team is fully staffed, I'll have more time to communicate about our projects and how they will interact with the community. I'm looking forward to it :)
Thanks,
-Toby
[1] https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/Quarterly_re... [2] https://www.mediawiki.org/wiki/Analytics/Prioritization_Planning
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Hi Henrik --
This is the coolest email I've gotten this week. I'm really excited for the new server.
I think we should be thanking you for building and hosting this service!
Best,
-Toby
On Thu, Apr 10, 2014 at 5:05 PM, Henrik Abelsson henrik@abelsson.comwrote:
Hi everyone,
So, quick update: I've received the server installed it and set up an instance of the code on and sent it on my hosting provider, and they've received it and are hooking it up. I'll migrate over stats.grok.se to the new server asap, hopefully over the weekend.
Hopefully it will be much faster for users, and GLAM usage in particular. I'm also feeling pretty motivated to code new features after you guys, and Toby in particular, have been so generous as to provide the stats service with hardware. So, I'm taking suggestions - anything in particular you'd like to see implemented on stats.grok.se first?
Many thanks! -henrik
On 09/04/14 17:45, Toby Negrin wrote:
Hi Everyone,
We had our quarterly review with WMF management last week. The minutes[1] are posted up on meta along with the deck we presented. (Thank you to Tilman for taking the minutes and helping post the slides)
Please take a look at the deck and let me know if you have any questions. In particular, I'd like to highlight our reprioritization[2] of our projects. We continue to focus on our Editor Engagement Vital Signs project and have added a couple of new projects, including taking over the Event Logging system from the Platform Team.
I want to call out the Page View API project specifically. Everyone on the team wants to work on this but we have prioritized other projects ahead of it. While this is challenging for everyone, Editor Growth remains the priority for the Foundation and Analytics needs to support this initiative.
In the meantime, we have worked with Henrik, the maintainer of stats.grok.se to help scale out this service. We've purchased a new machine and the initial performance numbers are very encouraging. We'll have more updates on this shortly.
In conclusion, now that the team is fully staffed, I'll have more time to communicate about our projects and how they will interact with the community. I'm looking forward to it :)
Thanks,
-Toby
[1] https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/Quarterly_re... [2] https://www.mediawiki.org/wiki/Analytics/Prioritization_Planning
Analytics mailing listAnalytics@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
On Thu, Apr 10, 2014 at 7:05 PM, Henrik Abelsson henrik@abelsson.comwrote:
Hi everyone,
So, quick update: I've received the server installed it and set up an instance of the code on and sent it on my hosting provider, and they've received it and are hooking it up. I'll migrate over stats.grok.se to the new server asap, hopefully over the weekend.
Hopefully it will be much faster for users, and GLAM usage in particular. I'm also feeling pretty motivated to code new features after you guys, and Toby in particular, have been so generous as to provide the stats service with hardware. So, I'm taking suggestions - anything in particular you'd like to see implemented on stats.grok.se first?
As an end user who manually datamines a lot of stuff for reports I write, my wish list would be to be able to get an open office document that would let me generate a report like https://en.wikinews.org/wiki/File:Spanish_competitors_at_the_IPC_Athletics_W... https://en.wikinews.org/wiki/File:IPC_NorAmCup.pdf . Some of these tools exist in isolation or I just do not know about them. :-/ https://toolserver.org/~magnus/ts2/treeviews/ and https://tools.wmflabs.org/glamtools/glamorous.php and http://tools.wmflabs.org/glamtools/baglama.php come to mind.
Desirable data is by day page views (for event tracking and understanding viewing patterns for content around a specific category) that can be looked at against the context of social media linking (total links to content on a daily basis, who is linking to it, how many followers they have or how many views the linked to content gets on a native site). It would also be useful to get some of this data mapped against total daily edits alongside daily views to get a better idea if getting people to collaborate on an article results in increased views. The total number of images added to a specific category during a certain timeframe, the number of uses of those images and the number of page views to images where those articles were used. Number of articles linking to a certain domain or URL across multiple language projects and page views to articles containing links to those. This information being easy to combine and tabulate more easily into one document for the purposes of explaining broader patterns would be useful.
Sincerely, Laura Hale
By 'total links' do you mean 'referer tracking from other websites'? Because I don't think we're doing that, and that makes me feel...very uncomfortable. It also wouldn't be at all reliable, because HTTPS strips the referer.
The other requests look interesting but relatively niche and difficult to instrument.
On 10 April 2014 17:57, Laura Hale laura@fanhistory.com wrote:
On Thu, Apr 10, 2014 at 7:05 PM, Henrik Abelsson henrik@abelsson.comwrote:
Hi everyone,
So, quick update: I've received the server installed it and set up an instance of the code on and sent it on my hosting provider, and they've received it and are hooking it up. I'll migrate over stats.grok.se to the new server asap, hopefully over the weekend.
Hopefully it will be much faster for users, and GLAM usage in particular. I'm also feeling pretty motivated to code new features after you guys, and Toby in particular, have been so generous as to provide the stats service with hardware. So, I'm taking suggestions - anything in particular you'd like to see implemented on stats.grok.se first?
As an end user who manually datamines a lot of stuff for reports I write, my wish list would be to be able to get an open office document that would let me generate a report like https://en.wikinews.org/wiki/File:Spanish_competitors_at_the_IPC_Athletics_W... https://en.wikinews.org/wiki/File:IPC_NorAmCup.pdf . Some of these tools exist in isolation or I just do not know about them. :-/ https://toolserver.org/~magnus/ts2/treeviews/ and https://tools.wmflabs.org/glamtools/glamorous.php and http://tools.wmflabs.org/glamtools/baglama.php come to mind.
Desirable data is by day page views (for event tracking and understanding viewing patterns for content around a specific category) that can be looked at against the context of social media linking (total links to content on a daily basis, who is linking to it, how many followers they have or how many views the linked to content gets on a native site). It would also be useful to get some of this data mapped against total daily edits alongside daily views to get a better idea if getting people to collaborate on an article results in increased views. The total number of images added to a specific category during a certain timeframe, the number of uses of those images and the number of page views to images where those articles were used. Number of articles linking to a certain domain or URL across multiple language projects and page views to articles containing links to those. This information being easy to combine and tabulate more easily into one document for the purposes of explaining broader patterns would be useful.
Sincerely, Laura Hale
-- twitter: purplepopple
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Total inbound links from Twitter, Facebook and Google +. There are API calls for that, and reach of content (not necessarily page views in and out themselves) is considered an important metric in journalism. Some of this is done by external non-WMF sources for all links but being able to easily have this data in a place where it can be gathered in bulk would be very useful. See http://topsy.com/trackback?url=http%3A%2F%2Fen.wikinews.org%2Fwiki%2FWikimed... one place. Page views in and of themselves are not a useful metric, especially given the demonstrated problems with accuracy.
Sincerely, Laura Hale
On Thu, Apr 10, 2014 at 8:45 PM, Oliver Keyes okeyes@wikimedia.org wrote:
By 'total links' do you mean 'referer tracking from other websites'? Because I don't think we're doing that, and that makes me feel...very uncomfortable. It also wouldn't be at all reliable, because HTTPS strips the referer.
The other requests look interesting but relatively niche and difficult to instrument.
On 10 April 2014 17:57, Laura Hale laura@fanhistory.com wrote:
On Thu, Apr 10, 2014 at 7:05 PM, Henrik Abelsson henrik@abelsson.comwrote:
Hi everyone,
So, quick update: I've received the server installed it and set up an instance of the code on and sent it on my hosting provider, and they've received it and are hooking it up. I'll migrate over stats.grok.se to the new server asap, hopefully over the weekend.
Hopefully it will be much faster for users, and GLAM usage in particular. I'm also feeling pretty motivated to code new features after you guys, and Toby in particular, have been so generous as to provide the stats service with hardware. So, I'm taking suggestions - anything in particular you'd like to see implemented on stats.grok.se first?
As an end user who manually datamines a lot of stuff for reports I write, my wish list would be to be able to get an open office document that would let me generate a report like https://en.wikinews.org/wiki/File:Spanish_competitors_at_the_IPC_Athletics_W... https://en.wikinews.org/wiki/File:IPC_NorAmCup.pdf . Some of these tools exist in isolation or I just do not know about them. :-/ https://toolserver.org/~magnus/ts2/treeviews/ and https://tools.wmflabs.org/glamtools/glamorous.php and http://tools.wmflabs.org/glamtools/baglama.php come to mind.
Desirable data is by day page views (for event tracking and understanding viewing patterns for content around a specific category) that can be looked at against the context of social media linking (total links to content on a daily basis, who is linking to it, how many followers they have or how many views the linked to content gets on a native site). It would also be useful to get some of this data mapped against total daily edits alongside daily views to get a better idea if getting people to collaborate on an article results in increased views. The total number of images added to a specific category during a certain timeframe, the number of uses of those images and the number of page views to images where those articles were used. Number of articles linking to a certain domain or URL across multiple language projects and page views to articles containing links to those. This information being easy to combine and tabulate more easily into one document for the purposes of explaining broader patterns would be useful.
Sincerely, Laura Hale
-- twitter: purplepopple
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Research Analyst Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
This is the best news of the week (at least!)!!!
On Fri, Apr 11, 2014 at 1:05 AM, Henrik Abelsson henrik@abelsson.comwrote:
Hi everyone,
So, quick update: I've received the server installed it and set up an instance of the code on and sent it on my hosting provider, and they've received it and are hooking it up. I'll migrate over stats.grok.se to the new server asap, hopefully over the weekend.
Hopefully it will be much faster for users, and GLAM usage in particular. I'm also feeling pretty motivated to code new features after you guys, and Toby in particular, have been so generous as to provide the stats service with hardware. So, I'm taking suggestions - anything in particular you'd like to see implemented on stats.grok.se first?
Many thanks! -henrik
On 09/04/14 17:45, Toby Negrin wrote:
Hi Everyone,
We had our quarterly review with WMF management last week. The minutes[1] are posted up on meta along with the deck we presented. (Thank you to Tilman for taking the minutes and helping post the slides)
Please take a look at the deck and let me know if you have any questions. In particular, I'd like to highlight our reprioritization[2] of our projects. We continue to focus on our Editor Engagement Vital Signs project and have added a couple of new projects, including taking over the Event Logging system from the Platform Team.
I want to call out the Page View API project specifically. Everyone on the team wants to work on this but we have prioritized other projects ahead of it. While this is challenging for everyone, Editor Growth remains the priority for the Foundation and Analytics needs to support this initiative.
In the meantime, we have worked with Henrik, the maintainer of stats.grok.se to help scale out this service. We've purchased a new machine and the initial performance numbers are very encouraging. We'll have more updates on this shortly.
In conclusion, now that the team is fully staffed, I'll have more time to communicate about our projects and how they will interact with the community. I'm looking forward to it :)
Thanks,
-Toby
[1] https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/Quarterly_re... [2] https://www.mediawiki.org/wiki/Analytics/Prioritization_Planning
Analytics mailing listAnalytics@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Thanks for doing a great job, Henrik!
On Fri, Apr 11, 2014 at 11:37 AM, Magnus Manske <magnusmanske@googlemail.com
wrote:
This is the best news of the week (at least!)!!!
On Fri, Apr 11, 2014 at 1:05 AM, Henrik Abelsson henrik@abelsson.comwrote:
Hi everyone,
So, quick update: I've received the server installed it and set up an instance of the code on and sent it on my hosting provider, and they've received it and are hooking it up. I'll migrate over stats.grok.se to the new server asap, hopefully over the weekend.
Hopefully it will be much faster for users, and GLAM usage in particular. I'm also feeling pretty motivated to code new features after you guys, and Toby in particular, have been so generous as to provide the stats service with hardware. So, I'm taking suggestions - anything in particular you'd like to see implemented on stats.grok.se first?
Many thanks! -henrik
On 09/04/14 17:45, Toby Negrin wrote:
Hi Everyone,
We had our quarterly review with WMF management last week. The minutes[1] are posted up on meta along with the deck we presented. (Thank you to Tilman for taking the minutes and helping post the slides)
Please take a look at the deck and let me know if you have any questions. In particular, I'd like to highlight our reprioritization[2] of our projects. We continue to focus on our Editor Engagement Vital Signs project and have added a couple of new projects, including taking over the Event Logging system from the Platform Team.
I want to call out the Page View API project specifically. Everyone on the team wants to work on this but we have prioritized other projects ahead of it. While this is challenging for everyone, Editor Growth remains the priority for the Foundation and Analytics needs to support this initiative.
In the meantime, we have worked with Henrik, the maintainer of stats.grok.se to help scale out this service. We've purchased a new machine and the initial performance numbers are very encouraging. We'll have more updates on this shortly.
In conclusion, now that the team is fully staffed, I'll have more time to communicate about our projects and how they will interact with the community. I'm looking forward to it :)
Thanks,
-Toby
[1] https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/Quarterly_re... [2] https://www.mediawiki.org/wiki/Analytics/Prioritization_Planning
Analytics mailing listAnalytics@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- undefined
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Thank you for the kind words :)
I just switched over stats.grok.se to query the new server for all pageview data from January 2014 on, the older data is still being copied over (it's still accessible, the web frontend will just transparently query the old server)
I tried very briefly with a few random latest30 URLs and the reported execution times seems to have gone from ~2s to ~20ms.
Let me know if you see anything that's broken!
-henrik
On 11/04/14 06:03, Alex Druk wrote:
Thanks for doing a great job, Henrik!
On Fri, Apr 11, 2014 at 11:37 AM, Magnus Manske <magnusmanske@googlemail.com mailto:magnusmanske@googlemail.com> wrote:
This is the best news of the week (at least!)!!! On Fri, Apr 11, 2014 at 1:05 AM, Henrik Abelsson <henrik@abelsson.com <mailto:henrik@abelsson.com>> wrote: Hi everyone, So, quick update: I've received the server installed it and set up an instance of the code on and sent it on my hosting provider, and they've received it and are hooking it up. I'll migrate over stats.grok.se <http://stats.grok.se> to the new server asap, hopefully over the weekend. Hopefully it will be much faster for users, and GLAM usage in particular. I'm also feeling pretty motivated to code new features after you guys, and Toby in particular, have been so generous as to provide the stats service with hardware. So, I'm taking suggestions - anything in particular you'd like to see implemented on stats.grok.se <http://stats.grok.se> first? Many thanks! -henrik On 09/04/14 17:45, Toby Negrin wrote:Hi Everyone, We had our quarterly review with WMF management last week. The minutes[1] are posted up on meta along with the deck we presented. (Thank you to Tilman for taking the minutes and helping post the slides) Please take a look at the deck and let me know if you have any questions. In particular, I'd like to highlight our reprioritization[2] of our projects. We continue to focus on our Editor Engagement Vital Signs project and have added a couple of new projects, including taking over the Event Logging system from the Platform Team. I want to call out the Page View API project specifically. Everyone on the team wants to work on this but we have prioritized other projects ahead of it. While this is challenging for everyone, Editor Growth remains the priority for the Foundation and Analytics needs to support this initiative. In the meantime, we have worked with Henrik, the maintainer of stats.grok.se <http://stats.grok.se> to help scale out this service. We've purchased a new machine and the initial performance numbers are very encouraging. We'll have more updates on this shortly. In conclusion, now that the team is fully staffed, I'll have more time to communicate about our projects and how they will interact with the community. I'm looking forward to it :) Thanks, -Toby [1] https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/Quarterly_reviews/Analytics/March_2014 [2] https://www.mediawiki.org/wiki/Analytics/Prioritization_Planning _______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org <mailto:Analytics@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/analytics_______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org <mailto:Analytics@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/analytics -- undefined _______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org <mailto:Analytics@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/analytics-- Thank you.
Alex Druk alex.druk@gmail.com mailto:alex.druk@gmail.com (775) 237-8550 Google voice
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
On Sat, Apr 12, 2014 at 7:35 AM, Dan Andreescu dandreescu@wikimedia.org wrote:
That couldn't have been just the hardware, you must've done some magic too :) Thank you so much Henrik, this is just awesome.
+1. Henrik, thank you for maintaining this indispensable service, and making it even better.
Erik
+1000 Henrik! Your service is one of the reasons why I love working on the English Wikipedia and Commons projects best, because I can link to you directly from the "View history" or "History" page.
To answer your earlier question about possible features, there is one service which I would love to see, and that is some checkmark to enable page views (per day or month) for a whole group of items in a category. I would still want to be able to check page views per category though. I am sure I am not the first person to ask for this, but it would be a great way to find the more popular items in the lists I work on.
Thanks again for your work, Jane
2014-04-13 0:21 GMT+02:00, Erik Moeller erik@wikimedia.org:
On Sat, Apr 12, 2014 at 7:35 AM, Dan Andreescu dandreescu@wikimedia.org wrote:
That couldn't have been just the hardware, you must've done some magic too :) Thank you so much Henrik, this is just awesome.
+1. Henrik, thank you for maintaining this indispensable service, and making it even better.
Erik
-- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics