Hi, today we are starting officially a new project to gather automatic metrics from the Wikimedia tech / MediaWiki community:
http://www.mediawiki.org/wiki/Community_metrics#Metrics_dashboard
We have a dashboard in an interim location, moving to Labs soon:
http://bitergia.com/projects/mediawiki-dashboard/browser/
The immediate steps (end of this week?) are:
* Moving to Labs. :) * Scan the sources we want to scan for git repositories, Bugzilla products and mailing lists.
Then we will follow with (end of this month?)
* Agreeing with the community what data to gather about contributors. * Polish the list of contributors e.g. assigning to people their different handlers.
Gerrit and IRC metrics are on the way (end of August?). I also expect improvements in the interface, based on our feedback... and our patches This dashboard is based on the open source projects Metrics Grimoire and VizGrimoire. Other organizations (prominently OpenStack) are using it as well so more features might come from other stakeholders.
We are working with Bitergia (maintainers or the Grimoire software and other FLOSS metrics related projects) as contractors. Álvaro del Castillo (CCed) is the main contact with the team.
The current contract will run during the next 12 months. The Engineering Community team of the WMF is pushing this effort, and I'm coordinating it. However, this clearly belongs to the Analytics team area. We have agreed to hand over the project at some point during the next 6-12 months.
We will use the Analytics mailing list to inform and discuss this project. Your feedback and help is welcome!
Hi!
First of all, I am very happy to start working with you doing analytics for Wikimedia software.
I have been reading this mailing list archives and your are doing amazing things in Wikimedia analytics.
El lun, 01-07-2013 a las 14:29 -0700, Quim Gil escribió:
Hi, today we are starting officially a new project to gather automatic metrics from the Wikimedia tech / MediaWiki community:
http://www.mediawiki.org/wiki/Community_metrics#Metrics_dashboard
We have a dashboard in an interim location, moving to Labs soon:
This is an alpha version of the metrics browser we are building for Mediawiki resources. During this week a new version will be uploaded and from it, we can start working together improving it.
The immediate steps (end of this week?) are:
- Moving to Labs. :)
- Scan the sources we want to scan for git repositories, Bugzilla
products and mailing lists.
The main goal is to analyze Mediawiki software so right now git repositories for "mediawiki/core" and all "mediawiki/extensions" (559 projects) are daily analyzed.
For Bugzilla, Mediawiki and Mediawiki Extensions product are being analyzed.
And for mailing lists we need to define the list together. Right now we are analyzing:
http://bitergia.com/projects/mediawiki-dashboard/browser/mls-repos.html
Then we will follow with (end of this month?)
- Agreeing with the community what data to gather about contributors.
- Polish the list of contributors e.g. assigning to people their
different handlers.
In VizGrimoireR we are using tools that help in the process to unifying identities between different data sources: git, bugzilla, mailing list and others. The idea is to have a final page where you can see a contributors activity in all project data sources.
And example:
http://bitergia.com/public/previews/VizGrimoireJS/browser/people.html?id=13&...
This is a feature in which we are working now but I am pretty confident we can have a first version for Mediawiki project before the end of this month.
Gerrit and IRC metrics are on the way (end of August?).
Yes. Gerrit metrics will be available soon. During this week a first version with global Gerrit metrics will be available.
I also expect improvements in the interface, based on our feedback... and our patches
Great!
This dashboard is based on the open source projects Metrics Grimoire and VizGrimoire. Other organizations (prominently OpenStack) are using it as well so more features might come from other stakeholders.
We are working with Bitergia (maintainers or the Grimoire software and other FLOSS metrics related projects) as contractors. Álvaro del Castillo (CCed) is the main contact with the team.
I will be active on this mailing list.
The current contract will run during the next 12 months. The Engineering Community team of the WMF is pushing this effort, and I'm coordinating it. However, this clearly belongs to the Analytics team area. We have agreed to hand over the project at some point during the next 6-12 months.
We will use the Analytics mailing list to inform and discuss this project. Your feedback and help is welcome!
Yes!
It is a pleasure to start working with you guys. Next email, updates about the new metrics browser.
Cheers
Quim Gil, 01/07/2013 23:29:
Hi, today we are starting officially a new project to gather automatic metrics from the Wikimedia tech / MediaWiki community:
http://www.mediawiki.org/wiki/Community_metrics#Metrics_dashboard
We have a dashboard in an interim location, moving to Labs soon:
Thanks. It seems my old comments still apply, here are they: http://article.gmane.org/gmane.science.linguistics.wikipedia.technical/65492
Nemo
The immediate steps (end of this week?) are:
- Moving to Labs. :)
- Scan the sources we want to scan for git repositories, Bugzilla
products and mailing lists.
Then we will follow with (end of this month?)
- Agreeing with the community what data to gather about contributors.
- Polish the list of contributors e.g. assigning to people their
different handlers.
Gerrit and IRC metrics are on the way (end of August?). I also expect improvements in the interface, based on our feedback... and our patches This dashboard is based on the open source projects Metrics Grimoire and VizGrimoire. Other organizations (prominently OpenStack) are using it as well so more features might come from other stakeholders.
We are working with Bitergia (maintainers or the Grimoire software and other FLOSS metrics related projects) as contractors. Álvaro del Castillo (CCed) is the main contact with the team.
The current contract will run during the next 12 months. The Engineering Community team of the WMF is pushing this effort, and I'm coordinating it. However, this clearly belongs to the Analytics team area. We have agreed to hand over the project at some point during the next 6-12 months.
We will use the Analytics mailing list to inform and discuss this project. Your feedback and help is welcome!
On 07/01/2013 10:43 PM, Federico Leva (Nemo) wrote:
Quim Gil, 01/07/2013 23:29:
Hi, today we are starting officially a new project to gather automatic metrics from the Wikimedia tech / MediaWiki community:
http://www.mediawiki.org/wiki/Community_metrics#Metrics_dashboard
We have a dashboard in an interim location, moving to Labs soon:
Thanks. It seems my old comments still apply, here are they: http://article.gmane.org/gmane.science.linguistics.wikipedia.technical/65492
Nemo, you are an excellent problem reporter and making you happy would be a very good benchmark in this project. :)
Could you please go to http://www.mediawiki.org/wiki/Talk:Community_metrics and provide more details about the problems you see and what would you expect to find instead? Or at least pick one problem and go for it.
The current sources are defined in the wiki page. Indeed, they shouldn't be restricted to MediaWiki core and extensions. I'm asking at wikitech-l which sources *shouldn't* be scanned, but other than that the default I want to propose is 'everything'.
Your criteria defining the mailing lists we should scan is more than welcome. I will surely agree with whatever list of lists you propose. :)
Thank you!
Hi,
On Mon, 2013-07-01 at 14:29 -0700, Quim Gil wrote:
I also expect improvements in the interface, based on our feedback...
Where would be the best feedback channel? Mailing list? Some component in Bugzilla? Or is it too early for this and I should wait?
Examples: * http://bitergia.com/public/previews/VizGrimoireJS/browser/people.html?id=13&... confuses me. Apart from using Spanish month names and assuming that "Commits per month" refers to the code repositories, I have not made any commits in 2011 but the graph says so. I did not even have a Wikimedia Gerrit account at that time. :) * http://bitergia.com/projects/mediawiki-dashboard/browser/its.html lists "Top its_closers last month". Does this only refer to tickets under the Bugzilla product "MediaWiki"? Looking at https://bugzilla.wikimedia.org/weekly-bug-summary.cgi?tops=10&days=32 and "Top 10 people who resolved the most reports in the last 32 days", I see very different names (but that's Bugzilla-wide and cross-product). * As people use different email addresses across mailing lists, Bugzilla, and Gerrit, I'm curious if there is a plan to map this accounts to each other. Somehow. * Same question for mapping resources to each other, e.g. project-specific mailing lists to the corresponding Git repository to the corresponding project or component in Bugzilla. When I played with Apache and GNOME projects a few months ago I tried to use their DOAP files to get metadata on resources, but I'm not aware of some similar concept in Wikimedia. * Personally I think I'm most curious about statistics how many users are active in a specific codebase repository, to identify bus factors. (When I ran "git log" on a full checkout of GNOME's Git repository a few months ago, one third of the projects were dead and had not seen any code commits in two years, excluding translation updates which are in subfolders.)
Cheers, andre
On 07/02/2013 04:49 AM, Andre Klapper wrote:
Where would be the best feedback channel? Mailing list? Some component in Bugzilla?
A wiki page? ;)
Let's start with http://www.mediawiki.org/wiki/Talk:Community_metrics . Hopefully after the expected bumpy first weeks things will just get pretty stable.
Note that Bitergia is developing at
https://github.com/Bitergia/mediawiki-dashboard
If you are a GitHub user you may want to send feedback/patches there. VizGrimoure and Metrics Grimoire have their own projects also in GitHub, just in case you want to go deeper upstream.
If after Wikimania we still feel like needing a more solid feedback channel then we can discuss Bugzilla. ok?
Or is it too early for this and I should wait?
Not too early, as long as reporters understand that things are just getting started and they are not meant to be perfect or even stable at this point. We are doing the steps this way precisely in order to get early feedback.
Thank you!
Hi!
El mar, 02-07-2013 a las 06:29 -0700, Quim Gil escribió:
On 07/02/2013 04:49 AM, Andre Klapper wrote:
Where would be the best feedback channel? Mailing list? Some component in Bugzilla?
A wiki page? ;)
Let's start with http://www.mediawiki.org/wiki/Talk:Community_metrics . Hopefully after the expected bumpy first weeks things will just get pretty stable.
Note that Bitergia is developing at
This is the production dashboard. We can do here minor fixes (HTML, CSS) that should be migrated upstream.
The real development is done in:
Web interface (JS+HTML+CSS): https://github.com/VizGrimoire/VizGrimoireJS/tree/mediawiki
R analytics: https://github.com/VizGrimoire/VizGrimoireR/tree/mediawiki
Data gathering tools: http://metricsgrimoire.github.io/
If you are a GitHub user you may want to send feedback/patches there.
Yes!
VizGrimoure and Metrics Grimoire have their own projects also in GitHub, just in case you want to go deeper upstream.
If after Wikimania we still feel like needing a more solid feedback channel then we can discuss Bugzilla. ok?
Great!
Or is it too early for this and I should wait?
Not too early, as long as reporters understand that things are just getting started and they are not meant to be perfect or even stable at this point. We are doing the steps this way precisely in order to get early feedback.
Correct.
Cheers
Thank you!
Hi!
El mar, 02-07-2013 a las 13:49 +0200, Andre Klapper escribió:
Hi,
On Mon, 2013-07-01 at 14:29 -0700, Quim Gil wrote:
I also expect improvements in the interface, based on our feedback...
Where would be the best feedback channel? Mailing list? Some component in Bugzilla? Or is it too early for this and I should wait?
I think starting from now, the feedback is more valuable once we have settled down a bit the dashboard.
Examples: * http://bitergia.com/public/previews/VizGrimoireJS/browser/people.html?id=13&... confuses me. Apart from using Spanish month names
This is going to be fixed. It is related to the env execution of R scripts that generates the data. Could you open an issue?
and assuming that "Commits per month" refers to the code repositories,
These are the commits you have done in all repositories analyzed (mediawiki core and all mediawiki extensions that are in wikimedia gerrit)
I have not made any commits in 2011 but the graph says so.
We should check it. The Unique identities scripts need to be tuned for Mediawiki. Maybe you have a wrong identity assigned to you. As soon as we start working in unique identities, I will check that.
Andre, if you want to open another issue great. I can do it for you!
I did not even have a Wikimedia Gerrit account at that time. :)
Clearly there is something wrong here.
* http://bitergia.com/projects/mediawiki-dashboard/browser/its.html lists "Top its_closers last month".Does this only refer to tickets under the Bugzilla product "MediaWiki"?
Not only Mediawiki, but also Mediawiki Extensions. You can see the repositories analyzed in the per repositories report:
http://bitergia.com/projects/mediawiki-dashboard/browser/its-repos.html
Looking at https://bugzilla.wikimedia.org/weekly-bug-summary.cgi?tops=10&days=32 and "Top 10 people who resolved the most reports in the last 32 days", I see very different names (but that's Bugzilla-wide and cross-product).
It is better to work with unique identities before doing this cross checking. I have cross check total numbers between Bugzilla and our data and they are correct.
* As people use different email addresses across mailing lists, Bugzilla, and Gerrit, I'm curious if there is a plan to map this accounts to each other. Somehow.
Sure. We have some script that helps in the unique identities mapping like this one:
https://github.com/VizGrimoire/VizGrimoireR/blob/newperiod/misc/unifypeople....
But in order to have a good mapping, there is manual work to be done. We will need Wikimedia help in this field.
Also we have reports per country. But we need to know people country in order to build them, so we should also try to gather this information.
In general, people level is something experimental in this Community Dashboard and I hope we can improve it in this project.
* Same question for mapping resources to each other, e.g. project-specific mailing lists to the corresponding Git repository to the corresponding project or component in Bugzilla. When I played with Apache and GNOME projects a few months ago I tried to use their DOAP files to get metadata on resources, but I'm not aware of some similar concept in Wikimedia.
Right now we have created mapping for some dashboard, like for example the one we created for OpenStack. We created by hand the mapping between git and its repos, in order to build reports together:
https://github.com/VizGrimoire/VizGrimoireJS/blob/openstack/browser/data/jso...
But we should improve how things are done. In a first approach we can reuse it if needed.
* Personally I think I'm most curious about statistics how many users are active in a specific codebase repository, to identify bus factors.
Pretty interesting. Once of the key issues in order to have success in this project is to identity the KPI (key performance indicators) for Mediawiki (in the fist step) community. And this is something pretty specific and we need your community input in order to define them. We will be proposing several metrics:
https://github.com/VizGrimoire/VizGrimoireJS-lib/blob/master/data/metrics.js...
we have others more advanced, but the platform is designed to make easy to add new metrics to data sources.
(When I ran "git log" on a full checkout of GNOME's Git repository a few months ago, one third of the projects were dead and had not seen any code commits in two years, excluding translation updates which are in subfolders.)
Thank you very much for your feedback Andre!
Cheers
Cheers, andre