Dear all,
El lun, 12-08-2013 a las 19:21 +0200, Quim Gil escribió:
你好, 近排點呀? [0]
At Wikimania there was an improvised lunch-meeting about community
metrics with Jesús González-Barahona (Bitergia), Sumana, RobLa and
myself. The main conclusion was that
http://korma.wmflabs.org needs to
show a very few key metrics that could drive decisions affecting our
strategy and resources.
Yes, it BI terminology, this metrics are the KPI (Key Performance
Indicators), so we can reuse this name.
Let's agree first on key factors to watch, in the scope of projects
deployed in Wikimedia servers:
* Are the teams more efficient processing contributions?
* Is the share of non-WMF contributions growing?
* Are WMF and non-WMF contributions treated equally?
* Are the attraction and retention of new contributors improving?
* Are we improving the sustainability of our community?
If those are the key factors, these could be the key metrics:
# Who is contributing merged code each quarter? More origins == Better.
## WMF / WMDE / other Wikimedia / companies / OSS projects / independents
## Location of contributors (based on provided data)
# Time to resolve Gerrit changes. Shorter == Better.
## Authored by WMF/WMDE employees vs independent. Should be the same.
## Merged vs rejected. Similar progress?
## Best projects vs bottlenecks.
# Queue of open Gerrit change requests in relation to total amount of
contributions. Shorter == Better.
## Same points as above.
# New contributors with 1 / 2-5 / 6+ changes submitted in the past 3
months. More == Better.
## % of merged / rejected.
## Who is growing / stable / vanishing?
# Age of contributors since they started in the project. Small and
regular decline == Better.
## Number of contributions from each age group in the past quarters.
## WMF / non-WMF ratio for each age group.
Quim, thinking about those metrics, they are all related to Gerrit and
centered around people more than activity.
Any thoughts about the other data sources? From git, the commits metrics
is a bit rough, but combining it with SLOC and files, is a good
indicator of activity in the development. Also the tendencies for this
metrics (week, month and year) are a good indicator of how the project
is evolving. For example, last period of 365 days indicators points out
that in general, the activity is growing in Wikimedia projects:
http://korma.wmflabs.org/browser/scm.html
63% in commits, 159% in authors and 103% in files.
Taking a look to this metrics, why commits are growing less than
authors? Thinking in the metrics and tendencies and relations, we can
reach pretty interesting questions to understanding in deep how
Wikimedia development is going.
Taking a look to ITS, one metric that I think we are missing is
*pending* issues (open issues but not closed). And this concept is
something that could be applied to other data sources: the pending work
to be done.
Tendencies and pending are two areas in which we can find interesting
metrics that could evolve to KPIs.
In ITS we have also the time to close evolution for the 99% of tickets,
95% of them, 50% and 25%. "Time to" is always a good SLA (Service Level
Agreement) indicator and at the end, SLA is a strategic position for the
community.
In MLS with the current metrics, we can see that around 50% messages
generate a thread (has some response), and for example, and interesting
metric is that each person send 20 messages (ratio between total
messages and senders). So in general, mailing list have feedback and
people participate with several messages. Here the "Time to" shows that
95% of messages are attended in less 20 days and that the time to
attention has grown last year. The care of the mailing lists is going
down?
In demographics we can see pretty good information about all data
sources community. Mailing list members are going down also (mailing
lists are used less? where is the support of the projects moving on?),
SCM new members are also going down but ITS members are growing. The
software products start to be mature and more work is centered around
quality?
I think Quim that the starting point that you propose is a good one,
specially the goal questions. The Goal (improve strategy decision and
resources management)/Question/Metrics is the right methodology to
follow. With this email I just want to broaden a bit the discussion.
Keep in mind that the deep review of all data in the current Community
Dashboard is pending, so take all the numbers above with a grain of
salt.
Cheers!
--
|\_____/| Alvaro del Castillo
[o] [o] acs(a)bitergia.com - CTO, Software Engineer
| V |
http://www.bitergia.com
| |
-ooo-ooo-