I was really happy to hear Damon, at the MediaWiki Developer Summit,
ask us how long we take to code review and whether we had communicated
a timeframe in which we promised to do it to our community. He quite
rightly stressed that this was vital for the survival of our
community. I spoke to one of our more new developers during the summit
and he also confessed to me that the reason he was an active volunteer
in our extension was that he got feedback on his code pretty quickly.
I had a few ideas about how to measure this so in my spare time I have
generated this report based on data from Gerrit patchsets using a
hacked together python script [1] which I hope will be if nothing else
an interesting artifact to talk about and generate some discussion.
Introducing:
https://www.mediawiki.org/wiki/Extension_health
To help you understand what you are reading, let's take Echo as an example:
Project: mediawiki/extensions/Echo
524 patches analysed (23 open, 501 merged)
Average review time: 29 days
Oldest open patch: (bug 41987) Updating tables indexes' names. (766
days) -
https://gerrit.wikimedia.org/r/40095
The average time for code to go from submitted to merged appears to be
29 days over a dataset of 524 patches, excluding all that were written
by the L10n bot. There is a patchset there that has been _open_ for
766 days - if you look at it it was uploaded on Dec 23, 2012 12:23 PM
is -1ed by me and needs a rebase.
There are many patches like this outside Echo. We should probably be
seeing those patchsets through to completion or abandoning them on the
basis that if it hasn't been merged in over 2 years it's probably not
important and shouldn't be clogging up people's review queues and
hiding other more important patchsets which need review.
The more troubling situation is when patches have been open for some
time and have not been reviewed at all or are awaiting for some
response... let's get better at this!
Help make this ecosystem better. I will rerun the script in a month
and see if we have improved. Note our average time to review across
all those projects seems to be 18 days. That's really not good. We can
do better. We will do better.
[1]
https://github.com/jdlrobson/GerritCommandLine