I was really happy to hear Damon, at the MediaWiki Developer Summit, ask us how long we take to code review and whether we had communicated a timeframe in which we promised to do it to our community. He quite rightly stressed that this was vital for the survival of our community. I spoke to one of our more new developers during the summit and he also confessed to me that the reason he was an active volunteer in our extension was that he got feedback on his code pretty quickly.
I had a few ideas about how to measure this so in my spare time I have generated this report based on data from Gerrit patchsets using a hacked together python script [1] which I hope will be if nothing else an interesting artifact to talk about and generate some discussion.
Introducing: https://www.mediawiki.org/wiki/Extension_health
To help you understand what you are reading, let's take Echo as an example:
Project: mediawiki/extensions/Echo 524 patches analysed (23 open, 501 merged) Average review time: 29 days Oldest open patch: (bug 41987) Updating tables indexes' names. (766 days) - https://gerrit.wikimedia.org/r/40095
The average time for code to go from submitted to merged appears to be 29 days over a dataset of 524 patches, excluding all that were written by the L10n bot. There is a patchset there that has been _open_ for 766 days - if you look at it it was uploaded on Dec 23, 2012 12:23 PM is -1ed by me and needs a rebase.
There are many patches like this outside Echo. We should probably be seeing those patchsets through to completion or abandoning them on the basis that if it hasn't been merged in over 2 years it's probably not important and shouldn't be clogging up people's review queues and hiding other more important patchsets which need review.
The more troubling situation is when patches have been open for some time and have not been reviewed at all or are awaiting for some response... let's get better at this!
Help make this ecosystem better. I will rerun the script in a month and see if we have improved. Note our average time to review across all those projects seems to be 18 days. That's really not good. We can do better. We will do better.