New Developers Quarterly Report's first edition

List overview All Threads
Download

newer

older

October 12: Strategy update -...

Books & Bytes – Issue 23, June –...

Srishti Sethi

17 Oct 2017 17 Oct '17

7:03 p.m.

Hello everyone, I would like to share the first edition of the New Developers Quarterly Report <https://www.mediawiki.org/wiki/New_Developers/Quarterly/2017-10> that the Developer Relations team has produced. This report covers metrics, survey analysis and lessons learned from new developers focused activities in the previous quarter (July-September 2017). If you have questions and feedback that you would like to share with us, please add them on the discussion <https://www.mediawiki.org/wiki/Talk:New_Developers/Quarterly/2017-10> page. To receive a notification when a new report is published, subscribe here <https://www.mediawiki.org/wiki/Special:Newsletter/11/subscribe>. We plan to release a report every quarter and take action items identified from the key findings for improving our existing methods and processes. The next release will be in January 2018. If you have any questions, comments, and concerns, we will be more than happy to hear them! Thanks, Srishti -- Srishti Sethi Developer Advocate Technical Collaboration team Wikimedia Foundation https://www.mediawiki.org/wiki/User:SSethi_(WMF)

Show replies by date

Brian Wolff

18 Oct 18 Oct

11:12 a.m.

New subject: [Wmfall] New Developers Quarterly Report's first edition

On Tue, Oct 17, 2017 at 7:03 PM, Srishti Sethi <ssethi(a)wikimedia.org> wrote:

...

Hello everyone, I would like to share the first edition of the New Developers Quarterly Report that the Developer Relations team has produced. This report covers metrics, survey analysis and lessons learned from new developers focused activities in the previous quarter (July-September 2017). If you have questions and feedback that you would like to share with us, please add them on the discussion page. To receive a notification when a new report is published, subscribe here. We plan to release a report every quarter and take action items identified from the key findings for improving our existing methods and processes. The next release will be in January 2018. If you have any questions, comments, and concerns, we will be more than happy to hear them! Thanks, Srishti

...

From the report:

...

Percentage of volunteers active one year (± 3 months) after their first contribution, out of all new volunteers attracted one year ago (between April–June >2016). (Source: Calculation on data) QoQ: -26.5%. YoY: -60.0%

That's kind of scary.... -- bawolff

Fæ

1:31 p.m.

New subject: [Wikitech-l] [Wmfall] New Developers Quarterly Report's first edition

On 18 October 2017 at 12:12, Brian Wolff <bawolff(a)gmail.com> wrote:

...

On Tue, Oct 17, 2017 at 7:03 PM, Srishti Sethi <ssethi(a)wikimedia.org> wrote:

From the report:

That's kind of scary.... -- bawolff

Does the minus symbol in "-60.0%" mean anything? Being a retention percentage, I do not understand how it can be negative unless potential volunteers are getting rejected at the door before they can sign-up. Could that be corrected? Weak figures are unsurprising, at least when compared to other percentages in the Wikiverse, like truly miniscule levels of new editor retention that have been measured from investing in edit-a-thons. However the first statement in the report of "we are attracting around 54 developers per quarter and retaining 8% of them", i.e. 4/54, feels low enough to have a review of whether the events to attract developers are worth doing in their current formats. The return on investment in terms of volunteer time and basic expenses, must make them "non-successes". P.S. while on the perennial issue of jargon, could we avoid "noticings"? It's a neologism nobody ever needed, though I appreciated the use of "taken with a grain of salt". Thanks, Fae -- faewik(a)gmail.com https://commons.wikimedia.org/wiki/User:Fae

Brian Wolff

5:32 p.m.

New subject: [Wmfall] New Developers Quarterly Report's first edition

Fae wrote:

...

My understanding is that this means that the rentention percentage was 60% (or is it percentage points?) less than it was this time last year. So its now 5%, but this time last year it was 12%. -- bawolff

Fæ

6:21 p.m.

New subject: [Wmfall] New Developers Quarterly Report's first edition

On 18 October 2017 at 18:32, Brian Wolff <bawolff(a)gmail.com> wrote:

...

Fae wrote:

Quim Gil

20 Oct 20 Oct

10:18 a.m.

New subject: [Wmfall] New Developers Quarterly Report's first edition

Hi, I am very happy to see that the New Developers Quarterly report is raising some interest. Yes, there are important problems of sustainability in our developer community that deserve attention. On Wed, Oct 18, 2017 at 8:21 PM, Fæ <faewik(a)gmail.com> wrote:

...

Suggestion, throw away the current plan and rather than using findings to create incremental improvement,[1] try something completely different before all the wheels fall off. I look forward to seeing some serious radical initiatives.

The good news is that we have done this already. :) The retention numbers for this quarter correspond to the newcomers who landed between ~April-September 2016. We can expect there developers attracted by our hackathons in Jerusalem and Esino Lario, and the corresponding Google Summer of Code and Outreachy rounds. It was by that time when the Technical Collaboration team at the Wikimedia Foundation (who co-organizes these activities with mentors and affiliates) was digging beyond our apparent success, deeper into the problem of developer retention. Then we started to think that we should focus on new developers, even if that meant less focus for our more experienced technical contributors. Since then, we have radically changed our plans and we are experimenting in various ways. You can find a comprehensive explanation in a blog post published last week: How Technical Collaboration is bringing new developers into the Wikimedia movement <https://blog.wikimedia.org/2017/10/05/technical-collaboration-new-developers/> Since we are discussing about new developers, let me also recommend you another blog published just yesterday: Towards building an African Wikimedia Developer Community <https://blog.wikimedia.org/2017/10/19/african-wikimedia-developer-community/> . Wikimedia volunteers and affiliates, we welcome your ideas and involvement! When our developer community grows, everybody benefits. -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

Lodewijk

5:57 p.m.

New subject: [Wmfall] New Developers Quarterly Report's first edition

Just trying to understand: this is the percentage change of a percentage? Or the percentage change of the absolute retention? (I would be particularly interested in the latter, as the former could be muddied by successful efforts to have more people make a first contribution) Lodewijk On Wed, Oct 18, 2017 at 11:21 AM, Fæ <faewik(a)gmail.com> wrote:

...

On 18 October 2017 at 18:32, Brian Wolff <bawolff(a)gmail.com> wrote:

Fae wrote:

Ah, thanks for the clarification. I have a background as a mathematician, but that report with second-order numbers had me foxed. Now I think I understand the stats, I probably correctly appreciate that whatever actions were taken in the last 12 months to retain volunteers were not "non-successes", they are super fantastic management team learning points for the coming year... Suggestion, throw away the current plan and rather than using findings to create incremental improvement,[1] try something completely different before all the wheels fall off. I look forward to seeing some serious radical initiatives. Links: 1. https://en.wikipedia.org/wiki/Boiling_frog Thanks, Fae -- faewik(a)gmail.com https://commons.wikimedia.org/wiki/User:Fae _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wik i/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l(a)lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

James Salsman

19 Oct 19 Oct

1:44 a.m.

New subject: [Wmfall] New Developers Quarterly Report's first edition

Brian Wolff wrote:

...

[The developer retention rate is] now 5%, but this time last year it was 12%.

It's currently 8% per https://www.mediawiki.org/wiki/New_Developers/Quarterly/2017-10#Key_findings And the time series is the third graph under https://www.mediawiki.org/wiki/New_Developers/Quarterly/2017-10#Key_findings ...the first two of which are more important and solid. The variance of the retention rate is high, so its downward trend isn't too serious, but it would be nice if we could increase it. Speaking of new developer retention, Brian, would you be willing to mentor Brij Mohan at https://en.wiktionary.org/wiki/Wiktionary:Grease_pit/2017/August#Pronunciat… ? I would prefer that an experienced developer take over because I have little experience with gadget scripting. Best regards, James

Gergo Tisza

21 Oct 21 Oct

8:29 a.m.

New subject: [Wmfall] New Developers Quarterly Report's first edition

Thanks Srishti & DevRel team, that was interesting, and it's great to see that developer retention gets the organizational focus it deserves. Some comments on the document: We currently struggle to keep track of information about new developers

...

such as project they worked on, their contact details (e.g., Phabricator, Gerrit, and Github username), etc.

FWIW this is a problem with all developers, not just new ones. For example after I find out who is the best person to ask about something, there isn't really way to find out their IRC account. (Or vice versa, know who I've just talked with on IRC.) Phabricator can be linked to mediawiki.org and wikitech but few people do both. And so on. The WMF has its private contact list, but even that is not always updated, and it's unaccessible to volunteers. It would be great to have a proper, public contact info management system somewhere. Re: surveys, it would be a missed opportunity to only survey new developers. The limitation in that is that only a small fraction of them become regulars, and it would be great to understand better what personality trait or circumstance determines whether a given person stays or leaves, but the only way to find that out is to survey people whom we already know remained (or left) about their experiences as new developers. Re: retention (the raw numbers for which can be found in T160430#3395405), apparently year-on-year for Q3 2017 is interpreted as "(developers who started in 2016 Q3 and were still active in 2017 Q3 / all developers who started in 2016 Q3) / (developers who started in 2015 Q3 and were still active in 2016 Q3 / all developers who started in 2015 Q3)" which is a weird definition. YoY should be the difference in retention between two full years, not between two quarters spaced one year apart. One thing that jumps out looking at the list of retained developers is how many of them are experienced Wikipedians (at a glance, something like two-thirds?). Which I guess is not that surprising, but I wonder to what extent is it reflected in the outreach strategy? On Tue, Oct 17, 2017 at 12:03 PM, Srishti Sethi <ssethi(a)wikimedia.org> wrote:

...

2419

days inactive

2423

days old

wikimedia-l@lists.wikimedia.org

Manage subscription

8 comments

7 participants

tags (0)

participants (7)

Brian Wolff
Fæ
Gergo Tisza
James Salsman
Lodewijk
Quim Gil
Srishti Sethi