[Labs-l] Fall 2015 Tool Labs user survey data published

[Bryan Davis]

On Wed, May 4, 2016 at 7:57 PM, Bryan Davis <bd808 at wikimedia.org> wrote:
> [0]: https://meta.wikimedia.org/wiki/Research:Annual_Tool_Labs_Survey

Apologies for that abrupt initial message, that was a great example of
hitting the wrong key in a mail client. :)

Between 2015-09-25 and 2015-10-08, the Wikimedia Foundation ran a
direct response user survey of registered Tool Labs users. 106 users
responded to the survey.

Based on responses to demographic questions, the average[1] respondent:
* Has used Tool Labs for 1-3 years
* Developed & maintains 1-3 tools
* Spends an hour or less a week using Tool Labs
* Programs using PHP and/or Python
* Does the majority of their work locally
* Uses source control
* Was not a developer or maintainer on Toolserver

[1]: "Average" here means a range of responses covering 50% or more of
responses to the question. This summarization is coarse, but useful as
a broad generalization. Detailed demographic response data will be
made available on wiki.

Qualitative questions:
64% agree that services have high reliability (up time).
69% agree that it is easy to write code and have it running on Tool Labs.
67% agree that they feel they are supported by the Tool Labs team when
they contact them via labs-l mailing list, #wikimedia-labs IRC
channel, or phabricator.
53% agree that they receive useful information via labs-announce /
labs-l mailing lists.
52% disagree that documentation is easy-to-find.
71% find the support they receive when using Tool Labs as good or
better than the support they received when using Toolserver.
50% disagree that Tool Labs documentation is comprehensive.
50% agree that Tool Labs documentation is clear.

Service usage:
45% use LabsDB often.
60% use webservices often.
54% use cronjobs often.
75% never use redis.
41% never use continuous jobs .

The survey included several free form response sections. Survey
participants were told that we would only publicly share their
responses or survey results in aggregate or anonymized form. The
freeform responses include comments broadly falling into these
categories:

Documentation (33 comments)
Stability and performance (18 comments)
Version control and Deployment (14 comments)
Logs, Metrics, and Monitoring (12 comments)
Package management (10 comments)
SGE (8 comments)
Database (7 comments)
Account/Tool creation (5 comments)
SSH (5 comments)
Other (11 comments)

Additonal details are available on meta [2].

[2]: https://meta.wikimedia.org/wiki/Research:Annual_Tool_Labs_Survey

Bryan
-- 
Bryan Davis              Wikimedia Foundation    <bd808 at wikimedia.org>
[[m:User:BDavis_(WMF)]]  Sr Software Engineer            Boise, ID USA
irc: bd808                                        v:415.839.6885 x6855