Hi all,
We've got the first DRAFT (sorry for shouting, but can't hurt to emphasize :)) of the annual goals for the engineering/product department up on mediawiki.org. We're now mid-point in the process, and will finalize through June.
https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals
Note that at this point in the process, teams have flagged inter-dependencies, but they've not necessarily been taken into account across the board, i.e. team A may say "We depend on X from team B" and team B may not have sufficiently accounted for X in its goals. :P Identifying common themes, shared dependencies, and counteracting silo tendencies is the main focus of the coming weeks. We may also add whole new sections for cross-functional efforts not currently reflected (e.g. UX standardization). Site performance will likely get its own section as well.
My own focus will be on fleshing out the overall narrative, aligning around organization-wide objectives, and helping to manage scope.
As far as quantitative targets are concerned, we will aim to set them where we have solid baselines and some prior experience to work with (a good example is Wikipedia Zero, where we now have lots of data to build targets from). Otherwise, though, our goal should be to _obtain_ metrics that we want to track and build targets from. This, in itself, is a goal that needs to be reflected, including expectations e.g. from Analytics.
Like last year, these goals won't be set in stone. At least on a quarterly basis, we'll update them to reflect what we're learning. Some areas (e.g. scary new features like Flow) are more likely to be significantly revised than others.
With this in mind: Please leave any comments/questions on the talk page (not here). Collectively we're smarter than on our own, so we do appreciate honest feedback:
- What are our blind spots? Obvious, really high priority things we're not paying sufficient attention to?
- Where are we taking on too much? Which projects/goals make no sense to you and require a stronger rationale, if they're to be undertaken at all?
- Which projects are a Big Deal from a community perspective, or from an architecture perspective, and need to be carefully coordinated?
These are all conversations we'll have in coming weeks, but public feedback is very helpful and may trigger conversations that otherwise wouldn't happen.
Please also help to carry this conversation into the wikis in coming weeks. Again, this won't be the only opportunity to influence, and I'll be thinking more about how the quarterly review process can also account for community feedback.
Warmly,
Erik
On Mon, Jun 9, 2014 at 9:24 PM, Erik Moeller erik@wikimedia.org wrote:
Hi all,
We've got the first DRAFT (sorry for shouting, but can't hurt to emphasize :)) of the annual goals for the engineering/product department up on mediawiki.org. We're now mid-point in the process, and will finalize through June.
https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals
Note that at this point in the process, teams have flagged inter-dependencies, but they've not necessarily been taken into account across the board, i.e. team A may say "We depend on X from team B" and team B may not have sufficiently accounted for X in its goals. :P Identifying common themes, shared dependencies, and counteracting silo tendencies is the main focus of the coming weeks. We may also add whole new sections for cross-functional efforts not currently reflected (e.g. UX standardization). Site performance will likely get its own section as well.
My own focus will be on fleshing out the overall narrative, aligning around organization-wide objectives, and helping to manage scope.
As far as quantitative targets are concerned, we will aim to set them where we have solid baselines and some prior experience to work with (a good example is Wikipedia Zero, where we now have lots of data to build targets from). Otherwise, though, our goal should be to _obtain_ metrics that we want to track and build targets from. This, in itself, is a goal that needs to be reflected, including expectations e.g. from Analytics.
Like last year, these goals won't be set in stone. At least on a quarterly basis, we'll update them to reflect what we're learning. Some areas (e.g. scary new features like Flow) are more likely to be significantly revised than others.
With this in mind: Please leave any comments/questions on the talk page (not here). Collectively we're smarter than on our own, so we do appreciate honest feedback:
- What are our blind spots? Obvious, really high priority things we're
not paying sufficient attention to?
- Where are we taking on too much? Which projects/goals make no sense
to you and require a stronger rationale, if they're to be undertaken at all?
- Which projects are a Big Deal from a community perspective, or from
an architecture perspective, and need to be carefully coordinated?
These are all conversations we'll have in coming weeks, but public feedback is very helpful and may trigger conversations that otherwise wouldn't happen.
Please also help to carry this conversation into the wikis in coming weeks. Again, this won't be the only opportunity to influence, and I'll be thinking more about how the quarterly review process can also account for community feedback.
Warmly,
Erik
Hi Erik,
Can you describe how specific engineering projects are helping to address the gender gap? Do you typically review user-facing projects with the gender gap specifically in mind? I notice that while there is a FOSS outreach program for women, there is nothing specific in the Wikimania Hackathon materials to suggest that an effort was made to attract women to this event. There is another hackathon planned for early next year - will engineering make the gender gap part of the goals of that event?
I also notice that the editor engagement projects don't specifically list the gender gap or women users. I think that testing new features (such as communication tools, notifications, teahouse-style innovations and others that bear specifically on interactions) with female user/focus groups would greatly improve our understanding of how these tools impact the gender gap. For example: If part of the plan for the growth team is to invite anonymous editors to sign up, why not tailor some of those invitations specifically to female anonymous editors? Then you could add a measure, retention of female editors, to that particular growth project. The results should be instructive.
Likewise, it would be nice to see gender-gap related goals within VisualEditor and the user experience groups goals, and to see some focus from the Analytics and Research teams on measuring and understanding the gap better. Needless to say its a little disappointing that the FOSS outreach program (which currently has 8 participants) is the only mention of women in the entire goals document, and the gender gap is never mentioned.
Nathan
If part of the plan for the growth team is to invite anonymous editors to sign up, why not tailor some of those invitations specifically to female anonymous editors? Then you could add a measure, retention of female editors, to that particular growth project. The results should be instructive.
At the risk of biting into something contentious... How would that work? What would the message say: "Have you considered creating an account? Its free. Doubly recommended if you're a women!" I have trouble imagining a gender-specific account creation invitation that doesn't sound creepy.
Additionally, actually measuring success rates by gender would require knowing people's gender. Its conceivable that requiring people to disclose their gender could have a negative impact on the gender gap. (Or maybe it wouldn't. I have no idea)
--bawolff
On Mon, Jun 23, 2014 at 3:45 PM, Brian Wolff bawolff@gmail.com wrote:
If part of the plan for the growth team is to invite anonymous editors to sign up, why not tailor some of those invitations specifically to female anonymous editors? Then you could add a measure, retention of female editors, to that particular growth project. The
results
should be instructive.
At the risk of biting into something contentious... How would that work? What would the message say: "Have you considered creating an account? Its free. Doubly recommended if you're a women!" I have trouble imagining a gender-specific account creation invitation that doesn't sound creepy.
Additionally, actually measuring success rates by gender would require knowing people's gender. Its conceivable that requiring people to disclose their gender could have a negative impact on the gender gap. (Or maybe it wouldn't. I have no idea)
--bawolff
I'm not sure what the messaging is planned to look like, but an appeal oriented towards women doesn't seem like an insurmountable obstacle. "Did you know only 10% of Wikipedia editors are women? If you are a woman reading this, we need your help!" And then you can track sign-ups through that particular message, figuring that a substantial proportion of them will actually be women.
And you could also ask, during sign-up, for people to self-identify confidentially. "We're trying to increase the proportion of our fellow editors who are women, would you mind telling us if you identify as female? [Y/N]". And then don't publish that anywhere except in aggregate.
(I'm not on wikimedia-l so this won't make it to that list unless someone forwards it; please feel free.)
wikitech-l and https://www.mediawiki.org/wiki/Talk:Wikimedia_Engineering/2014-15_Goals seem like good places to talk about whether we should have gendergap-related goals for Wikimedia Foundation engineering for 2014-2015. To discuss whether particular we could add or modify specific bits of functionality (e.g., the signup page, VE) to better appeal to underrepresented groups and help them start contributing, I suggest we use the Editor Engagement list:
https://lists.wikimedia.org/mailman/listinfo/ee
I also want to note that the new grant-funded project to gather better data on the gender gap will produce a midpoint report in September-October of 2014, and a final report in January 2015. The researcher will gather and aggregate data that Engineering could use: "This data may also be used to guide the design of future interventions or technology enhancements that seek to address the gap."
https://meta.wikimedia.org/wiki/Grants:IEG/Women_and_Wikipedia
So perhaps we could set up checkpoints for our engineering teams in October and January to look at those reports and use the new data to inform their work.
Sumana Harihareswara Senior Technical Writer Wikimedia Foundation
On Mon, Jun 23, 2014 at 10:28 AM, Nathan nawrich@gmail.com wrote:
On Mon, Jun 9, 2014 at 9:24 PM, Erik Moeller erik@wikimedia.org wrote:
Hi all,
We've got the first DRAFT (sorry for shouting, but can't hurt to emphasize :)) of the annual goals for the engineering/product department up on mediawiki.org. We're now mid-point in the process, and will finalize through June.
https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals
Note that at this point in the process, teams have flagged inter-dependencies, but they've not necessarily been taken into account across the board, i.e. team A may say "We depend on X from team B" and team B may not have sufficiently accounted for X in its goals. :P Identifying common themes, shared dependencies, and counteracting silo tendencies is the main focus of the coming weeks. We may also add whole new sections for cross-functional efforts not currently reflected (e.g. UX standardization). Site performance will likely get its own section as well.
My own focus will be on fleshing out the overall narrative, aligning around organization-wide objectives, and helping to manage scope.
As far as quantitative targets are concerned, we will aim to set them where we have solid baselines and some prior experience to work with (a good example is Wikipedia Zero, where we now have lots of data to build targets from). Otherwise, though, our goal should be to _obtain_ metrics that we want to track and build targets from. This, in itself, is a goal that needs to be reflected, including expectations e.g. from Analytics.
Like last year, these goals won't be set in stone. At least on a quarterly basis, we'll update them to reflect what we're learning. Some areas (e.g. scary new features like Flow) are more likely to be significantly revised than others.
With this in mind: Please leave any comments/questions on the talk page (not here). Collectively we're smarter than on our own, so we do appreciate honest feedback:
- What are our blind spots? Obvious, really high priority things we're
not paying sufficient attention to?
- Where are we taking on too much? Which projects/goals make no sense
to you and require a stronger rationale, if they're to be undertaken at all?
- Which projects are a Big Deal from a community perspective, or from
an architecture perspective, and need to be carefully coordinated?
These are all conversations we'll have in coming weeks, but public feedback is very helpful and may trigger conversations that otherwise wouldn't happen.
Please also help to carry this conversation into the wikis in coming weeks. Again, this won't be the only opportunity to influence, and I'll be thinking more about how the quarterly review process can also account for community feedback.
Warmly,
Erik
Hi Erik,
Can you describe how specific engineering projects are helping to address the gender gap? Do you typically review user-facing projects with the gender gap specifically in mind? I notice that while there is a FOSS outreach program for women, there is nothing specific in the Wikimania Hackathon materials to suggest that an effort was made to attract women to this event. There is another hackathon planned for early next year - will engineering make the gender gap part of the goals of that event?
I also notice that the editor engagement projects don't specifically list the gender gap or women users. I think that testing new features (such as communication tools, notifications, teahouse-style innovations and others that bear specifically on interactions) with female user/focus groups would greatly improve our understanding of how these tools impact the gender gap. For example: If part of the plan for the growth team is to invite anonymous editors to sign up, why not tailor some of those invitations specifically to female anonymous editors? Then you could add a measure, retention of female editors, to that particular growth project. The results should be instructive.
Likewise, it would be nice to see gender-gap related goals within VisualEditor and the user experience groups goals, and to see some focus from the Analytics and Research teams on measuring and understanding the gap better. Needless to say its a little disappointing that the FOSS outreach program (which currently has 8 participants) is the only mention of women in the entire goals document, and the gender gap is never mentioned.
Nathan _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
As an update on the goals process for WMF engineering, we've begun fleshing out out the top priorities for the first quarter. Going forward, we'll aim to call out the top priorities for each quarter as we approach it, to create more shared visibility into the most urgent and high-impact projects we're working on.
I've decided for now to use a division between "User-Impacting Changes" and "Cross-Functional Platform and Process Improvements". The intent of calling out both areas is to ensure that important organizational priorities don't fall off our collective radar. At the management level, the intent is for us to pay special attention to the priorities called out in this manner, and this may also impact our willingness to request help from across the organization if necessary to support these priorities, at least in Q1.
I've merged the current draft into the goals document, here: https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals#Top_depar...
Once again, this is draft and marked as such. The "Impact" column will include links to relevant metrics once those are a bit more solid; if you look further down in the document you'll see that these are being refined and tweaked in multiple areas right now.
A little bit of rationale for some items that may surprise you:
- I've decided to list HHVM as the top priority in both categories. This is because a) it's a very complex undertaking from an engineering perspective and requires significant coordination across development & operations, b) it's probably the biggest change regarding how code gets executed in production since we adopted PHP in the first place, c) the expected performance benefits for many uncached logged-in user operations are very significant (I defer to the team to quantify before throwing out estimates).
This is also indicative of the importance we're attaching to site performance. There's no question that performance is directly correlated with user engagement, and it's appropriate that we spend significant effort in this area.
- We're elevating SUL finalisation ( https://www.mediawiki.org/wiki/SUL_finalisation ) to a top priority, and I've classified it as user-impacting. This is because it's on the critical path for making it easier to develop cross-site functionality (as long as we have to deal with the edge case of "detached accounts", certain features that work across wikis are just trickier to implement), and one of those long term issues of technical debt we've been kicking down the road for years. It's also a pretty complex project -- if it goes wrong and we mess up our account database, we're in big trouble. So we want to make sure we have lots of eyeballs on this from a technical and community management perspective. We may not completely wrap up in Q1 since we need to give users whose accounts are affected significant warning time, which is just elapsed time we can't shorten.
- Front-end code standardization is called out as a top priority. We really need to dig ourselves out of the mess of having disjointed templating systems, widget libraries, and JS frameworks across our codebase if we want to increase development velocity and UX consistency. I'm prepared to sacrifice short term development velocity on other projects in order to make this happen.
- The content API that Gabriel is working on ( https://www.mediawiki.org/wiki/Requests_for_comment/Content_API ) is called out as a top priority. This is because the Parsoid output (for which the content API will be a high performance front-end) is now getting to the point where it's starting to become plausible to increasingly use it not just for VisualEditor, but also for views as well. The potential here are performance benefits across the board: for logged-in users in general by consistently relying on fast, cached output; for users loading VisualEditor by giving them most of the payload required to edit in read mode; for users saving through VisualEditor by potentially turning the wikitext transformation into a post-save asynchronous process and thereby making saves near instantaneous.
Moreover, it will put us on the long term path towards possibly using HTML5 as MediaWiki's native format, supporting HTML5-only wikis, and more. And it will be valuable for third party re-use and re-processing of Wikimedia content for a multitude of use cases. Last but not least, it's also a great use case for a service-oriented architecture including REST APIs & good/clean API documentation.
In short, this is a big deal, and it has lots and lots of architectural implications -- so raising the visibility on this is intended to get more people to actually think through what all of this means for the future of MediaWiki.
Other elements of the prioritization shouldn't be surprising: Phabricator is a big deal, and it's coming; Mobile (including new contributory features) and VE (including a really awesome new citations experience we're prepping for) continue to be high on the agenda, and we need to standardize and improve our quantitative measures across the board. Growth is going to work on new user acquisition in Q1 which didn't quite make the cut for the top 5 because it's not as broadly impacting as some of the other things listed here, but still a big deal -- as are other things not called out here. The intent isn't to demote anyone's work, but to give ourselves shared awareness of some of the most mission-critical things that are going on.
If you do feel the prioritization is off, leave a note on the talk page. It's still draft and we reserve the right to correct and adjust as we learn things, but we'll wrap the Q1 prioritization by July 1.
Thanks!
Erik
wikitech-l@lists.wikimedia.org