TLDR: How fast should new content (maps) features be rolled out, and how ready should they be? Constant but smaller improvements seems better.
As we gradually roll out maps to wider and wider audience, I would like to get some feedback on how we approach new feature roll-outs.
Frankly, WMF has botched a number of releases in the past. In a way, this is great, because it means both WMF and volunteers are still eager to improve Wikipedia, we are still trying to make things better. On the other hand, it is never a good thing to irritate the most important group - our community. So there is always a compromise: when and what to roll out, how to make it least disruptive vs how to improve the usability and the content quality.
For wikis, there are two types of improvements: user interface and content. User interface features change how one views and edits site's content, so any change immediately affects everyone. We try to mitigate it with "Beta features" -- logged in users may enable new functionality before it is enabled by default for everyone, but the vast majority of the readers are not logged in, so when enabled, it is still a serious and instant change. Maps are not really a part of interface because they only appear as part of the page content, which is fully controlled by the community.
Content features allow editors to add new type of content - maps, graphs, sheet music, text formatting or new types of templates. There is no way to hide a new content feature behind a "beta flag" because everyone sees the same content, but content features are not disruptive because they depend on the editors to add them to the pages. Community has full control, and if it does not like a feature, or if it feels the feature is not ready yet, it will not be used. The only time content changes are disruptive is when the support for a widely used feature changes or gets disabled. This is clearly not the case with the maps roll out on Wikipedia.
The WOW effect, and marketing in general, have both positive and negative effect on a feature roll out. If a feature is quietly enabled, only the more engaged community members will experiment with it and discuss the feature's best usage, give feedback on how to improve it, and eventually enable it at its own pace. A massive marketing of a feature would attract a lot of attention and expedite adaption, but may also create some amount of negativity if the community feels the feature is not yet ready.
I also feel it is very dangerous to delay releasing new features until they are perfect. We (developers/PMs/...) may think we know what feature is needed, but most likely we are wrong. If we delay, we may spend a lot of resources on polishing something that is not needed. Instead, by releasing early, community's feedback would put us back on the right track. Yes, it may not be as good, but at least we will quickly change direction, producing a truly needed feature that can be polished later. Of course this is much easier to do with the content features rather than user interface changes (hence the UI's "feature flag").
In light of this, I feel it is better to continuously roll-out small content-related features without much publicity (e.g. Village pump is OK, blog might be less so), and continuously improve based on community feedback. Once the feature has been out for some time and there is a general consensus that the feature is good, we can start the marketing push. This approach creates less stress on the community lesions, developers, and servers. Feedback is received in smaller portions and can be properly acted on.
This discussion is a bit generic, I'm not sure I understand what's the real issue at hand. But it's good to discuss. I'll jump to what's IMHO the main point.
Yuri Astrakhan, 05/11/2016 19:26:
Maps are not really a part of interface because they only appear as part of the page content, which is fully controlled by the community.
On the other hand, maps are most often used via templates. Such templates usually have a long history and often their maintainers have long since disappeared. Most of the time, it's hard to implement any change simply because the editors have other, more pressing things to look after.
If you select some widely used templates on a few dozens wikis, and code their replacement for the community, you can have a deployment which is both gradual and (locally) massive. Of course it's important to not impose yourself and at the same time take responsibility of monitoring the effects and adjusting the templates later if issues arise, although this is usually not the developers' concern.
Nemo
Le 2016-11-05, Yuri Astrakhan a écrit :
TLDR: How fast should new content (maps) features be rolled out, and how ready should they be? Constant but smaller improvements seems better.
Hello Yuri,
(please note I read the whole text, but...) your discussion of the question is interesting, but it sounds a little too theoretical to me. Could you be more precise and give actual instances of the map features you have in mind ?
Guillaume and Federico, thanks! I might have been too generic :) Basically I would like to get a general feel for
* are we enabling new content-oriented (no impact until editors add it to the pages) features too fast or too slow? And that mostly means maps - should we make maps absolutely perfect before giving it into the hands of the community?
* should we enable new features earlier, in a more unfinished state, so that community can comment on things earlier, and possibly tell us if we are going in the wrong direction? Or should we release them later, so new features are more polished from the start, but possibly less needed (miss the mark) and would require considerable resources to rework?
On Sat, Nov 5, 2016 at 6:17 PM Guillaume Allegre allegre.guillaume@free.fr wrote:
Le 2016-11-05, Yuri Astrakhan a écrit :
TLDR: How fast should new content (maps) features be rolled out, and how ready should they be? Constant but smaller improvements seems better.
Hello Yuri,
(please note I read the whole text, but...) your discussion of the question is interesting, but it sounds a little too theoretical to me. Could you be more precise and give actual instances of the map features you have in mind ?
-- ° /\ Guillaume Allègre OpenStreetMap France http://www.openstreetmap.fr /~~/\ Allegre.Guillaume@free.fr Wikipédia Wiktionnaire Wikimédia-Commons Wikidata / /~~\ tél. 04.76.63.26.99 Des contenus partagés libres et collaboratifs
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Le 2016-11-05, Yuri Astrakhan a écrit :
Guillaume and Federico, thanks! I might have been too generic :) Basically I would like to get a general feel for
- are we enabling new content-oriented (no impact until editors add it to
the pages) features too fast or too slow? And that mostly means maps - should we make maps absolutely perfect before giving it into the hands of the community?
- should we enable new features earlier, in a more unfinished state, so
that community can comment on things earlier, and possibly tell us if we are going in the wrong direction? Or should we release them later, so new features are more polished from the start, but possibly less needed (miss the mark) and would require considerable resources to rework?
As a (little) contributor, I would like to have more frequent, little increases and new features, (RERO https://en.wikipedia.org/wiki/Release_early,_release_often). But I can't speak from a reader-only point of view.
BTW, I was thrilled to have Kartographer/maplink available in Wikipedia 2 months ago, and I had a little burst of propaganda endeavour concerning this feature, with a little blog post(1) in french and some talk on WP project pages which could use it (mainly monuments and natural preservation areas). I'd like to know if there is a way to list all the pages using maplink/mapframe on a version of Wikipedia, to help figure out the popularity of the feature, and its evolution.
(1) https://gallaxie.wordpress.com/2016/09/21/wikipedia-wikidata-et-openstreetma...
I am very happy that we are able to add new type of contents (especially maps) to Wikipedia articles. I would like to see more content of the map layer(s) itself and more options and features of these tools.
I would suggest to release the new features only as (opt in) beta feature (for registered users) for a longer time, until most of the bugs are discovered, and then I would roll out as a stable feature for everybody. Maybe this is a slower way, but safer and cause less dissatisfaction in the community and about Wikipedia.
Best, Samat
On Sat, Nov 5, 2016 at 11:38 PM, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
Guillaume and Federico, thanks! I might have been too generic :) Basically I would like to get a general feel for
- are we enabling new content-oriented (no impact until editors add it to
the pages) features too fast or too slow? And that mostly means maps - should we make maps absolutely perfect before giving it into the hands of the community?
- should we enable new features earlier, in a more unfinished state, so
that community can comment on things earlier, and possibly tell us if we are going in the wrong direction? Or should we release them later, so new features are more polished from the start, but possibly less needed (miss the mark) and would require considerable resources to rework?
Samat, it is not really possible to enable *content* features for logged-in users only. An article content is shown to everyone, so if an editor adds a map to an article, everyone would see that map. On the other hand, everyone will be able to see if it's broken, and remove it right away, without any involvement from developers. The whole thing would be in the hands of the community - deciding what to use, usage policies, etc.
On Sun, Nov 6, 2016 at 5:51 AM Samat samat78@gmail.com wrote:
I am very happy that we are able to add new type of contents (especially maps) to Wikipedia articles. I would like to see more content of the map layer(s) itself and more options and features of these tools.
I would suggest to release the new features only as (opt in) beta feature (for registered users) for a longer time, until most of the bugs are discovered, and then I would roll out as a stable feature for everybody. Maybe this is a slower way, but safer and cause less dissatisfaction in the community and about Wikipedia.
Best, Samat
On Sat, Nov 5, 2016 at 11:38 PM, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
Guillaume and Federico, thanks! I might have been too generic :) Basically I would like to get a general feel for
- are we enabling new content-oriented (no impact until editors add it to
the pages) features too fast or too slow? And that mostly means maps - should we make maps absolutely perfect before giving it into the hands of the community?
- should we enable new features earlier, in a more unfinished state, so
that community can comment on things earlier, and possibly tell us if we are going in the wrong direction? Or should we release them later, so new features are more polished from the start, but possibly less needed (miss the mark) and would require considerable resources to rework?
Maps-l mailing list Maps-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l
Yuri,
Last half a year I am experimenting with rewriting some of the Commons templates to take advantage of Wikidata, which recently allows arbitrary access from Commons. Some of Commons templates and modules are used on 40M pages (compare to 5M articles on English Wikipedia). Changes to such templates will trigger refresh of all those pages so they should be kept to the minimum; on the other hand large changes to templates or modules used on many pages are more likely to break something and will be quickly reverted. As a result me and others use approach of occasional medium size changes, which gradually change the software. That also allows us to observe the results, and look for potential issues, which we try to correct quickly, before millions of pages are refreshed over a period of weeks.
One of the templates updated that way is Commons {{Location}} and {{Object location}} templates, which are written in Lua [7], and which now have link to Yuri’s Kartographer extension [1]. Enabling that and being able to work with Kartographer maps immediately allow discovery of various issues [2] and possible extensions [3] which would improve the software. Similarly, enabling comparison of Commons and Wikidata coordinates allow us to find mismatches, see [[Category:Pages with local coordinates and mismatching wikidata coordinates]] [4] and working on resolving them quickly proved that it is really hard to pinpoint true location of some object. For example Church of the Exaltation of the Holy Cross in Yekaterinburg [4] where Commons location is 2 km away from Wikidata location. One of them is wrong but which one? Easy way to find out would be to see map with geolocation of images in the church category. Apparently {{Geogroup}} template does that, so now I hope to merge it with {{Object location}} so one change naturally guides the need for the future changes. I also proposed to extend Kartographer to be able to do the same so I do not have to be switching between 2 OSM maps [3], but that might take longer. Need for those changes would be really hard to predict if one was waiting for the perfection before the final software release.
Hope that answers some of your questions. By the way, to other map enthusiasts: I would appreciate help with fixing pages with mismatched coordinates between Wikidata and Commons [4].
Jarek T. User:Jarekt
[1] https://www.mediawiki.org/wiki/Extension:Kartographer [2] https://phabricator.wikimedia.org/T149427 [3] https://phabricator.wikimedia.org/T149280 [4] https://commons.wikimedia.org/wiki/Category:Pages_with_local_coordinates_and... [4] https://commons.wikimedia.org/wiki/Category:Church_of_the_Exaltation_of_the_... [5] https://commons.wikimedia.org/wiki/Category:Church_of_the_Exaltation_of_the_... [6] https://commons.wikimedia.org/wiki/Template:Geogroup [7] https://commons.wikimedia.org/wiki/Module:Coordinates
From: Maps-l [mailto:maps-l-bounces@lists.wikimedia.org] On Behalf Of Yuri Astrakhan Sent: Saturday, November 05, 2016 2:26 PM To: Discovery; Maps Subject: [Maps-l] Maps, community, and enabling disruptive tech
TLDR: How fast should new content (maps) features be rolled out, and how ready should they be? Constant but smaller improvements seems better.
As we gradually roll out maps to wider and wider audience, I would like to get some feedback on how we approach new feature roll-outs.
Frankly, WMF has botched a number of releases in the past. In a way, this is great, because it means both WMF and volunteers are still eager to improve Wikipedia, we are still trying to make things better. On the other hand, it is never a good thing to irritate the most important group - our community. So there is always a compromise: when and what to roll out, how to make it least disruptive vs how to improve the usability and the content quality.
For wikis, there are two types of improvements: user interface and content. User interface features change how one views and edits site's content, so any change immediately affects everyone. We try to mitigate it with "Beta features" -- logged in users may enable new functionality before it is enabled by default for everyone, but the vast majority of the readers are not logged in, so when enabled, it is still a serious and instant change. Maps are not really a part of interface because they only appear as part of the page content, which is fully controlled by the community.
Content features allow editors to add new type of content - maps, graphs, sheet music, text formatting or new types of templates. There is no way to hide a new content feature behind a "beta flag" because everyone sees the same content, but content features are not disruptive because they depend on the editors to add them to the pages. Community has full control, and if it does not like a feature, or if it feels the feature is not ready yet, it will not be used. The only time content changes are disruptive is when the support for a widely used feature changes or gets disabled. This is clearly not the case with the maps roll out on Wikipedia.
The WOW effect, and marketing in general, have both positive and negative effect on a feature roll out. If a feature is quietly enabled, only the more engaged community members will experiment with it and discuss the feature's best usage, give feedback on how to improve it, and eventually enable it at its own pace. A massive marketing of a feature would attract a lot of attention and expedite adaption, but may also create some amount of negativity if the community feels the feature is not yet ready.
I also feel it is very dangerous to delay releasing new features until they are perfect. We (developers/PMs/...) may think we know what feature is needed, but most likely we are wrong. If we delay, we may spend a lot of resources on polishing something that is not needed. Instead, by releasing early, community's feedback would put us back on the right track. Yes, it may not be as good, but at least we will quickly change direction, producing a truly needed feature that can be polished later. Of course this is much easier to do with the content features rather than user interface changes (hence the UI's "feature flag").
In light of this, I feel it is better to continuously roll-out small content-related features without much publicity (e.g. Village pump is OK, blog might be less so), and continuously improve based on community feedback. Once the feature has been out for some time and there is a general consensus that the feature is good, we can start the marketing push. This approach creates less stress on the community lesions, developers, and servers. Feedback is received in smaller portions and can be properly acted on.
Disclaimer: I'm an Agile Coach at the WMF, and have worked with Discovery a lot, but only a little bit with the interactive team. These thoughts are my own; I am not speaking for the foundation, nor for any department or team.
On Sat, Nov 5, 2016 at 6:26 PM, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
TLDR: How fast should new content (maps) features be rolled out, and how ready should they be? Constant but smaller improvements seems better.
My tl;dr: "It's complicated."
For those who may have been confused about the context of this email, it is a continuation of some internal team/vertical conversations we have been having. Those debates have not been as simple as "fast vs. slow" or "release incrementally or wait until it is perfect". Everyone I have spoken to is in favor of fast and incremental development.
So there is always a compromise: when and what to roll out, how to make it least disruptive vs how to improve the usability and the content quality.
I agree, although other groups besides communities can be disrupted. Notably other WMF teams like legal, Visual Editor, ops, and others. Finding the right balance between all the stakeholders can be challenging.
Maps are not really a part of interface because they only appear as part of
the page content, which is fully controlled by the community.
This is a reasonable basic framing, but there is a lot of grey. The moment an editor adds a map to a page, that map becomes part of the interface. Any changes to the maps code will alter the user experience of that page. If Visual Editor needs a new feature to support maps, then all editors (even first-time/casual editors) will experience the pros and cons of that new editing feature.
content features are not disruptive because they depend on the editors to add them to the pages.
I disagree with that blanket statement, for reasons given above.
Community has full control, and if it does not like a feature, or if it feels the feature is not ready yet, it will not be used.
"Community" is vague and amorphous. Some editors might love a content features, while some readers might hate it. Or some people might love this month's release, but hate the update a month later. Once a feature is in use, removing even small aspects of it can be very contentious.
I also feel it is very dangerous to delay releasing new features until they are perfect.
I believe everyone in Discovery, and most of Engineering, agree with that statement. "Release early and often" is pretty well-accepted here, I think. The question becomes: How early, and how often?
In light of this, I feel it is better to continuously roll-out small content-related features without much publicity (e.g. Village pump is OK, blog might be less so), and continuously improve based on community feedback. Once the feature has been out for some time and there is a general consensus that the feature is good, we can start the marketing push. This approach creates less stress on the community liaisons, developers, and servers. Feedback is received in smaller portions and can be properly acted on.
A given feature could be somewhere on the complete <-> incomplete spectrum, but it is also somewhere on the buggy <-> bugfree spectrum. Releasing incomplete features quickly is a pretty easy sell. Releasing buggy features can be problematic. In addition to "features" and "bugs", there are legal, scalability, reliability, usability, and accessibility issues. They all combine into the magic question: Is some code *ready* to be released.
In order to answer that question, the team needs some processes and norms. Design research, focus group testing, RFCs, conversations with various communities, Automated testing, manual testing, load testing, staging environments, beta features, and small-wiki rollouts can all be part of a strategy to release quality software quickly. The interactive team is still relatively new, and has been moving very fast, so it is still figuring out its strategy with all of this.
As for publicity: I can understand not blogging about a new feature until it has proven its value. But my own naive inclination is to be very open in communicating about upcoming plans, imminent features, and just-released features. Village pump is one channel, and mailing lists would be another. Of course, phabricator is also a channel, but it is extremely difficult for most people outside the development team to follow. Personally, I think communities should have a chance to know what is coming rather than only being able to respond after it has been deployed.
To conclude, these are useful conversations about important topics. There are no easy answers, and generally there are not single "correct" answers. Let's acknowledge that it's messy.
Kevin Smith Agile Coach, Wikimedia Foundation