(reposting: held in moderation because of recipient list)
I am posting this to wikitech-l, ee-l, and will cross-post this to https://bugzilla.wikimedia.org/show_bug.cgi?id=35306 because it's important to keep frank discussions in the open.
When I use to loaded words like "back-dooring" etc, I believe that no malice was intended and the discussions so far have been in good faith from all parties. I think people have a valid concern and want it addressed and are wondering honestly how decisions have been made. In particular, my decision to not allow the MassMessage Extension to roll out onto MediaWiki last week, since that occurred during a meeting that didn't even involve or derive have consultation (except ex-post-facto) with any product manager or engineer here.
Here is why I am inijating this thread:
1) https://wikitech.wikimedia.org/w/index.php?title=Deployments&diff=83188&...
2) On Sep 30, 2013, at 4:25 PM, MZMcBride z@mzmcbride.com wrote:
Hi Fabrice, Terry, and Howie,
https://bugzilla.wikimedia.org/show_bug.cgi?id=35306#c19 is awaiting feedback (if you have any).
MZ
3) https://wikitech.wikimedia.org/w/index.php?title=Deployments&diff=84903&...
…
We have two separate but related bugs here:
https://bugzilla.wikimedia.org/show_bug.cgi?id=35306 https://bugzilla.wikimedia.org/show_bug.cgi?id=52723
bug 35306 is an tracking bug MZMcBride posted for a solution to deal with EdwardsBot. I believe it dates to around the time I was first employed almost two years ago.
bug 52723 is a recent bug to deploy Legoktm's MassMessage extension as a solution to bug 35306, it is less than a month old there has been little public discussion, and until it appeared on the deploy calendar last week, I wasn't even aware of its existence.
The problem with moving on bug 52723 as a solution to bug 35306 is it commits Features Engineering to maintaining an extension that is a stop gap solution with no or very little discussion in a manner that doesn't serve a broad strategic goal about how messages, notifications, etc. should be handled on the wikis. To the first, maybe there is something wrong with my e-mail client, but I have yet to find this discussion on wikitech-l or anywhere outside the bug.
Because of the above, my view is that this solution is being back-doored in and just moves the "technical debt" from one sheet (the community and tool labs) to another that has even less capability. I am biased against that because the latter sheet (WMF Features Engineering) is my responsibility. This is just my view, I'm open for us coming to consensus on a solution for this bug, but what I have seen is not consensus.
It is along these lines that, I asked to remove MassMessage from the deploy calendar when it was added to the deploy calendar without discussion from Features, Design, or Product last week. After discussion during that Friday meeting among the EPMs, I compromised to let it to go out on two test wikis, but not on mediawiki. Nobody made the case that it should go out on mediawiki. I demurred because no person at the WMF, including me as Director of Features Engineering, should fiat a decision when unaware of the status of discussions involved.
But let frank: if this had been a WMF employee writing this extension, it wouldn't have made it to even the test wikis by a country mile. In fact, a lot of extensions have been written by employees and either have extensive discussion, review, and buy-in (e.g. GuidedTours), or have not been deployed (e.g. Etherpad, the org chart, BetaFeatures) even though much more work has been done on them.
I also don't like that WMF resources in Platform and Design are being spent to review something that has had no adequate discussion over in the annual plan, in anyone's 20% time, in any cross-team discussion, or public discussion on wikitech-l (There was a last minute thread in the Design list, I am not on the design list, nor should I be, and the Creative Director is new and the team is just trying to get their sea legs and some consideration to that needs to be done here). Furthermore, after further discussion, nearly all of Product felt I should not have compromised earlier because the following situation might occur: Having gotten it onto "the cluster" people would then move to back-door it into deployment on the basis that it's already on the cluster. Their prediction is occurring right now. This is a bogus argument because nobody agreed this should be on the cluster, it's just that nobody has reviewed it in a thorough enough manner to shout "NO!"
If necessary, I'm going to shout "NO!" at this moment.
Having said that, there is the larger issue of bug 35306 which has been sitting there unsolved for a while as well as the smaller issue of the month's work Legoktm and MZMcBride have already put into Bug 52723 and [[mw:Extension:MassMessage]] possibly going to waste if I keep blocking it. Besides, MZ is sitting there trying to hold everything up on his own shoulders with EdwardsBot, and we all know it.
…
Let me address the functionality overlap with Echo:
LanguageEngineering built their own parallel message/notification system before Echo that lives on Meta today. They commit to supporting their own parallel message/notification system, and I'm okay with it living outside Echo (where it currently does), but there is an understanding that they've basically committed to that work with no support from Features for the duration that it does live outside Echo.
In those lines, I'm glad that Platform has helped out here, but unless Platform is committing to support MassMessage indefinitely into the future and not just provide one-time security and deploy assistance, I'm not okay with them adding to Features work even though they've been super helpful to MZMcBride and Legoktm. If Dan Garry is willing to commit Platform to support MassMessage, I'll think the same precedent we've done with LE applies here.
Furthermore, before *in-echo, outside-echo, use-echo) for a solution to be bug 35306, it needs to actually exhibit product discipline. The WMF gets panned for having a "agile processes" but not actually doing so, but we do have some process and we need to at least apply the same product discipline that we apply everywhere else to this bug.
For example, features in MassMessage and EdwardsBot need to be addressed in a Must-Have, Should-Have, Could-Have, Won't-Have (MoSCoW) framework. Features like "mass delivery from a list" are probably a Must-Have; features like "cross-wiki notification" are probably a should-have, other features like "public over private notifications", "page-centric over user-centric" or "longer stream notifications" are either not a must-have or could have or are should-haves to be done outside Echo by a different service (bot) using the Echo API or some extension thereof. All that is a product issue, and I nor anyone in my team is in product. Those decisions should be done in discussion with the Product team and they should not be disintermediated from it, which they have been.
I understand that many people would like to see a solution to Bug 35306 would be great to have. I'd like to see a better Signpost notification system and a more generalized solution for newsletter delivery also! I'd also like a pony. But we have already committed resources and continue to commit resources without discussion from the people (Product Design, not Features Engineering) who have been tasked with this responsibility and are very good at these sort of thing. I hope everyone participating on this bug can see that dropping this bomb on a newly hired associate product manager is simply not cool on so many levels.
…
Here is my suggestion:
(This is thinking, not a directive so don't think of this as definitive or final, I'm seeking consensus here.)
First, bug 35306 is a long-standing request. I think it's important we get headway on this, but I hope others will be understanding if it doesn't happen immediately, so long as there is a commitment for this to happen.
(For perspective, Flow was first proposed years ago, and was added to the annual plan almost two years ago before their first actual development sprint was completed (end of this week!). Echo was first deployed almost 8 months ago and is still not out on all the wikis. BetaFeatures has been in discussion for months and is still not deployed, nor is the commitment to maintain inside Features and that has caused a lot of issues. Fixing things right right takes time because consensus takes time and open discussion.)
There is an RfP for student developer time for legoktm for things like this (Finding a solution for Bug 35306 but not [[mw:Extension:MassMessage]] because MassMessage would not be deployed if it was my own engineer). Benny Situ has been spending all his time supporting [[mw:Extension:Echo]] when he should balancing [[mw:Flow]] development time into Echo bug fixes and needs to spend at least one sprint (two weeks) solely getting up to speed on Flow before doing enhancements for [[wm:Echo]]. Furthermore, Echo is not deployed everywhere yet and is still rolling out (though I've been pushing up the timetable on this as I feel we're too slow here), so it can't reach the places that EdwardsBot cat.
After the above happen, I'd like Benny and Kunal to work together to add some of the functionality of EdwardsBot into Echo for mass message delivery. Because of this, I'm moving bug 35306 into Echo as an enhancement and raising it's priority.
In the meantime I'll be OK with leaving MassMessage on test and test2 wikis because the alternative would be to remove it from the cluster entirely. The experiences and code Kunal derives by that can inform the enhancement to Echo, as well as things it already does that find themselves outside Echo (Echo does not and should not post to talk pages). So I figure two stages:
1) wait for some things to happen: legoktm to get an RfP, Echo to be on all wikis, and Benny to do an entire Flow only sprint and balancing his time more effectively wrt Echo. 2) MoSCoW other features of MassMessage/EdwardsBot for integration into Echo 3) Enhance and deploy a first pass to Echo to allow some sort of mass notification from a wiki list 4) Some sort of cross wiki notification enhancement (requires a design pass) 5) Discuss how to implement must-have or should-have features of EdwardsBot that can't live in Echo (permanent log of events)? 6) Add those to plan and be done.
The above can occur independently or in parallel to the above. If Product thinks it cool to commit Platform's constrained engineering resources to deploying and supporting MassMessage Extension forever and use it to take advantage of Echo when the features roll out in some undefined future, consider this thinking moot or at least orthogonal to MassMessage. IMO, it is bad enough that something important like BetaFeatures is without a home, but my answer from Features is "No" for MassMessage. If this was my own engineer, I'd raise hell with them for this and yell at their Product Manager for not being a good steward of Platform's time.
Take care,
terry
terry chay 최태리 Director of Features Engineering Wikimedia Foundation “Imagine a world in which every single human being can freely share in the sum of all knowledge. That's our commitment.”
p: +1 (415) 839-6885 x6832 m: +1 (408) 480-8902 e: tchay@wikimedia.org i: http://terrychay.com/ w: http://meta.wikimedia.org/wiki/User:Tychay aim: terrychay
On Oct 3, 2013, at 3:15 PM, Terry Chay tchay@wikimedia.org wrote:
The above can occur independently or in parallel to the above. If Product thinks it cool to commit Platform's constrained engineering resources to deploying and supporting MassMessage Extension forever and use it to take advantage of Echo when the features roll out in some undefined future, consider this thinking moot or at least orthogonal to MassMessage. IMO, it is bad enough that something important like BetaFeatures is without a home, but my answer from Features is "No" for MassMessage. If this was my own engineer, I'd raise hell with them for this and yell at their Product Manager for not being a good steward of Platform's time.
Ori has stepped up and is willing to commit his time to help Legoktm in supporting MassMessage on an ongoing basis. According to the current standard of the above, there is no longer any block on bug 52723.
Time still has to be made to handle the rollout, but I'm assuming what is already in-process in Platform, and Ori can assist beyond that.
The requirement for extensions to be deployed has ALWAYS been that Features Engineering sign off on a commitment to maintain the extension. I've relaxed it to be: "If another engineering department in collaboration with product is willing to commit to it, Features will not block." I'd like to someday relax this further to: "If any engineering department or community development in collaboration with the other core competencies (features, product, design, and community) are willing to commit to it, no one should block." We are not there yet, and trying to rail against that is winning a battle at the cost of the war breaking down these silos and having shared ownership and responsibility in our organization, in our community, and in our movement.
Please realize the current state of affairs is that new Extension deploy can be done under the following frameworks:
1) The traditional "Features signs off on all extensions" that has always been in place. For Features to sign off on this, like with our own engineering staff, we require collaboration and buy-in in affected areas like the product managers and designers who will lose productivity or might be retasked on this. We must be good stewards of each-others times, and I would not have permitted (and will not permit) my engineers to task Platform and Design engineers for MassMessage in the manner that has happened to date (water under the bridge at this point). 2) The above can and has been be bypassed on a directive from the VPE as was the case for BetaFeatures. 3) Like LanguageEngineering's use of TranslationNotifications instead of Echo, if another team consisting of engineering, product, and design is willing to commit to supporting on an ongoing basis, then extension deploy proceeds without the traditional rule of discussing Features. Note, in this example we have an assumption that the feature set of MassMessage is as-is so it currently doesn't involve Product or Design, nor does it require more engineering resources inside the WMF beyond Ori who is currently untasked.
We still eventually want to reach the point where the criteria is not the amalgam of rules above but a simpler one based on intent, expertise-sharing and consensus-building: "If any engineering department or community developer in collaboration with the other core competencies (engineering, product, design, and community) are willing to commit to ongoing maintenance of a feature, then no one group should block it."
We haven't reached there yet, and please realize that when me make exceptions as in this case, it compromises the larger picture of our need to break down these silos and create shared ownership and responsibility in the WMF, in our communities, and in the movement as a whole.
Take care,
terry
terry chay 최태리 Director of Features Engineering Wikimedia Foundation “Imagine a world in which every single human being can freely share in the sum of all knowledge. That's our commitment.”
p: +1 (415) 839-6885 x6832 m: +1 (408) 480-8902 e: tchay@wikimedia.org i: http://terrychay.com/ w: http://meta.wikimedia.org/wiki/User:Tychay aim: terrychay
On Thu, Oct 3, 2013 at 6:50 PM, Terry Chay tchay@wikimedia.org wrote:
The requirement for extensions to be deployed has ALWAYS been that Features Engineering sign off on a commitment to maintain the extension. I've relaxed it to be: "If another engineering department in collaboration with product is willing to commit to it, Features will not block." I'd like to someday relax this further to: "If any engineering department or community development in collaboration with the other core competencies (features, product, design, and community) are willing to commit to it, no one should block." We are not there yet, and trying to rail against that is winning a battle at the cost of the war breaking down these silos and having shared ownership and responsibility in our organization, in our community, and in our movement.
I don't think this assertion is true, nor is the future it describes. We've been deploying extensions for a long time since before Features Engineering existed. Heck, I can think of a half dozen that we've deployed in the last year or so that Features didn't have anything to do with at all :)
As long as someone's willing to maintain it, someone's willing to deploy it, it satisfies a technical need and/or has community consensus and most importantly has passed technical review then I'm always in favor of deploying extensions that people want to get live. It's incredibly encouraging especially to our volunteer devs to see the fruits of their labors actually being used :)
-Chad
On Fri, Oct 4, 2013 at 9:07 AM, Chad innocentkiller@gmail.com wrote:
[snip] It's incredibly encouraging especially to our volunteer devs to see the fruits of their labors actually being used :)
Indeed - and Extension:ShortUrl which I wrote as a volunteer getting deployed made me immensely happy. Also made the folks at the (often neglected?) Indian language communities where it was deployed happy. Let's not bottleneck 'what we can deploy' with 'how many people we are paying'.
I support strongly Chad comment.
The motivation of most of volunteers like me is to see their code deployed. There are a lot of extensions like ProofreadPage that have been written and are maintained without nearly any help from the WMF Feature department. If you assert something like "everything should be controlled by WMF Feature department", requesting volunteer work has, I think, no more any sense and it's a sentence of death for project like Wikisource that relies strongly on community written and maintained extensions.
Sorry for these hard words but it's, I think, the success in the long term of some projects like Wikisource that is at stake.
Thomas (User:Tpt)
Le 4 oct. 2013 à 07:45, Yuvi Panda yuvipanda@gmail.com a écrit :
On Fri, Oct 4, 2013 at 9:07 AM, Chad innocentkiller@gmail.com wrote:
[snip] It's incredibly encouraging especially to our volunteer devs to see the fruits of their labors actually being used :)
Indeed - and Extension:ShortUrl which I wrote as a volunteer getting deployed made me immensely happy. Also made the folks at the (often neglected?) Indian language communities where it was deployed happy. Let's not bottleneck 'what we can deploy' with 'how many people we are paying'. --
So I asked Erik offline for some clarification on this thread, since a number of people seem worried. To summarize:
* there is no existing or planned rule that WMF Features dept must sign off on all extensions prior to deployment * there is no existing or planned rule that WMF Features dept must maintain all extensions prior to deployment * there is no existing or planned rule that WMF Features dept can choose to waive these requirements, because they don't exist in the first place
Existing and continued-to-be-planned policy is that someone, somewhere has to maintain stuff to be deployed, but there's no specific rule on who.
-- brion
On Thu, Oct 3, 2013 at 6:50 PM, Terry Chay tchay@wikimedia.org wrote:
On Oct 3, 2013, at 3:15 PM, Terry Chay tchay@wikimedia.org wrote:
The above can occur independently or in parallel to the above. If
Product thinks it cool to commit Platform's constrained engineering resources to deploying and supporting MassMessage Extension forever and use it to take advantage of Echo when the features roll out in some undefined future, consider this thinking moot or at least orthogonal to MassMessage. IMO, it is bad enough that something important like BetaFeatures is without a home, but my answer from Features is "No" for MassMessage. If this was my own engineer, I'd raise hell with them for this and yell at their Product Manager for not being a good steward of Platform's time.
Ori has stepped up and is willing to commit his time to help Legoktm in supporting MassMessage on an ongoing basis. According to the current standard of the above, there is no longer any block on bug 52723.
Time still has to be made to handle the rollout, but I'm assuming what is already in-process in Platform, and Ori can assist beyond that.
The requirement for extensions to be deployed has ALWAYS been that Features Engineering sign off on a commitment to maintain the extension. I've relaxed it to be: "If another engineering department in collaboration with product is willing to commit to it, Features will not block." I'd like to someday relax this further to: "If any engineering department or community development in collaboration with the other core competencies (features, product, design, and community) are willing to commit to it, no one should block." We are not there yet, and trying to rail against that is winning a battle at the cost of the war breaking down these silos and having shared ownership and responsibility in our organization, in our community, and in our movement.
Please realize the current state of affairs is that new Extension deploy can be done under the following frameworks:
- The traditional "Features signs off on all extensions" that has always
been in place. For Features to sign off on this, like with our own engineering staff, we require collaboration and buy-in in affected areas like the product managers and designers who will lose productivity or might be retasked on this. We must be good stewards of each-others times, and I would not have permitted (and will not permit) my engineers to task Platform and Design engineers for MassMessage in the manner that has happened to date (water under the bridge at this point). 2) The above can and has been be bypassed on a directive from the VPE as was the case for BetaFeatures. 3) Like LanguageEngineering's use of TranslationNotifications instead of Echo, if another team consisting of engineering, product, and design is willing to commit to supporting on an ongoing basis, then extension deploy proceeds without the traditional rule of discussing Features. Note, in this example we have an assumption that the feature set of MassMessage is as-is so it currently doesn't involve Product or Design, nor does it require more engineering resources inside the WMF beyond Ori who is currently untasked.
We still eventually want to reach the point where the criteria is not the amalgam of rules above but a simpler one based on intent, expertise-sharing and consensus-building: "If any engineering department or community developer in collaboration with the other core competencies (engineering, product, design, and community) are willing to commit to ongoing maintenance of a feature, then no one group should block it."
We haven't reached there yet, and please realize that when me make exceptions as in this case, it compromises the larger picture of our need to break down these silos and create shared ownership and responsibility in the WMF, in our communities, and in the movement as a whole.
Take care,
terry
terry chay 최태리 Director of Features Engineering Wikimedia Foundation “Imagine a world in which every single human being can freely share in the sum of all knowledge. That's our commitment.”
p: +1 (415) 839-6885 x6832 m: +1 (408) 480-8902 e: tchay@wikimedia.org i: http://terrychay.com/ w: http://meta.wikimedia.org/wiki/User:Tychay aim: terrychay
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Thu, Oct 3, 2013 at 6:50 PM, Terry Chay tchay@wikimedia.org wrote:
We still eventually want to reach the point where the criteria is not the amalgam of rules above but a simpler one based on intent, expertise-sharing and consensus-building: "If any engineering department or community developer in collaboration with the other core competencies (engineering, product, design, and community) are willing to commit to ongoing maintenance of a feature, then no one group should block it."
I'd actually say that this paragraph more accurately describes both historical and current practice than your preceding enumeration, except that certain competencies (product/design) have historically been under-represented in the dev process and should be more consistently looped in now that we have more folks who wear those hats (both within WMF and beyond).
Erik
Terry Chay wrote:
[a very long e-mail about the MassMessage extension]
Hi Terry.
Have you ever used or tested the MassMessage MediaWiki extension? If so, when and where specifically?
MZMcBride
wikitech-l@lists.wikimedia.org