On Fri, Mar 8, 2019 at 9:04 AM Amir Sarabadani ladsgroup@gmail.com wrote:

" Everything is open-source and as non-profit, there's always resource constraint." So if only wikiworld were for profit, then it wouldn't be resource-constrained? Therein could lie the solution of all problems, not just those of wikiworld.

Pe vineri, 8 martie 2019, Amir Sarabadani ladsgroup@gmail.com a scris:

Hey, I'm not WMF so I'm not the best one to answer the question but I think your statement is overgeneralizing. Some teams have more resource constraints than the other ones and treating all of WMF as a big monolith doesn't seem to be a good approach. I think you should be more precise and give a more clear statement on what do you think is wrong.

Several things: * the bug backlog has been steadily increasing in all phabricator reports I have seen (I don't read them all, so some decreases might have occurred occasionally, but the trend is there) * feature development is prioritized over bug fixes (read: once a feature goes into maintenance, good luck getting a fix without bribing someone) * after Andre stopped being bug wrangler, nobody else took the job of clarifying user requests, closing obvious duplicates etc.

I'm sure there are legitimate reasons for these problems, the question is what can be done to improve the situation?

Two other things to note: 1- As a developer who loves to fix bugs, the reason I can't sometimes fix a bug is that it's not clearly defined, and/or there's no proper instruction to reproduce. Don't always blame the other party.

This is bound to happen when 99% of your users are non-technical. It would be great if you (globally) could take some time to ask for clarifications when you feel they are required.

2- Everything is open-source and as non-profit, there's always resource

constraint. If it's really important to you, feel free to make a patch and the team would be always more than happy to review.

No, not always. There are over 4600 open reviews, some 5 years old. There are some reasons why one would want to keep a review opened, but very few IMHO. You also need to consider the fact that open reviews might discourage people from proposing new fixes. And again, 99% of MW users are non-technical.

All the best, Strainu

Best

On Fri, Mar 8, 2019 at 1:31 PM John Erling Blad jeblad@gmail.com wrote:

...

The backlog for bugs are pretty large (that is an understatement), even for bugs with know fixes and available patches. Is there any real plan to start fixing them? Shall I keep telling the community the bugs are "tracked"?

/jeblad

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

-- Amir _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Fri, 8 Mar 2019 at 18:04, Strainu strainu10@gmail.com wrote:

Several things:

• the bug backlog has been steadily increasing in all phabricator reports I

have seen (I don't read them all, so some decreases might have occurred occasionally, but the trend is there)

• feature development is prioritized over bug fixes (read: once a feature

goes into maintenance, good luck getting a fix without bribing someone)

• [snip]

I'm sure there are legitimate reasons for these problems, the question is what can be done to improve the situation

Also should be on the list: Sometimes bugs have a known fix that isn't being rolled out, in favour of a larger more fundamental restructuring (demanding even more resources).

2- Everything is open-source and as non-profit, there's always resource constraint. If it's really important to you, feel free to make a patch and the team would be always more than happy to review.

Wikipedia is the core product, and the users are the primary customers. When a group of core customers request a change, then the service provider should respond. Whether the service provider is a non-profit doesn't really matter, the business model is not part of the functional requirement. The service provider should simply make sure the processes function properly.

If the service provider has resource constraints, then it must scale the services until it gets a reasonable balance, but that does not seem to be the case here. It is more like there are no process or the process is defunc.

The strange thing is; for many projects the primary customers aren't even part of a stakeholder group, the devs in the groups defines themselves as the "product user group". That tend to skew development from bugs to features. Perhaps that is what happen in general here, too much techies that believe they are the primary customers.

Without the editors there would be no content, and thus no readers, and without readers there would be no use for the software provided. So is the actual users subsidizing the software? Definitely yes! The content is the primary reason why we have readers. The software is just a tool to provide the content in an accessible form to the readers.

Whether an editor is a customer by subsidizing the product directly or indirectly is not much of a concern, as long as there will be no subsidizing at all, from any party – ever, without the content.

The primary customer of the software is the editors, but the primary customer of the content is the readers.

On Tue, Mar 12, 2019 at 2:18 AM David Barratt dbarratt@wikimedia.org wrote:

A customer, by definition (https://en.wikipedia.org/wiki/Customer) exchanges something of value (money) for a product or service.

That does not mean that a freemium model ( https://en.wikipedia.org/wiki/Freemium) is not a valid business model. However, if there is no exchange of value, the person consuming the free version of the product or service, is not (yet) a customer.

If MediaWiki is the thing we give away for free, what do we charge money for? Are our customers successfully subsidizing our free (as in beer) software?

On Mon, Mar 11, 2019 at 7:33 PM John Erling Blad jeblad@gmail.com wrote:

...

...

2- Everything is open-source and as non-profit, there's always resource constraint. If it's really important to you, feel free to make a patch

and

...

the team would be always more than happy to review.

Wikipedia is the core product, and the users are the primary customers. When a group of core customers request a change, then the service provider should respond. Whether the service provider is a non-profit doesn't really matter, the business model is not part of the functional requirement. The service provider should simply make sure the processes function properly.

If the service provider has resource constraints, then it must scale the services until it gets a reasonable balance, but that does not seem to be the case here. It is more like there are no process or the process is defunc.

The strange thing is; for many projects the primary customers aren't even part of a stakeholder group, the devs in the groups defines themselves as the "product user group". That tend to skew development from bugs to features. Perhaps that is what happen in general here, too much techies that believe they are the primary customers.

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

I'm going to make a few points that I think will respond to some comments that I've read, and I will try to organize some of my previous points so that they're easier to follow.

1. My impression is that there's agreement that there is a huge backlog.

2. I think that there's consensus that the backlog is a problem.

3. I think that we're collectively unsure how to address the backlog, or how to analyze the backlog so that everyone has shared situational awareness, but we collectively seem to agree that the backlog should be addressed.

If any of the above is wrong, please feel free to say so.

Regarding my own opinions only, I personally am frustrated regarding multiple issues:

a. that there's been discussion for years about technical debt,

b. that WMF's payroll continues to grow, and while I think that more features are getting developed, the backlog seems to be continuing to grow,

c. that WMF, which has the largest budget of any Wikimedia entity, is not more transparent regarding how it spends money and what is obtained from that spending;

d. that although I think that the Community Liaisons help a lot with communications, there remains much room for improvement of communications. One of my larger frustrations is that the improvements regarding communications have not been more extensive throughout all of WMF.

e. that WMF retains the ability to veto community decisions regarding decisions such as deployments of features, but the community has little ability to veto WMF decisions. I think that staff as individuals and collectively have become more willing to negotiate and yield ground in the past few years, which is good, but I remain concerned that these are informal rather than formal changes.

f. that I think that some of what WMF does is good and I want to support those activities, but there are other actions and inactions of WMF that I don't understand or with which I disagree. Conflicts can be time consuming and frustrating for me personally, and my guess is that others might feel the same, including some people in WMF. I don't know how to solve this. I realize that some people might say "Then you should leave", and I regularly consider that option, but Wikipedia and the sister projects do a lot of good, so I'm torn. This is very much my personal issue and I don't expect to discuss it more on Wikitech-l, but it's a factor in how I think about this email thread, which is why I mention it.

I hope that this email provides some clarity and is useful for this discussion.

Pine ( https://meta.wikimedia.org/wiki/User:Pine )

On Tue, 2019-03-12 at 20:34 +0000, Pine W wrote:

1. My impression is that there's agreement that there is a huge backlog.

Phabricator is public. Anyone can propose and report anything. Hence the number of ideas, bugs, feature requests is usually higher than the number of available developers (paid or not) needed to work on them. Hence the number of tasks which will remain unresolved grows. Because in theory someone could always show up and provide a patch.

If you know a larger free and open source software project where the number of resolved (not: declined) tickets per month/year/etc is higher than the number of open(ed) tickets, I'd be curious to know.

2. I think that there's consensus that the backlog is a problem.

No, why?

There are likely quite some ideas that don't make much sense to prioritize and fix (out of project scope, time consuming because of required huge architecture changes, increased test complexity and maintenance costs after adding yet another preference, etc etc).

And many ideas and bugs that will not get fixed (limited number of available developers, different individual and group priorities) until you (or someone else) writes code if you're really interested in seeing that fixed. (If that idea is considered 'in scope' - see above.)

And disappointment *if* someone makes a decision to decline a request. And followup discussion to challenge someone's decision which takes time that could have been spent to work on tasks that someone considers more important. In practice, people and time are limited resources.

andre

I get an impression from this thread that the problem is not really the size of the backlog, but rather certain individual tasks that sit in said backlog rather than being worked on, and which according to John are actually major issues.

It's hard to disagree or agree with this without knowing which tasks it is actually about though.

Hi!

1. My impression is that there's agreement that there is a huge backlog.

Obviously, there is a backlog. As for it being "huge", it's subjective, for someone who has experience with long-running projects, having thousands of issues in the bug tracker is nothing out of the ordinary. Does it make the backlog "huge"? I don't know, depends of what is "huge".

2. I think that there's consensus that the backlog is a problem.

I'm not sure where such a consensus came from. Of course, bugs not being resolved immediately is not the ideal situation - ideally, the bug would be fixed within hours of submitting it. Nobody thinks it can really happen. Any large popular long-running project has more bugs and wishlist items than it can implement. There are always more users than developers and more ideas than time. Thus, the backlog. Of course, ignoring the backlog completely would be the problem, but I don't think we have this situation. It's likely we could do better. But I think we know the backlog exists, and its existence by itself is not a problem, or at least not a problem that can be realistically solved for such a huge project.

Regarding my own opinions only, I personally am frustrated regarding multiple issues:

a. that there's been discussion for years about technical debt,

I'm not sure why it's the source of frustration for you. Having discussion about technical debt is great. Of course, it should also lead to fixing the said debt - which I think is happening. But expecting that starting some magic date we stop having technical debt or the need to address it as realistic as deciding starting today our code won't have bugs anymore.

b. that WMF's payroll continues to grow, and while I think that more features are getting developed, the backlog seems to be continuing to grow,

Of course. How it could be any other way? With more features, come more places that can have bugs (you don't expect WMF to be the only software developing organization in the Multiverse that writes code without bugs?) and once people start using them, they inevitably ask for improvements and have ideas on how to extend it, thus adding more tasks. Expecting that more functionality would lead to less issues in the bug tracker is contrary to what I have experienced over my whole career - it never happened, unless the project is effectively dead and the users have moved away.

f. that I think that some of what WMF does is good and I want to support those activities, but there are other actions and inactions of WMF that I don't understand or with which I disagree. Conflicts can be time consuming> and frustrating for me personally, and my guess is that others might feel the same, including some people in WMF. I don't know how to solve this. I

I don't think it's possible to "solve" this. Unless you are appointed the Supreme Dictator of WMF, there always would be things that WMF does and you disagree. And so would be the case for every other person who cares about what WMF does. We just need to find things that we can do that a lot of people can use and not too many people disagree, but there's no way to guarantee you won't disagree with anything (for any value of "you"). I think we already have processes for finding this kind of kinda-consensus-even-though-not-completely. As all processes, they are not ideal. They can be improved with specific suggestions. But expecting that nobody (and any specific person in particular) would ever think "WMF is totally wrong in doing this!" is not realistic. Reasonable people can and do disagree. Reasonable people also can work through disagreements and find common interests and ways to contribute to mutual benefit. I think that's what we're trying to do here.

On Wed, 2019-03-13 at 01:22 +0100, John Erling Blad wrote:

What frustrates me the most are

• bugs found by the editor community, that has obvious simple fixes,

which isn't acted upon for several years

• new features that isn't fully tested, and you have to answer in the

community about stuff you rather want to throw out

• new features and changes that are advertised but never implemented

Could you please provide a specific example (link) for the last item?

The first one is perhaps the one most easily fixed. I believe WMF could set up either an official bug squad

In my understanding a bug squad does not write patches but triages tickets. Maybe you meant a patchsquad or Gerrit wrangler (in case you refer to *existing* simple fixes, which is not clear from your words). Not sure why there is call to some authority ("WMF") here.

or use bug bounties to speed up fixing of bugs. I tend to believe bug bounties works best, but it would be really nice to know that bugs are handled in an orderly fashion by a bug squad.

See https://phabricator.wikimedia.org/T88265#1870218 for risks and (dis)advantages of bug bounties. Note that throwing more 'external' developers onto code does not necessarily "speed up fixing of bugs". Often to the contrary.

When introducing new features make a help page at Meta or Mediawiki, and link to the page from the feature.

Looking at the beta features section at https://meta.wikimedia.org/wiki/Special:Preferences#mw-prefsection-betafeatu... all beta features have "Discussion" and "Information" links. What you are suggesting already happens.

On that page make a visible link "Don't panic!" and link to the issue tracker. Don't expect the users to figure out which extension provides the specific feature, they don't have a clue. For all important issues make an estimated fix time, and if no one works on the issue say so.

When nobody works on an issue, Phabricator's "Assigned To" field usually shows "None". What you are suggesting already happens.

Cookie licking can happen though (means: someone assigns a ticket to themselves and then does not work on it). It's up to each assignee to occasionally check which tasks are (still) assigned to them: https://phabricator.wikimedia.org/maniphest/query/5INV_avtCreJ/#R

Don't assume the users understand fancy wording about "need volunteer". Need volunteer for what? Making coffee?

Some features are described in Phabricator, which is fine, but some of them has extensive cookie licking which could give someone the impression that you actually will implement the feature.

Could you please provide a specific example (link) for this?

That often leads to users asking about the feature, and when it will arrive at their project. When it does not arrive users gets upset. If you are working on something, say it, but also be very clear if something has gone into you personal freezer.

Cheers, andre

Pe vineri, 8 martie 2019, bawolff bawolff+wn@gmail.com a scris:

"tracked" does not mean someone is planning to work on it. This could be for a lot of reasons, maybe the bug is unclear, maybe its not obvious what a good way to fix is, maybe nobody cares (This sounds harsh, but the simple truth is, different things have different people caring about them, and some parts just don't have anyone).

This is not really a paid vs unpaid thing. Volunteer projects have a big backlog of bugs. Commercial projects also have a backlog or things they just don't intend to fix (although usually big commercial projects keep the list of bug reports secret).

How many successful commercial projects leave customer issues unresolved for years because they're working on something else now? I can name a few which used to do that because they were monopolies, but even those improved eventually, pressured by the market. There are companies that require weekly reports of progress on customer issues, others that don't release until all bugs are closed one way or another etc.

The discussion at https://lists.gt.net/wiki/wikitech/889489 is relevant, I believe. The request there was to not decline low-priority issues that might be resolved by volunteers and this clearly increases the number of open bugs (as I said, there are good reasons for that :) ). There were a number of proposals on how to track such issues so that reporters have a clear image of the status of the bugs. Have any of them been tried by at least one of the teams at wmf? If so, is there a way to share the results with other teams? If not, how can we convince the wmf to give them a chance?

Strainu

I really think its no different from Wikipedia. https://en.wikipedia.org/wiki/Wikipedia:Backlog isn't getting any smaller. That's just the natural way of things. Its a bit easier to yell {{sofixit}} on wiki than it is to yell it about technical tasks, as technical stuff by their very nature require specialized knowledge (Although i would argue that lots of tasks on wiki also require specialized knowledge). At the end of the day, to get a task fixed, someone who knows how to do it (Or is willing to learn how to do it) needs to be interested in doing it.

-- Brian

On Fri, Mar 8, 2019 at 12:31 PM John Erling Blad jeblad@gmail.com wrote:

...

The backlog for bugs are pretty large (that is an understatement), even for bugs with know fixes and available patches. Is there any real plan to start fixing them? Shall I keep telling the community the bugs are "tracked"?

/jeblad

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

This discussion touches on a number of my frustrations. If someone is in a bad mood then they might want to postpone reading my comments below.

As far as I know, WMF wants to advertise itself as being a provider of infrastructure for fundraising purposes, and wants to exercise absolute power over technical matters (see, for example, Superprotect). I think that it would be difficult to reconcile these factors with an attempt by WMF to disclaim responsibility for deficiencies in the technical infrastructure that WMF created and remains in use on Wikimedia sites. (I think that this does not generally extend to bots, tools, etc. that were not primarily created by WMF.)

Regarding "There's an anti-pattern here: we all have a big mailing list discussion, agree there's a problem, agree that the Foundation should solve the problem, then ask again in a year what they did even though they didn't actually say they'd do anything about it. That's not a healthy dynamic.": if WMF doesn't say that it will fix a widely known problem, that is not necessarily an excuse for not fixing it by a year later, and may also indicate a failure by WMF to clearly communicate what it *won't* fix.

It's true that for profit companies can sometimes ignore important bugs and have poor customer service, but when a provider is not a monopoly then customers who are unhappy can (with varying amounts of expense and pain) change providers.

Other organizations being sloppy does not imply that WMF should follow their bad example or make excuses that WMF is probably no worse than others.

I don't agree that the technical space is owned by all of us. WMF never apologized for Superprotect, and there is nothing to stop WMF from making arbitrary decisions against community consensus other than the threats of (1) community members quitting in substantial numbers and (2) bad press coverage. Also, WMF's technical services are one of the primary justifications for WMF's budget.

I believe that WMF does a lot of good for the world, but it also has a lot of room for improvement, and hearing excuses (or getting no substantive responses) regarding the same problems year after year gets old, particularly as WMF's payroll continues to grow.

These subjects are frustrating and depressing for me, so let me close on a more positive note. I am glad that we have these discussions in public, and the strategy process may be a way to make progress. Also, I think that WMF has improved the infrastructure over the years. For example, I like the New Wikitext Editor, and I think that VisualEditor is now a good option for many use cases. Citoid is wonderful. Wikimedia sites generally seem to be more performant than in years past. The Search Platform team seems to do a lot of good work. I like the new edit filters in the watchlist.

So, much good has been done, and there remains much to improve.

Pine ( https://meta.wikimedia.org/wiki/User:Pine )

On Sat, Mar 9, 2019, 4:13 AM Dan Garry (Deskana) djgwiki@gmail.com wrote:

On Sat, 9 Mar 2019 at 11:26, Strainu strainu10@gmail.com wrote:

...

How many successful commercial projects leave customer issues unresolved for years because they're working on something else now?

Almost all of them, they just keep it secret. Companies pay millions of dollars each year for support packages, even after having paid for software in the first place, specifically because otherwise their support issues may not be answered in a timely fashion, or even answered at all. I don't think comparing us to commercial products makes much sense in this context.

...

There were a number of proposals on how to track such issues so that reporters have a clear image of the status of the bugs. Have any of them been tried by at least one of the teams at wmf? If so, is there a way to share the results with other teams? If not, how can we convince the wmf to give them a chance?

I don't agree with shifting responsibility onto the Wikimedia Foundation. There's an anti-pattern here: we all have a big mailing list discussion, agree there's a problem, agree that the Foundation should solve the problem, then ask again in a year what they did even though they didn't actually say they'd do anything about it. That's not a healthy dynamic.

The technical space is owned by all of us, so if we, as a technical community, decide this is important to us, then we can look at the problem and try to tackle it, and then figure out how the Wikimedia Foundation could catalyse that.

Dan _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Strainu,

I, too, am glad for the discussion!

On Sat, 9 Mar 2019 at 22:31, Strainu strainu10@gmail.com wrote:

Let me start with a simple question, to put the references to wmf into context. You keep talking below about volunteer developers and how they can take over any project.

I'm confused by this. I didn't mention volunteer teams taking over projects at all, and I don't think that'd work except in very rare and limited circumstances. I was talking about people helping with bug triage on Phabricator.

While that's true, how many fully-volunteer teams are there? How does that number compare to the number of wmf teams? Am I right to assume the ratio is hugely in favor of wmf teams? Note: teams, not developers, since decisions on project management are usually done at team level.

See above; this wasn't what I meant.

In my experience in b2b contracts they don't keep it a secret, they usually have SLAs they respect, but ok, let's leave it at that.

Yes, I have more to say about this, but this would be tangential to this discussion. :-)

Responsibility for what? Developing and hosting MediaWiki? Helping communities concentrate on creating and attracting content without having to work around bugs? I'm sorry, but that's precisely one of the responsibilities of the wmf and this is what's discussed here.

Well, in your earlier emails in this thread, you mentioned the bug backlog steadily increasing, so that was what I was talking about. Is that not what you were talking about in your subsequent emails?

This is one thing that we agree on: nobody committed on anything. Ever. That's why I asked above: what does it take to have someone (anyone) at the WMF act upon these discussions?

My role in the Wikimedia tech community is tech ambassador above all else, so I'm caught in the middle here: I have to explain new features and technical decisions to people who don't care about php, js or server performance , but I also feel obligated to relay their requirements, as I see them, to the development team. This second process does not happen as smoothly as it should.

It's not healthy to ignore discussion after discussion and claim it's a community issue. It's not. It's a governance issue and it's growing every day.

I agree. It's not a community issue, it's a movement-wide one. I don't know how to solve it.

The projects belong to the community at large, not just the technical subcommunity. They are the ones affected by the bugs and also they are the ones that need our support. So why should they be ignored in taking this decision?

I'm confused by this too. I wasn't talking about ownership of the Wikimedia projects, I was again talking about the bug backlog, which anyone is welcome to get involved in simply by registering an account.

My proposal is to begin the discussion here: how can we better relay issues that are more important to communities than new features? How can we have a "community whishlist for bugs"?

The community wishlist explicitly accepts requests to fix bugs, as well requests for new features. So, is what you're asking for some process to supplement that?

Dan

Also, the Tech team at the Foundation is investing in Technical Engagement team who I hope will be (amongst other things) become advocates for the tech debt that affects our communities.

Best regards,

Victoria

Sent from my iPhone

On Mar 9, 2019, at 6:28 PM, bawolff bawolff+wn@gmail.com wrote:

Regarding:

...

My proposal is to begin the discussion here: how can we better relay issues that are more important to communities than new features? How can we have a "community whishlist for bugs"?

Well fundamentally it starts with making a list.

This is basically a lobbying discussion right. People think WMF should do more of X. Lobbying discussions are more successful the more specific they are. Having a list of the top 20 worse bugs is something you could convince people to do something about. Even something like /WMF spends too much time on new features and not enough time on maintenance/bug fixing/, is something you could convince people to change, if you for example knew how much time WMF currently spends on bug fixing, and you have an idea of how much time you think they should be spending. Even if management doesn't agree with your proposal, it would at least be specific enough to debate.

When these discussions start from vague places, like there's too many bugs, is when they go nowhere. Even if WMF stopped everything else it was doing, and worked solely on bugs, I doubt they would fix every bug in existence. (We can't all be TeX!), and attempting to do that would be a bad idea.

Change happens when stuff is measurable, and people can work towards a goal. Failing that, change happens when people can be held accountable. Objective measures are needed.

-- Brian

...

On Sat, Mar 9, 2019 at 10:31 PM Strainu strainu10@gmail.com wrote:

Dan,

Thank you for your response. I appreciate far more someone disagreeing with me than someone ignoring me :)

Let me start with a simple question, to put the references to wmf into context. You keep talking below about volunteer developers and how they can take over any project. While that's true, how many fully-volunteer teams are there? How does that number compare to the number of wmf teams? Am I right to assume the ratio is hugely in favor of wmf teams? Note: teams, not developers, since decisions on project management are usually done at team level.

Pe sâmbătă, 9 martie 2019, Dan Garry (Deskana) djgwiki@gmail.com a scris:

...

...

On Sat, 9 Mar 2019 at 11:26, Strainu strainu10@gmail.com wrote:

How many successful commercial projects leave customer iss

To blame WMF for having a huge backlog is wrong imho. Most development work is done by volunteers, and while I do believe more can be done from both sides but blaming it all on WMF is wrong.

-- Devin “Zppix” CCENT Volunteer Wikimedia Developer Africa Wikimedia Developers Member and Mentor Volunteer Mozilla Support Team Member (SUMO) Quora.com Partner Program Member enwp.org/User:Zppix **Note: I do not work for Wikimedia Foundation, or any of its chapters. I also do not work for Mozilla, or any of its projects. **

On Mar 9, 2019, at 11:40 PM, Victoria Stavridou-Coleman victoria@gocolemans.com wrote:

Re reading this now on the ground in Austin, reminds me not to send emails in a hurry from an airplane! So trying again - hopefully more grammatically sound this time!

The Tech Engagement team (which includes Wikimedia Cloud Services) in the Tech department is investing in a developer advocacy team who I hope will (amongst other things) speak on behalf of the communities that are affected by tech debt.

All the best,

Victoria

...

On Mar 9, 2019, at 6:39 PM, Victoria Coleman victoria@gocolemans.com wrote:

Also, the Tech team at the Foundation is investing in Technical Engagement team who I hope will be (amongst other things) become advocates for the tech debt that affects our communities.

Best regards,

Victoria

Sent from my iPhone

...

On Mar 9, 2019, at 6:28 PM, bawolff bawolff+wn@gmail.com wrote:

Regarding:

...

My proposal is to begin the discussion here: how can we better relay issues that are more important to communities than new features? How can we have a "community whishlist for bugs"?

Well fundamentally it starts with making a list.

This is basically a lobbying discussion right. People think WMF should do more of X. Lobbying discussions are more successful the more specific they are. Having a list of the top 20 worse bugs is something you could convince people to do something about. Even something like /WMF spends too much time on new features and not enough time on maintenance/bug fixing/, is something you could convince people to change, if you for example knew how much time WMF currently spends on bug fixing, and you have an idea of how much time you think they should be spending. Even if management doesn't agree with your proposal, it would at least be specific enough to debate.

When these discussions start from vague places, like there's too many bugs, is when they go nowhere. Even if WMF stopped everything else it was doing, and worked solely on bugs, I doubt they would fix every bug in existence. (We can't all be TeX!), and attempting to do that would be a bad idea.

Change happens when stuff is measurable, and people can work towards a goal. Failing that, change happens when people can be held accountable. Objective measures are needed.

-- Brian

...

On Sat, Mar 9, 2019 at 10:31 PM Strainu strainu10@gmail.com wrote:

Dan,

Thank you for your response. I appreciate far more someone disagreeing with me than someone ignoring me :)

Let me start with a simple question, to put the references to wmf into context. You keep talking below about volunteer developers and how they can take over any project. While that's true, how many fully-volunteer teams are there? How does that number compare to the number of wmf teams? Am I right to assume the ratio is hugely in favor of wmf teams? Note: teams, not developers, since decisions on project management are usually done at team level.

Pe sâmbătă, 9 martie 2019, Dan Garry (Deskana) djgwiki@gmail.com a scris:

...

...

On Sat, 9 Mar 2019 at 11:26, Strainu strainu10@gmail.com wrote:

How many successful commercial projects leave customer iss

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Wow, this thread has become quite huge (which is a good problem to have!). Trying to repond to a few messages in a single email.

În dum., 10 mar. 2019 la 01:23, Dan Garry (Deskana) djgwiki@gmail.com a scris:

I'm confused by this. I didn't mention volunteer teams taking over projects at all, and I don't think that'd work except in very rare and limited circumstances. I was talking about people helping with bug triage on Phabricator.

Got it. Glad we agree that project maintenance is the responsibility of the WMF. :) Bug triage (and even solving) is a great way of implicating volunteer developers, the problem is what do you do with bugs on projects where there are no volunteers. My take is that the paid teams that have ownership of those products should offer a certain degree of maintenance at least to widely deployed extensions (i.e. on all versions of a project). The (negative) example I like to give is Content Translation, were all bugfixes on v1 were stopped until v2 was developed, process that was delayed by at least a year if not 2 compared to initial estimates. During this time tech ambassadors were simply left with the task of explaining to users what they can do to save their work (hint: nothing). That just shouldn't happen.

...

The projects belong to the community at large, not just the technical subcommunity. They are the ones affected by the bugs and also they are the ones that need our support. So why should they be ignored in taking this decision?

I'm confused by this too. I wasn't talking about ownership of the Wikimedia projects, I was again talking about the bug backlog, which anyone is welcome to get involved in simply by registering an account.

Me too. Prioritization should involve (as much as possible, and certainly more than now) people reporting bugs, regardless of their ability to provide a patch for the bug they file. If we can go further into the communities, even better.

...

My proposal is to begin the discussion here: how can we better relay issues that areSee above - "anyone is welcome" is not enough. more important to communities than new features? How can we have a "community whishlist for bugs"?

The community wishlist explicitly accepts requests to fix bugs, as well requests for new features. So, is what you're asking for some process to supplement that?

A totally new process is not needed nor desirable. Improvements could be easily done that would improve the overall results.

The main problem I see with the community wishlist is that it's a process beside the normal process, not part of it. The dedicated team takes 10 bugs and other developers another ~10. I think we would be much better off if each team at the WMF would also take the top ranked bug on their turf and solve it and bump the priority of all the other bugs by one (e.g. low->medium). One bug per year per team means at least 18 bugs (at least if [1] is up to date) or something similar.

As a side note, even this process has degraded over time: check out how empty is the list for 2017 [2] compared to 2016. [3]

[1] https://meta.wikimedia.org/wiki/Template:Wikimedia_Foundation_departments [2] https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2017/Results [3] https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2016/Results

În dum., 10 mar. 2019 la 07:42, Victoria Stavridou-Coleman victoria@gocolemans.com a scris:

Re reading this now on the ground in Austin, reminds me not to send emails in a hurry from an airplane! So trying again - hopefully more grammatically sound this time!

The Tech Engagement team (which includes Wikimedia Cloud Services) in the Tech department is investing in a developer advocacy team who I hope will (amongst other things) speak on behalf of the communities that are affected by tech debt.

That should help, as long as they speak within the normal development process and not in parallel. Looking forward to seeing the official announcement.

În dum., 10 mar. 2019 la 02:28, bawolff bawolff+wn@gmail.com a scris:

Regarding:

...

My proposal is to begin the discussion here: how can we better relay issues that are more important to communities than new features? How can we have a "community whishlist for bugs"?

Well fundamentally it starts with making a list.

Change happens when stuff is measurable, and people can work towards a goal. Failing that, change happens when people can be held accountable. Objective measures are needed.

What a wonderful world that would be! Unfortunately, all too often I feel that objective measures (such as "+1" on bugs, duplicates etc.) have no effect on prioritization.

if you for example knew how much time WMF currently spends on bug fixing, and you have an idea of how much time you think they should be spending. Even if management doesn't agree with your proposal, it would at least be specific enough to debate.

I have no way of finding that, do I? If there is one, I would be very curious to learn of it.

În lun., 11 mar. 2019 la 16:58, Bartosz Dziewoński matma.rex@gmail.com a scris:

In my experience WMF teams usually have a way to distinguish "bugs we're going to work on soon" and "bugs we're not planning to work on, but we'd accept patches". This is usually public in Phabricator, but not really documented.

Yes, and also not uniform between teams and prone to change on each team reorg. Having a list would would be great, but I'm not sure how maintainable it is. Would you like to start one? ;)

În mar., 12 mar. 2019 la 01:29, John Erling Blad jeblad@gmail.com a scris:

It seems like some projects simply put everything coming from external sources into deep freezer or add "need volunteer". If they respond at all. In some cases it could be that the projects are defunc.

No deployed project should be considered defunct. But I agree with others - naming them would help. Let me start: - ContentTranslation v1 (obsolete now, has been unmaintained for 2 years while in production) - UploadWizard (2 with high priority, 40 with normal, a few dozens low, hundreds more untriaged): this is the project that got us out of the "overloading the lang parameter for customizing the uploader" era, the project that is used by millions of people every year, including during every photo contest

I encourage you all to add more examples.

Thanks to all who chimed in on the subject. Strainu

Hopefully I can say this in a way that is mutually acceptable.

For me this discussion is difficult but enlightening. I think that this is the most difficult Wikimedia discussion in which I have been a participant so far this year. I don't know what agreements will emerge from this thread, if any. But the enlightenment is useful, and I hope that other people also feel that they are learning something new or are being heard on issues that are important to them.

I am thinking that ideally we would all be in a room together to have this discussion. Perhaps a second best choice would be an online meeting regarding the subject of technical debt. We could schedule a meeting via Doodle. It's okay if people don't want to participate, but I'd like to suggest an online meeting as an option.

Pine ( https://meta.wikimedia.org/wiki/User:Pine )

On Wed, Mar 13, 2019 at 3:02 PM Strainu strainu10@gmail.com wrote:

The main problem I see with the community wishlist is that it's a process beside the normal process, not part of it. The dedicated team takes 10 bugs and other developers another ~10. I think we would be much better off if each team at the WMF would also take the top ranked bug on their turf and solve it and bump the priority of all the other bugs by one (e.g. low->medium). One bug per year per team means at least 18 bugs (at least if [1] is up to date) or something similar.

Community Tech is seven people and they do ten wishlist requests a year. (Granted, they do other things too, but the wishlist is their main focus.) So you are proposing to reallocate on average 1-2 months per year for every team to work on wishlist wishes. That's about two million dollars of donor money. How confident are you that the wishlist is actually a good way of estimating the impact of tasks, outside of the narrow field where editors have personal experience (ie. editing tools)?

What a wonderful world that would be! Unfortunately, all too often I

feel that objective measures (such as "+1" on bugs, duplicates etc.) have no effect on prioritization.

Leaving aside how objective those measures are, in my when the task is related to a product owned by a team, they are aware and take it into account. (Which does not necessarily mean they agree, of course.) A lot of production components don't really have an owner, though. (Or only do to the extent that there is someone who can be pulled away from their normal job if the thing catches fire.) That's just the reality of running the website we have with the capacity we have - the alternative would be undeploying things or not starting new projects for a long time. The Wikimedia movement happens to be in the middle of its strategic planning process, so if you want to argue for either of these things, this is a good time to do it. (Not a good place, though.)

- UploadWizard (2 with high priority, 40 with normal, a few dozens

low, hundreds more untriaged): this is the project that got us out of the "overloading the lang parameter for customizing the uploader" era, the project that is used by millions of people every year, including during every photo contest

UploadWizard is not in active development currently. If you want to argue that the Multimedia team should be reassigned to work on it (and drop the Structured Data on Commons project), or some other rearrangement should be made, that's a specific proposal that can be meaningfully discussed. (Probably not here, though - that's a matter of movement strategy, not a technical decision. So maybe better suited to wikimedia-l.) Saying that some project should be picked up, without saying what should be dropped to make space, is an easy thing to do. Not all that useful though.

(As an aside, I'd love to see more people self-organize to get more say in how priorities are decided. If you look at the discussion pages on WMF annual plans, movement strategy and so on, they do not give the impression of a community that's seriously interested in its own future.)

Perhaps a better example would be the Drupal community who has a total of ~1,071,600 issues and ~282,350 of those are open https://www.drupal.org/project/issues and they have several organizations https://www.drupal.org/organizations working on the software.

I do not understand how a large backlog is a problem. It is not an indication of anything.

On Fri, Mar 15, 2019 at 12:25 PM Strainu strainu10@gmail.com wrote:

În joi, 14 mar. 2019 la 22:23, Gergő Tisza gtisza@gmail.com a scris:

...

About backlogs in general, Chromium is probably the biggest open-source Google repo; that has currently 940K tickets, 60K of which

are

...

open, and another 50K have been auto-archived after a year of inactivity. (As others have pointed out, having a huge backlog and ruthlessly closing tasks that do not get on the roadmap are the only two realistic options, and the latter does have its advantages, no one here seems to be in favor of it.) We have 220K tasks in total, 40K of which are open, so that's in the same ballpark

That's an overstatement: 18% (not counting bugs closed as declined) is almost double to 11%. If you're going this route, we're doing much worse than Chromium.

...

On Wed, Mar 13, 2019 at 3:02 PM Strainu strainu10@gmail.com wrote:

...

The main problem I see with the community wishlist is that it's a process beside the normal process, not part of it. The dedicated team takes 10 bugs and other developers another ~10. I think we would be much better off if each team at the WMF would also take the top ranked bug on their turf and solve it and bump the priority of all the other bugs by one (e.g. low->medium). One bug per year per team means at least 18 bugs (at least if [1] is up to date) or something similar.

Community Tech is seven people and they do ten wishlist requests a year. (Granted, they do other things too, but the wishlist is their main

focus.)

...

So you are proposing to reallocate on average 1-2 months per year for

every

...

team to work on wishlist wishes. That's about two million dollars of

donor

...

money. How confident are you that the wishlist is actually a good way of estimating the impact of tasks, outside of the narrow field where editors have personal experience (ie. editing tools)?

I'm 99.9% sure the wishlist is relevant in at least half the categories (Admins&patrollers, Bots&Gadgets, Editing, Notifications, Programs&Events, Watchlists, Wikidata, Wikisource, Wiktionary) and very likely (80%) also for Anti-harassment, Categories and Maps.

I'm not sure how you arrived at the $2M figure (even 36 months of dev time - 18 teams, 2 man-months/team - only add up to ~$400K, unless Glasdoor is waaay off on the salaries there [2]), but presumably going down on the list will also surface bugs and not only features, which will take less time to solve. Investing an additional 1% of the revenue into this seems reasonable to me.

[2] https://www.glassdoor.com/Salary/Wikimedia-Foundation-Salaries-E38331.htm

...

UploadWizard is not in active development currently.

I did not claim (or asked) that it was. What I said is that it is an important part of the infrastructure and that it should be maintained properly. I also said I will try to come up with a more detailed critique later on and see if it has any result.

Strainu

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Sat, 2019-03-16 at 17:22 +0200, Strainu wrote:

That being said, their org stats are pretty awsome, is there any way to obtain similar stats from Phabricator/Gerrit (at least by email domain if nothing else)?

See https://www.mediawiki.org/wiki/Community_metrics

andre

În dum., 17 mar. 2019 la 23:22, Gergo Tisza gtisza@wikimedia.org a scris:

On Sat, Mar 16, 2019 at 8:23 AM Strainu strainu10@gmail.com wrote:

...

A large backlog by itself is not alarming. A growing one for components deployed to WMF sites is. It indicates insufficient attention is given to ongoing maintenance of projects after they are no longer "actively developed", which in turn creates resentment with the reporters.

It really doesn't. The backlog is the contact surface between stuff that exists and stuff that doesn't; all the things we don't have but which seem realistically within reach. As functionality expands, that surface expands too. It is a normal process.

Except functionality doesn't expand for not actively developed products, but the backlog does.

(We do have projects which are basically unmaintained. Those are not typically the ones producing lots of new tasks though, since most relevant issues have been filed already. And realistically the choice is between having poorly maintained components and having far less components. Would undeploying UploadWizard, for example, reduce resentment? I don't think so.)

It's all relative: if UW would be undeployed in favor of a different component that would cover some of the stuff lacking from UW, than I don't think we'd see much resentment. I would personally love to see regular code stewardship reviews for every deployed components which haven't had one in 2-3 years. After a couple of such iterations, I'm pretty sure we'd have a non-negligible number of extensions undeployed. Would that lead to resentment? Sure, but I don't think the level would be comparable. The main problem I see is there is no good way to decide how important something is beyond usage metrics.

First of all, I want to say that I wholeheartedly agree with everything tgr wrote.

Regarding Pine's question on technical debt.

Technical debt is basically a fancy way of saying something is "icky". It is an inherently subjective notion, and at least for me, how important technical debt is depends a lot on how much my subjective sensibilities on what is icky matches whoever is talking about technical debt.

So yes, I think everyone agrees icky stuff is bad, but sometimes different people have different ideas on what is icky and how much ickiness the icky things contain. Furthermore there is a trap one can fall into of only fixing icky stuff, even if its only slightly icky, which is bad as then you don't actually accomplish anything else. As with everything else in life, moderation is the best policy (imo).

-- Brian

On Mon, Mar 18, 2019 at 8:17 PM Pine W wiki.pine@gmail.com wrote:

Hi,

First, I'll respond to Scott's comment that " A secondary issue is that too much wiki dev is done by WMF/WMFDE employees (IMO); I don't think the current percentages lead to an overall healthy open source community. But (again in my view) the first step to nurturing and growing our non-employee contributors is to make sure their patches are timely reviewed."

I'll make a distinction between two types of proposals:

a, Offload development work from WMF onto volunteers, and b, Grow and support the population of developers..

The first type of proposal is likely to get a cold reception from me, but I'm more supportive of the second. I don't know how many non-Wikimedia organizations which use MediaWiki software have staff that contribute in significant ways to WMF in the forms of software, time, or money, but growing the significance of those contributions sounds like a good idea. I also like programs such as GSoC and Outreachy, and for WMF providing support for volunteer devs who create tools, bots, etc. on their own initiative.

Second, returning to the subject of technical debt, my understanding was that WMF staff were concerned for years about the accumulation of technical debt, but in this thread I get the impression that WMF staff has changed their minds. Am I misunderstanding something? If the consensus opinion among the staff changed, how and why did that change happen?

Thanks,

Pine ( https://meta.wikimedia.org/wiki/User:Pine ) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Monday, March 18, 2019, John Erling Blad jeblad@gmail.com wrote:

On Mon, Mar 18, 2019 at 10:52 PM bawolff bawolff+wn@gmail.com wrote:

...

First of all, I want to say that I wholeheartedly agree with everything

tgr

...

wrote.

Regarding Pine's question on technical debt.

Technical debt is basically a fancy way of saying something is "icky". It is an inherently subjective notion, and at least for me, how important technical debt is depends a lot on how much my subjective sensibilities

on

...

what is icky matches whoever is talking about technical debt.

So yes, I think everyone agrees icky stuff is bad, but sometimes

different

...

people have different ideas on what is icky and how much ickiness the

icky

...

things contain. Furthermore there is a trap one can fall into of only fixing icky stuff, even if its only slightly icky, which is bad as then

you

...

don't actually accomplish anything else. As with everything else in life, moderation is the best policy (imo).

-- Brian

To set degree of ickyness you need a stakeholdergroup, which is often just the sales department. When you neither have a stakeholder group or sales department you tend to end up with ickyness set by the devs, and then features win over bugs. Its just the way things are.

I believe the ickyness felt by the editors must be more visible to the devs, and the actual impact the devs do on bugs to lower the ickyness must be more visible to the editors.

Technical debt is by definition "ickyness felt by devs". It is a thing that can be worked on. It is not the only thing to be worked on, nor should it be, but it is one aspect of the system to be worked on. If its ignored it makes it really hard to fix bugs because then devs wont understand how the software works. If tech debt is worked on at the expense of everything else, that is bad too (like cleaning your house for a week straight without stopping to eat-bad outcomes) By definition it is not new features nor is it ickyness felt by users. It might help with bugs felt by users as often they are the result of devs misunderstanding what is going on, but that is a consequence not the thing itself.

Sales dept usually dont advocate for bug fixing as that doesnt sell products, new features do, so i dont know why you are bringing them up. They also dont usually deal with technical debt in the same way somebody who has never been to your house cant give you effective advice on how to clean it.

That said, fundamentally you want user priorities (or at least *your* priorities. Its unclear if your priorities reflect the user base at large) to be taken into consideration when deciding developer priorities? Well step 1 is to define what you want. The wmf obviously tries to figure out what is important to users, and its pretty obvious in your view they are failing. Saying people are working on the wrong thing without saying what they should work on instead is a self-fulfiling prophecy.

-- Brian

All software development costs someone something. Software does not change without someone paying something for it (even if that is just their own time, time has value).

To that end, technical debt, is like any other software change. There is no difference between solving technical debt, fixing bugs, or creating new features. All of these changes must have a "business value" associated with them. If you cannot justify the value of the change, why are you doing it? If you can, then you know what it is worth.

A lot of technical debt has a business case that it makes future software development slower. Or to borrow some of the examples from this thread, it takes longer to find something in your home, if your home is not organized. Likewise, it takes longer to fix bugs and develop new features if the code is not organized and provides a pleasant developer experience.

It is up to individual developers to surface the issues that have the most value to the business (from a technical or user perspective) and it is up to the product owners to make a determination of what truly has the most value to the business. In other words, what gets Wikimedia the greatest bang for the buck? I am thankful that it is not up to me to answer that question. :)

On Tue, Mar 19, 2019 at 11:49 AM John Erling Blad jeblad@gmail.com wrote:

On Tue, Mar 19, 2019 at 12:53 PM bawolff bawolff+wn@gmail.com wrote:

...

Technical debt is by definition "ickyness felt by devs". It is a thing

that

...

can be worked on. It is not the only thing to be worked on, nor should it be, but it is one aspect of the system to be worked on. If its ignored it makes it really hard to fix bugs because then devs wont understand how

the

...

software works. If tech debt is worked on at the expense of everything else, that is bad too (like cleaning your house for a week straight

without

...

stopping to eat-bad outcomes) By definition it is not new features nor is it ickyness felt by users. It might help with bugs felt by users as often they are the result of devs misunderstanding what is going on, but that

is

...

a consequence not the thing itself.

The devs is not the primary user group, and they never will be. An editor is a primary user, and (s)he has no idea where the letters travels or how they are stored. A reader is a primary user, and likewise (s)he has no idea how the letters emerge on the screen.

The devs are just one of several in a stakeholder group, and focusing solely on whatever ickyness they feel is like building a house by starting calling the plumber.

...

Sales dept usually dont advocate for bug fixing as that doesnt sell products, new features do, so i dont know why you are bringing them up. They also dont usually deal with technical debt in the same way somebody who has never been to your house cant give you effective advice on how to clean it.

A sales dep is in contact with the customer, which is a primary user of the product. If you don't like using the sales department, then say you have a support desk that don't report bugs. Without anyone reporting the bugs the product is dead.

Actually this is the decade old fight over "who owns the product". The only solution is to create a real stakeholder group.

...

That said, fundamentally you want user priorities (or at least *your* priorities. Its unclear if your priorities reflect the user base at

large)

...

to be taken into consideration when deciding developer priorities? Well step 1 is to define what you want. The wmf obviously tries to figure out what is important to users, and its pretty obvious in your view they are failing. Saying people are working on the wrong thing without saying what they should work on instead is a self-fulfiling prophecy.

Not going to answer this, it is an implicit blame game

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Second, returning to the subject of technical debt, my understanding was that WMF staff were concerned for years about the accumulation of technical debt, but in this thread I get the impression that WMF staff has changed their minds. Am I misunderstanding something?

Last year has seen a lot of focus on Technical Debt. WMF also has a core platform team now, which finally allows a more sustainable chipping away at some of the technical debt. Lastly our CI tools now help to gradually clean up technical debt as well. All this has showed that fixing technical debt works and can be done (even for MediaWiki). So this is why you observe a bit more relaxed attitude to this. There are still loads of problems and things that need fixing, but there is light on the horizon so people are less panicky about it.

DJ

On Mon, Mar 18, 2019 at 9:17 PM Pine W wiki.pine@gmail.com wrote:

Hi,

First, I'll respond to Scott's comment that " A secondary issue is that too much wiki dev is done by WMF/WMFDE employees (IMO); I don't think the current percentages lead to an overall healthy open source community. But (again in my view) the first step to nurturing and growing our non-employee contributors is to make sure their patches are timely reviewed."

I'll make a distinction between two types of proposals:

a, Offload development work from WMF onto volunteers, and b, Grow and support the population of developers..

The first type of proposal is likely to get a cold reception from me, but I'm more supportive of the second. I don't know how many non-Wikimedia organizations which use MediaWiki software have staff that contribute in significant ways to WMF in the forms of software, time, or money, but growing the significance of those contributions sounds like a good idea. I also like programs such as GSoC and Outreachy, and for WMF providing support for volunteer devs who create tools, bots, etc. on their own initiative.

Second, returning to the subject of technical debt, my understanding was that WMF staff were concerned for years about the accumulation of technical debt, but in this thread I get the impression that WMF staff has changed their minds. Am I misunderstanding something? If the consensus opinion among the staff changed, how and why did that change happen?

Thanks,

Pine ( https://meta.wikimedia.org/wiki/User:Pine ) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Tue, Mar 19, 2019 at 4:16 PM Gergő Tisza gtisza@gmail.com wrote:

On Mon, Mar 18, 2019 at 3:01 PM Derk-Jan Hartman < d.j.hartman+wmf_ml@gmail.com> wrote:

...

Last year has seen a lot of focus on Technical Debt. WMF also has a core platform team now, which finally allows a more sustainable chipping away

at

...

some of the technical debt.

Yeah. Having tech debt is never great but what gets people concerned is when it just grows and grows, and management dismisses concerns because it is always more important to have the next feature out quickly. We used to have a bit of that problem, but IMO there have been lots of positive changes in the last two years or so, and there is now a credible organization-wide effort now to get debt under control (mainly looking at the Platform Evolution program here). Having the core platform team also helped a lot, and in my impression some other teams that had in the past focused on fast feature iteration have also been given more space to do things right.

Thanks very much for the explanations regarding that point.

One of my continuing concerns is that, as far as I know, no one has a way of reliably quantifying the scale of the technical debt or measuring how it is changing over time. It sounds like the internal view in WMF is that the situation is improving, which is good to hear. However, I would prefer to have a way to measure technical debt and how it is changing. If that information is available then I think that making decisions about resources, priorities, etc. will involve less guesswork. I think that this proposal could align with WMF's work on code health.(See https://www.mediawiki.org/wiki/Code_Health).

Pine ( https://meta.wikimedia.org/wiki/User:Pine )

:) Structured data exists regarding many other subjects such as books and magazines. I would think that a similar approach could be taken to technical debt. I realize that development tasks have properties and interactions that change over time, but I think that having a better quantitative understanding of the backlog would be good and would likely improve the quality of planning and resourcing decisions.

To use an analogy, as far as I know there is no project management system that reliably produces accurate estimates of the time and resources required to complete large projects, but I think that attempting to use a project management system is, in many cases, better than trying to execute a large project without a project management system.

Pine ( https://meta.wikimedia.org/wiki/User:Pine )

On Wed, Mar 20, 2019, 3:32 PM Nick Wilson (Quiddity) nwilson@wikimedia.org wrote:

Pine, please see the exact (quite precise) definition of https://en.wiktionary.org/wiki/technical_debt https://en.wikipedia.org/wiki/Technical_debt https://martinfowler.com/bliki/TechnicalDebt.html I.e. Technical debt is Not at all equivalent to "bugs". The topic is a tangential one. Software can work perfectly fine for end-users even if it has a lot of "technical debt", it is just a pain for developers to change anything in it or connected to it because the code has complex issues (it's a mess, or imperfectly architected at a higher-level, or "icky", or other factors). It is not possible to measure, and is somewhat subjective in nature.

Thanks for correcting me regarding the definition. That helps. (One of these days I will probably write something that will reveal a deep ignorance of a Wikimedia topic, and I'm sure that I will hear about it. Making errors is one way to learn, although it is a way that I often make an effort to avoid.)

Overall this thread is going in circles, and I recommend dropping it here. There are several good suggestions above if anyone wants to put effort into actual solutions.

It sounds like we have different perspectives. However, get the impression that people are getting tired of the this topic, so I'll move on.

Pine ( https://meta.wikimedia.org/wiki/User:Pine )

I think it was doomed to fail as soon as people argued that an organization with an ~$80m annual budget had too many "resource constraints" to address a backlog of bugs in its core product. That happened in the first five or so replies to the thread!

On Sun, Mar 24, 2019 at 10:05 PM John Erling Blad jeblad@gmail.com wrote:

It is a strange discussion, especially as it is now about how some technical debts are not _real_ technical debts. You have some code, and you change that code, and breakage emerge both now and for future projects. That creates a technical debt. Some of it has a more pronounced short time effect (user observed bugs), and some of has a more long term effect (it blocks progress). At some point you must fix all of them.

On Thu, Mar 21, 2019 at 11:10 PM Pine W wiki.pine@gmail.com wrote:

...

It sounds like we have different perspectives. However, get the

impression

...

that people are getting tired of the this topic, so I'll move on.

I don't think this will be solved, so "move on" seems like an obvious choice.

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Wed, Mar 20, 2019 at 2:08 PM Pine W wiki.pine@gmail.com wrote:

:) Structured data exists regarding many other subjects such as books and magazines. I would think that a similar approach could be taken to technical debt. I realize that development tasks have properties and interactions that change over time, but I think that having a better quantitative understanding of the backlog would be good and would likely improve the quality of planning and resourcing decisions.

Focusing on metrics is something bad managers tend to do when they don't have the skills or knowledge to determine the actual value of the work. It's a famous anti-pattern. I'll refer you to the classic Spolsky article: https://www.joelonsoftware.com/2002/07/15/20020715/

On Sat, Mar 16, 2019 at 8:23 AM Strainu strainu10@gmail.com wrote:

...

A large backlog by itself is not alarming. A growing one for components deployed to WMF sites is. It indicates insufficient attention is given to ongoing maintenance of projects after they are no longer "actively developed", which in turn creates resentment with the reporters.

On Sun, Mar 17, 2019 at 10:22 PM Gergo Tisza gtisza@wikimedia.org wrote:

It really doesn't. The backlog is the contact surface between stuff that exists and stuff that doesn't; all the things we don't have but which seem realistically within reach. As functionality expands, that surface expands too. It is a normal process.

This isn't quite right, it only hold in some kind of simplified and idealized environment.

There are several axis, not only what exist. For example existing and non-existing features might be on the same axis, while it is hard to say that functional vs non-functional code is on the same axis. If you say these two are on the same axis, "stuff that exists", then you end up arguing fixing bugs would be a problem as it expands the feature space, thus will increase the total space and then increase the technical debt.

This will imply that introducing a critical bug will solve the technical debt, as the contact space will collapse. Fairly an acceptable solution! ;D

Hi!

In my experience WMF teams usually have a way to distinguish "bugs we're going to work on soon" and "bugs we're not planning to work on, but we'd accept patches". This is usually public in Phabricator, but not really documented.

There's the "Need volunteer" tag that I think can be used for that.

On Tue, 2019-03-12 at 00:29 +0100, John Erling Blad wrote:

It seems like some projects simply put everything coming from external sources into deep freezer or add "need volunteer". If they respond at all. In some cases it could be that the projects are defunc.

What's the expectation based on that there should always be a response? If a bug report has all info needed to allow someone to reproduce and work on it, anyone is free to pick it up and work on it if anyone is interested in working on it. No further response needed.

Cheers, andre

Is the Wikimedia Foundation responsible for people's emotions?

On Thu, Mar 14, 2019 at 10:51 AM John Erling Blad jeblad@gmail.com wrote:

Yes, there should always be a response to all bugs. Without a response the impression in the reporting wiki-community would be "nobody cares about our bug reports".

Someone in the community finds a bug, and it is posted and discussed in the community. Then another one writes a report in a task at Phabricator, but nothing further happen. A couple of months later the first one ask again about the bug, but does not get a satisfactory answer, and gets angry. This usually happen in cycles of a few months to a year. We must somehow break those cycles, they are bad and disruptive and creates a "us and them" attitude.

Users from the wiki-communities don't visit Phabricator to see all those small administrative tasks, they see the notes from the official and unofficial tech ambassadors, and they see the changes in the "tracked" templates. The templates are only changed when the bugs are closed for whatever reason, which could take years. Creating additional manual interventions does not work, the process must be simpler and more efficient.

On Thu, Mar 14, 2019 at 1:23 PM Andre Klapper aklapper@wikimedia.org wrote:

...

On Tue, 2019-03-12 at 00:29 +0100, John Erling Blad wrote:

...

It seems like some projects simply put everything coming from external sources into deep freezer or add "need volunteer". If they respond at all. In some cases it could be that the projects are defunc.

What's the expectation based on that there should always be a response? If a bug report has all info needed to allow someone to reproduce and work on it, anyone is free to pick it up and work on it if anyone is interested in working on it. No further response needed.

Cheers, andre -- Andre Klapper | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Thu, 2019-03-14 at 15:50 +0100, John Erling Blad wrote:

Yes, there should always be a response to all bugs. Without a response the impression in the reporting wiki-community would be "nobody cares about our bug reports".

Someone in the community finds a bug, and it is posted and discussed in the community. Then another one writes a report in a task at Phabricator, but nothing further happen. A couple of months later the first one ask again about the bug, but does not get a satisfactory answer, and gets angry. This usually happen in cycles of a few months to a year. We must somehow break those cycles, they are bad and disruptive and creates a "us and them" attitude.

I've seen it a few times on wiki village pumps or wiki article talk pages that someone points out something and then nobody else replied (or "nobody cared", as you call it). And then people "get angry" as you call it.

Do you manage to reply to all and each post in your local wiki community, or how do you deal with this problem?

andre

Stas made a point that I was considering too, although from a different perspective.

One of the issues may be a difference between end user expectations and what the resources are available to fulfill those expectations.

Communications requires time and mental bandwidth, both of which are limited resources for everyone, including WMF staff and end users. Also, there are financial considerations regarding how people use their time.

From the end user perspective, reporting a bug and then having nothing

happen, or getting an initial reply but later seeing that a bug appears to stall for months or years, may be frustrating depending on the nature of the bug and the patience of the user. I think that communicating with users regarding when bugs will likely be fixed would be helpful. I think that some of that happens already, but there's more that can be done. There are probably ways to automate some of these communications to a degree.

On the larger scale, I don't know whether it's possible to get a good large scale understanding of all of the open tasks in Phabricator. I speculate that teams might be able to create semi-automated reports regarding their own teams' tasks. To get a larger view of the situation in Phabricator might require combining the unique outputs of the reports from individual teams. By having a big picture view of the situation I hope that we could improve our collective situational awareness regarding tasks, including open feature requests and technical debt. Also, by creating snapshots of the results of the same type of combined report over a period of months or years, maybe we could get a sense of how technical debt is changing over time.

To summarize: I am thinking that a two pronged approach would be good, one regarding communications regarding the status of individual bugs, and one regarding a big picture analysis of technical debt.

I realize that there would be costs of time and money for both of those approaches. Automation can help with both.

My guess is that managing thousands of bugs in an continuous development environment is challenging in the best of circumstances. I am somewhat familiar with the end user experience and financial considerations (both of which motivated me to participate in this thread), but I'm not an expert in software engineering for a product that is on the scale of a top 10 website.

What do others think regarding these proposals?

Thanks for the good discussion.

Pine ( https://meta.wikimedia.org/wiki/User:Pine )

John,

Multiple people on this thread have pointed out that you are not giving specifics bugs that are being ignored by WMF (and should not be). Can you provide some examples?

Chico Venancio

Em qua, 13 de mar de 2019 às 17:01, John Erling Blad jeblad@gmail.com escreveu:

This is like an enormous sinkhole, with people standing on the edge, warning about the sinkhole. All around people are saying "we must do something"! Still the sinkhole slowly grows larger and larger. People placing warning signs "Sinkhole ahead". Others notifying neighbors about the growing sinkhole. But nobody does anything about the sinkhole itself.

I doubt this will be fixed.

John

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Blame games does not fix faulty processes. You fix a sinkhole by figuring out where the water comes from and where it goes.

Why do we have bugs that isn't handled for years? Why is it easier to get a new feature than fixing an old bug?

Google had a problem with unfixed bugs, and they started identifying the involved developers each time the build was broken. That is pretty harsh, but what if devs somehow was named when their bugs were mentioned? What if there were some kind of public statistic? How would the devs react to being identified with a bug? Would they fix the bug, or just be mad about it? Devs at some of Googles teams got mad, but in the end the code were fixed. Take a look at "GTAC 2013 Keynote: Evolution from Quality Assurance to Test Engineering" [1]

What if we could show information from the bugs in Phabricator in a "tracked" template at other wiki-projects, identifying the team responsible and perhaps even the dev assigned to the bug? Imagine the creds the dev would get when the bug is fixed! Because it is easy to loose track of pages with "tracked" templates we need some other means to show this information, and our "public monitor" could be a special page with the same information.

We say we don't want voting over bugs, but by saying that we refuse getting stats over how many users a specific bug hits, and because of that we don't get sufficient information (metrics) to make decisions about specific bugs. Some bugs (or missing features) although changes how users are doing specific things, how do we handle that?

What if users could give a "this hits me too" from a "tracked" template. That would give a very simple metric on how important it is to fix a problem. To make this visible to the wiki-communities the special page could be sorted on this metric. Of course the devs would have completely different priorities, but this page would list the wiki-communities priorities.

It would be a kind of blame game, but it would also give the devs an opportunity to get sainthood by fixing annoying bugs.

[1] https://www.youtube.com/watch?v=nyOHJ4GR4iU from 32:20

On Wed, Mar 13, 2019 at 11:49 PM Andre Klapper aklapper@wikimedia.org wrote:

On Wed, 2019-03-13 at 21:01 +0100, John Erling Blad wrote:

...

This is like an enormous sinkhole, with people standing on the edge, warning about the sinkhole. All around people are saying "we must do something"! Still the sinkhole slowly grows larger and larger. People placing warning signs "Sinkhole ahead". Others notifying neighbors about the growing sinkhole. But nobody does anything about the sinkhole itself.

And repeating the same thing over and over again while repeatedly ignoring requests to be more specific won't help either...

andre

Andre Klapper | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Sorry, but I try to point out that the process is broken and give a few examples on how to fix the process.

On Thu, Mar 14, 2019 at 1:20 PM Andre Klapper aklapper@wikimedia.org wrote:

On Thu, 2019-03-14 at 12:35 +0100, John Erling Blad wrote:

...

Blame games does not fix faulty processes.

Hmm, why is this thread called "Question to WMF" instead of "Question to developers"?

...

Why do we have bugs that isn't handled for years?

Basically: Because you did not fix these bugs. Longer version: https://www.mediawiki.org/wiki/Bug_management/Development_prioritization

...

Why is it easier to get a new feature than fixing an old bug?

{{Citation needed}}. If that was the case: Because your priority was to write code for a new feature instead of fixing an old bug. Longer version: https://www.mediawiki.org/wiki/Bug_management/Development_prioritization

...

Google had a problem with unfixed bugs, and they started identifying the involved developers each time the build was broken. That is pretty harsh, but what if devs somehow was named when their bugs were mentioned? What if there were some kind of public statistic? How would the devs react to being identified with a bug? Would they fix the bug, or just be mad about it? Devs at some of Googles teams got mad, but in the end the code were fixed. Take a look at "GTAC 2013 Keynote: Evolution from Quality Assurance to Test Engineering" [1]

Not really - I see 60000 open bug reports in Chromium, for example: https://bugs.chromium.org/p/chromium/issues/list (Only if you want to imply that only "Google" was responsible for fixing all bugs in that free and open source project, of course.)

...

What if we could show information from the bugs in Phabricator in a "tracked" template at other wiki-projects, identifying the team responsible and perhaps even the dev assigned to the bug? Imagine the creds the dev would get when the bug is fixed! Because it is easy to loose track of pages with "tracked" templates we need some other means to show this information, and our "public monitor" could be a special page with the same information.

Feel free to extend https://www.mediawiki.org/wiki/Template:Tracked

...

We say we don't want voting over bugs, but by saying that we refuse getting stats over how many users a specific bug hits, and because of that we don't get sufficient information (metrics) to make decisions about specific bugs.

I disagree. Different people see different priorities. Longer version: https://www.mediawiki.org/wiki/Bug_management/Development_prioritization

...

What if users could give a "this hits me too" from a "tracked" template. That would give a very simple metric on how important it is to fix a problem.

It does not, because software development is not a popularity contest: https://www.mediawiki.org/wiki/Bug_management/Development_prioritization Voting would create expectations that nobody will fulfill.

Cheers, andre -- Andre Klapper | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Mar 13, 2019, at 6:48 PM, Andre Klapper <aklapper@wikimedia.org mailto:aklapper@wikimedia.org> wrote:

On Wed, 2019-03-13 at 21:01 +0100, John Erling Blad wrote:

...

... But nobody does anything about the sinkhole itself.

And repeating the same thing over and over again while repeatedly ignoring requests to be more specific won't help either...

Here’s a specific example, created in 2015:

https://phabricator.wikimedia.org/T116145 https://phabricator.wikimedia.org/T116145

A bug fix was provided years ago but never accepted or rejected. It’s the first and last MediaWiki bug ever assigned to me. I’ve just unassigned myself.

In cases like this, remarks like “Because you did not fix these bugs” and “... anyone is free to pick it up and work on it ... No further response needed” miss the point. When a bug fix is provided, but nobody with authority to accept or reject it ever does so, that’s a failure on the part of those who have authority, not on the part of those who are able and willing to fix bugs. Sure, volunteers are “free” to waste their time!

You need to use and share your authority more effectively, to “be bold” with accepting and rejecting bug fixes. Authorize more people to accept or reject bug fixes. Assign each proposed bug fix to one such person, starting with the oldest bugs. Then hold those people accountable. You don’t lack volunteers, you lack volunteers with authority.

Best wishes,

Tom

Wenlin Institute, Inc. SPC (a Social Purpose Corporation) 文林研究所社会目的公司 Software for Learning Chinese E-mail: wenlin@wenlin.com mailto:wenlin@wenlin.com Web: http://www.wenlin.com http://www.wenlin.com/ Telephone: 1-877-4-WENLIN (1-877-493-6546) ☯

I'll echo Andre here: the specific problem of patches from new volunteer devs which don't get timely responses is a real issue, and one which we have attempted to address (as Andre described) but an area we could probably still use additional ideas, accountability, etc for.

A secondary issue is that too much wiki dev is done by WMF/WMFDE employees (IMO); I don't think the current percentages lead to an overall healthy open source community. But (again in my view) the first step to nurturing and growing our non-employee contributors is to make sure their patches are timely reviewed. --scott

On Sat, Mar 16, 2019, 10:54 PM Andre Klapper aklapper@wikimedia.org wrote:

Hi and thanks for joining the discussion!

On Sat, 2019-03-16 at 20:37 -0400, Thomas Eugene Bishop wrote:

...

Here’s a specific example, created in 2015:

  https://phabricator.wikimedia.org/T116145 <

https://phabricator.wikimedia.org/T116145%3E

A bug fix was provided years ago but never accepted or rejected. It’s the first and last MediaWiki bug ever assigned to me. I’ve just unassigned myself.

In cases like this, remarks like “Because you did not fix these bugs” and “... anyone is free to pick it up and work on it ... No further response needed” miss the point. When a bug fix is provided, but nobody with authority to accept or reject it ever does so, that’s a failure on the part of those who have authority, not on the part of those who are able and willing to fix bugs. Sure, volunteers are “free” to waste their time!

You need to use and share your authority more effectively, to “be bold” with accepting and rejecting bug fixes. Authorize more people to accept or reject bug fixes. Assign each proposed bug fix to one such person, starting with the oldest bugs. Then hold those people accountable. You don’t lack volunteers, you lack volunteers with authority.

I fully agree. I was referring to bug reports in my emails.

Code review is an area in which Wikimedia is very frustrating. There are regular emails about patches by new contributors awaiting review [1] but that obviously only covers a small group of contributors. And while we recently started to have code stewardship reviews [2] to fill some gaps in the list of responsible persons and teams [3] per code base, we for example still lack meaningful statistics how big the code review problem is, in general and per team.

andre

[1] https://lists.wikimedia.org/pipermail/wikitech-l/2019-March/091632.html [2] https://www.mediawiki.org/wiki/Code_stewardship_reviews [3] https://www.mediawiki.org/wiki/Developers/Maintainers -- Andre Klapper | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Sat, Mar 16, 2019 at 5:37 PM Thomas Eugene Bishop < thomasbishop@wenlin.com> wrote:

A bug fix was provided years ago but never accepted or rejected. It’s the first and last MediaWiki bug ever assigned to me. I’ve just unassigned myself.

https://phabricator.wikimedia.org/T149639https://phabricator.wikimedia.org/T... In cases like this, remarks like “Because you did not fix these bugs” and “... anyone is free to pick it up and work on it ... No further response needed” miss the point. When a bug fix is provided, but nobody with authority to accept or reject it ever does so, that’s a failure on the part of those who have authority, not on the part of those who are able and willing to fix bugs. Sure, volunteers are “free” to waste their time!

The code review backlog is a genuine problem (I'd say it's in the top 3 problems we have, along with lack of good documentation, and well-structured testable code). It's entirely unrelated to the task backlog and the other topics in this thread, though. There has been plenty of discussion on it and various attempts at addressing (you can see some in T78768 [1], or in various Wikimedia Developer Summit sessions such as T149639 [2]). Unfortunately without much result so far, but the problem is definitely no lack of awareness. (I'd argue that lack of organizational focus / commitment *is* a problem, so making your voice heard in the various planning processes would be helpful. wikitech-l is not a great place for that, though.)

You need to use and share your authority more effectively, to “be bold”

with accepting and rejecting bug fixes. Authorize more people to accept or reject bug fixes. Assign each proposed bug fix to one such person, starting with the oldest bugs. Then hold those people accountable. You don’t lack volunteers, you lack volunteers with authority.

Being able to accept bug fixes effectively means being able to deploy code to Wikimedia production, which has security and robustness implications. So there are some limits on how widely we can distribute that authority. That said, we are probably more conservative than we should be, and nominating new reviewers [3] is one of the more useful things one could do.

[1] https://phabricator.wikimedia.org/T78768 [2] https://phabricator.wikimedia.org/T149639 [3] https://www.mediawiki.org/wiki/Gerrit/Privilege_policy#Requesting_Gerrit_pri...