There always be gaps in ownership and there will be some critical software left to individuals to keep the lights on. It's an ideal we need to move towards but we might never reach.

That being said, it really doesn't need to be this bad.

Here is a non-exhaustive list of critical tech currently in production without any maintainers:

- All of the authorization stack (2FA, Central Auth (otherwise known as SUL), authentication in mw, OAuth, etc.) - All of the multimedia stack (from upload, to the video player, to the mw file backend, media handler, metadata extraction, etc.) - FlaggedRevs aka Pending changes, a critical tool in the workflow of patrollers - Abusefilter, the tool that prevents vandalism from being saved. I can't stress how important this is. - SecurePoll, critical to community resilience. - Deletion workflow - Mailman (mailing lists), the very same infra that is sending and storing this email. - User preferences - Gadgets infra - Beta cluster: The most important human testing infra before bugs hit production. - SpamBlacklist/TitleBlacklist

You can go and check: https://www.mediawiki.org/wiki/Developers/Maintainers and https://phabricator.wikimedia.org/project/board/3144/query/open/ to see frustrations and open requests for years.

And even if a software would have an owner, it used to be that the team was under so much pressure to produce new things instead of maintenance that the software would practically be without a maintainer (or worse, as even volunteers couldn't unofficially take the role). I can example a few.

Am Mo., 17. Apr. 2023 um 02:51 Uhr schrieb Gergő Tisza gtisza@gmail.com:

On Sat, Apr 15, 2023 at 7:49 AM AntiCompositeNumber < anticompositenumber@gmail.com> wrote:

...

Agreed. It has long been the case that infrastructure critical to the operation of the various wikis has been left without a clear maintainer, or has been maintained only in the volunteer time of a single staffer already fulfilling a full-time role. Teams would be dissolved or reassigned to completely different projects after completion, without the ability and/or willingness to even review patches. That assumes that the team doing the work wasn't made up of contractors who departed the Foundation when the project was "completed", taking their knowledge of it with them.

This was a major factor in causing the technical debt problem, and must be addressed to have any chance of solving it.

At some point we will have to admit that we have created a feature set many times larger than we have the capacity to actively maintain and improve. Either we make software development cheaper somehow (move the WMF to Romania or something), or we cut some of the non-software spending (but we already spend 50%+ of movement funds on software, and we'd have to increase capacity way more than by a factor of two to maintain all our code), or we undeploy most current features, or we'll have to put up with most things being unmaintained, which is the status quo. That's not to say we can't be smarter about it (e.g. microservices are a great way to have maintenance overhead spin even more out of control) or that maintenance efforts couldn't be better prioritized (e.g. the lack of maintainership of our authentication stack is somewhat wild), but fundamentally changing the current mode of operation (where most things are deployed and then abandoned to work on the next thing) is a pipe dream IMO. _______________________________________________ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

Either we make software development cheaper somehow (move the WMF to Romania or something)

Hiring in countries with the worst labour laws and cheapest minimum wages is totally immoral. Especially in a community where equity is part of our culture we must endeavour to ensure that employees/contractors regardless of where they live paid fairly and equally subject to skills and responsibilities of the role. WMF already has many employees that are based in countries where such immoral employment conditions dominate.

On Tue, 18 Apr 2023 at 05:49, Dan Garry (Deskana) djgwiki@gmail.com wrote:

I agree with much of what Amir has said here, except one little bit...

On Mon, 17 Apr 2023 at 20:52, Amir Sarabadani ladsgroup@gmail.com wrote:

...

And even if a software would have an owner, it used to be that the team was under so much pressure to produce new things instead of maintenance that the software would practically be without a maintainer (or worse, as even volunteers couldn't unofficially take the role). I can example a few.

I think pressure on a team to deliver new things is *one* reason why this situation has come about, but it's far from being the only one. Here's a few others off the top of my head:

• Owning so many things that even if there was zero pressure to

deliver new features, the team still couldn't maintain everything that they own.

• Incredibly powerful and incredibly complex features that teams are

afraid of touching lest they break them and make community members angry.

• Conservatism and fear of community outrage causing reluctance to

deprecate functionality.

• Lack of understanding of the impact of the feature.
• Lack of a clear roadmap (a list of bug reports and feature requests

is not a roadmap).

There's more but those are some that come to the top of my head. And, not everyone one of those always applies to every situation, e.g. I definitely don't think all of the items in your list should be deprecated!

This causes the path of least resistance to be, for everyone involved, to leave things in limbo and hope for the best.

Dan _______________________________________________ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

Hiring people because they are in such countries as the basis for saving money is morally bankrupt, yet we'll happily draw from the pool of donations that primarily come from those more expensive countries. Much like we talk about equity but decide that some places arent worth engaging in because its too far to travel leaving others to shoulder the burden of travel.

On Tue, 18 Apr 2023 at 19:35, Felipe Schenone schenonef@gmail.com wrote:

Yet in some countries, like mine, paying for food, renting a place, buying a house, etc. is far cheaper than in the US, so paying a lower salary (in USD) wouldn't amount to a lower standard of living at all, and doesn't feel immoral, at least to me.

On Tue, Apr 18, 2023 at 8:00 AM Gnangarra gnangarra@gmail.com wrote:

...

Either we make software development cheaper somehow (move the WMF to

...

Romania or something)

Hiring in countries with the worst labour laws and cheapest minimum wages is totally immoral. Especially in a community where equity is part of our culture we must endeavour to ensure that employees/contractors regardless of where they live paid fairly and equally subject to skills and responsibilities of the role. WMF already has many employees that are based in countries where such immoral employment conditions dominate.

...

On Tue, 18 Apr 2023 at 05:49, Dan Garry (Deskana) djgwiki@gmail.com wrote:

...

I agree with much of what Amir has said here, except one little bit...

On Mon, 17 Apr 2023 at 20:52, Amir Sarabadani ladsgroup@gmail.com wrote:

...

And even if a software would have an owner, it used to be that the team was under so much pressure to produce new things instead of maintenance that the software would practically be without a maintainer (or worse, as even volunteers couldn't unofficially take the role). I can example a few.

I think pressure on a team to deliver new things is *one* reason why this situation has come about, but it's far from being the only one. Here's a few others off the top of my head:

• Owning so many things that even if there was zero pressure to

deliver new features, the team still couldn't maintain everything that they own.

• Incredibly powerful and incredibly complex features that teams are

afraid of touching lest they break them and make community members angry.

• Conservatism and fear of community outrage causing reluctance to

deprecate functionality.

• Lack of understanding of the impact of the feature.
• Lack of a clear roadmap (a list of bug reports and feature

requests is not a roadmap).

There's more but those are some that come to the top of my head. And, not everyone one of those always applies to every situation, e.g. I definitely don't think all of the items in your list should be deprecated!

This causes the path of least resistance to be, for everyone involved, to leave things in limbo and hope for the best.

Dan _______________________________________________ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

-- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar'

---

Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

---

Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

Just to put things into perspective, in Argentina, earning USD 4000 a month means you're the fucking king. You can rent almost any place you want, buy food and all necessities, eat out everyday, and have enough left over to buy some land or a house in a few years. By contrast, a quick Google search suggests that renting a 1-bedroom apartment in NYC costs around USD 4000, while in Silicon Valley costs around USD 2500. I may be wrong, but judging from https://meta.wikimedia.org/wiki/Wikimedia_Foundation_salaries I can see that nowadays, WMF salaries don't go below USD 200,000 per year, or USD 16,000 a month.

Rather than morally bankrupt, I'd argue that bringing salaries of even USD 5000 per month to people in countries like mine would be an economic bonanza and a smart use of resources, a win-win situation. Regarding labor laws, many non-US countries, like mine, have quite stringent labor laws (such as Argentina, due to a long history of syndicalism). Perhaps it's just a matter of finding countries that balance both criteria. I'm not sure that expanding development to cheaper countries is the solution to all of WMF software problems, but I think it could help a lot.

On Tue, Apr 18, 2023 at 8:55 AM Gnangarra gnangarra@gmail.com wrote:

or the 3am meetings

On Tue, 18 Apr 2023 at 19:49, Gnangarra gnangarra@gmail.com wrote:

...

Hiring people because they are in such countries as the basis for saving money is morally bankrupt, yet we'll happily draw from the pool of donations that primarily come from those more expensive countries. Much like we talk about equity but decide that some places arent worth engaging in because its too far to travel leaving others to shoulder the burden of travel.

On Tue, 18 Apr 2023 at 19:35, Felipe Schenone schenonef@gmail.com wrote:

...

Yet in some countries, like mine, paying for food, renting a place, buying a house, etc. is far cheaper than in the US, so paying a lower salary (in USD) wouldn't amount to a lower standard of living at all, and doesn't feel immoral, at least to me.

On Tue, Apr 18, 2023 at 8:00 AM Gnangarra gnangarra@gmail.com wrote:

...

Either we make software development cheaper somehow (move the WMF to

...

Romania or something)

Hiring in countries with the worst labour laws and cheapest minimum wages is totally immoral. Especially in a community where equity is part of our culture we must endeavour to ensure that employees/contractors regardless of where they live paid fairly and equally subject to skills and responsibilities of the role. WMF already has many employees that are based in countries where such immoral employment conditions dominate.

...

On Tue, 18 Apr 2023 at 05:49, Dan Garry (Deskana) djgwiki@gmail.com wrote:

...

I agree with much of what Amir has said here, except one little bit...

On Mon, 17 Apr 2023 at 20:52, Amir Sarabadani ladsgroup@gmail.com wrote:

...

And even if a software would have an owner, it used to be that the team was under so much pressure to produce new things instead of maintenance that the software would practically be without a maintainer (or worse, as even volunteers couldn't unofficially take the role). I can example a few.

I think pressure on a team to deliver new things is *one* reason why this situation has come about, but it's far from being the only one. Here's a few others off the top of my head:

• Owning so many things that even if there was zero pressure to

deliver new features, the team still couldn't maintain everything that they own.

• Incredibly powerful and incredibly complex features that teams

are afraid of touching lest they break them and make community members angry.

• Conservatism and fear of community outrage causing reluctance to

deprecate functionality.

• Lack of understanding of the impact of the feature.
• Lack of a clear roadmap (a list of bug reports and feature

requests is not a roadmap).

There's more but those are some that come to the top of my head. And, not everyone one of those always applies to every situation, e.g. I definitely don't think all of the items in your list should be deprecated!

This causes the path of least resistance to be, for everyone involved, to leave things in limbo and hope for the best.

Dan _______________________________________________ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

-- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar'

---

Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

---

Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

-- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar'

-- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar'

---

Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

The point here is that getting rid of staff in one country, purely to hire someone in another country to do the same job for significantly less, is morally bankrupt. The same issue goes with hiring new people solely from selected countries for the same reason. We are a global community and that means we should seek out people regardless of their location or the labour laws.

That also translates to other areas of the community, where we chose not to even consider achieving equity in global activities. We need to do more to balance equity through other means.

Yes this has drifted away from the pure software issues, being morally loose "because we can" should never be the solution to any problem.

On Tue, 18 Apr 2023 at 20:30, The Cunctator cunctator@gmail.com wrote:

I think one of the key lessons of software development is that infinite money doesn't necessarily lead to good software development. I think the work the new leadership is showing to address the structural flaws will go a long way. There's certainly nothing immoral about a global non-profit having an international staff. It is certainly true that wealth and salary differentials are a challenge for any international organization, and should be approached from a perspective of solidarity and mutuality. But like many things in our world, it will always be a source of tension.

On Tue, Apr 18, 2023, 8:23 AM Felipe Schenone schenonef@gmail.com wrote:

...

Just to put things into perspective, in Argentina, earning USD 4000 a month means you're the fucking king. You can rent almost any place you want, buy food and all necessities, eat out everyday, and have enough left over to buy some land or a house in a few years. By contrast, a quick Google search suggests that renting a 1-bedroom apartment in NYC costs around USD 4000, while in Silicon Valley costs around USD 2500. I may be wrong, but judging from https://meta.wikimedia.org/wiki/Wikimedia_Foundation_salaries I can see that nowadays, WMF salaries don't go below USD 200,000 per year, or USD 16,000 a month.

Rather than morally bankrupt, I'd argue that bringing salaries of even USD 5000 per month to people in countries like mine would be an economic bonanza and a smart use of resources, a win-win situation. Regarding labor laws, many non-US countries, like mine, have quite stringent labor laws (such as Argentina, due to a long history of syndicalism). Perhaps it's just a matter of finding countries that balance both criteria. I'm not sure that expanding development to cheaper countries is the solution to all of WMF software problems, but I think it could help a lot.

On Tue, Apr 18, 2023 at 8:55 AM Gnangarra gnangarra@gmail.com wrote:

...

or the 3am meetings

On Tue, 18 Apr 2023 at 19:49, Gnangarra gnangarra@gmail.com wrote:

...

Hiring people because they are in such countries as the basis for saving money is morally bankrupt, yet we'll happily draw from the pool of donations that primarily come from those more expensive countries. Much like we talk about equity but decide that some places arent worth engaging in because its too far to travel leaving others to shoulder the burden of travel.

On Tue, 18 Apr 2023 at 19:35, Felipe Schenone schenonef@gmail.com wrote:

...

Yet in some countries, like mine, paying for food, renting a place, buying a house, etc. is far cheaper than in the US, so paying a lower salary (in USD) wouldn't amount to a lower standard of living at all, and doesn't feel immoral, at least to me.

On Tue, Apr 18, 2023 at 8:00 AM Gnangarra gnangarra@gmail.com wrote:

...

Either we make software development cheaper somehow (move the WMF > to Romania or something)

Hiring in countries with the worst labour laws and cheapest minimum wages is totally immoral. Especially in a community where equity is part of our culture we must endeavour to ensure that employees/contractors regardless of where they live paid fairly and equally subject to skills and responsibilities of the role. WMF already has many employees that are based in countries where such immoral employment conditions dominate.

> On Tue, 18 Apr 2023 at 05:49, Dan Garry (Deskana) djgwiki@gmail.com wrote:

> I agree with much of what Amir has said here, except one little > bit... > > On Mon, 17 Apr 2023 at 20:52, Amir Sarabadani ladsgroup@gmail.com > wrote: > >> And even if a software would have an owner, it used to be that the >> team was under so much pressure to produce new things instead of >> maintenance that the software would practically be without a maintainer (or >> worse, as even volunteers couldn't unofficially take the role). I can >> example a few. >> > > I think pressure on a team to deliver new things is *one* reason > why this situation has come about, but it's far from being the only one. > Here's a few others off the top of my head: > > - Owning so many things that even if there was zero pressure to > deliver new features, the team still couldn't maintain everything that they > own. > - Incredibly powerful and incredibly complex features that teams > are afraid of touching lest they break them and make community members > angry. > - Conservatism and fear of community outrage causing reluctance > to deprecate functionality. > - Lack of understanding of the impact of the feature. > - Lack of a clear roadmap (a list of bug reports and feature > requests is not a roadmap). > > There's more but those are some that come to the top of my head. > And, not everyone one of those always applies to every situation, e.g. I > definitely don't think all of the items in your list should be deprecated! > > This causes the path of least resistance to be, for everyone > involved, to leave things in limbo and hope for the best. > > Dan > _______________________________________________ > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, > guidelines at: > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... > To unsubscribe send an email to > wikimedia-l-leave@lists.wikimedia.org

-- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar'

---

Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

---

Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

-- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar'

-- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar'

---

Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

---

Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

---

Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

So leave them to rot because their standard of living is low and their government is crap? Right.

From: Gnangarra [mailto:gnangarra@gmail.com] Sent: 18 April 2023 13:49 To: Wikimedia Mailing List Subject: [Wikimedia-l] Re: [Wikitech-l] Re: Reflecting on my listening tour

Hiring people because they are in such countries as the basis for saving money is morally bankrupt, yet we'll happily draw from the pool of donations that primarily come from those more expensive countries. Much like we talk about equity but decide that some places arent worth engaging in because its too far to travel leaving others to shoulder the burden of travel.

On Tue, 18 Apr 2023 at 19:35, Felipe Schenone schenonef@gmail.com wrote:

Yet in some countries, like mine, paying for food, renting a place, buying a house, etc. is far cheaper than in the US, so paying a lower salary (in USD) wouldn't amount to a lower standard of living at all, and doesn't feel immoral, at least to me.

On Tue, Apr 18, 2023 at 8:00 AM Gnangarra gnangarra@gmail.com wrote:

Either we make software development cheaper somehow (move the WMF to Romania or something)

Hiring in countries with the worst labour laws and cheapest minimum wages is totally immoral. Especially in a community where equity is part of our culture we must endeavour to ensure that employees/contractors regardless of where they live paid fairly and equally subject to skills and responsibilities of the role. WMF already has many employees that are based in countries where such immoral employment conditions dominate.

On Tue, 18 Apr 2023 at 05:49, Dan Garry (Deskana) djgwiki@gmail.com wrote:

I agree with much of what Amir has said here, except one little bit...

On Mon, 17 Apr 2023 at 20:52, Amir Sarabadani ladsgroup@gmail.com wrote:

And even if a software would have an owner, it used to be that the team was under so much pressure to produce new things instead of maintenance that the software would practically be without a maintainer (or worse, as even volunteers couldn't unofficially take the role). I can example a few.

I think pressure on a team to deliver new things is one reason why this situation has come about, but it's far from being the only one. Here's a few others off the top of my head:

* Owning so many things that even if there was zero pressure to deliver new features, the team still couldn't maintain everything that they own. * Incredibly powerful and incredibly complex features that teams are afraid of touching lest they break them and make community members angry. * Conservatism and fear of community outrage causing reluctance to deprecate functionality. * Lack of understanding of the impact of the feature. * Lack of a clear roadmap (a list of bug reports and feature requests is not a roadmap).

There's more but those are some that come to the top of my head. And, not everyone one of those always applies to every situation, e.g. I definitely don't think all of the items in your list should be deprecated!

This causes the path of least resistance to be, for everyone involved, to leave things in limbo and hope for the best.

Dan

_______________________________________________ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

And an international organisation hiring more people in a city where employment conditions are so expensive you can’t do the job properly is not?

Also no-one was recommending firing the current incumbents, or not that I noticed anyway. Cheers, P

From: Gnangarra [mailto:gnangarra@gmail.com] Sent: 20 April 2023 10:18 To: Wikimedia Mailing List Subject: [Wikimedia-l] Re: [Wikitech-l] Re: Reflecting on my listening tour

No, what I said was that firing people in one country then hiring some else in another country with the same skills to do the same job just because employment conditions are cheaper is morally bankrupt. With that just hiring from certain countries because its cheapest with the least amount of conditions, is also morally bankrupt.

On Thu, 20 Apr 2023 at 16:02, Peter Southwood peter.southwood@telkomsa.net wrote:

So leave them to rot because their standard of living is low and their government is crap? Right.

From: Gnangarra [mailto:gnangarra@gmail.com] Sent: 18 April 2023 13:49 To: Wikimedia Mailing List Subject: [Wikimedia-l] Re: [Wikitech-l] Re: Reflecting on my listening tour

Hiring people because they are in such countries as the basis for saving money is morally bankrupt, yet we'll happily draw from the pool of donations that primarily come from those more expensive countries. Much like we talk about equity but decide that some places arent worth engaging in because its too far to travel leaving others to shoulder the burden of travel.

On Tue, 18 Apr 2023 at 19:35, Felipe Schenone schenonef@gmail.com wrote:

Yet in some countries, like mine, paying for food, renting a place, buying a house, etc. is far cheaper than in the US, so paying a lower salary (in USD) wouldn't amount to a lower standard of living at all, and doesn't feel immoral, at least to me.

On Tue, Apr 18, 2023 at 8:00 AM Gnangarra gnangarra@gmail.com wrote:

Either we make software development cheaper somehow (move the WMF to Romania or something)

Hiring in countries with the worst labour laws and cheapest minimum wages is totally immoral. Especially in a community where equity is part of our culture we must endeavour to ensure that employees/contractors regardless of where they live paid fairly and equally subject to skills and responsibilities of the role. WMF already has many employees that are based in countries where such immoral employment conditions dominate.

On Tue, 18 Apr 2023 at 05:49, Dan Garry (Deskana) djgwiki@gmail.com wrote:

I agree with much of what Amir has said here, except one little bit...

On Mon, 17 Apr 2023 at 20:52, Amir Sarabadani ladsgroup@gmail.com wrote:

And even if a software would have an owner, it used to be that the team was under so much pressure to produce new things instead of maintenance that the software would practically be without a maintainer (or worse, as even volunteers couldn't unofficially take the role). I can example a few.

I think pressure on a team to deliver new things is one reason why this situation has come about, but it's far from being the only one. Here's a few others off the top of my head:

* Owning so many things that even if there was zero pressure to deliver new features, the team still couldn't maintain everything that they own. * Incredibly powerful and incredibly complex features that teams are afraid of touching lest they break them and make community members angry. * Conservatism and fear of community outrage causing reluctance to deprecate functionality. * Lack of understanding of the impact of the feature. * Lack of a clear roadmap (a list of bug reports and feature requests is not a roadmap).

There's more but those are some that come to the top of my head. And, not everyone one of those always applies to every situation, e.g. I definitely don't think all of the items in your list should be deprecated!

This causes the path of least resistance to be, for everyone involved, to leave things in limbo and hope for the best.

Dan

_______________________________________________ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/... To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org

This is not really about Selena's email but it's so nerdy I still want to talk about what tech debt is and what it isn't.

The word tech debt at the beginning meant a very specific thing: When an engineer takes a shortcut to deliver a feature faster. Daniel has a pretty good essay on tech debt: https://www.mediawiki.org/wiki/The_Tech_Debt_Trap

But the term slowly became so overused that engineers used it to refer to any invisible work they couldn't tie into a feature. At this point, people refer to tech debt as the general maintenance practically. And that led to non-tech people to slowly disregard maintenance work because everything is tech debt now.

What tech debt is not (this is not a MECE list):

- Keep the lights on: OS upgrades, security updates, hw maintenance, incidents, ... - Bitrot: The code stays the same but the underlying technology changes, the usage pattern changes, requirements changes, etc. - Over-engineering: This is way more common than you think. Building an overly complex palace that no one can maintain anymore because no one understands it and everyone is afraid to remove anything. This is sorta the opposite of tech debt, people spent more resources than they should and it's still hurting us. - Plain bad engineering: People make mistakes, typos are easily fixable while architectural mistakes are not, sometimes we know it because of hindsight, sometimes it was obvious from the beginning. - Limiting architecture: Many architectural decisions come in the form of trade-offs, a classic example is performance vs. security. If someone wants something and it's not possible due to architectural limitations, it doesn't necessarily mean a bad thing. We sacrificed something in favor of the other. - Code hygiene: Improvements on readability and maintainability of the code. - Scale/Performance/Security considerations: I have seen that people see a prototype and want it in Wikipedia right now. I understand it but any solution on such a large scale comes with all sorts of edge cases and hidden costs. People bash MediaWiki for being too complex and messy but anything you build at the scale of Wikipedia is going to be complex and messy. If a solution takes years, sometimes that's the only option.

What I want to say is that while we do have a tech debt problem in Wikimedia, we also have a lot of bitrot problems too, or any other "slowing down development" kind of problem. Some cases it's fixable, in some cases it is not. What is needed is more resources on maintenance which is happening and is making me extremely happy. Whether we call that paying back tech debt or not.

Am Fr., 14. Apr. 2023 um 23:44 Uhr schrieb Brian Wolff bawolff@gmail.com:

Perhaps i am hyperfocused on technical debt in the sense of improving the abstractions used in mediawiki. The phrasing around sustainability especially leads me in that direction. However, technical debt is certainly a broad concept and can mean a lot of things.

The common thread in the examples you cited seem to be things that have fallen through the ownership cracks. I'm not sure its the case that engineers aren't incentivized to fix these, so much as there are no natural engineers to be incentivized due to team structure (scribunto is an exception but i would disagree with that task's inclusion for reasons that get off topic). On the other hand, perhaps a different incentivization structure would encourage people to branch out more.

I think it is especially telling that 3 (or 4 even) of these tasks are multimedia related, given that wmf hasn't had a multimedia team in a very long time [SDC does not count], and definitely not one focused on the backend. There are quite a few multimedia related areas in desperate need of love (it is 2023 and video uploads are limited to 4gb with the flakiest upload process known to man).

It was also pointed out to me on irc, that many critical workflows in the community depend on toolforge tools that have very limited volunteer maintainership. A sort of https://xkcd.com/2347/ situation. Just because they're not "production" we often pretend they don't exist. Regardless of how we label them, in the end it doesn't make a difference to the end user, and the fragility of that ecosystem is a form of technical debt that is often overlooked.

So i guess it all depends on what is meant by "technical debt"

Brian

On Friday, April 14, 2023, Andre Klapper aklapper@wikimedia.org wrote:

...

On Thu, 2023-04-13 at 20:06 -0700, bawolff wrote:

...

...

"I think there are lots of promising opportunities to incentivise people to pay off technical debt and make our existing stack more sustainable. Right now there are no incentives for engineers in this regard."

Interesting. Personally to me, it can sometimes feel like we never stop talking about technical debt. While I think paying off technical debt is important, at times I feel like we've swung in the opposite direction where we are essentially rewriting things for the sake of rewriting things.

"Technical debt" spontaneously brings the following items to my little mind. They should not be about rewriting but rather "maintenance":

• librsvg for SVG rendering is a five year old version: https://phabricator.wikimedia.org/T193352 / https://phabricator.wikimedia.org/T265549
• Graph extension on old Vega version 2.6.3: see subtasks of https://phabricator.wikimedia.org/T292341
• Scribunto extension on old Lua version 5.1 (last 5.1.x release was in 2012): https://phabricator.wikimedia.org/T178146
• 3D extension on a five year old three.js library in https://phabricator.wikimedia.org/T277930#7636129
• Removing the OpenStackManager extension from wikitech.wm.org: https://phabricator.wikimedia.org/T161553
• Removing WVUI from MediaWiki core: https://phabricator.wikimedia.org/T310244
• Replacing jsduck with JSDoc3 across all Wikimedia code bases: https://phabricator.wikimedia.org/T138401
• Undeploy VipsScaler from Wikimedia wikis: https://phabricator.wikimedia.org/T290759
• https://phabricator.wikimedia.org/T151393 (a non-public task)

This is not a complete list. Plus there are also separate "waiting for someone to make a decision" and "improving communicating & documenting already-made decisions" categories which would be different lists.

Of course there might be valid reasons not to look into some of this technical debt (other higher priorities, high risk, complexity, etc).

Cheers, andre

-- Andre Klapper (he/him) | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/ _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org

https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

---

Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/