There always be gaps in ownership and there will be some critical software left to individuals to keep the lights on. It's an ideal we need to move towards but we might never reach. That being said, it really doesn't need to be this bad. Here is a non-exhaustive list of critical tech currently in production without any maintainers: - All of the authorization stack (2FA, Central Auth (otherwise known as SUL), authentication in mw, OAuth, etc.) - All of the multimedia stack (from upload, to the video player, to the mw file backend, media handler, metadata extraction, etc.) - FlaggedRevs aka Pending changes, a critical tool in the workflow of patrollers - Abusefilter, the tool that prevents vandalism from being saved. I can't stress how important this is. - SecurePoll, critical to community resilience. - Deletion workflow - Mailman (mailing lists), the very same infra that is sending and storing this email. - User preferences - Gadgets infra - Beta cluster: The most important human testing infra before bugs hit production. - SpamBlacklist/TitleBlacklist You can go and check: https://www.mediawiki.org/wiki/Developers/Maintainers and https://phabricator.wikimedia.org/project/board/3144/query/open/ to see frustrations and open requests for years. And even if a software would have an owner, it used to be that the team was under so much pressure to produce new things instead of maintenance that the software would practically be without a maintainer (or worse, as even volunteers couldn't unofficially take the role). I can example a few. Am Mo., 17. Apr. 2023 um 02:51 Uhr schrieb Gergő Tisza <gtisza(a)gmail.com>om>: -- Amir (he/him)

Hiring in countries with the worst labour laws and cheapest minimum wages is totally immoral. Especially in a community where equity is part of our culture we must endeavour to ensure that employees/contractors regardless of where they live paid fairly and equally subject to skills and responsibilities of the role. WMF already has many employees that are based in countries where such immoral employment conditions dominate. On Tue, 18 Apr 2023 at 05:49, Dan Garry (Deskana) <djgwiki(a)gmail.com> wrote: > I agree with much of what Amir has said here, except one little bit... > On Mon, 17 Apr 2023 at 20:52, Amir Sarabadani <ladsgroup(a)gmail.com> wrote: >> And even if a software would have an owner, it used to be that the team >> was under so much pressure to produce new things instead of maintenance >> that the software would practically be without a maintainer (or worse, as >> even volunteers couldn't unofficially take the role). I can example a few. >> I think pressure on a team to deliver new things is *one* reason why this > situation has come about, but it's far from being the only one. Here's a > few others off the top of my head: > - Owning so many things that even if there was zero pressure to > deliver new features, the team still couldn't maintain everything that they > own. > - Incredibly powerful and incredibly complex features that teams are > afraid of touching lest they break them and make community members angry. > - Conservatism and fear of community outrage causing reluctance to > deprecate functionality. > - Lack of understanding of the impact of the feature. > - Lack of a clear roadmap (a list of bug reports and feature requests > is not a roadmap). > There's more but those are some that come to the top of my head. And, not > everyone one of those always applies to every situation, e.g. I definitely > don't think all of the items in your list should be deprecated! > This causes the path of least resistance to be, for everyone involved, to > leave things in limbo and hope for the best. > Dan > _______________________________________________ > Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines > at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org… > To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org -- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar'

Hiring people because they are in such countries as the basis for saving money is morally bankrupt, yet we'll happily draw from the pool of donations that primarily come from those more expensive countries. Much like we talk about equity but decide that some places arent worth engaging in because its too far to travel leaving others to shoulder the burden of travel. On Tue, 18 Apr 2023 at 19:35, Felipe Schenone <schenonef(a)gmail.com> wrote: -- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar'

Just to put things into perspective, in Argentina, earning USD 4000 a month means you're the fucking king. You can rent almost any place you want, buy food and all necessities, eat out everyday, and have enough left over to buy some land or a house in a few years. By contrast, a quick Google search suggests that renting a 1-bedroom apartment in NYC costs around USD 4000, while in Silicon Valley costs around USD 2500. I may be wrong, but judging from https://meta.wikimedia.org/wiki/Wikimedia_Foundation_salaries I can see that nowadays, WMF salaries don't go below USD 200,000 per year, or USD 16,000 a month. Rather than morally bankrupt, I'd argue that bringing salaries of even USD 5000 per month to people in countries like mine would be an economic bonanza and a smart use of resources, a win-win situation. Regarding labor laws, many non-US countries, like mine, have quite stringent labor laws (such as Argentina, due to a long history of syndicalism). Perhaps it's just a matter of finding countries that balance both criteria. I'm not sure that expanding development to cheaper countries is the solution to all of WMF software problems, but I think it could help a lot. On Tue, Apr 18, 2023 at 8:55 AM Gnangarra <gnangarra(a)gmail.com> wrote:

The point here is that getting rid of staff in one country, purely to hire someone in another country to do the same job for significantly less, is morally bankrupt. The same issue goes with hiring new people solely from selected countries for the same reason. We are a global community and that means we should seek out people regardless of their location or the labour laws. That also translates to other areas of the community, where we chose not to even consider achieving equity in global activities. We need to do more to balance equity through other means. Yes this has drifted away from the pure software issues, being morally loose "because we can" should never be the solution to any problem. On Tue, 18 Apr 2023 at 20:30, The Cunctator <cunctator(a)gmail.com> wrote: -- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar'

So leave them to rot because their standard of living is low and their government is crap? Right. From: Gnangarra [mailto:gnangarra@gmail.com] Sent: 18 April 2023 13:49 To: Wikimedia Mailing List Subject: [Wikimedia-l] Re: [Wikitech-l] Re: Reflecting on my listening tour Hiring people because they are in such countries as the basis for saving money is morally bankrupt, yet we'll happily draw from the pool of donations that primarily come from those more expensive countries. Much like we talk about equity but decide that some places arent worth engaging in because its too far to travel leaving others to shoulder the burden of travel. On Tue, 18 Apr 2023 at 19:35, Felipe Schenone &lt;schenonef(a)gmail.com&gt; wrote: Yet in some countries, like mine, paying for food, renting a place, buying a house, etc. is far cheaper than in the US, so paying a lower salary (in USD) wouldn't amount to a lower standard of living at all, and doesn't feel immoral, at least to me. On Tue, Apr 18, 2023 at 8:00 AM Gnangarra &lt;gnangarra(a)gmail.com&gt; wrote: Either we make software development cheaper somehow (move the WMF to Romania or something) Hiring in countries with the worst labour laws and cheapest minimum wages is totally immoral. Especially in a community where equity is part of our culture we must endeavour to ensure that employees/contractors regardless of where they live paid fairly and equally subject to skills and responsibilities of the role. WMF already has many employees that are based in countries where such immoral employment conditions dominate. On Tue, 18 Apr 2023 at 05:49, Dan Garry (Deskana) &lt;djgwiki(a)gmail.com&gt; wrote: I agree with much of what Amir has said here, except one little bit... On Mon, 17 Apr 2023 at 20:52, Amir Sarabadani &lt;ladsgroup(a)gmail.com&gt; wrote: And even if a software would have an owner, it used to be that the team was under so much pressure to produce new things instead of maintenance that the software would practically be without a maintainer (or worse, as even volunteers couldn't unofficially take the role). I can example a few. I think pressure on a team to deliver new things is one reason why this situation has come about, but it's far from being the only one. Here's a few others off the top of my head: * Owning so many things that even if there was zero pressure to deliver new features, the team still couldn't maintain everything that they own. * Incredibly powerful and incredibly complex features that teams are afraid of touching lest they break them and make community members angry. * Conservatism and fear of community outrage causing reluctance to deprecate functionality. * Lack of understanding of the impact of the feature. * Lack of a clear roadmap (a list of bug reports and feature requests is not a roadmap). There's more but those are some that come to the top of my head. And, not everyone one of those always applies to every situation, e.g. I definitely don't think all of the items in your list should be deprecated! This causes the path of least resistance to be, for everyone involved, to leave things in limbo and hope for the best. Dan _______________________________________________ Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org… To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org -- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar' _______________________________________________ Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org… To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org _______________________________________________ Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org… To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org -- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar' <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> Virus-free. <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> www.avg.com

And an international organisation hiring more people in a city where employment conditions are so expensive you can’t do the job properly is not? Also no-one was recommending firing the current incumbents, or not that I noticed anyway. Cheers, P From: Gnangarra [mailto:gnangarra@gmail.com] Sent: 20 April 2023 10:18 To: Wikimedia Mailing List Subject: [Wikimedia-l] Re: [Wikitech-l] Re: Reflecting on my listening tour No, what I said was that firing people in one country then hiring some else in another country with the same skills to do the same job just because employment conditions are cheaper is morally bankrupt. With that just hiring from certain countries because its cheapest with the least amount of conditions, is also morally bankrupt. On Thu, 20 Apr 2023 at 16:02, Peter Southwood &lt;peter.southwood(a)telkomsa.net&gt; wrote: So leave them to rot because their standard of living is low and their government is crap? Right. From: Gnangarra [mailto:gnangarra@gmail.com] Sent: 18 April 2023 13:49 To: Wikimedia Mailing List Subject: [Wikimedia-l] Re: [Wikitech-l] Re: Reflecting on my listening tour Hiring people because they are in such countries as the basis for saving money is morally bankrupt, yet we'll happily draw from the pool of donations that primarily come from those more expensive countries. Much like we talk about equity but decide that some places arent worth engaging in because its too far to travel leaving others to shoulder the burden of travel. On Tue, 18 Apr 2023 at 19:35, Felipe Schenone &lt;schenonef(a)gmail.com&gt; wrote: Yet in some countries, like mine, paying for food, renting a place, buying a house, etc. is far cheaper than in the US, so paying a lower salary (in USD) wouldn't amount to a lower standard of living at all, and doesn't feel immoral, at least to me. On Tue, Apr 18, 2023 at 8:00 AM Gnangarra &lt;gnangarra(a)gmail.com&gt; wrote: Either we make software development cheaper somehow (move the WMF to Romania or something) Hiring in countries with the worst labour laws and cheapest minimum wages is totally immoral. Especially in a community where equity is part of our culture we must endeavour to ensure that employees/contractors regardless of where they live paid fairly and equally subject to skills and responsibilities of the role. WMF already has many employees that are based in countries where such immoral employment conditions dominate. On Tue, 18 Apr 2023 at 05:49, Dan Garry (Deskana) &lt;djgwiki(a)gmail.com&gt; wrote: I agree with much of what Amir has said here, except one little bit... On Mon, 17 Apr 2023 at 20:52, Amir Sarabadani &lt;ladsgroup(a)gmail.com&gt; wrote: And even if a software would have an owner, it used to be that the team was under so much pressure to produce new things instead of maintenance that the software would practically be without a maintainer (or worse, as even volunteers couldn't unofficially take the role). I can example a few. I think pressure on a team to deliver new things is one reason why this situation has come about, but it's far from being the only one. Here's a few others off the top of my head: * Owning so many things that even if there was zero pressure to deliver new features, the team still couldn't maintain everything that they own. * Incredibly powerful and incredibly complex features that teams are afraid of touching lest they break them and make community members angry. * Conservatism and fear of community outrage causing reluctance to deprecate functionality. * Lack of understanding of the impact of the feature. * Lack of a clear roadmap (a list of bug reports and feature requests is not a roadmap). There's more but those are some that come to the top of my head. And, not everyone one of those always applies to every situation, e.g. I definitely don't think all of the items in your list should be deprecated! This causes the path of least resistance to be, for everyone involved, to leave things in limbo and hope for the best. Dan _______________________________________________ Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org… To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org -- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar' _______________________________________________ Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org… To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org _______________________________________________ Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org… To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org -- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar' <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> Error! Filename not specified. Virus-free. <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> www.avg.com _______________________________________________ Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org… To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org -- Boodarwun Gnangarra 'ngany dabakarn koorliny arn boodjera dardoon ngalang Nyungar koortaboodjar'

This is not really about Selena's email but it's so nerdy I still want to talk about what tech debt is and what it isn't. The word tech debt at the beginning meant a very specific thing: When an engineer takes a shortcut to deliver a feature faster. Daniel has a pretty good essay on tech debt: https://www.mediawiki.org/wiki/The_Tech_Debt_Trap But the term slowly became so overused that engineers used it to refer to any invisible work they couldn't tie into a feature. At this point, people refer to tech debt as the general maintenance practically. And that led to non-tech people to slowly disregard maintenance work because everything is tech debt now. What tech debt is not (this is not a MECE list): - Keep the lights on: OS upgrades, security updates, hw maintenance, incidents, ... - Bitrot: The code stays the same but the underlying technology changes, the usage pattern changes, requirements changes, etc. - Over-engineering: This is way more common than you think. Building an overly complex palace that no one can maintain anymore because no one understands it and everyone is afraid to remove anything. This is sorta the opposite of tech debt, people spent more resources than they should and it's still hurting us. - Plain bad engineering: People make mistakes, typos are easily fixable while architectural mistakes are not, sometimes we know it because of hindsight, sometimes it was obvious from the beginning. - Limiting architecture: Many architectural decisions come in the form of trade-offs, a classic example is performance vs. security. If someone wants something and it's not possible due to architectural limitations, it doesn't necessarily mean a bad thing. We sacrificed something in favor of the other. - Code hygiene: Improvements on readability and maintainability of the code. - Scale/Performance/Security considerations: I have seen that people see a prototype and want it in Wikipedia right now. I understand it but any solution on such a large scale comes with all sorts of edge cases and hidden costs. People bash MediaWiki for being too complex and messy but anything you build at the scale of Wikipedia is going to be complex and messy. If a solution takes years, sometimes that's the only option. What I want to say is that while we do have a tech debt problem in Wikimedia, we also have a lot of bitrot problems too, or any other "slowing down development" kind of problem. Some cases it's fixable, in some cases it is not. What is needed is more resources on maintenance which is happening and is making me extremely happy. Whether we call that paying back tech debt or not. Am Fr., 14. Apr. 2023 um 23:44 Uhr schrieb Brian Wolff &lt;bawolff(a)gmail.com&gt;om>: -- Amir (he/him)