Thank you for your feedback. I think we have a very large and open corpus of documented incidents. As said, the Wikidata workshop is not a good target conference. Following the reference

[4]: https://doi.org/10.1109/ICSE.2013.6606583

I think a paper on that would better fit https://conf.researchr.org/track/icse-2024/icse-2024-software-engineering-in...

I will continue updating the paper on Overleaf

https://www.overleaf.com/read/swswtbdyyhmg

All the best Moritz

On Thu, Jun 8, 2023 at 7:57 PM Tyler Cipriani tcipriani@wikimedia.org wrote:

On Thu, Jun 8, 2023 at 12:40 AM Physikerwelt wiki@physikerwelt.de wrote:

...

Hi all,

is there any research on common causes of Wikimedia production errors?

Based on recent examples, I plan to analyze and discuss how production errors could be avoided. I am considering submitting a short paper on that to the Wikidata workshop, with the deadline Thursday, 20 July 2023 Website: https://wikidataworkshop.github.io/2023/ However, there might be better suitable venues.

We (Release Engineering) file production-error tasks as part of the weekly train and collect some data in the "train-stats" repo on GitLab[0]. Additionally, Timo Tijhof's "production excellence" blog posts and emails to this list may be of interest to you[1].

The "train-stats" repo collects data for "software defect prediction" based on the use of "FixCaches" or "BugCaches."[2] Each week, we record changes that fix bugs (i.e., the change uses the git trailer `Bug: TXXX` and gets backported to a currently deployed branch). The theory (per the paper linked above) is that the more often a file needs a fix, the more likely it is to cause future bugs. I have an extremely convoluted query to show the list of commonly backported files[3].

Problems with this data:

• Many of these files are frequently touched files vs error-prone files (e.g., "composer.json")
• Looking at the count of backports for each file means newer files are less likely to be represented
• "Lower level" files may be overrepresented (although, that's probably to be expected)

In 2013, a case study used data like this inside Google and found it to be fairly accurate at predicting future bugs[4].

Also, in the case study, whenever a developer edited a file that was present in their fixCache, researchers added a bot-generated note to the patch in their code review tool.Their developers found this note unhelpful: developers already knew these files were problematic—the warning just caused confusion.

Based on that, in March 2020, we created the "Risky Change Template"[5]. My thinking was: if developers already know what's risky, then they can flag it in the train task for the week[6]. At the time, I hoped this would reduce the total version deployment time (although I have no data on that).

I hope some of this helps!

– Tyler

...

I am also open to collaboration on this effort. If you are interested in a joint paper, drop me an email until the end of this week.

All the best Moritz _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

---

Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/