[Wikitech-l] Engineering Productivity Quarterly Roundup for Q1 - 20/21

Greetings! *TL;DR* We’re doing lots of engineering productivity work this year, which is highlighted below in this new quarterly report. For Q1, highlights include: - the GitLab consultation <https://www.mediawiki.org/wiki/GitLab_consultation> [1] is closed and results are being assessed; - Code Health Objectives are being set up with all engineering teams, nearly 20% of production code bases across three teams are covered; - A call for help with a burgeoning number of errors in production <https://lists.wikimedia.org/pipermail/wikitech-l/2020-September/093833.html> [2] went out; - New resources for discussing and improving local dev environments, including an email list <https://lists.wikimedia.org/mailman/listinfo/local-dev> [3] and a wiki page <https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Local_Dev_Updates> [4]; - A new Phabricator Video Tutorial Series <https://commons.wikimedia.org/wiki/Category:Phabricator_Tutorial_Series> [5] was unveiled, and; - CI now updates your deployment-charts <https://phabricator.wikimedia.org/phame/post/view/208/ci_now_updates_your_deployment-charts/> [6] *Longer Version* In the annual plan for this fiscal year, there are a number of commitments to address and improve engineering productivity across our technical teams in the organization, and this Engineering Productivity Quarterly Roundup is being launched with the goal of sharing and celebrating the progress that is being made as we chip away at this essential work. As a bit of background, we made a number of commitments across our engineering organization in both Technology and Product to take action on improving the health of our code bases, educating teams on how to approach code health, and improving systems that help with streamlining the overall engineering workflow. Let’s take a look at what happened in Q1 to forward these commitments: GitLab Consultation <https://www.mediawiki.org/wiki/GitLab_consultation> [1] During the month of September, Engineering Productivity Director Greg Grossmeier shepherded a consultation period to assess opinions on the use of GitLab as a code review system to replace Gerritt. This was launched in response to outcomes from the Developer Satisfaction Survey <https://www.mediawiki.org/wiki/Developer_Satisfaction_Survey/2020> [7] that indicated dissatisfaction with Gerritt, especially in the volunteer community. Comments poured in about the pros and cons from both internal and external engineers, and results are being compiled now. Stay tuned for an announcement later this month on the outcome! Code Health <https://www.mediawiki.org/wiki/Code_Health> [8] Objectives JR Branaa, Engineering Manager of the Quality & Test team, is leading the charge on the goal of setting Code Health Objectives (CHOs) with all engineering teams in the organization by the end of the fiscal year. The CHOs are the result of an evolution of learning and understanding wrangled by JR and the Code Health Group <https://www.mediawiki.org/wiki/Code_Health_Group> [9] over the past couple of years, and in order to make tracks on what has been surfaced, a KR was created to “educate teams on and define Code Health Objectives for our production-deployed code repositories (20% in Q1, 40% in Q2, 70% in Q3, 100% in Q4) and report and iterate on them with code owners on a quarterly basis.” These CHOs will be different for every team and are mainly crafted to address areas including technical debt, test coverage, bug backlogs, documentation, and code complexity, among others. JR deserves a huge THANK YOU for his persistence in holding and driving this important body of work! Appreciation also goes out to the teams that have already been working on their CHOs with JR: Platform Engineering Team, Search Platform and Parsing, which represent nearly 20% of our production code bases. But wait, there’s more! Part of the code health initiative includes education, and you can learn about opportunities in Q2 for you to join workshops on a sign-up basis. The next one is scheduled for October 22 and will be on Test Driven Development <https://www.mediawiki.org/w/index.php?title=Code_Health_Group/projects/DevEd/Workshops#Test_Driven_Development_Bowling_Kata_Workshop> [10] with training delivered by Zbyszko Papierski from Search Platform. Call to Help with Errors in Production <https://lists.wikimedia.org/pipermail/wikitech-l/2020-September/093833.html> [2] Tyler Cipriani, Engineering Manager for the Release Engineering team, put a call out asking for help to reduce the number of errors showing up in production. If you haven’t done so yet, take a look at the Phab board <https://phabricator.wikimedia.org/tag/wikimedia-production-error/> [11] where these errors are captured as tasks and see if there are any that your team needs to or can address. Tyler noted that the number of new tasks tagged as “Production Error” are outpacing the number of tasks closed in a given week, which is clearly not sustainable. Please give Release Engineering a hand in wrangling these! And, if you’re not quite sure where to start, and/or digging into logs is a challenge for you, check out this presentation from Timo on How to Logstash <https://drive.google.com/file/d/1WzvqRNEpg9ndTq9RLHSzCH43vAOiQnUn/view> [12] for some guidance. A New Focus on an Old Problem: Local Development Environments Vagrant? Docker? Bare metal Mediawiki installs? Get involved in the evolution of local development environment improvements. Release Engineering has put a stake in the ground with a new local-dev mailing list <https://lists.wikimedia.org/mailman/listinfo/local-dev> [3] and Local Dev Updates wiki page <https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Local_Dev_Updates> [4] dedicated to updating and improving our local development environments, with a particular focus on Docker. In addition to being informed through these channels, you are invited to share your thoughts and contributions regarding this critical and complex tooling. Instead of giving in to your struggles and frustrations, step up and help make it better! Thanks to Jeena Huneidi and Brennen Bearnes on the Rel Eng team for spearheading the effort, and a special shout out to Adam Shoreland for his pioneering work on Mediawiki-Docker. Phabricator Video Tutorial Series Debut <https://commons.wikimedia.org/wiki/Category:Phabricator_Tutorial_Series> [5] Have you ever wondered who this mysterious “AKlapper” subscriber on every Phabricator ticket is? Why, it’s Developer Advocate Andre Klapper, of course! Without his tireless attention to Phab and it’s constant influx of tickets, well, we would have only chaos. In addition to scanning an insane number of tickets on a daily basis, Andre also works hard to educate users on how to use the system through the Phab Help Page <https://www.mediawiki.org/wiki/Phabricator/Help> [13] where folks can get answers to their questions <https://www.mediawiki.org/wiki/Talk:Phabricator/Help> [14], live Q&A sessions every other Tuesday for real-time help (look on the WMF Staff Calendar for these), and now, most recently, a set of tutorial videos <https://commons.wikimedia.org/wiki/Category:Phabricator_Tutorial_Series> [15] covering the basic concepts of Phabricator. Thank you, Andre, for maintaining order and consistency and teaching us all how to follow in your giant footsteps! Bits and Bobs Other informative things you may have missed: - CI now updates your deployment-charts <https://phabricator.wikimedia.org/phame/post/view/208/ci_now_updates_your_deployment-charts/> [6], a blog post by Jeena Huneidi from Release Engineering detailing “Another reason to migrate your service to the pipeline!” - Production Excellence #23: July & August 2020 <https://phabricator.wikimedia.org/phame/post/view/204/production_excellence_23_july_august_2020/> [16], Timo Tijof’s monthly summary of incidents, trends, and appreciations focused on striving for operational excellence. - Phabricator Monthly Statistics <https://lists.wikimedia.org/pipermail/wikitech-l/2020-September/093802.html> [17], an automated email from Phab Rick Aytor detailing the month’s Phabricator stats, including accounts created, tasks created, task statuses and other interesting productivity indicators. If you would like anything added to this roundup report in the future, please send your suggestions to Erika Bjune (ebjune(a)wikimedia.org). Thank you all for your hard work and attention to improving the effectiveness of our engineering foundations! [1] https://www.mediawiki.org/wiki/GitLab_consultation [2] https://lists.wikimedia.org/pipermail/wikitech-l/2020-September/093833.html [3] https://lists.wikimedia.org/mailman/listinfo/local-dev [4] https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Local_Dev… [5] https://commons.wikimedia.org/wiki/Category:Phabricator_Tutorial_Series [6] https://phabricator.wikimedia.org/phame/post/view/208/ci_now_updates_your_d… [7] https://www.mediawiki.org/wiki/Developer_Satisfaction_Survey/2020 [8] https://www.mediawiki.org/wiki/Code_Health [9] https://www.mediawiki.org/wiki/Code_Health_Group [10] https://www.mediawiki.org/w/index.php?title=Code_Health_Group/projects/DevE… [11] https://phabricator.wikimedia.org/tag/wikimedia-production-error/ [12] https://drive.google.com/file/d/1WzvqRNEpg9ndTq9RLHSzCH43vAOiQnUn/view [13] https://www.mediawiki.org/wiki/Phabricator/Help [14] https://www.mediawiki.org/wiki/Talk:Phabricator/Help [15] https://commons.wikimedia.org/wiki/Category:Phabricator_Tutorial_Series [16] https://phabricator.wikimedia.org/phame/post/view/204/production_excellence… [17] https://lists.wikimedia.org/pipermail/wikitech-l/2020-September/093802.html *Erika Bjune *(she/her) VP of Engineering Wikimedia Foundation <https://wikimediafoundation.org/>