First, apologies for not announcing this last week. A short work week coupled with a new fiscal year delayed this until today.
tl;dr: Wikimedia will be moving to a self-hosted (in our datacenter(s)) GitLab Community Edition (CE) installation for both code review and continuous integration (CI).
Longer: We are making a change to our code review and continuous integration (CI) systems in response to a complex set of inputs (Developer Satisfaction Survey[0], passing comments/critiques, an evaluation of replacement continuous integration infrastructure[1], feedback from leaders in the Foundation, etc) and evaluation conversations with Wikimedia Technology department leadership (eg CTO, VPs) and representatives from Wikimedia SRE, Architecture, Core Platform, Product, Security, and Technical Engagement teams. In those conversations with Technology department leadership, coordinated by our CTO Grant Ingersoll, we determined that an RFC was not needed for this decision[2].
We plan to replace Gerrit and Zuul+Jenkins (the software that powers our CI system) with GitLab. We hope that making a move to GitLab now will address most of the concerns and desires that are able to be addressed via software alone.
We join a growing list of other free/open knowledge organizations using a self-hosted GitLab installation and look forward to the added benefits of working together with them.
The project portal page lives at: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/GitLab It is a living documentation portal and we will continue to update it as we go. The Talk: page is also available for your feedback and questions (and is already being used). We hope everyone will join in to make this transition as successful as possible.
Notably, I would like to point people to the conversation[3] about evaluating pull-request style workflows and finding a recommended workflow for our projects. While pull-request workflows are much more common in the wider software development world{{cn}} we would like to provide as much guidance as reasonable so that we are using our tools to the best of their ability.
Here is the list of stakeholders as we have them now. In a RACI[4] model these would be Consulted. These are the people the Wikimedia Release Engineering team will be consulting with most closely as they get this going. * SRE/Service Ops - Mark and delegate(s) * Security - Chase P and Scott B * Core Platform Team - TBD * Technical Engagement - TBD * Product - Daniel C
“TBD” means we have asked for a representative and we’re waiting to hear back on confirmation. We also have a few other non-WMF groups that we have already reached out to or will be shortly to include in that list; feel free to ping me with other suggestions.
What does this mean for you as you do your work today? Right now, nothing. But as we set up the new GitLab installation we will be looking for early adopters to give us feedback as we work on improvements. As we start to migrate more users and repositories we will strive to help everyone as much as possible and reasonable. It should go without saying that this includes completely volunteer projects using the shared infrastructure.
The full timeline and plan will be posted to the above project portal page.
For the avoidance of doubt, this does not impact issue/task management; that will remain in Phabricator.
Thank you,
Greg
PS: Please let me know where else this announcement should be sent.
[0] https://www.mediawiki.org/wiki/Developer_Satisfaction_Survey/2020 [1] https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/CI_Futures... [2] see also: https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Charter#Areas_w... [3] https://www.mediawiki.org/wiki/Topic:Vpbwawb4lkdy89ym [4] https://en.wikipedia.org/wiki/Responsibility_assignment_matrix
I feel positive about every aspect of this announcement. I've enjoyed my own experiments with GitLab and its integrated CI. It's a huge relief that we'll be able to move towards a feature-branch strategy.
Even as a paid developer with 8 years of wiki experience, Gerrit is still an obstacle to my work and a challenge to use "correctly". Similarly, tinkering with our custom CI framework has been fun and rewarding, but I understand it's less fun for the small circle of people doing the daily maintenance.
The only detail which makes me nervous is that GitLab seems to lack diversity on their Board and upper management, so they may be prone to diseases endemic to Silicon Valley such as sudden corporate takeover or a shift to more aggressive profiteering. Hopefully self-hosting will insulate ourselves from this kind of turmoil, or at least gives us a few years of runway to migrate away as needed.
Kind regards, Adam W.
On 7/6/20 7:39 PM, Greg Grossmeier wrote:
First, apologies for not announcing this last week. A short work week coupled with a new fiscal year delayed this until today.
tl;dr: Wikimedia will be moving to a self-hosted (in our datacenter(s)) GitLab Community Edition (CE) installation for both code review and continuous integration (CI).
Longer: We are making a change to our code review and continuous integration (CI) systems in response to a complex set of inputs (Developer Satisfaction Survey[0], passing comments/critiques, an evaluation of replacement continuous integration infrastructure[1], feedback from leaders in the Foundation, etc) and evaluation conversations with Wikimedia Technology department leadership (eg CTO, VPs) and representatives from Wikimedia SRE, Architecture, Core Platform, Product, Security, and Technical Engagement teams. In those conversations with Technology department leadership, coordinated by our CTO Grant Ingersoll, we determined that an RFC was not needed for this decision[2].
We plan to replace Gerrit and Zuul+Jenkins (the software that powers our CI system) with GitLab. We hope that making a move to GitLab now will address most of the concerns and desires that are able to be addressed via software alone.
We join a growing list of other free/open knowledge organizations using a self-hosted GitLab installation and look forward to the added benefits of working together with them.
The project portal page lives at: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/GitLab It is a living documentation portal and we will continue to update it as we go. The Talk: page is also available for your feedback and questions (and is already being used). We hope everyone will join in to make this transition as successful as possible.
Notably, I would like to point people to the conversation[3] about evaluating pull-request style workflows and finding a recommended workflow for our projects. While pull-request workflows are much more common in the wider software development world{{cn}} we would like to provide as much guidance as reasonable so that we are using our tools to the best of their ability.
Here is the list of stakeholders as we have them now. In a RACI[4] model these would be Consulted. These are the people the Wikimedia Release Engineering team will be consulting with most closely as they get this going.
- SRE/Service Ops - Mark and delegate(s)
- Security - Chase P and Scott B
- Core Platform Team - TBD
- Technical Engagement - TBD
- Product - Daniel C
“TBD” means we have asked for a representative and we’re waiting to hear back on confirmation. We also have a few other non-WMF groups that we have already reached out to or will be shortly to include in that list; feel free to ping me with other suggestions.
What does this mean for you as you do your work today? Right now, nothing. But as we set up the new GitLab installation we will be looking for early adopters to give us feedback as we work on improvements. As we start to migrate more users and repositories we will strive to help everyone as much as possible and reasonable. It should go without saying that this includes completely volunteer projects using the shared infrastructure.
The full timeline and plan will be posted to the above project portal page.
For the avoidance of doubt, this does not impact issue/task management; that will remain in Phabricator.
Thank you,
Greg
PS: Please let me know where else this announcement should be sent.
[0] https://www.mediawiki.org/wiki/Developer_Satisfaction_Survey/2020 [1] https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/CI_Futures... [2] see also: https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Charter#Areas_w... [3] https://www.mediawiki.org/wiki/Topic:Vpbwawb4lkdy89ym [4] https://en.wikipedia.org/wiki/Responsibility_assignment_matrix _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hi Gregg,
Thanks for the details.
PS: Please let me know where else this announcement should be sent.
I think it would be a good idea to send this to the Wikitech Ambassadors wikitech-ambassadors@lists.wikimedia.org list.
If not already done, mentioning in Tech News [https://meta.m.wikimedia.org/wiki/Tech/News] would be a good thing too.
Hi Greg,
On 06-07-2020 19:39, Greg Grossmeier wrote:
First, apologies for not announcing this last week. A short work week coupled with a new fiscal year delayed this until today.
tl;dr: Wikimedia will be moving to a self-hosted (in our datacenter(s)) GitLab Community Edition (CE) installation for both code review and continuous integration (CI).
tl;dr: WMF decides to do a major change without any community consultation. Community members are upset. More at https://www.mediawiki.org/wiki/Topic:Vpbt50rwxgb2r6qn
Maarten
wikitech-l@lists.wikimedia.org