Hey folks,
In case you did not see the update from Tyler already[0], both Gerrit and GitLab will stay around. The TL;dr is that there's a few repositories that must stay in Gerrit (from our perspective, most notably puppet.git), but for the rest of our repositories we're free to choose which code host we want to use. Here's a quick proposal what to do:
Our Toolforge related repositories are mostly on GitLab, and they're making heavy use of GitLab's CI features. I think keeping those there is the best option for now, and we should move Striker and labs/toollabs.git there for consistency.
The wmcs-cookbooks repo should stay in Gerrit. That repository is primarily used by SREs in conjunction with the Puppet repository which is staying in Gerrit. Similarly I think we should move the new Cloud VPS tofu-infra repository to Gerrit, as that's also used for SRE workflows and the ability to merge individual patches in a stack is useful there similar to how it is on the Puppet repository.
For metricsinfra, we should either migrate the tofu-provisioning repository from GitLab to Gerrit (which is my preference), or migrate the prometheus-* repos from Gerrit to GitLab to keep everything related to that project in one place.
Finally, I think we should move the few repositories we have canonically on GitHub to GitLab.
Thoughts? I'm happy to draft a formal decision request for my proposals, although I'm hoping this is simple and uncontroversial enough to not require one.
[0]: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/t...
Taavi
-- Taavi Väänänen (he/him) Site Reliability Engineer, Cloud Services Wikimedia Foundation
On 06/25 10:52, Taavi Väänänen wrote:
Hey folks,
In case you did not see the update from Tyler already[0], both Gerrit and GitLab will stay around. The TL;dr is that there's a few repositories that must stay in Gerrit (from our perspective, most notably puppet.git), but for the rest of our repositories we're free to choose which code host we want to use. Here's a quick proposal what to do:
Our Toolforge related repositories are mostly on GitLab, and they're making heavy use of GitLab's CI features. I think keeping those there is the best option for now, and we should move Striker and labs/toollabs.git there for consistency.
The wmcs-cookbooks repo should stay in Gerrit. That repository is primarily used by SREs in conjunction with the Puppet repository which is staying in Gerrit. Similarly I think we should move the new Cloud VPS tofu-infra repository to Gerrit, as that's also used for SRE workflows and the ability to merge individual patches in a stack is useful there similar to how it is on the Puppet repository.
I would move those to gitlab too, as they are meant to be used by toolforge roots, not only SRE (hopefully from cloudcumin at some point).
I don't see anything preventing us from doing 1 commit per MR on gitlab (if we want), or not, that's up to us more than the tooling (gitlab just gives more freedom in that respect).
The only reason I see to leave things in gerrit is if we want to make sure that only SREs have rights on them, otherwise gitlab offers us more control on how to give access to non-SREs and how to run CI jobs.
For metricsinfra, we should either migrate the tofu-provisioning repository from GitLab to Gerrit (which is my preference), or migrate the prometheus-* repos from Gerrit to GitLab to keep everything related to that project in one place.
Sounds good for me moving it to gerrit, this one is for sure only used by SREs, though I'm ok moving the others to gitlab instead (no strong preference).
Finally, I think we should move the few repositories we have canonically on GitHub to GitLab.
+1
Thoughts? I'm happy to draft a formal decision request for my proposals, although I'm hoping this is simple and uncontroversial enough to not require one.
Taavi
-- Taavi Väänänen (he/him) Site Reliability Engineer, Cloud Services Wikimedia Foundation _______________________________________________ Cloud-admin mailing list -- cloud-admin@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/
The wmcs-cookbooks repo should stay in Gerrit.
No strong preference on this one, I'm fine if it stays in Gerrit but also if it's moved.
Similarly I think we should move the new Cloud VPS tofu-infra repository to Gerrit
I think the GitLab CI would make it easier to implement automatic Tofu plans on MR, if we sort out authentication.
For metricsinfra, we should either migrate the tofu-provisioning repository from GitLab to Gerrit (which is my preference), or migrate the prometheus-* repos from Gerrit to GitLab to keep everything related to that project in one place.
I agree we should keep the project in one place, but same as above: GitLab CI might be handy for Tofu repos.
Finally, I think we should move the few repositories we have canonically on GitHub to GitLab.
Maybe this last point deserves a separate discussion. I assume we're talking about https://github.com/toolforge/ or do we have other ones? Are they "community projects" or "WMCS-managed projects"? Who are the admins of the "toolforge" GitHub org? Are the projects there using GitHub Actions and how difficult is it to migrate them to GitLab CI?
Francesco
On Tue, Jun 25, 2024 at 10:42 AM David Caro dcaro@wikimedia.org wrote:
On 06/25 10:52, Taavi Väänänen wrote:
Hey folks,
In case you did not see the update from Tyler already[0], both Gerrit and GitLab will stay around. The TL;dr is that there's a few repositories that must stay in Gerrit (from our perspective, most notably puppet.git), but for the rest of our repositories we're free to choose which code host we want to use. Here's a quick proposal what to do:
Our Toolforge related repositories are mostly on GitLab, and they're making heavy use of GitLab's CI features. I think keeping those there is the best option for now, and we should move Striker and labs/toollabs.git there for consistency.
The wmcs-cookbooks repo should stay in Gerrit. That repository is primarily used by SREs in conjunction with the Puppet repository which is staying in Gerrit. Similarly I think we should move the new Cloud VPS tofu-infra repository to Gerrit, as that's also used for SRE workflows and the ability to merge individual patches in a stack is useful there similar to how it is on the Puppet repository.
I would move those to gitlab too, as they are meant to be used by toolforge roots, not only SRE (hopefully from cloudcumin at some point).
I don't see anything preventing us from doing 1 commit per MR on gitlab (if we want), or not, that's up to us more than the tooling (gitlab just gives more freedom in that respect).
The only reason I see to leave things in gerrit is if we want to make sure that only SREs have rights on them, otherwise gitlab offers us more control on how to give access to non-SREs and how to run CI jobs.
For metricsinfra, we should either migrate the tofu-provisioning repository from GitLab to Gerrit (which is my preference), or migrate the prometheus-* repos from Gerrit to GitLab to keep everything related to that project in one place.
Sounds good for me moving it to gerrit, this one is for sure only used by SREs, though I'm ok moving the others to gitlab instead (no strong preference).
Finally, I think we should move the few repositories we have canonically on GitHub to GitLab.
+1
Thoughts? I'm happy to draft a formal decision request for my proposals, although I'm hoping this is simple and uncontroversial enough to not require one.
Taavi
-- Taavi Väänänen (he/him) Site Reliability Engineer, Cloud Services Wikimedia Foundation _______________________________________________ Cloud-admin mailing list -- cloud-admin@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/
-- David Caro SRE - Cloud Services Wikimedia Foundation https://wikimediafoundation.org/ PGP Signature: 7180 83A2 AC8B 314F B4CE 1171 4071 C7E1 D262 69C3
"Imagine a world in which every single human being can freely share in the sum of all knowledge. That's our commitment." _______________________________________________ Cloud-admin mailing list -- cloud-admin@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/
On Tue, Jun 25, 2024 at 5:19 AM Francesco Negri fnegri@wikimedia.org wrote:
Finally, I think we should move the few repositories we have canonically on GitHub to GitLab.
Maybe this last point deserves a separate discussion. I assume we're talking about https://github.com/toolforge/ or do we have other ones? Are they "community projects" or "WMCS-managed projects"?
https://github.com/toolforge/ started as a place to collect community projects, but did then become the "official" home of PAWS and Quarry when they left Yuvi's personal github account for a shared space. I think there may be a couple of additional origin repos there now as well.
Who are the admins of the "toolforge" GitHub org?
https://github.com/orgs/toolforge/people?query=role%3Aowner
Are the projects there using GitHub Actions and how difficult is it to migrate them to GitLab CI?
This sort of feels like a Rook question...
* https://github.com/toolforge/paws/tree/main/.github/workflows * https://github.com/toolforge/quarry/tree/main/.github/workflows * https://github.com/toolforge/superset-deploy/tree/main/.github/workflows * https://github.com/toolforge/tf-infra-test/tree/main/.github/workflows * https://github.com/toolforge/github-actions/tree/main/.github/workflows
Bryan
Are the projects there using GitHub Actions and how difficult is it to
migrate them to GitLab CI?
Can we run `docker build` in gitlab yet? That has been the main technical blocker in migrating. I believe there were a few other smaller things, but that one at least was a show stopper.
On Tue, Jun 25, 2024 at 1:11 PM Bryan Davis bd808@wikimedia.org wrote:
On Tue, Jun 25, 2024 at 5:19 AM Francesco Negri fnegri@wikimedia.org wrote:
Finally, I think we should move the few repositories we have canonically on GitHub to GitLab.
Maybe this last point deserves a separate discussion. I assume we're talking about https://github.com/toolforge/ or do we have other ones? Are they "community projects" or "WMCS-managed projects"?
https://github.com/toolforge/ started as a place to collect community projects, but did then become the "official" home of PAWS and Quarry when they left Yuvi's personal github account for a shared space. I think there may be a couple of additional origin repos there now as well.
Who are the admins of the "toolforge" GitHub org?
https://github.com/orgs/toolforge/people?query=role%3Aowner
Are the projects there using GitHub Actions and how difficult is it to
migrate them to GitLab CI?
This sort of feels like a Rook question...
- https://github.com/toolforge/paws/tree/main/.github/workflows
- https://github.com/toolforge/quarry/tree/main/.github/workflows
- https://github.com/toolforge/superset-deploy/tree/main/.github/workflows
- https://github.com/toolforge/tf-infra-test/tree/main/.github/workflows
- https://github.com/toolforge/github-actions/tree/main/.github/workflows
Bryan
Bryan Davis Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808 _______________________________________________ Cloud-admin mailing list -- cloud-admin@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/
On Tue, Jun 25, 2024 at 11:19 AM Vivian Rook vrook@wikimedia.org wrote:
Are the projects there using GitHub Actions and how difficult is it to migrate them to GitLab CI?
Can we run `docker build` in gitlab yet? That has been the main technical blocker in migrating. I believe there were a few other smaller things, but that one at least was a show stopper.
In https://phabricator.wikimedia.org/T357612#9706890 Jelto said that it _should_ be possible to build with Docker via the "untrusted" runners. "Untrusted" here means that they do not have the ability to push images into the prod docker registry. I would guess the next step would be for someone to try and prove if it is possible or not. I think that the worst case answer would end up being that WMCS needs to setup a dedicated runner to build the images for PAWS.
Bryan
On 06/25 11:32, Bryan Davis wrote:
On Tue, Jun 25, 2024 at 11:19 AM Vivian Rook vrook@wikimedia.org wrote:
Are the projects there using GitHub Actions and how difficult is it to migrate them to GitLab CI?
Can we run `docker build` in gitlab yet? That has been the main technical blocker in migrating. I believe there were a few other smaller things, but that one at least was a show stopper.
In https://phabricator.wikimedia.org/T357612#9706890 Jelto said that it _should_ be possible to build with Docker via the "untrusted" runners. "Untrusted" here means that they do not have the ability to push images into the prod docker registry. I would guess the next step would be for someone to try and prove if it is possible or not. I think that the worst case answer would end up being that WMCS needs to setup a dedicated runner to build the images for PAWS.
Yep, we might want to eventually have our own runner also to do things like startup lima-kilo and run functional tests, or generate the lima-kilo VM template (so you don't have to set most of the stuff up on the first run locally and download images/package/etc. on every rebuild).
Bryan
Bryan Davis Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808 _______________________________________________ Cloud-admin mailing list -- cloud-admin@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/
cloud-admin@lists.wikimedia.org