I've asked previously though haven't got much of an answer, are we going to upset people by moving away from github? Particularly PAWS people?
What would the shared CI stuff look like? I'll describe more on Thursday but PAWS is now setup to do pretty much everything on a PR. The main reusable thing is the container build/push bits. Would we be trying to coalesce around a standard way of doing that? Along with other shared checking or the like?
On Fri, Mar 25, 2022 at 9:45 AM Arturo Borrero Gonzalez < aborrero@wikimedia.org> wrote:
Hi there,
I've been playing this morning with gitlab.w.o and CI, and here is my proposal on how to work with it:
- have all gitlab-ci related stuff consolidated into a single repository
[0], this includes: ** generic gitlab-ci.yaml config files as required [1] ** Dockerfiles for the images used in the above gitlab-ci.yaml files [2]
- in each repo that requires CI (likely all), instead of having to
re-write the gitlab-ci.yaml file every time, "include" it from the repo above. [3]
- due to upstream docker registry ratelimits, we need to do some heavy
caching in our docker registry (docker-registry.tools.wmflabs.org), which even involves scp the base docker image from ones laptop (something like [4]) because you can't even pull the base images at docker.io from tools-docker-imagebuilder-01.
To see a live demo/example of this, here is a successful tox job for a python 3.9 repository:
https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-framework-api/-/jobs...
I prefer if we have our own CI/CD stuff in a separate repo/docker registry for now. I don't think there is a unified effort for this unlike in gerrit.
I think this kind of CI stuff is one of the main missing pieces that was previously preventing us from adopting gitlab for good.
PD: there are a bunch of things to automate here, like base image maintenance and such. Will wait to see if this proposed workflow is something we're interested in.
regards.
[0] https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci [1]
https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci/-/blob/main/py3.9-bu... [2]
https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci/-/blob/main/py3.9-bu... [3]
https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-framework-api/-/blob... [4]
https://stackoverflow.com/questions/23935141/how-to-copy-docker-images-from-...
-- Arturo Borrero Gonzalez Site Reliability Engineer Wikimedia Cloud Services Wikimedia Foundation _______________________________________________ Cloud-admin mailing list -- cloud-admin@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/