Hi there,

I've been playing this morning with gitlab.w.o and CI, and here is my
proposal on how to work with it:

* have all gitlab-ci related stuff consolidated into a single repository
[0], this includes:
** generic gitlab-ci.yaml config files as required [1]
** Dockerfiles for the images used in the above gitlab-ci.yaml files [2]

* in each repo that requires CI (likely all), instead of having to
re-write the gitlab-ci.yaml file every time, "include" it from the repo
above. [3]

* due to upstream docker registry ratelimits, we need to do some heavy
caching in our docker registry (docker-registry.tools.wmflabs.org),
which even involves scp the base docker image from ones laptop
(something like [4]) because you can't even pull the base images at
docker.io from tools-docker-imagebuilder-01.

To see a live demo/example of this, here is a successful tox job for a
python 3.9 repository:
https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-framework-api/-/jobs/11682

I prefer if we have our own CI/CD stuff in a separate repo/docker
registry for now. I don't think there is a unified effort for this
unlike in gerrit.

I think this kind of CI stuff is one of the main missing pieces that was
previously preventing us from adopting gitlab for good.

PD: there are a bunch of things to automate here, like base image
maintenance and such. Will wait to see if this proposed workflow is
something we're interested in.

regards.

[0] https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci
[1]
https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci/-/blob/main/py3.9-buster-tox/gitlab-ci.yaml
[2]
https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci/-/blob/main/py3.9-buster-tox/Dockerfile
[3]
https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-framework-api/-/blob/main/.gitlab-ci.yml
[4]
https://stackoverflow.com/questions/23935141/how-to-copy-docker-images-from-one-host-to-another-without-using-a-repository

--
Arturo Borrero Gonzalez
Site Reliability Engineer
Wikimedia Cloud Services
Wikimedia Foundation
_______________________________________________
Cloud-admin mailing list -- cloud-admin@lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/