On 03/25 10:03, Vivian Rook wrote:
I've asked previously though haven't got much
of an answer, are we going to
upset people by moving away from github? Particularly PAWS people?
We can always ask, though I would wait until we have a clear idea and a tested
process/setup.
What would the shared CI stuff look like? I'll describe more on Thursday
but PAWS is now setup to do pretty much everything on a PR. The main
reusable thing is the container build/push bits.
So for what I'm seeing (and guessing) the idea would be to have some skeleton gitlab
ci tasks to do some common stuff
(run tox, run npm ci, docker build, push to toolforge, ...) and then include each of them
from the project's gitlab ci,
overriding whatever specifics the project needs.
Would we be trying to
coalesce around a standard way of doing that? Along with other shared
checking or the like?
I'd say that that would be the biggest benefit, allowing reusage and easing
maintenance.
On Fri, Mar 25, 2022 at 9:45 AM Arturo Borrero Gonzalez <
aborrero(a)wikimedia.org> wrote:
Hi there,
I've been playing this morning with gitlab.w.o and CI, and here is my
proposal on how to work with it:
* have all gitlab-ci related stuff consolidated into a single repository
[0], this includes:
** generic gitlab-ci.yaml config files as required [1]
** Dockerfiles for the images used in the above gitlab-ci.yaml files [2]
* in each repo that requires CI (likely all), instead of having to
re-write the gitlab-ci.yaml file every time, "include" it from the repo
above. [3]
* due to upstream docker registry ratelimits, we need to do some heavy
caching in our docker registry (
docker-registry.tools.wmflabs.org),
which even involves scp the base docker image from ones laptop
(something like [4]) because you can't even pull the base images at
docker.io from tools-docker-imagebuilder-01.
To see a live demo/example of this, here is a successful tox job for a
python 3.9 repository:
https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-framework-api/-/job…
I prefer if we have our own CI/CD stuff in a separate repo/docker
registry for now. I don't think there is a unified effort for this
unlike in gerrit.
I think this kind of CI stuff is one of the main missing pieces that was
previously preventing us from adopting gitlab for good.
PD: there are a bunch of things to automate here, like base image
maintenance and such. Will wait to see if this proposed workflow is
something we're interested in.
regards.
[0]
https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci
[1]
https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci/-/blob/main/py3.9-b…
[2]
https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci/-/blob/main/py3.9-b…
[3]
https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-framework-api/-/blo…
[4]
https://stackoverflow.com/questions/23935141/how-to-copy-docker-images-from…
--
Arturo Borrero Gonzalez
Site Reliability Engineer
Wikimedia Cloud Services
Wikimedia Foundation
_______________________________________________
Cloud-admin mailing list -- cloud-admin(a)lists.wikimedia.org
List information:
https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/
--
*Vivian Rook (They/Them)*
Site Reliability Engineer
Wikimedia Foundation <https://wikimediafoundation.org/>
_______________________________________________
Cloud-admin mailing list -- cloud-admin(a)lists.wikimedia.org
List information:
https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/
--
David Caro
SRE - Cloud Services
Wikimedia Foundation <https://wikimediafoundation.org/>
PGP Signature: 7180 83A2 AC8B 314F B4CE 1171 4071 C7E1 D262 69C3
"Imagine a world in which every single human being can freely share in the
sum of all knowledge. That's our commitment."