Le 15/09/2022 à 08:55, Robert Vogel via Wikitech-l a écrit :
@Hashar: Just wondering, are there any plans to change
CI configs
after moving development/code-review from gerrit to gitlab [1]? Maybe
some file living in the repo directly rather than using a centralized
repo like `integration/config`?
[1]
https://www.mediawiki.org/wiki/GitLab/Gerrit_to_GitLab
Hello,
In short it is complicated. Essentially, YES, GitLab provides a way to
define your CI jobs directly in the code repository (see Gitlab CI quick
start <https://docs.gitlab.com/ee/ci/quick_start/>).
In longer form it is even more complicated. Given I like wall of texts,
below is some historical context regarding migrating MediaWiki CI. It is
rather long form and most can probably stop reading at this point.
*Current CI*
The test infrastructure for MediaWiki is build with:
- Zuul a workflow system (X happens in Gerrit, do A, B, C tasks, report
to Gerrit)
- Jenkins freestyle jobs, which you can really think as a library of
shell scripts running shell scripts serially. The jobs are mostly
invoking commands inside containers (docker run <some args>)
- Docker images to provide the execution environment (such as the same
php version and php extensions Wikimedia uses in production)
The Zuul version we use is from 2017 and has all the CI configuration
centralized (the infamous integration/config repository) though in
practice the job/container delegates a lot of the logic to CI entry
points
<https://www.mediawiki.org/wiki/Continuous_integration/Entry_points>
such as composer test. Zuul got frozen just before a large rework by
upstream. It is around that time we:
- moved from a shared build environment to Docker images
- I wrote Quibble <https://doc.wikimedia.org/quibble/> to simplify the
execution stack (instead of a hundred of scripts scattered in three
different repositories)
*A digression about GitLab*
My grand idea at the time was to be able to trigger a Kubernetes job
with a list of parameters (provided by Zuul and consumed by Quibble) and
an execution environment (a Docker image). Then later on find out a way
to integrate with the our tool to create container image on the fly: the
novel Blubber <https://wikitech.wikimedia.org/wiki/Blubber>. The upgrade
to Zuul, phase out of Jenkins and moving the execution from WMCS to a
Kubernetes cluster, those did not happen. The foundation had other
technical priorities and to be fair we did not have the human resources
to achieve any of it.
A bit later we formed a working group for the future of CI, you can see
the requirements at
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/CI_Future…
, and a feedback request on wikitech-l
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/GOCP6D456X4JPF6T2FB6J34SW32YFFE7/>.
Different solutions got evaluated and I have required Zuul version 3 to
be included in the evaluation at least for comparison. Defining the CI
workflow in the code repository is referred to as "*self serve CI*".
That became popular with Travis CI. Zuul and Jenkins projects
implemented similar feature around 2017 as well (Zuul version 3 and
Jenkins with pipeline as code).
Proof of concepts for 3 different CI systems were build in 2019 one got
picked (Argo) and we had a nice proposal:
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Seakeeper…
. It is a good read as to what we envisioned to push.
Due to internal tensions at the foundation (others wanted to adopt
GitLab, some *really* hate Gerrit, others would rather use GitHub), our
Seakeeper proposal got shot at some point late in 2019. I was not in
any of this discussions since I had other matters to deal with and went
briefly out of job due to some Kafkaesque
<https://en.wikipedia.org/wiki/Franz_Kafka> legal situation. Eventually
Covid has hit early in 2020. In short I don't think I had the mental
state to be involved or consulted in any of those discussions (nor do I
think anyone would have thought of reaching out to the /de facto/
internal CI expert, but I am exaggerating and ranting at this point. I
apologize).
GitLab entered the foundation annual plan for July 2020 - June 2021.Â
You can read the public announce at
https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…
. There were some backslashes about lack of a public consultation (and
rightfully so) but in the end the decision had already been made. To my
knowledge all people who were actively pushing for GitLab have since
left the foundation or moved to different departments.
*Self serve CI!*
All of that to say that, in GitLab, one can define the CI workflow
directly from the repository. I think the documentation is
https://docs.gitlab.com/ee/ci/pipelines/merge_request_pipelines.html and
that system is entirely different than our current
Zuul/Jenkins/Container images. We can surely reuse the existing images
with minor adaptations, but everything else has to be migrated and most
probably rewritten from scratch (unless we get smart enough to convert
the Zuul / Jenkins jobs yaml definition to Gitlab CI pipelines).
*How are we going to migrate MediaWiki to GitLab?*
Well that is a good question, and in all transparency nobody knows.
During the GitLab consultation I have warned the GitLab community
edition we envisioned is missing critical features ( some at
https://www.mediawiki.org/wiki/Topic:Vv0c2rqyzfy1dos2 ) and internally I
and others very boldly warned about the huge cost of migration with
either or both of:
- a radical change of how we develop MediaWiki (lack of gating changes,
no cross repositories dependencies, lack of search)
- a lot of tooling on top of GitLab
For MediaWiki those discussions haven't started yet.
*Hopeful conclusion*
The conclusion is that moving the CI definitions toward the code
repositories is definitely a feature we seek. GitLab projects already
have self serve CI. Before GitLab we had achieved it for MediaWiki
services with PipelineLib/Blubber
<https://wikitech.wikimedia.org/wiki/PipelineLib> and there is work
going on right now to add it to GitLab. Once completed, that would open
the migration of MediaWiki services to GitLab. For MediaWiki the future
remains to be written though, one way or the other it would be easier
for developers to define the tests workflows.
cheers,
Antoine "hashar" Musso