Hello,
We need to upgrade the operating system on the server hosting the CI systems (Zuul, Jenkins). We will switch the services from contint1001 (eqiad) to contint2001 (codfw) which has been upgraded to Buster.
The maintenance is scheduled for Monday May 11st 14:00 CEST (12:00 UTC), just after the European SWAT window.
The maintenance window is TWO HOURS, but probably the switch will be complete in a shorter time frame.
During the maintenance, CI will not be available: no tests will be run and events (code-review +2, rebase, new patchsets) will not be captured.
The beta cluster will NOT be automatically updated.
If a change has to happen on the configuration repositories:
* operations/puppet : locally run: bundle update && bundle exec rake test * operations/mediawiki-config : locally run: composer install && composer test
Then manually V+2 and Submit the change.
After the maintenance, Antoine will retrigger most events.
The related changes are:
https://gerrit.wikimedia.org/r/c/operations/dns/+/594480 https://gerrit.wikimedia.org/r/c/operations/puppet/+/594477
The related ticket is: https://phabricator.wikimedia.org/T224591
Synchronization will happen on IRC in #wikimedia-operations
Later we will switch back from contint2001 to contint1001. The goal is to get rid of jessie servers and run fully on buster.
cheers,
On Thu, May 7, 2020 at 5:20 PM Antoine Musso hashar@free.fr wrote: [CUT]
If a change has to happen on the configuration repositories:
- operations/puppet : locally run: bundle update && bundle exec rake test
If you have docker installed, you can also run the script $puppet_dir/utils/run_ci_locally.sh
that, unsurprisingly, runs the same tests as CI does, in the same container.
Cheers, Giuseppe
Hello,
We tried the upgrade and unfortunately had to rollback.
The operation itself went smoothly (rsync, puppet, dns) but we have hit an issue with Jenkins no more being able to allocate an instance to run jobs on.
I suspect it might be related to the java upgrade (8 to 11). I will try to find the root cause and we will schedule a new maintenance window.
For patches missing a CI result, you can retrigger it by commenting in Gerrit "recheck" or for changes that had Code-Review +2, remove your vote and apply it again to trigger the test and merge.
Sorry for the inconvenience.
wikitech-l@lists.wikimedia.org