Hello,
Running all tests for MediaWiki and matching what CI/Jenkins is running has been a constant challenge for everyone, myself included. Today I am introducing Quibble, a python script that clone MediaWiki, set it up and run test commands.
It is a follow up to the Vienna Hackathon in 2017. We had a lot of discussion to make the CI jobs reproducible on a local machine and to unify the logic at a single place. Today, I have added a few jobs to mediawiki/core.
An immediate advantage is that they run in Docker containers and will start running as soon as an execution slot is available. That will be faster than the old jobs (suffixed with -jessie) that had to wait for a virtual machine to be made available.
A second advantage, is one can exactly reproduce the build on a local computer and even hack code for a fix up.
The setup guide is available from the source repository (integration/quibble.git): https://gerrit.wikimedia.org/r/plugins/gitiles/integration/quibble/
The minimal example would be:
git clone https://gerrit.wikimedia.org/r/p/integration/quibble cd quibble python3 -m pip install -e . quibble
A few more details are available in this post on the QA list: https://lists.wikimedia.org/pipermail/qa/2018-April/002699.html
Please give it a try and send issues, support requests to Phabricator: https://phabricator.wikimedia.org/tag/quibble/
Next week I will polish up support for MediaWiki extensions and skins and eventually Quibble will take over most of the CI jobs running for MediaWiki related projects.