Hello,
I have published a draft RFC about testing MediaWiki core and the extensions all together in a single job. As a first step limited to the extensions deployed on the Wikimedia cluster.
That would let us catch tricky dependencies such as an incompatible change in Mantle breaking Flow and MobileFrontend. Currently, a change made to Mantle does not run the tests of other extensions depending on it. The RFC aims to solve it
From the RFC:
We will first present how Zuul establish states of repositories for a given patchset, then the utility that makes it trivial to reproduce such a state on a Jenkins slave taking for example the mediawiki/core and mediawiki/vendor tight integration that is being used today. Finally we will propose to extend such system to all MediaWiki extensions deployed on the Wikimedia cluster.
The link:
https://www.mediawiki.org/wiki/RFC/Extensions_continuous_integration