XZise added a comment.
In fact there are also tests which don't need a site at all. Those together with the
static sites could run the tests only once per version.
TASK DETAIL
https://phabricator.wikimedia.org/T70671
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign
<username>.
To: XZise
Cc: pywikipedia-bugs, XZise, jayvdb