Antoine Musso hashar+wmf@free.fr wrote:
Earlier today I have slightly changed our review / test workflow on mediawiki/core.git . That will let us do more tests and scale things in the long term.
The new workflow is described on the wiki (including a flowchart):
https://www.mediawiki.org/wiki/Continuous_integration/Workflow
How it works?
Whenever a patch is submitted, Jenkins will attempt to merge the patchset against the latest master and abort whenever it fails asking for the submitter to rebase the patchset.
Jenkins will then verify the PHP files are valid and run JSHint for the Javascript files.
If everything is fine, jenkins-bot will vote Code-Review +1 to let everyone know that the change look fine. Else, it will vote Verified -1 preventing the patchset from being submitted.
That is all. The slow unit tests are no more run on patchset submission.
[...]
I'm with Erik on this one who posted on the talk page in March:
| I understand the concern, but if we're creating a world | where tests have to wait for developer review, we're doing | CI the wrong way around. The whole point of automated | testing is to minimize the need for human review, so we | should aim to run as many tests as possible as early as | possible. Both test performance and security considerations | are problems to be gradually resolved in aiming for highest | possible test execution for all code that gets submitted, | and minimizing the need for human review on code which is | obviously broken.
Why not just add (more) slaves? Computing power is much cheaper than developer time.
Tim