Antoine Musso <hashar+wmf(a)free.fr> wrote:
Earlier today I have slightly changed our review /
test workflow on
mediawiki/core.git . That will let us do more tests and scale things in
the long term.
The new workflow is described on the wiki (including a
flowchart):
How it works?
Whenever a patch is submitted, Jenkins will attempt to
merge the
patchset against the latest master and abort whenever it fails asking
for the submitter to rebase the patchset.
Jenkins will then verify the PHP files are valid and
run JSHint for the
Javascript files.
If everything is fine, jenkins-bot will vote
Code-Review +1 to let
everyone know that the change look fine. Else, it will vote Verified -1
preventing the patchset from being submitted.
That is all. The slow unit tests are no more run on
patchset submission.
[...]
I'm with Erik on this one who posted on the talk page in
March:
| I understand the concern, but if we're creating a world
| where tests have to wait for developer review, we're doing
| CI the wrong way around. The whole point of automated
| testing is to minimize the need for human review, so we
| should aim to run as many tests as possible as early as
| possible. Both test performance and security considerations
| are problems to be gradually resolved in aiming for highest
| possible test execution for all code that gets submitted,
| and minimizing the need for human review on code which is
| obviously broken.
Why not just add (more) slaves? Computing power is much
cheaper than developer time.
Tim