[QA] Fwd: [Wikitech-l] CI and cross repository dependencies

[Željko Filipin]

Forwarding to QA list, in case somebody is subscribed to wikitech-l.

Željko

---------- Forwarded message ----------
From: Antoine Musso <hashar+wmf at free.fr>
Date: Thu, Nov 19, 2015 at 2:19 PM
Subject: [Wikitech-l] CI and cross repository dependencies
To: wikitech-l at lists.wikimedia.org


Hello,

We often have the case of a change to an extension depending on a
pending patch to MediaWiki core.  I have upgraded our CI scheduler -
Zuul - a couple weeks ago and it now supports marking dependencies even
in different repositories.


Why does it matter?  To make sure the dependency is fulfilled one
usually either:

* CR-2 the patch until dependent change is merged
* write a test that exercise the required patch in MediaWiki.

With the first solution (lack of test), once both are merged, nothing
prevent one from cherry picking a patch without its dependent patch.
For example for MediaWiki minor releases or Wikimedia deployment branches.

When a test covers the dependency, it will fail until the dependent one
is merged which is rather annoying.


Zuul now recognizes the header 'Depends-On' in git messages, similar to
'Change-Id' and 'Bug'.  'Depends-On' takes as parameter a change-id and
multiple ones can be added.

When a patch is proposed in Gerrit, Zuul looks for Gerrit changes
matching the 'Depends-On' and verify whether any are still open. In such
a case, it will craft references for the open patches so all the
dependencies can be tested as if they got merged.


Real world example
------------------

The ContentTranslation extension is tested with the Wikidata one and was
not passing the test.  Wikidata created a patch and we did not want to
merge it until we confirm the ContentTranslation one is passing properly.

The Wikidata patch is https://gerrit.wikimedia.org/r/#/c/252227/
Change-Id: I0312c23628d706deb507b5534b868480945b6163

On ContentTranslation we indicated the dependency:
https://gerrit.wikimedia.org/r/#/c/252172/1..2//COMMIT_MSG
+ Depends-On: I0312c23628d706deb507b5534b868480945b6163

Which is the Wikidata patch.


Zuul:
* received the patch for ContentTranslation
* looked up the change-id and found the Wikidata
* created git references in both repo to point to the proper patches

Jenkins:
* zuul-cloner cloned both repos and fetched the references created by
the Zuul service
* run tests
* SUCCESS

That confirmed us the Wikidata patch was actually fixing the issue for
ContentTranslation. Hence we CR+2 both and all merged fine.


Please take a moment to read upstream documentation:

http://docs.openstack.org/infra/zuul/gating.html#cross-repository-dependencies

Wikidata/ContentTranslation task:
https://phabricator.wikimedia.org/T118263


--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
Wikitech-l at lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/qa/attachments/20151119/c45c9591/attachment.html>