Hello,
After much hassle with Jenkins, ant and PHPUnit, I finally managed to run an extension PHPUnit test suite under Jenkins.
The winner is TitleBlacklist for which results are available via: https://integration.mediawiki.org/ci/job/Ext-TitleBlacklist/
The Jenkins job is setup to report the build status back in Gerrit.
If you see any issue, please fill in a bug under Testing infrastructure.
TODO: Job does not run PHP lint yet.
Are there any plans to move more extensions (or at least those with unit tests) under Jenkins? Because I know one of my extension (TwoFactorAuthentication) has unit tests.
*--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerromeo@gmail.com
On Wed, Aug 29, 2012 at 10:18 AM, Antoine Musso hashar+wmf@free.fr wrote:
Hello,
After much hassle with Jenkins, ant and PHPUnit, I finally managed to run an extension PHPUnit test suite under Jenkins.
The winner is TitleBlacklist for which results are available via: https://integration.mediawiki.org/ci/job/Ext-TitleBlacklist/
The Jenkins job is setup to report the build status back in Gerrit.
If you see any issue, please fill in a bug under Testing infrastructure.
TODO: Job does not run PHP lint yet.
-- Antoine "hashar" Musso
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Le 29/08/12 16:24, Tyler Romeo a écrit :
Are there any plans to move more extensions (or at least those with unit tests) under Jenkins? Because I know one of my extension (TwoFactorAuthentication) has unit tests.
Ultimately, all extensions hosted on WMF git server will be integrated in Jenkins. If you get PHPUnit test, I will add a job for it as soon as the current jobs are stable enough.
Ultimately, all extensions hosted on WMF git server will be integrated in Jenkins. If you get PHPUnit test, I will add a job for it as soon as the current jobs are stable enough.
Awesome!
*--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerromeo@gmail.com
On Wed, Aug 29, 2012 at 10:47 AM, Antoine Musso hashar+wmf@free.fr wrote:
Le 29/08/12 16:24, Tyler Romeo a écrit :
Are there any plans to move more extensions (or at least those with unit tests) under Jenkins? Because I know one of my extension (TwoFactorAuthentication) has unit tests.
Ultimately, all extensions hosted on WMF git server will be integrated in Jenkins. If you get PHPUnit test, I will add a job for it as soon as the current jobs are stable enough.
-- Antoine "hashar" Musso
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hey,
Ultimately, all extensions hosted on WMF git server will be integrated
in Jenkins. If you get PHPUnit test, I will add a job for it as soon as the current jobs are stable enough.
Thanks for your work on this! :)
Cheers
-- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. --
On Wed, Aug 29, 2012 at 11:01 AM, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey,
Ultimately, all extensions hosted on WMF git server will be integrated
in Jenkins. If you get PHPUnit test, I will add a job for it as soon as the current jobs are stable enough.
Thanks for your work on this! :)
Indeed! I should've said so in my first e-mail.
-Chad
Ultimately, all extensions hosted on WMF git server will be integrated in Jenkins. If you get PHPUnit test, I will add a job for it as soon as the current jobs are stable enough.
At the risk of sounding unoriginal, _zomg this is amazing_.
Thanks a bunch, Antoine!
On Wed, Aug 29, 2012 at 10:18 AM, Antoine Musso hashar+wmf@free.fr wrote:
Hello,
After much hassle with Jenkins, ant and PHPUnit, I finally managed to run an extension PHPUnit test suite under Jenkins.
The winner is TitleBlacklist for which results are available via: https://integration.mediawiki.org/ci/job/Ext-TitleBlacklist/
The Jenkins job is setup to report the build status back in Gerrit.
If you see any issue, please fill in a bug under Testing infrastructure.
Question: why does the config for non-extension tests attempt to load extensions? -Parser and -Misc both seem to be failing due to a broken inclusion of Wikibase.
Core tests should be run without any extensions.
-Chad
On 29.08.2012, 18:27 Chad wrote:
Question: why does the config for non-extension tests attempt to load extensions? -Parser and -Misc both seem to be failing due to a broken inclusion of Wikibase.
Core tests should be run without any extensions.
Ideally, core tests should include as many extension tests as possible to prevent core from silently breaking extensions.
On Wed, Aug 29, 2012 at 10:53 AM, Max Semenik maxsem.wiki@gmail.com wrote:
On 29.08.2012, 18:27 Chad wrote:
Question: why does the config for non-extension tests attempt to load extensions? -Parser and -Misc both seem to be failing due to a broken inclusion of Wikibase.
Core tests should be run without any extensions.
Ideally, core tests should include as many extension tests as possible to prevent core from silently breaking extensions.
That's why we write tests for extensions. You could very easily write two extensions that produce conflicting output--core should not break because of this.
-Chad
Core tests should be run without any extensions.
Ideally, core tests should include as many extension tests as possible to prevent core from silently breaking extensions.
That's why we write tests for extensions. You could very easily write two extensions that produce conflicting output--core should not break because of this.
I think we have three different things going on here.
* Unit tests for extensions to prevent core changes from breaking extensions * Unit tests for core (with extensions enabled) to prevent core changes from breaking extensions. * Unit tests for core (with extensions enabled) to prevent conflicting extensions from breaking the core.
Le 29/08/12 16:27, Chad a écrit :
Question: why does the config for non-extension tests attempt to load extensions? -Parser and -Misc both seem to be failing due to a broken inclusion of Wikibase.
The -Parser and -Misc jobs are triggered by both the MediaWiki core job and the one testing the Wikidata branch. I originally thought it was a good idea to a job dedicated to a PHPUnit group, I will end up creating a job dedicated to testing the Wikidata branch.
Core tests should be run without any extensions.
Fully agree. We can later create a job to test core + the extension deployed on the wmf and another one for a Semantic MediaWiki setup.
I've tried to do this for translate ext last week - so here are a couple of questions: 1. is successfully runnig the test a requirement to successfully score on gerrit? (i.e. how is gerrit integrated) 2. does the extension need to include php unit?
On Thu, Aug 30, 2012 at 9:43 AM, Antoine Musso hashar+wmf@free.fr wrote:
Le 29/08/12 16:27, Chad a écrit :
Question: why does the config for non-extension tests attempt to load extensions? -Parser and -Misc both seem to be failing due to a broken inclusion of Wikibase.
The -Parser and -Misc jobs are triggered by both the MediaWiki core job and the one testing the Wikidata branch. I originally thought it was a good idea to a job dedicated to a PHPUnit group, I will end up creating a job dedicated to testing the Wikidata branch.
Core tests should be run without any extensions.
Fully agree. We can later create a job to test core + the extension deployed on the wmf and another one for a Semantic MediaWiki setup.
-- Antoine "hashar" Musso
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Le 30/08/12 10:11, Oren Bochman a écrit :
- is successfully runnig the test a requirement to successfully score on
gerrit? (i.e. how is gerrit integrated)
When a change is submitted in Gerrit a notification is sent to Jenkins. If a job is setup for the repository, it will be triggered. For extension tests the process is: - copy a snapshot of the current master version of MediaWiki - clone latest master version of the extension - apply the change - load the extension entry point (Foobar/Foobar.php by convention) - Run PHPUnit against the 'extensions' directory - Test results are aggregate and publish under Jenkins - If all tests are successful, Jenkins mark the change in Gerrit as Verified +2. Else it marks it as Verified -2
- does the extension need to include php unit?
The PHPUnit software is installed on the continuous integration server. The only thing you need in the extension is to have your tests in a PHP file ending with 'Test.php'.
The TitleBlacklist is an example.
cheers,
On Thu, Aug 30, 2012 at 1:11 AM, Oren Bochman orenbochman@gmail.com wrote:
I've tried to do this for translate ext last week - so here are a couple of questions:
- is successfully runnig the test a requirement to successfully score on
gerrit? (i.e. how is gerrit integrated)
Jenkins sets a Verified +1 or -1 on the change, depending on whether the tests succeed or not, just like for core. A V+1 is required to merge a change. So to merge any change into an extension, the tests (that is, the core tests and any extension tests) have to pass. But the extension isn't required to have tests, it's just that you can't submit changes that break the core tests or break any tests that are present in the extension. Existing extensions without any tests or awareness of phpunit will continue to work just fine, as long as they don't try to do crazy stuff that breaks a test in core.
- does the extension need to include php unit?
No, all of the phpunit wrapper stuff is in MediaWiki core. Adding tests to extensions is simple, see https://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Writing_unit_tests_fo... .
Roan
Excellent! Can't wait to get MobileFrontend using this.
Great work Antoine.
On Thu, Aug 30, 2012 at 12:59 PM, Roan Kattouw roan.kattouw@gmail.com wrote:
On Thu, Aug 30, 2012 at 1:11 AM, Oren Bochman orenbochman@gmail.com wrote:
I've tried to do this for translate ext last week - so here are a couple of questions:
- is successfully runnig the test a requirement to successfully score on
gerrit? (i.e. how is gerrit integrated)
Jenkins sets a Verified +1 or -1 on the change, depending on whether the tests succeed or not, just like for core. A V+1 is required to merge a change. So to merge any change into an extension, the tests (that is, the core tests and any extension tests) have to pass. But the extension isn't required to have tests, it's just that you can't submit changes that break the core tests or break any tests that are present in the extension. Existing extensions without any tests or awareness of phpunit will continue to work just fine, as long as they don't try to do crazy stuff that breaks a test in core.
- does the extension need to include php unit?
No, all of the phpunit wrapper stuff is in MediaWiki core. Adding tests to extensions is simple, see https://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Writing_unit_tests_fo... .
Roan
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org