tl;dr below, frustration ahead.
I've been trying to get Jenkins to run the parsertests of LabeledSectionTransclusion, but this has been an incredibly frustrating experience. It shouldn't be too hard, right? Well, read on...
I thought I'd start with the existing 'testextensions' jenkins job [1], which is failing. The reason it is failing is because of this:
*12:02:56* + php tests/phpunit/phpunit.php --log-junit junit-phpunit-ext.xml -- extensions/LabeledSectionTransclusion*12:02:56* Unexpected non-MediaWiki exception encountered, of type "PHPUnit_Framework_Exception"*12:02:56* exception 'PHPUnit_Framework_Exception' with message 'Neither "extensions/LabeledSectionTransclusion.php" nor "extensions/LabeledSectionTransclusion.php" could be opened.' in /usr/share/php/PHPUnit/Util/Skeleton/Test.php:100
After installing PHPUnit (on itself quite a frustrating experience), I get this response instead:
valhallasw@lisilwen:~/src/core$ php tests/phpunit/phpunit.php extensions/LabeledSectionTransclusion PHPUnit 3.7.10 by Sebastian Bergmann.
Configuration read from /home/valhallasw/src/core/tests/phpunit/suite.xml
Time: 0 seconds, Memory: 18.50Mb
No tests executed!
Which suggests it might be a problem with the Jenkins configuration. Of course, the tests are also not running, but this is a second issue.
So, where is the Jenkins configuration? Not in Jenkins itself. After logging in in Jenkins, there /is/ suddenly a new option 'Workspace' (why is that disabled for logged-out users?), which shows the extension is checked out in the workspace - so that doesn't explain why phpunit is unable to locate the tests.
So let's check the configuration. [2] suggests it should be at [3]. Ugh, gitweb, and no github mirror. Under 'jobs', there are some extensions, but not LST. There is MediaWiki-Tests-Extensions, which has some XML files that, at least for me, are incomprehensible. And are they even relevant to this? In any case, they don't tell me why PHPUnit is unable to find the LST directory, even though it's right there [4]. I give up.
By the way - I just realized why the job is not in the Jenkins repository:
valhallasw@lisilwen:~/src/jenkins$ grep -R * -e 'mwext'
jobs/.gitignore:/mwext-*
*Really?*
Okay, so then how do I get PHPUnit to run the parsertests? There is the NewParserTest class, so I was guessing adding this to LabeledSectionTransclusion.php would work:
class LSTParserTest extends NewParserTest {
function __construct()
{
parent::__construct();
$this->setParserTestFile(__DIR__ . "/lstParserTests.txt");
}
};
Well, that does, er, absolutely nothing. [5] suggests I should move it to a seperate file and add the following:
$wgHooks['UnitTestsList'][] = 'efFruitsRegisterUnitTests';function efFruitsRegisterUnitTests( &$files ) { $testDir = dirname( __FILE__ ) . '/'; $files[] = $testDir . 'AppleTest.php'; return true;}
what it *doesn't* tell you is that this is actually irrelevant when you run php tests/phpunit/phpunit.php extensions/LabeledSectionTransclusion - and it's not really clear to me for when it is relevant. After some struggling, apparently it detects tests because the filename *ends in Test.php* (which is documented, well, no-where. After moving the test case there, it still didn't quite work, but after some more fiddling, something is working!
Okay, so now I have a test case that actually runs when I call the above command. So, let's push it for review [6] - but that doesn't actually make Jenkins run the test cases.... so I still don't know whether this actually fixes it.
[1] https://integration.mediawiki.org/ci/job/mwext-LabeledSectionTransclusion-te... [2] http://www.mediawiki.org/wiki/Continuous_integration/Jenkins [3] https://gerrit.wikimedia.org/r/gitweb?p=integration/jenkins.git;a=tree [4] https://integration.mediawiki.org/ci/job/mwext-LabeledSectionTransclusion-te... [5] http://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Writing_unit_tests_for...
So, tl,dr: debugging jenkins failures and getting phpunit to work is a frustrating experience, because of a lack of documentation and a lack of 'obviousness' (there is not one 'obvious' way to do it). I've updated [5] with my experiences, but I still have no idea how the magic that is Jenkins is configured and how it works. It would be great if there would be better documentation on that.
Some specific things that I think should be fixed: - Jenkins should not show new (read-only) options to users - make them public instead - The Jenkins configuration should also be mirrored on github - The mwext jobs should be in git (preferrably, because then it's easy to check them on-line), *or* there should be clear documentation on how to generate those jobs... - There should be documentation on how the configuration is actually used - and preferrably on how to debug failures.
Thanks.
Merlijn