Le 10/02/2016 22:06, Greg Grossmeier a écrit :
Hello all,
I want to apologize for the current slowness in our CI stack. Everything
is working fine in the sense that nothing is broken.
We simply did not adequately prepare for the switch to php5.5 being the
new minimum version for MediaWiki. This resulted in a large influx of
jobs on a set of our Jenkins executers that was not a large enough pool.
Timo and Antoine have created more executers to help mitigate [0].
More explicitly: due to package version availability in Ubuntu, our
php5.3 jobs run on our pool of Ubuntu Precis hosts, while the
php5.5/hhvm jobs run on our pool of Ubuntu Trusty hosts.
Both sets of jobs are in our pipeline to migrate to the new Nodepool
system that allows burstable availability of executers.
Again, apologies for the delays but know that the jobs will complete
just not as fast as you are accustomed to.
Greg
[0]
https://phabricator.wikimedia.org/T126423
Hello,
As a follow, yesterday we created new instance turns out 2GB memory is
not enough to run some of the MediaWiki jobs. Since yesterday you might
have encountered errors such as:
* Access denied for user 'jenkins_mw0'@'localhost'
* Disk I/O Error
* File that went away mysteriously
* Long delays in jobs
etc
A few minutes ago I have deleted all those 2GB slaves and spawned bigger
ones. Provisioning is ongoing as I am writing. Will get them pooled in
a couple/three hours.
I do not have proof: but PHP 5.5 on Trusty might have its share of issue
as well when stress tests with our test suites. After all we never
massively ran tests against 5.5 ;-}
cheers and be careful!
--
Antoine "hashar" Musso