While the migration from Cloudbees was completed earlier this month, the MobileFrontend jobs were migrated off of Cloudbees over one month ago.
Chris McMahon sent an email announcing this on June 6 [0]. In addition, Chris said 'The tests for MF on beta labs running in headless Firefox under xvfb are reliably green as of today and we'll be working to keep them that way.'
However, those tests have been consistently failing since early June. After digging through MobileFrontend test failures done in particular by Jon and Juliusz,
it's clear that the consistency of the failures are related to architectural/infrastructural issues. Jon brought this up on June 9 [1], with a response from Zeljko on June 28 [2] mentioning that the issue was known, but that they haven't had time to debug the problem. Fast forward two weeks, and Juliusz resurfaced the problem with this thread since we haven't heard any additional information in regards to resolving the issue, while dealing with a high degree of noise from build failures.
We really appreciate all of the hard work that release/qa/platform/etc has put into this and we understand that resolving issues takes time. When we had more reliable builds, we found the automated browser tests to be incredibly valuable. We want to regain that value so that we can again more reliably catch issues before they find their way to production.
Is there anyone currently owning or willing to own digging into and resolving these issues? Can we get any kind of timeline for resolving this - even if it's just in regards to when the issue will be able to be investigated? In the mean time, let's remove the mobile web team from the failure notifications until such time that the builds are reliable and we can depend on a better signal-to-noise ratio.
As an aside - I didn't know the migration work to Cloudbees was complete until it was addressed on this thread. It looks like the only announcement about it was made on the qa list, where I and many folks affected by this change are not subscribed. Please make announcements about things as significant as this on broad-reaching lists like wikitech-l.