Wow. This feature did actually become broken. Jenkins emails became useful again! \o/
Anyone want to submit a patch to fix this?
FAILURE: MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox Build #484 (Thu, 10 Apr 2014 01:00:07 +0000)
Test Result
1 failed, 0 skippedFailed Tests
Test Name Duration Age Nearby pages.Page preview 45 sec 1 All Tests
Package Duration Fail Skip Total (root) 31 min 1 0 64 Wikitext Editor (TEST RUN ON WIKIPEDIA 1 min 5 sec 0 0 3