Chris, this sounds really cool. Can you point us to some specs about how the test environment is set up (what is the architecture like, what services are running, etc)? How closely does it emulate the production environment? Does the beta labs environment provide load balanced squid/varnish caching layers, configured similarly to the produciton cluster? If not, is that something we can hope to see? Is the setup something that we can package up and easily deploy to new instances in labs?
One of the biggest challenges I've had testing code is not being able to easily test what will happen when requests are handled by squid or varnish (particularly for mobile where request routing is more complicated) - to the point where there are something we kinda just have to test in production. I cannot wait until this is no longer the case :)
Also, how can other projects/extensions start getting automatically pushed to the beta labs setup?
Thanks! Arthur
On Fri, Sep 28, 2012 at 8:48 AM, Chris McMahon cmcmahon@wikimedia.orgwrote:
When I was hired as QA Lead almost seven months ago, WMF lacked a test environment where
- code was routinely deployed ahead of production
- the test environment emulated the production environment closely
- aspects of the test environment (config, permissions, etc.) could be
easily and reliably manipulated for testing purposes
Today I am happy to announce that beta labs fulfills those needs.
Beta labs is intended to host the upcoming release of Mediawiki, plus those extensions scheduled for deployment to production, for the purpose of testing and investigation.
As of a little while ago, Mediawiki, AFTv5, New Pages Feed/Page Curation, and UploadWizard are being deployed to beta labs from git automatically and reliably. The configurations for those extensions are also being managed in git. The environment itself is managed via puppet, and emulates production to the greatest extent possible. Many many thanks to Antoine Musso for making this possible.
As of this week, all these extensions are up, running, and configured to be useful. Note that they are not perfect, just useful. For example, right now on beta enwiki both AFTv4 and AFTv5 input forms appear on the same page in many cases, because I was experimenting with what happens when these extensions are not configured correctly. Some actions from the Page Curation toolbar never complete. As these glitches become important to testing, we will get them working correctly, and likely will find out some interesting things about the software along the way.
The timing for this announcement is excellent, because new QA Engineers will be joining WMF soon (more on that next week), and beta labs will be a prime target for the browser-level end-to-end automated tests we will shortly be creating. Also, we have been wanting to retire the 'prototype' host for some time, and having AFTv5 etc. on beta labs should make that possible.
In summary, beta labs is up and running with current code for Mediawiki and critical extensions, and at this point the best way to improve beta labs is to use it.
http://en.wikipedia.beta.wmflabs.org/wiki/Special:ArticleFeedbackv5 http://en.wikipedia.beta.wmflabs.org/wiki/Special:NewPagesFeed http://commons.wikimedia.org/wiki/Special:UploadWizard _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l