Re: [Wikitech-l] About outreach and tech events (as suggested by Sumana!)

I'm the QA Lead for WMF, and I can say a little about what we've done so far, and things we'd like to do in the future. In May we collaborated with the Weekend Testers (Americas)[1] in an online test exercise to validate the new frequent rollout schedule of new software to all of the Wikipedia wikis. WTA meets on the first Saturday of every month, and conveniently, we had deployed the latest version to all of the wikis except English Wikipedia at that time, so we had quite a few professional software testers looking for anomalies in the new wiki software, using English Wikipedia as an oracle for correct behavior. It went well, but the scope was a bit ambitious, so we were lucky that the testers were very professional. In June we collaborated online with Openhatch.org to validate a near-final version of the new Article Feedback system. This was a much more focused exercise, and it went really well, we found a number of real issues with AFT that we were able to address before rolling it out widely. Some of participants from the previous test event with WTA helped out, so there was a mix of skill and experience among the testers, and several people remarked about how they had not expected to have so much fun. We would like to do some more sessions like these. One strong suggestion is to have a test event addressing outstanding Bugzilla issues for particular extensions. This could be an ongoing exercise, either in collaboration with other groups or as a pure-Wikipedia exercise. I have discussed doing this with the leader of WTA who is also one of the instructors of the Association for Software Testing[2] 'Bug Advocacy' course, but haven't pursued it much farther than that. In the long run we would like these sorts of exercises to foster a critical spirit among Wikipedia users, improve the quality of issue reporting and follow-up in Bugzilla, build liaisons with communities like WTA and Openhatch that would not otherwise exist, and foster a general sense that all the Wikipedia software can be tested at any time, and Bugzilla is always open for all sorts of improvements. So now to address what you actually said :-)... Although I've read my share of unit tests in many languages, I'm not expert at it, nor do I have a background in PHP. But I am nearly certain that our existing unit test arrangements could be improved in many ways. Threads on the subject show up on this list from time to time, and I think I can say accurately that we could make better use of mocks and stubs instead of e.g. real database tables, we could do more TDD, our code coverage is probably not very high, and improving that coverage would entail not only writing more unit tests, but also refactoring existing code to make it more testable. I think improving unit testing would be a great ongoing project. [1] http://weekendtesting.com/chapters/america [2] http://www.associationforsoftwaretesting.org/training/courses/