Hello,
Some localizable messages in MediaWiki features and extensions have very complicated, ambiguous and error-prone wording. A big example for that would be FlaggedRevs, but there are others. MediaWiki also has ability for changing messages according to number, gender and other grammatical features. It's not enough to just translate a list of strings there - the translation must also be tested in context.
The right way to do it is to have a test suite: A document that walks the translator through all the use cases of the feature so that all possible messages permutations and combinations appear.
There are plenty of methodologies and tools for testing code, as well as books and courses that teach them. I tried to search for set methodologies that focus on testing localization, but until now i didn't find almost anything.
So, two questions:
1. Are there currently any tests in the MediaWiki test suite that focus on localization?
2. Does anyone on this list know about testing methodologies that focus on localization?
PS: I wrote a few things about it in the English Wikipedia article [[Software testing#Internationalization and localization]], although i admit that it's at least partly original research based on my own experience.
-- Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי http://aharoni.wordpress.com "We're living in pieces, I want to live in peace." - T. Moore
On Mar 22, 2011, at 1:32 PM, Amir E. Aharoni wrote:
- Are there currently any tests in the MediaWiki test suite that
focus on localization?
The MediaWiki PHPUnit test suites are still very much incomplete, and have yet to test a fraction of the MediaWiki code. That said, there are tests that test the wfMessage() function and the Message class, including the various translations. So if I am understanding your question correctly, the answer is somewhat yes.
-X!
Hello,
The right way to do it is to have a test suite: A document that walks the translator through all the use cases of the feature so that all possible messages permutations and combinations appear.
Possibly, Selenium tests could be helpful. A Selenium test basically acts as a remote control for a browser. So a test could walk through most of the cases that produces localized messages. Since a Selenium test is client side, though, it cannot (re)produce all the errors on a server. So, for example, a missing connection to a database could not be triggered. Another thing we might have to consider is how to set breakpoints to pause the Selenium test in order to allow a translator to check the messages.
Cheers, Markus
-----Ursprüngliche Nachricht----- Von: wikitech-l-bounces@lists.wikimedia.org [mailto:wikitech-l-bounces@lists.wikimedia.org] Im Auftrag von Soxred93 Gesendet: Dienstag, 22. März 2011 18:51 An: Wikimedia developers Betreff: Re: [Wikitech-l] testing of localization
On Mar 22, 2011, at 1:32 PM, Amir E. Aharoni wrote:
- Are there currently any tests in the MediaWiki test suite that
focus on localization?
The MediaWiki PHPUnit test suites are still very much incomplete, and have yet to test a fraction of the MediaWiki code. That said, there are tests that test the wfMessage() function and the Message class, including the various translations. So if I am understanding your question correctly, the answer is somewhat yes.
-X!
_______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Markus Glaser glaser@hallowelt.biz wrote:
Hello,
The right way to do it is to have a test suite: A document that walks the translator through all the use cases of the feature so that all possible messages permutations and combinations appear.
Possibly, Selenium tests could be helpful. A Selenium test basically acts as a remote control for a browser. So a test could walk through most of the cases that produces localized messages. Since a Selenium test is client side, though, it cannot (re)produce all the errors on a server. So, for example, a missing connection to a database could not be triggered. Another thing we might have to consider is how to set breakpoints to pause the Selenium test in order to allow a translator to check the messages.
That would be wonderful. (I don't think PHPUnit is the way to go, since translators would only get checks whether message looks like it *should*, not the context).
When translating a complex extension, like FlaggedRevs or AbuseFilter you have to develop quite a large dictionary of concepts that should be somehow consistent. This attempt at consitency somehow leads to unclear messages as seen in the real life.
So having a possibility to have a pre-flight test of the translation (or even watch the demo of the original in action) is something Selenium could deinitely help. In many cases, translators do not have permission to experience some interface in the live environment (CheckUser, AbuseFilter, etc.). Could Selenium solve this problem as well? Is it just mocking up the interface or do I need a instance behind that is somehow setup somewhere?
//Marcin
Marcin Cieslak wrote:
So having a possibility to have a pre-flight test of the translation (or even watch the demo of the original in action) is something Selenium could deinitely help. In many cases, translators do not have permission to experience some interface in the live environment (CheckUser, AbuseFilter, etc.). Could Selenium solve this problem as well? Is it just mocking up the interface or do I need a instance behind that is somehow setup somewhere?
//Marcin
It needs a tests instance.
Στις 23-03-2011, ημέρα Τετ, και ώρα 02:03 +0100, ο/η Platonides έγραψε:
Marcin Cieslak wrote:
So having a possibility to have a pre-flight test of the translation (or even watch the demo of the original in action) is something Selenium could deinitely help. In many cases, translators do not have permission to experience some interface in the live environment (CheckUser, AbuseFilter, etc.). Could Selenium solve this problem as well? Is it just mocking up the interface or do I need a instance behind that is somehow setup somewhere?
//Marcin
It needs a tests instance.
I've been beating this drum for years (that translators need to be able to see their message(s) in context). But having to watch an automated test suite exercise all of the features of an extension in order to check the 5 new messages you are translating, seems less than efficient.
What would be nice for a first take would be if, from translatewiki, one could click a link that would take one to a list of screens that show the message in question, with the text highlighted even. Even if the only thing we had to work from was a display of the English language texts in context, that would still be a big step up from where we are now.
Ariel
2011/3/23 Ariel T. Glenn ariel@wikimedia.org:
Στις 23-03-2011, ημέρα Τετ, και ώρα 02:03 +0100, ο/η Platonides έγραψε:
Marcin Cieslak wrote:
So having a possibility to have a pre-flight test of the translation (or even watch the demo of the original in action) is something Selenium could deinitely help. In many cases, translators do not have permission to experience some interface in the live environment (CheckUser, AbuseFilter, etc.). Could Selenium solve this problem as well? Is it just mocking up the interface or do I need a instance behind that is somehow setup somewhere?
//Marcin
It needs a tests instance.
I've been beating this drum for years (that translators need to be able to see their message(s) in context). But having to watch an automated test suite exercise all of the features of an extension in order to check the 5 new messages you are translating, seems less than efficient.
What would be nice for a first take would be if, from translatewiki, one could click a link that would take one to a list of screens that show the message in question, with the text highlighted even. Even if the only thing we had to work from was a display of the English language texts in context, that would still be a big step up from where we are now.
Yes, and the other way around is also possible and probably even easier to implement - from a wiki to translatewiki. I just opened an enhancement suggestion about it:
https://bugzilla.wikimedia.org/show_bug.cgi?id=28191
-- Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי http://aharoni.wordpress.com "We're living in pieces, I want to live in peace." - T. Moore
Hello,
What would be nice for a first take would be if, from translatewiki, one could click a link that would take one to a list of screens that show the message in question, with the text highlighted even. Even if the only thing we had to work from was a display of the English language texts in context, that would still be a big step up from where we are now.
I think, Selenium provides a screenschot feature. So if we can get Selenium to dump screens and name them according to a sensible scheme, that could produce the material you ask for. This could then be done for the English language as a kind of template screen set and also for the target language, at certain points, e.g. before a new minor release comes out.
Cheers, Markus
-----Ursprüngliche Nachricht----- Von: wikitech-l-bounces@lists.wikimedia.org [mailto:wikitech-l-bounces@lists.wikimedia.org] Im Auftrag von Ariel T. Glenn Gesendet: Mittwoch, 23. März 2011 07:04 An: Wikimedia developers Betreff: Re: [Wikitech-l] testing of localization
Στις 23-03-2011, ημέρα Τετ, και ώρα 02:03 +0100, ο/η Platonides έγραψε:
Marcin Cieslak wrote:
So having a possibility to have a pre-flight test of the translation (or even watch the demo of the original in action) is something Selenium could deinitely help. In many cases, translators do not have permission to experience some interface in the live environment (CheckUser, AbuseFilter, etc.). Could Selenium solve this problem as well? Is it just mocking up the interface or do I need a instance behind that is somehow setup somewhere?
//Marcin
It needs a tests instance.
I've been beating this drum for years (that translators need to be able to see their message(s) in context). But having to watch an automated test suite exercise all of the features of an extension in order to check the 5 new messages you are translating, seems less than efficient.
What would be nice for a first take would be if, from translatewiki, one could click a link that would take one to a list of screens that show the message in question, with the text highlighted even. Even if the only thing we had to work from was a display of the English language texts in context, that would still be a big step up from where we are now.
Ariel
_______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Markus Glaser glaser@hallowelt.biz wrote:
Hello,
What would be nice for a first take would be if, from translatewiki, one could click a link that would take one to a list of screens that show the message in question, with the text highlighted even. Even if the only thing we had to work from was a display of the English language texts in context, that would still be a big step up from where we are now.
I think, Selenium provides a screenschot feature. So if we can get Selenium to dump screens and name them according to a sensible scheme, that could produce the material you ask for. This could then be done for the English language as a kind of template screen set and also for the target language, at certain points, e.g. before a new minor release comes out.
That's worth looking at. Actually we could have complete walkthroughs via many different workflows in the MediaWiki, useful also for training or documentation purposes. I wonder how many developers after spending many hours on reworking code have time and energy to update guides like this:
https://secure.wikimedia.org/wikipedia/meta/wiki/Help:CentralNotice
What I am doing now is deploying a small test wiki on my box or on the Polish toolserver, enabled the specific extension there and click through it. But how many translators can do this on their own? Or even a better question, how can I share those one-time setups with other translators, since this is mostly a one-off effort? Or is test.wikimedia.org enough for everybody?
Of course, the problem is in preparing those "walkthroughs" - how to cover many variants of possible configuration (many extensions show/hide elements of the UI depending on various $wgVariables) as well as preparing a reasonable set of test data to be imported (having empty CheckUser or AbuseFilter interfaces is pretty useless). But having an automated screenshot machine gun would improve things a lot. Gee, I wish I attended one of the talks that Markus gave last year about Selenium.
//Marcin
wikitech-l@lists.wikimedia.org