When the Vector skin became available, I tried it on my home wiki, pt.wikipedia, and noticed that a great deal of its interface was still in English. So I went to translatewiki.net and translated the remaining strings to Portuguese. Then I waited, and waited.. and I am waiting until today, and the skin still has the English strings on it. It's been almost a month.
This is bad for several reasons. On this specific context, it means that non-English users of the Vector skin, which is supposed to increase usability, will actually have potentially more trouble using it simply because it is using a foreign language.
On a more general stance, this is also bad for translators, since we don't have as much motivation to contribute when our translations lay unused for so much time. It's exactly one of the arguments that was used a lot to oppose the FlaggedRevisions extension: the immediacy of the edits going live is what makes wikis so compelling. (disclaimer: I'm actualy in favor of flagged revs; I would trade some immediacy for more stability. But not if the delay means a month!)
It's also bad for MediaWiki in general, since the expansion of its language support grows in a much slower pace.
I understand why it was chosen not to always run bleeding edge versions of the software on the live Wikimedia wikis. But the LocalisationUpdate was created precisely as a workaround to this, i.e, to allow updating the localisation without needing to update the software.
So my question is: why is it not enabled yet on most Wikimedia wikis?
Waldir Pimenta wrote:
I understand why it was chosen not to always run bleeding edge versions of the software on the live Wikimedia wikis. But the LocalisationUpdate was created precisely as a workaround to this, i.e, to allow updating the localisation without needing to update the software.
So my question is: why is it not enabled yet on most Wikimedia wikis?
The LocalisationUpdate extension is slow, with a significant performance loss per page view due to DB queries, and it's unnecessary, because the same effect can be had with a script that runs svn up periodically.
-- Tim Starling
Hoi, Why then is svn up not run every day on the Wikimedia Foundation's servers ?? Thanks, GerardM
2009/8/13 Tim Starling tstarling@wikimedia.org
Waldir Pimenta wrote:
I understand why it was chosen not to always run bleeding edge versions
of
the software on the live Wikimedia wikis. But the LocalisationUpdate was created precisely as a workaround to this, i.e, to allow updating the localisation without needing to update the software.
So my question is: why is it not enabled yet on most Wikimedia wikis?
The LocalisationUpdate extension is slow, with a significant performance loss per page view due to DB queries, and it's unnecessary, because the same effect can be had with a script that runs svn up periodically.
-- Tim Starling
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Hoi, The problem with svn up is that it does not take into account that the software run in production is not the same version that exists in SVN. Consequently you could update messages that are no longer the same. What LocalisationUpdate does is verify if the message in English in SVN is the same as the one that is currently running. When the messages are exactly the same, it follows that the localised messages are also the same.
Because the LocalisationUpdate updates from SVN, it will update the messages that were seen by the translatewiki.net developers.
When you say the LocalisationUpdate is slow, the current update process from SVN is slow. This however only needs to run once a day really. When this is done when there is not much traffic anyway, it does not matter. What does matter is that the results of a message found as a result of LocalisationUpdate end up in the l10ncache. Thanks, GerardM
2009/8/13 Tim Starling tstarling@wikimedia.org
Waldir Pimenta wrote:
I understand why it was chosen not to always run bleeding edge versions
of
the software on the live Wikimedia wikis. But the LocalisationUpdate was created precisely as a workaround to this, i.e, to allow updating the localisation without needing to update the software.
So my question is: why is it not enabled yet on most Wikimedia wikis?
The LocalisationUpdate extension is slow, with a significant performance loss per page view due to DB queries, and it's unnecessary, because the same effect can be had with a script that runs svn up periodically.
-- Tim Starling
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Gerard Meijssen wrote:
Hoi, The problem with svn up is that it does not take into account that the software run in production is not the same version that exists in SVN. Consequently you could update messages that are no longer the same. What LocalisationUpdate does is verify if the message in English in SVN is the same as the one that is currently running. When the messages are exactly the same, it follows that the localised messages are also the same.
Apparently that's not a problem for the release branches, which often receive backported message updates from translatewiki.net. Why can't the same backporting be done for the wmf-deployment branch?
-- Tim Starling
Hoi, When a back port is created, There is *a lot of manual work* involved in preparing the messages that are relevant to a specific release. As it is, translatewiki.net localises the latest development versions of software and this is not what runs in production. The key functionality of LocalisationUpdate is that it ensures that the messages it imports are compatible with the version of MediaWiki that runs in that instance of the software, including extensions.
Probably LocalisationUpdate could work with the 1.15 stable version of MediaWiki. This would be really valuable for all the MediaWiki installations out there that have "another" language as default. It would even be an incentive to them to localise MediaWiki at translatewiki.net because they do not have to wait for the next release. Thanks, GerardM
2009/8/13 Tim Starling tstarling@wikimedia.org
Gerard Meijssen wrote:
Hoi, The problem with svn up is that it does not take into account that the software run in production is not the same version that exists in SVN. Consequently you could update messages that are no longer the same. What LocalisationUpdate does is verify if the message in English in SVN is the same as the one that is currently running. When the messages are exactly
the
same, it follows that the localised messages are also the same.
Apparently that's not a problem for the release branches, which often receive backported message updates from translatewiki.net. Why can't the same backporting be done for the wmf-deployment branch?
-- Tim Starling
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Gerard Meijssen wrote:
Hoi, When a back port is created, There is *a lot of manual work* involved in preparing the messages that are relevant to a specific release. As it is, translatewiki.net localises the latest development versions of software and this is not what runs in production. The key functionality of LocalisationUpdate is that it ensures that the messages it imports are compatible with the version of MediaWiki that runs in that instance of the software, including extensions.
Wouldn't the release branches be improved if you ported that functionality from LocalisationUpdate to the backport automation scripts run on translatewiki.net? And then we'd be able to get translation updates with a simple "svn up" instead of adding a complex, tightly-integrated extension to the main MediaWiki instance on Wikimedia.
-- Tim Starling
Hoi, Three things:
- Perfection is the enemy of the good; this works now - This works for all extensions and MediaWiki supported in the WMF SVN and localised in translatewiki.net - This is tightly integrated with what has been localised in translatewiki.net
There are two benefits to this approach
- it motivates the people that localise - it provides timely support for our localisation effort - It works inside the WMF and outside, it is self contained
Again, the scripts for the back port ONLY work for MediaWiki itself This is not good enough because we need localisations for our extensions as well. The LocalisationUpdate functionality has been discussed before we started its development with Brion and Siebrand. This works and it does not require extra work from Siebrand.. A really important KPI in my opinion.
As you may perceive from the reactions in this list we are wasting a lot of effort from our volunteers. Implementing LocalisationUpdate is important because it demonstrates that we value the effort and the importance of the localisation work that is done at translatewiki.net. Thanks, GerardM
2009/8/13 Tim Starling tstarling@wikimedia.org
Gerard Meijssen wrote:
Hoi, When a back port is created, There is *a lot of manual work* involved in preparing the messages that are relevant to a specific release. As it is, translatewiki.net localises the latest development versions of software
and
this is not what runs in production. The key functionality of LocalisationUpdate is that it ensures that the messages it imports are compatible with the version of MediaWiki that runs in that instance of
the
software, including extensions.
Wouldn't the release branches be improved if you ported that functionality from LocalisationUpdate to the backport automation scripts run on translatewiki.net? And then we'd be able to get translation updates with a simple "svn up" instead of adding a complex, tightly-integrated extension to the main MediaWiki instance on Wikimedia.
-- Tim Starling
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Tim Starling <tstarling@...> writes:
The LocalisationUpdate extension is slow, with a significant performance loss per page view due to DB queries,
I can't reproduce that locally. After installing LocalisationUpdate and visiting a few pages, I get:
LocalisationCache::isExpired(en): cache for en expired due to GlobalDependency LocalisationCache::recache: got localisation for en from source SQL: BEGIN DatabaseBase::query: Writes done: DELETE FROM `l10n_cache` WHERE lc_lang = 'en' SQL: DELETE /* LCStore_DB::startWrite Catrope */ FROM `l10n_cache` WHERE lc_lang = 'en' SQL: INSERT /* LCStore_DB::set Catrope */ INTO `l10n_cache` ...
Presumably this is LU invalidating the l10ncache. This does not happen on a second or subsequent page view, though. Instead, I get
MessageCache::load: Loading en... got from global cache
and I see messages being pulled from the l10n_cache table.
If you can reproduce these extra queries locally, please tell me how.
It's true that there's a known issue with the update script being slow, but that shouldn't be too bad since it's only supposed to be run once every 6, 12 or 24 hours or something.
and it's unnecessary, because the same effect can be had with a script that runs svn up periodically.
Gerard already explained that that has undesirable side effects.
Roan Kattouw (Catrope)
Roan Kattouw wrote:
Tim Starling <tstarling@...> writes:
The LocalisationUpdate extension is slow, with a significant performance loss per page view due to DB queries,
I can't reproduce that locally. After installing LocalisationUpdate and visiting a few pages, I get:
LocalisationCache::isExpired(en): cache for en expired due to GlobalDependency LocalisationCache::recache: got localisation for en from source SQL: BEGIN DatabaseBase::query: Writes done: DELETE FROM `l10n_cache` WHERE lc_lang = 'en' SQL: DELETE /* LCStore_DB::startWrite Catrope */ FROM `l10n_cache` WHERE lc_lang = 'en' SQL: INSERT /* LCStore_DB::set Catrope */ INTO `l10n_cache` ...
Presumably this is LU invalidating the l10ncache. This does not happen on a second or subsequent page view, though. Instead, I get
MessageCache::load: Loading en... got from global cache
and I see messages being pulled from the l10n_cache table.
I don't think those queries have anything to do with LocalisationUpdate.
If you can reproduce these extra queries locally, please tell me how.
I removed the hook LocalisationUpdate used in r52503, so it doesn't do anything at all. You can reproduce it by reverting to MediaWiki 1.15.
I've now committed the rewrite of LocalisationUpdate I promised in the commit message of r52503. I held off on it because I wasn't convinced that it's a useful solution for anything.
It's true that there's a known issue with the update script being slow, but that shouldn't be too bad since it's only supposed to be run once every 6, 12 or 24 hours or something.
and it's unnecessary, because the same effect can be had with a script that runs svn up periodically.
I'm not talking about the update script speed, I'm talking about the MessageNotInMwNs, which was (before r52503) called from a common case of wfMsg() and typically did a DB query.
-- Tim Starling
Tim Starling <tstarling@...> writes:
Presumably this is LU invalidating the l10ncache. This does not happen on a second or subsequent page view, though. Instead, I get
MessageCache::load: Loading en... got from global cache
and I see messages being pulled from the l10n_cache table.
I don't think those queries have anything to do with LocalisationUpdate.
Ah, I now see that the invalidation was caused by adding LocalisationUpdate's localisations. I already knew the selects on l10n_cache were not LU-related.
If you can reproduce these extra queries locally, please tell me how.
I removed the hook LocalisationUpdate used in r52503, so it doesn't do anything at all. You can reproduce it by reverting to MediaWiki 1.15.
I've now committed the rewrite of LocalisationUpdate I promised in the commit message of r52503. I held off on it because I wasn't convinced that it's a useful solution for anything.
Thanks.
Roan Kattouw (Catrope)
Hi,
when I've read Tim's mail, I got really angry. Why would we translate anything on the translatewiki, when it's ignored for a month (or longer)?? When one has to correct some translation, then it's double work. I have to correct it on my local wiki, and on translatewiki. We copied the almost immediately translated strings of Vector skin to huwiki, because it was really embarrassing that after a month the skin is still english, but we already started push out the beta. And after update, we should delete the messages that we had copied to our local wiki. It's nonsense...
Gerard wrote on wikitech-l that the translate extension is not far away. Then that...
We just want a solution instead of unnecessary double/triple work.
Ákos Szabó (Glanthor Reviol)
Hello, Waldir.
Waldir Pimenta wrote:
When the Vector skin became available, I tried it on my home wiki, pt.wikipedia, and noticed that a great deal of its interface was still in English. So I went to translatewiki.net and translated the remaining strings to Portuguese. Then I waited, and waited.. and I am waiting until today, and the skin still has the English strings on it. It's been almost a month.
Thank you for the translation work. As a usability team leader, I appreciate your attention on Vector and the new toolbar especially. We, the usability team, get really excited when the text in UI are translated into new languages. We pushed lots of text especially for the beta landing page and survey. I see the Portuguese pages are fully translated for the survey. Thank you.
Since the first release of usability features (Acai) on July 1st, there has been minimum of weekly software update to Acai. And we updated the deployment software with up-to-date translation on August 6th for all languages. In the case for Portuguese, the text for the toolbar, the beta landing page, and the survey was updated. However the translation for tabs didn't get the update. We will look into why the update didn't occur.
As for LocalisationUpdate, there has been a discussion in this thread already, so I won't get into too much details. But we are planning to start testing it in the usability prototype sites for the next round of release. http://usability.wikimedia.org/wiki/Prototype It would be great if we can work with translators to test LocalisationUpdate so that the translators can actually confirm their translation appear promptly after the translation in traslatewiki.net.
When you see a portion of translation does not appear in UI, please drop me a note. We often think that the translation behind, and if that is not the case we will look into why the update did not occur. Hopefully this kind of gap will be gone, once the automation via LocalisationUpdate is available. But until then, feel free to write to me directly. My email address is nkomura@wikimedia.org.
Cheers,
- Naoko
Thanks, Naoko.
I indeed haven't been looking at all places; as I saw the tabs still untranslated, I assumed the rest was still so. It's already very good to have the everything but the tabs localised, but of course, I'll be looking forward to see the fully translated skin live :)
As you suggest, I'll contact you directly in the future should I come across any similar issues. Thanks for the feedback, and keep up the great work!
Waldir
On Thu, Aug 13, 2009 at 4:13 PM, Naoko Komura nkomura@wikimedia.org wrote:
Hello, Waldir.
Waldir Pimenta wrote:
When the Vector skin became available, I tried it on my home wiki, pt.wikipedia, and noticed that a great deal of its interface was still in English. So I went to translatewiki.net and translated the remaining
strings
to Portuguese. Then I waited, and waited.. and I am waiting until today,
and
the skin still has the English strings on it. It's been almost a month.
Thank you for the translation work. As a usability team leader, I appreciate your attention on Vector and the new toolbar especially. We, the usability team, get really excited when the text in UI are translated into new languages. We pushed lots of text especially for the beta landing page and survey. I see the Portuguese pages are fully translated for the survey. Thank you.
Since the first release of usability features (Acai) on July 1st, there has been minimum of weekly software update to Acai. And we updated the deployment software with up-to-date translation on August 6th for all languages. In the case for Portuguese, the text for the toolbar, the beta landing page, and the survey was updated. However the translation for tabs didn't get the update. We will look into why the update didn't occur.
As for LocalisationUpdate, there has been a discussion in this thread already, so I won't get into too much details. But we are planning to start testing it in the usability prototype sites for the next round of release. http://usability.wikimedia.org/wiki/Prototype It would be great if we can work with translators to test LocalisationUpdate so that the translators can actually confirm their translation appear promptly after the translation in traslatewiki.net.
When you see a portion of translation does not appear in UI, please drop me a note. We often think that the translation behind, and if that is not the case we will look into why the update did not occur. Hopefully this kind of gap will be gone, once the automation via LocalisationUpdate is available. But until then, feel free to write to me directly. My email address is nkomura@wikimedia.org.
Cheers,
- Naoko
-- Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Naoko Komura <nkomura@...> writes:
As for LocalisationUpdate, there has been a discussion in this thread already, so I won't get into too much details. But we are planning to start testing it in the usability prototype sites for the next round of release. http://usability.wikimedia.org/wiki/Prototype It would be great if we can work with translators to test LocalisationUpdate so that the translators can actually confirm their translation appear promptly after the translation in traslatewiki.net.
This Friday I enabled LocalisationUpdate on our sandbox [1]. It's running 4 days old software [2], but LU updates the translations every hour. The update starts at nn:17 and takes about 7 minutes, which means everything should be up-to-date at nn:24. Please note that only those translations that have been committed to SVN by TranslateWiki staff will be included in the updates, and only if the English version of the message hasn't changed.
Roan Kattouw (Catrope)
[1] http://prototype.wikimedia.org/sandbox [2] http://prototype.wikimedia.org/sandbox/Special:Version
Because we haven't had a chance to finish final review & fixups on it. Should be done in the next few days. (Why are there like 500 replies to this from people we've already talked to directly and know what's up?)
-- brion
On 8/13/09 11:43 AM, Brion Vibber wrote:
Because we haven't had a chance to finish final review& fixups on it. Should be done in the next few days. (Why are there like 500 replies to this from people we've already talked to directly and know what's up?)
Quick status update -- Tim made some good fixes this morning which tidy up the basic issues w/ compatibility and cache read performance. I've been doing some review & cleanup in consultation w/ Roan over the course of the day which should make it work on our deployment branch.
We've got another fix that's needed to avoid duplicating the update database across our hundreds of wikis which should be pretty easy, then we may want to do another pass over the update speed issue if we can improve that.
(Right now the updates run relatively slowly, which could be much improved; at least some of that was introduced with some security fixes, but that can probably be sped up a lot without going back to the original, less safe method.)
I'm reasonably confident we can roll this out for testing by tomorrow, and if all is well it can be rolled out generally quite soon.
-- brion
wikimedia-l@lists.wikimedia.org