Toolserver-l

toolserver-l@lists.wikimedia.org

1612 discussions

Toolserver Intuition - Tech spec (Toolserver goes I18N!)
by Krinkle 05 Apr '11

05 Apr '11

So, as announced in the previous mailing (Agreed, I should've sent this one first...). Regarding the technical implementation. First a few points that you are likely aware of, that have stopped, disabled or scared off doing this untill now. ie. creating a dedicated solution that can scale for more than just 1 tool or group of tools – And is easy to use for translators as well for developers. -- Past issues / obstacles * Separated Toolserver SVN repositories ** For MediaWiki extensions, TranslateWiki can easily translate things and can sync in a single commit because it has a partial check out of a single SVN repository. Having to add all ts-account repos to TW's configuration is way too much work and pretty much not-done. Not to mention the fact that most tools are either not in SVN at all, or are maintained outside SVN and pushed towards SVN for public source viewing every once in a while. * Language of choice ** Users want to set their choice once and not have to search re-do it for all tools independently and have to find the right place to 'do it' on everybody's tool. Nor do we want to click the same link again every visit. And developers prefer not to have userlang-parameters dangling in the url and have to make sure it's preserved throughout the app with every link. This can (and sometimes is) solved by using cookies (one example is Luxo's contributions tool that sets a cookie) * Prevent vandalism but also slow-down and other down sides of a regular MediaWiki page. ** When translated on one wiki-page (ie. at Meta-Wiki such as Magnus' implementation, which I think is the best implementation so far) there isn't a good translation-oriented workflow for translators or developers. Of course pages could be protected by sysops - and then have to be updated from another page on request. And then there is FlaggedRevs. But neither are not optimized towards translators (ie. there's no way to FUZZY a message, or see translation suggestions from services like Google Translate, or a description of the message while modifying it (aka "/qqq"). Nor is it not ideal (if not impossible) to keep track of changes to the original message without having to manually check it (eg. perform "FUZZY"). And no easy workflow to translate many messages at once. This is all taken care of by Extension:Translate [4] on TranslateWiki. * Fallback languages ** Aside from the management involved, fallback is also an important point. It shouldn't be required that a translator has to translate everything at once or nothing at all. Some implementations around would fail if a new message wasn't available in a translated language yet. Other implementation created a way to fallback to English if there was no message-key in the selected language. But I haven't seen any real fine grained fallback (such falling back from NDS (Low German) to DE (German), or from ACE (Aceh) to ID (Indonesian) for untranslated messages, like MediaWiki does). * Universal messages ** Some messages like 'welcome', 'login', 'submit' etc. are generic and should not required to be duplicated around everywhere for each tool. Eventhough TranslateWiki has {{Identical}}, it save work by just having a group of generic messages. Tools that are mostly data or visually oriented may not even have to request to be added to the project if they only need a few of these generic messages to control the input form. * Keeping up with latest versions ** Another implementation currently around (the only one that actually uses TranslateWiki afaik) is done through a fake extension named ToolserverTools [1] in Wikimedia SVN. For the translators side this was perfect (since they could use the TranslateWiki workflow they know and love). But not so for the developers. Messages all had a prefix, and in order to actually use them in the tools some wheel-reinvention took place (like getting the message from the array in the correct language, providing fallback, replacing variables like $1/$2, etc.). Also they still had to re-create a way for users to choose a language, store it, remember it and apply it the next visit. Lots of wasted time. And of course staying up2date with the latest version in Wikimedia SVN was sometimes forgotten and translators are known to get especially motivated if there is no work required from the developer to put the new translations into use (ie. TranslateWiki having the ability to push the updates and there being no extra action required). We could have everybody create a cronjob to update their svn-checkout of the messages file from the "/extensions/ToolserverTools/" directory in SVN, but that's not ideal either. -- All of the above have been solved with my proposal. Either because of the fact that it's powered via TranslateWiki, or because it's taken care of by the central i18n system. -- Tool developer workflow: I'll describe how the system would work from a tool developers point of view. [3] So here's what you'd do to make it work, three easy steps: 1) The toolserver tool developer includes a single php file (eg. / p_i18n/ToolStart.php). This makes the i18n class available. 2) A new instance of the class is created like $I18N = new TsIntuition( 'mygroup' ); 3) Messages can now be returned with either _('message-key') [2] or $TSi18n->msg( 'message-key', $options ). The msg() function can optionally be passed a text domain name (or 'group name' if you will) as second argument to get a message from a different group eg. the group 'general' for messages like 'welcome', 'login', 'submit' etc. Or an array if you need multiple options like escape, variable replacement etc. (more on that in a minute). -- Other features Although the I18N class will be able to do a lot more, this above is the core principle. Here's a list of items in no particular order for other things that it will have: * Variable replacement ($1, $2, etc.) $welcome = $I18N->msg('welcomeback', array( 'variables' => array( $username, $lastvisit ) ) from [[Toolserver:Mytool-welcomeback]] which contains "Welcome back $1 (last visit: $2)". * Fallback languages: If a message is not found in the current user language, a fallback will be used. And if that one isn't found English is used. * Getting language names (eg. de -> 'Deutsch', en -> English) is built- in. Currently uses a copy MediaWiki's Names.php, could be made to use sql.toolserver.language if that is preferable but I think it's good this way) * Escaping (ie. options = array( escape => html ) * Automated updates: Since the messages are file-stored in the messages-directory of the tool. There's no need to keep track or update anything. ToolStart.php will load the appropriate class from the correct file, and when initializing the class and using msg(), will load needed message files on demand. * Direction (Get which direction a language is, LTR or RTL). Handy when constructing your <html> element: <html dir="{$I18N->getDir()}" lang="{$I18N->getLang()}"> * Automatic detection and remembering of the right user language (users can choose a language from a central i18n preferences page. This is stored as a cookie and (if no cookies available, in session). It can still be overridden by using the userlang GET parameter [2]. One can also pass the desired message language to the getMsg() function to force a certain language for one message. * No prefixes or collisions for MediaWiki messages: To avoid conflicts with other tools, message-keys are automatically prefixed with the name of the group. So you won't have to prefix every key internally to avoid conflicts with messages of another Toolserver tool. Also (still in talks with TranslateWiki) we're planning to put them in a dedicated namespace and not in the MediaWiki:-namespace on TranslateWiki. Example: * A message at [[translatewiki:Toolserver:Luxocontris-usernotfound]] * will be available through $I18N->msg( 'usernotfound' ), assuming $i18n = new TsIntuition( 'luxocontris' ) * otherwise $I18N->msg('usernotfound', 'luxocontris'); * Localize other fronts as well: There's several popular tools out there that have an additional (or only) front-end via JavaScript implementation on a wiki. Since the i18n system will have an API that has a JSON-output format with callback (JSONP) you can get the messages in there as well. * API: When not in PHP (ie. JavaScript or Python) you can do queries (GET or POST) like api.php? action=getmessages&group=luxo&message=foobar|lorem|ipsum|logout|login &format=json&callback=myTool.initLang * More.. (see design specification on Toolserver wiki) [5] -- TranslateWiki I'm currently in talks with TranslateWiki how to best set up the syncing system. Although initial chat with Nikerabbit didn't bring up any expected problem (as it's fairly similar other projects they translate), it still needs to be set up. I expect to have something going within one or two weeks. The source files have been added to Wikimedia SVN [6] and checked out in the TsIntuition directory [7] at the Toolserver. -- Documentation / design specification The initial concept for the class has been documented at Toolserver Wiki [5]. Most of it has already been implemented in SVN [6] and can be tested. The implemention is subject to change based on feedback from you. -- Already translated The following tools have been translated already. Log in at the toolserver and look at their source to learn how they work: * http://toolserver.org/~krinkle/TsIntuition/ * http://toolserver.org/~jarry/svgtranslate/ * http://toolserver.org/~krinkle/getWikiAPI.php -- Krinkle [1] http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/ToolserverTools/ [2] Yes, there's a way to disable the global function _() if you don't want it or have a function named like that already. [3] Right now this is only for PHP tools (which are the most common), but I'm currently working on providing an API to make this available in other formats as well (ie. jsonp-callback for usage in javascript gadgets. Some toolserver tools interact with a wiki-side javascript companion. And a format that can be easily loaded into languages like Python (xml/json). I will focus on that as soon as the initial system is up and running. [4] http://www.mediawiki.org/wiki/Extension:Translate [5] https://wiki.toolserver.org/view/Toolserver_Intuition [6] http://svn.wikimedia.org/viewvc/mediawiki/trunk/tools/ToolserverI18N/ [7] http://toolserver.org/~krinkle/TsIntuition/

15 36

Toolserver goes i18n! (Naming)
by Krinkle 28 Mar '11

28 Mar '11

Hi all, As some of you may have heard, I'm currently working on a system that will make it super easy for developers on the toolserver to localize their tools with almost no effort at all. More about the techinical aspects of this early next week (or poke me on IRC). I'm currently checking with TranslateWiki what the best way is for them to handle it and finding a balance so that Toolserver developers will have minimal efforts and can focus on the functionality (instead of on the localization), and for TranslateWiki to not have to dig into any kind of technical code. In this thread I'm looking for a suitable name for the system. A few ideas I came up with are below. Let me know what you think about them and/or send in some other ones you think are great! If you like playing with words, this is your call to get creative! I'm planning to pick the initial name [0] this weekend, so don't wait! I've got three =names=, and about a dozen ==abbrevations/nicknames to get you started. = Internationalization (i18n) for Tools' User Interfaces at the Toolserver == Name: Ituit (or ITUIT) (pronounce: " I tweet " or " Étui ") Story: People communicate by talking, as birds do by tweeting - preferably in their native language while using the tools. == Name: Toolserver Intuition Origin: The word "Ituition" is ITUIT (previous name above) + " ion ". Story: Peoples first "ituition" is to speak in their native language. = Toolserver Localization System When abbreviated as TLS it'll be confusing with the existing TLS/SSL technology [2][3]. But the following may be possible derivatives / abbreviations : * Tolocsy (or Tolloxy) * Toolsy * Tolcsy * .. = Internationalization for Toolserver's User Interface == Name: Intoyui (pronounce: " Into you " or " Into you and I ") Story: The i18n system is into you / your native language! == Name: ITUI (pronounce: " i twee " or " Étui " [1][2] ) Story: This i18n is like a bag (French: etui) of messages. == Name: IntoUI (pronounce: " Into U ('you') I ") Story: The tool intergrates into the UI (user interface) of toolserver tools. -- Those were the ideas I had so far. Personally I like the rol of "Toolserver Intuition" and "ITUI" (etui). What do you like and why ? Don't let any of the above limit your imagination, feel free to share any ideas you may have regarding the name. Note: For questions, suggestions or other comments on the system itself, please respond to the mailing I'll send out early next week. I'm currently investigating what aspects to look at and how we can best intergrate this with TranslateWiki in an as simple, flexible yet solid way as possible. Feel free to poke me at IRC [4] -- Krinkle [0] I say initial name because we might change it later on, but I prefer not to (due to account-, database and class names etc.). [1] http://en.wiktionary.org/wiki/étui [2] http://en.wikipedia.org/wiki/Transport_Layer_Security [3] http://en.wikipedia.org/wiki/TLS [4] https://wiki.toolserver.org/view/IRC irc://irc.freenode.net/wikimedia-toolserver

4 3

Urgent help needed
by Hydriz Wikipedia 27 Mar '11

27 Mar '11

Hi guys from Toolserver, My account was created a few days ago. Everything ran well, but not SVN. When I tried committing my changes, SVN replied: Couldn't perform atomic initialization What seems to be the error? I have done all the steps correctly and worked using both Ubuntu and Windows, but the problem still exists. Help is greatly appreciated! Regards,Hydrizhttp://simple.wikipedia.org/wiki/User:Hydriz

1 0

Changes to mail forwarding
by River Tarnell 26 Mar '11

26 Mar '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, During the next maintenance window (March 7th), we will make some changes to how mail forwarding works. This will not affect most users, but you should read this mail anyway, especially if you make use of .forward files. A summary of the changes you might need to make are at the end of this mail. At present, all mail to user(a)toolserver.org is sent to your LDAP alias; if you don't have an address set in LDAP, mail is sent to nightshade and processed locally (which might include using .forward files). Locally generated mail, e.g. cron, is handled by .forward; users with email addresses set in LDAP have "username(a)toolserver.org" in $HOME/.forward so it's forwarded properly. After this change, mail to user(a)toolserver.org and mail to user(a)ANYMACHINE.toolserver.org will be handled as follows: * If $HOME/.forward exists, mail will be delivered to the address in that file; * If not, mail will be delivered to the user's LDAP email address, if any if set; * If none is set, mail will be delivered locally on either clematis or hawthorn (*not* nightshade). In addition, mail to user+foo(a)toolserver.org and user+foo(a)ANYMACHINE.toolserver.org will be accepted, where "foo" can be any string. This will be handled in the same way, except that $HOME/.forward+foo will be checked instead of $HOME/.forward. This allows you to process mail differently depending on the address (see below for an example). In summary, you should make these changes after the maintenance: * If you do not do anything "special" with your mail, and you have an LDAP mail address set, and your .forward file contains "username(a)toolserver.org", then you do not need to do anything. We will delete your .forward file during the maintenance, since it's no longer necessary. This applies to the vast majority of users. * If you have an LDAP email address set, and your .forward file contains a different address, you should delete your .forward file after the maintenance. We will not do this for you. * If you do not have an LDAP email address set, and you have a .forward file containing an email address, you do not have to do anything. However, you *should* run 'setmail' to set an LDAP email address, then delete your .forward file. * If you do not have an LDAP email address set, and you do not have a .forward file, then currently your mail will be delivered locally on nightshade. After the maintenance, your mail will be delivered locally on either hawthorn or clematis. We do not support this configuration, and you should set an email address in LDAP so your mail is forwarded. However, if you insist on having mail delivered locally, you can continue to do so. * If you currently rely on accepting mail to user+foo(a)machine.toolserver.org and use .forward+foo to funnel this mail to a program, then you do not have to do anything. However, you *should* instead send mail to user+foo(a)toolserver.org, which does not rely on a particular machine being up. Note that if you do this, the incoming mail will be processed on either clematis or hawthorn. The explanation of these changes is more complicated than the actual changes. The result should be that mail processing is much simpler and more obvious to users. - river. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1pIIQACgkQIXd7fCuc5vKOkACfYd3na6TlCLOpuOiXWDEVBB1y sdkAnilzSrqxYCgL0WalfYlwSt68yf4U =2El7 -----END PGP SIGNATURE-----

4 6

Toolserver goes I18N! (Naming)
by Krinkle 25 Mar '11

25 Mar '11

Hi all, As some of you may have heard, I'm currently working on a system that will make it super easy for developers on the toolserver to localize their tools with almost no effort at all. More about the techinical aspects of this early next week (or poke me on IRC). I'm currently checking with TranslateWiki what the best way is for them to handle it and finding a balance so that Toolserver developers will have minimal efforts and can focus on the functionality (instead of on the localization), and for TranslateWiki to not have to dig into any kind of technical code. In this thread I'm looking for a suitable name for the system. A few ideas I came up with are below. Let me know what you think about them and/or send in some other ones you think are great! If you like playing with words, this is your call to get creative! I'm planning to pick the initial name [0] this weekend, so don't wait! I've got three =names=, and about a dozen ==abbrevations/nicknames to get you started. = Internationalization (i18n) for Tools' User Interfaces at the Toolserver == Name: Ituit (or ITUIT) (pronounce: " I tweet " or " Étui ") Story: People communicate by talking, as birds do by tweeting - preferably in their native language while using the tools. == Name: Toolserver Intuition Origin: The word "Ituition" is ITUIT (previous name above) + " ion ". Story: Peoples first "ituition" is to speak in their native language. = Toolserver Localization System When abbreviated as TLS it'll be confusing with the existing TLS/SSL technology [2][3]. But the following may be possible derivatives / abbreviations : * Tolocsy (or Tolloxy) * Toolsy * Tolcsy * .. = Internationalization for Toolserver's User Interface == Name: Intoyui (pronounce: " Into you " or " Into you and I ") Story: The i18n system is into you / your native language! == Name: ITUI (pronounce: " i twee " or " Étui " [1][2] ) Story: This i18n is like a bag (French: etui) of messages. == Name: IntoUI (pronounce: " Into U ('you') I ") Story: The tool intergrates into the UI (user interface) of toolserver tools. -- Those were the ideas I had so far. Personally I like the rol of "Toolserver Intuition" and "ITUI" (etui). What do you like and why ? Don't let any of the above limit your imagination, feel free to share any ideas you may have regarding the name. Note: For questions, suggestions or other comments on the system itself, please respond to the mailing I'll send out early next week. I'm currently investigating what aspects to look at and how we can best intergrate this with TranslateWiki in an as simple, flexible yet solid way as possible. Feel free to poke me at IRC [4] -- Krinkle [0] I say initial name because we might change it later on, = but I prefer not to (due to account-, database and class names etc.). [1] http://en.wiktionary.org/wiki/étui [2] http://en.wikipedia.org/wiki/Transport_Layer_Security [3] http://en.wikipedia.org/wiki/TLS [4] https://wiki.toolserver.org/view/IRC irc://irc.freenode.net/wikimedia-toolserver

1 0

Acceptability of MVC framework, general-use tools
by Jesse (Pathoschild) 20 Mar '11

20 Mar '11

Hello, I'm moving some external tools to the Toolserver. I have some questions to help decide which tools are appropriate. 1. The terms of use <https://wiki.toolserver.org/view/Rules> prohibit "large web applications"; does this prevent the use of an MVC framework like Kohana<http://kohanaframework.org/> to serve tool pages? The framework has a fairly low footprint, is maintained by an active community, and provides some nice security features (like rule-based input validation<http://kohanaframework.org/3.1/guide/kohana/security/validation>) which I'd like to use in my scripts. I plan to use some URL-rewriting to serve the pages, so there's no direct access to the underlying framework or MVC files. 2. The terms require that tools "be related to a designated affiliate project". Does this prevent small general-use tools useful to Wikimedians, like regular expression manipulation or set operations? -- Yours cordially, Jesse (Pathoschild)

2 1

A strange thing in huwiki database
by Mihajlo Andjelkovic 18 Mar '11

18 Mar '11

Today I did some analysis over latest revisions on huwiki and there I stumbled on something that surprised me. I believed that revids were given sequentially, so that lower revision id implies an earlier date, and higher revision id implies a later date. Thus, all edits having id greater than 6.000.000 would be no older than august 2009 on huwiki. However, the following revids are anomalies to this, being set 5-6 years back in comparison to their surrounding revids: 8764880, 2004 8764883, 2005 8764884, 2005 8764885, 2005 8764886, 2005 8764887, 2005 8764904, 2004 8764905, 2004 8764906, 2005 8764907, 2005 8764908, 2005 Example: http://hu.wikipedia.org/w/index.php?title=Ornithopoda&oldid=8764883 I don't really want to ask anything, I hope I pointed out something interesting. However, if there be any comments on this, shot away. :) M

8 7

Registration open for the Hackathon in Berlin, May 13-15
by Daniel Kinzler 11 Mar '11

11 Mar '11

Hi all! Wikimedia Germany invites anyone interested in improving MediaWiki to come and join us at or third developer meet-up. Like the last two years, it's going to be awesome! Unlike the last two years, there will be more hacking and less talking - it'll be a Hackathon, not a BarCamp. We'll meet on May 13 to 15, in Berlin, on the 4th floor of the betahaus coworking space <http://betahaus.de/>. There will not be an entrance fee, but registration is mandatory and now open: <http://de.amiando.com/hackathon2011>. Registration will close on April 10. If you like to attend, please register in time! More information can be found at <http://www.mediawiki.org/wiki/Berlin_Hackathon_2011>. The Berlin Hackathon 2011 is an opportunity for MediaWiki hackers to come together, squash bugs and write crazy new features. Our main focus this time around will probably be: * Improving usability / accessibility * Interactive Maps * Fixing the parser * WMF Ops (new data center, virtualization) * Supporting the Wiki Loves Monuments image hunt * Squashing bugs If you have different ideas, please let us know: <http://www.mediawiki.org/wiki/Berlin_Hackathon_2011#Topics> The Hackathon will be hosting the Language committee and Wiki loves Monuments group. There is a limited number of seats reserved for these groups and if you belong to one of them, you should receive an invitation code soon. If you have any doubts or questions, contact us at <hackathon(a)wikimedia.de>. We’re excited to see you in Berlin, your Hackathon Team Daniel Kinzler (Program Coordinator) Nicole Ebber (Logistics) Cornelius Kibelka (Assistant)

1 0

sql-s1 replag
by River Tarnell 09 Mar '11

09 Mar '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Due to the current high replag on thyme, I've switched sql-s1 to rosemary. If your tools require a user database and you haven't updated them to use sql-s1-user instead, you will find your databases are missing. Once the replag is down to a reasonable level I'll switch it back. - river. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk13HAsACgkQIXd7fCuc5vIvdQCffgfjFv9OKmvpoWNY/0+LiTqy IeUAniODzwDbdQG7jcf3Jp/3awc40gLR =MBKi -----END PGP SIGNATURE-----

2 1

Schema changes, now
by River Tarnell 08 Mar '11

08 Mar '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, We're currently applying schema changes from WMF for a MediaWiki upgrade. To reduce replication lag, we will first apply them to an inactive server for each cluster, then switch users to that server and apply them on the other one. The first part has started now, and we will be switching in an hour or two. During the second part, user databases will be unavailable until we finish the changes and switch back. This may take several hours to finish. The exception is s2/s5, which only has one server; these clusters will be lagged during the change. - river. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk11bNYACgkQIXd7fCuc5vIIowCgqKiJvPxtr+B19TRDrbJTy/70 fHQAn3CW69mS7tQPWrRxdMTKYnuFM5St =DpI/ -----END PGP SIGNATURE-----

1 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Toolserver-l