Lots of pages on mediawiki.org are pretty much uneditable because they're strewn with <translate> spanning multiple paragraphs that make VisualEditor completely unusable and the source editor very difficult to use.
I'm trying to update documentation, and I'm seriously thinking of regexing out all the <translate> stuff so I can actually edit the wiki pages.
This is probably not a good thing.
https://phabricator.wikimedia.org/T131516
-- brion
Proper VE/Parsoid support for the <translate> extension has been wanted for a while now. It hasn't made a list of quarterly goals yet. Help wanted, certainly. --scott On Apr 1, 2016 12:32 PM, "Brion Vibber" bvibber@wikimedia.org wrote:
Lots of pages on mediawiki.org are pretty much uneditable because they're strewn with <translate> spanning multiple paragraphs that make VisualEditor completely unusable and the source editor very difficult to use.
I'm trying to update documentation, and I'm seriously thinking of regexing out all the <translate> stuff so I can actually edit the wiki pages.
This is probably not a good thing.
https://phabricator.wikimedia.org/T131516
-- brion _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, Apr 1, 2016 at 7:35 PM, C. Scott Ananian cananian@wikimedia.org wrote:
Proper VE/Parsoid support for the <translate> extension has been wanted for a while now. It hasn't made a list of quarterly goals yet. Help wanted, certainly.
This is a HUGELY important dogfooding goal; we need to be able to edit the wiki about our own product!
Anything we can do to help out?
-- brion
So based on notes from Niklas on the bug, it looks like there are two main problems:
1) VE treats <translate>...</translate> chunks as uneditable extension blobs.
2) Some/many pages use <translate>...</translate> incorrectly: spanning paragraphs, markup level boundaries, etc.
So problem 1) can be improved in two ways: 1a) Have VE treat <translate>...</translate> as more or less like plaintext or a div or something 1b) Much fancier sub-document editing
Fixing problem 2) means refactoring broken pages in ways that presumably would require at least some of their translations to be re-created.
I don't know whether 2) is caused by problems in the translation interface, or by people further editing the pages without a detailed understanding of how the <translate> and <!--T:123--> markup bits work, or a combination of both.
-- brion
On Fri, Apr 1, 2016 at 7:39 PM, Brion Vibber bvibber@wikimedia.org wrote:
On Fri, Apr 1, 2016 at 7:35 PM, C. Scott Ananian cananian@wikimedia.org wrote:
Proper VE/Parsoid support for the <translate> extension has been wanted for a while now. It hasn't made a list of quarterly goals yet. Help wanted, certainly.
This is a HUGELY important dogfooding goal; we need to be able to edit the wiki about our own product!
Anything we can do to help out?
-- brion
I think improving VE to understand translate, and expanding the training/documentation for translate are the best next steps. This is also a problem on Meta-Wiki.
Removing the translate setup from the pages seems problematic as we're trying to make info available in languages beyond just English.
-greg
_______________ Sent from my iPhone - a more detailed response may be sent later.
On Apr 1, 2016, at 9:49 AM, Brion Vibber bvibber@wikimedia.org wrote:
So based on notes from Niklas on the bug, it looks like there are two main problems:
- VE treats <translate>...</translate> chunks as uneditable extension
blobs.
- Some/many pages use <translate>...</translate> incorrectly: spanning
paragraphs, markup level boundaries, etc.
So problem 1) can be improved in two ways: 1a) Have VE treat <translate>...</translate> as more or less like plaintext or a div or something 1b) Much fancier sub-document editing
Fixing problem 2) means refactoring broken pages in ways that presumably would require at least some of their translations to be re-created.
I don't know whether 2) is caused by problems in the translation interface, or by people further editing the pages without a detailed understanding of how the <translate> and <!--T:123--> markup bits work, or a combination of both.
-- brion
On Fri, Apr 1, 2016 at 7:39 PM, Brion Vibber bvibber@wikimedia.org wrote:
On Fri, Apr 1, 2016 at 7:35 PM, C. Scott Ananian cananian@wikimedia.org wrote:
Proper VE/Parsoid support for the <translate> extension has been wanted for a while now. It hasn't made a list of quarterly goals yet. Help wanted, certainly.
This is a HUGELY important dogfooding goal; we need to be able to edit the wiki about our own product!
Anything we can do to help out?
-- brion
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Arlo actually implemented the Parsoid side of this: https://phabricator.wikimedia.org/T50891 We still need VE support: https://phabricator.wikimedia.org/T55974 --scott
On Fri, Apr 1, 2016 at 1:00 PM, Gregory Varnum gregory.varnum@gmail.com wrote:
I think improving VE to understand translate, and expanding the training/documentation for translate are the best next steps. This is also a problem on Meta-Wiki.
Removing the translate setup from the pages seems problematic as we're trying to make info available in languages beyond just English.
-greg
Sent from my iPhone - a more detailed response may be sent later.
On Apr 1, 2016, at 9:49 AM, Brion Vibber bvibber@wikimedia.org wrote:
So based on notes from Niklas on the bug, it looks like there are two
main
problems:
- VE treats <translate>...</translate> chunks as uneditable extension
blobs.
- Some/many pages use <translate>...</translate> incorrectly: spanning
paragraphs, markup level boundaries, etc.
So problem 1) can be improved in two ways: 1a) Have VE treat <translate>...</translate> as more or less like
plaintext
or a div or something 1b) Much fancier sub-document editing
Fixing problem 2) means refactoring broken pages in ways that presumably would require at least some of their translations to be re-created.
I don't know whether 2) is caused by problems in the translation
interface,
or by people further editing the pages without a detailed understanding
of
how the <translate> and <!--T:123--> markup bits work, or a combination
of
both.
-- brion
On Fri, Apr 1, 2016 at 7:39 PM, Brion Vibber bvibber@wikimedia.org
wrote:
On Fri, Apr 1, 2016 at 7:35 PM, C. Scott Ananian <
cananian@wikimedia.org>
wrote:
Proper VE/Parsoid support for the <translate> extension has been wanted for a while now. It hasn't made a list of quarterly goals yet. Help
wanted,
certainly.
This is a HUGELY important dogfooding goal; we need to be able to edit
the
wiki about our own product!
Anything we can do to help out?
-- brion
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Friday, April 1, 2016, C. Scott Ananian cananian@wikimedia.org wrote:
Arlo actually implemented the Parsoid side of this: https://phabricator.wikimedia.org/T50891
<3
We still need VE support: https://phabricator.wikimedia.org/T55974
Cool, will concentrate my efforts there. :)
I hope we can figure out a nice intermediate behavior that keeps the annotations and doesn't act too surprising to the user. I can't dedicate huge amounts of time to it but would be happy to help where I can.
-- brion
--scott
On Fri, Apr 1, 2016 at 1:00 PM, Gregory Varnum <gregory.varnum@gmail.com javascript:;> wrote:
I think improving VE to understand translate, and expanding the training/documentation for translate are the best next steps. This is
also
a problem on Meta-Wiki.
Removing the translate setup from the pages seems problematic as we're trying to make info available in languages beyond just English.
-greg
Sent from my iPhone - a more detailed response may be sent later.
On Apr 1, 2016, at 9:49 AM, Brion Vibber <bvibber@wikimedia.org
javascript:;> wrote:
So based on notes from Niklas on the bug, it looks like there are two
main
problems:
- VE treats <translate>...</translate> chunks as uneditable extension
blobs.
- Some/many pages use <translate>...</translate> incorrectly: spanning
paragraphs, markup level boundaries, etc.
So problem 1) can be improved in two ways: 1a) Have VE treat <translate>...</translate> as more or less like
plaintext
or a div or something 1b) Much fancier sub-document editing
Fixing problem 2) means refactoring broken pages in ways that
presumably
would require at least some of their translations to be re-created.
I don't know whether 2) is caused by problems in the translation
interface,
or by people further editing the pages without a detailed understanding
of
how the <translate> and <!--T:123--> markup bits work, or a combination
of
both.
-- brion
On Fri, Apr 1, 2016 at 7:39 PM, Brion Vibber <bvibber@wikimedia.org
wrote:
On Fri, Apr 1, 2016 at 7:35 PM, C. Scott Ananian <
cananian@wikimedia.org javascript:;>
wrote:
Proper VE/Parsoid support for the <translate> extension has been
wanted
for a while now. It hasn't made a list of quarterly goals yet. Help
wanted,
certainly.
This is a HUGELY important dogfooding goal; we need to be able to edit
the
wiki about our own product!
Anything we can do to help out?
-- brion
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org javascript:; https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org javascript:; https://lists.wikimedia.org/mailman/listinfo/wikitech-l
-- (http://cscott.net) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org javascript:; https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, Apr 1, 2016 at 10:49 AM, Brion Vibber bvibber@wikimedia.org wrote:
- Some/many pages use <translate>...</translate> incorrectly: spanning
paragraphs, markup level boundaries, etc.
The documentation says to do it that way: https://www.mediawiki.org/wiki/Help:Extension:Translate/Page_translation_exa... Is that incorrect?
On Tue, Apr 5, 2016 at 9:12 AM, Ryan Kaldari rkaldari@wikimedia.org wrote:
On Fri, Apr 1, 2016 at 10:49 AM, Brion Vibber bvibber@wikimedia.org wrote:
- Some/many pages use <translate>...</translate> incorrectly: spanning
paragraphs, markup level boundaries, etc.
The documentation says to do it that way:
https://www.mediawiki.org/wiki/Help:Extension:Translate/Page_translation_exa... Is that incorrect?
Ah yes, I went over this in more detail with Niklas on the bug earlier and it turns out to be more complicated than I had originally thought!
The header/section-spanning _looks_ weird and has some edge cases where it fails (some section edit cases can break), but is the most reliable way right now to do it as the other ways break much more visibly. So the documentation is correct in that those are the best practices we have available right now.
-- brion
On Apr 1, 2016 18:32, "Brion Vibber" bvibber@wikimedia.org wrote:
Lots of pages on mediawiki.org are pretty much uneditable because they're strewn with <translate> spanning multiple paragraphs that make
VisualEditor
completely unusable and the source editor very difficult to use.
I'm trying to update documentation, and I'm seriously thinking of regexing out all the <translate> stuff so I can actually edit the wiki pages.
This is probably not a good thing.
This is not only on mediawiki.org but also on meta. I d not unhappy if this replaced by nothing.
Rupert
Is there any way to just flag pages to not get translated? These tags are basically the main reason I've given up on documenting any of my extensions/skins.
On 01/04/16 16:31, Brion Vibber wrote:
Lots of pages on mediawiki.org are pretty much uneditable because they're strewn with <translate> spanning multiple paragraphs that make VisualEditor completely unusable and the source editor very difficult to use.
I'm trying to update documentation, and I'm seriously thinking of regexing out all the <translate> stuff so I can actually edit the wiki pages.
This is probably not a good thing.
https://phabricator.wikimedia.org/T131516
-- brion _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I'm not too aware of the history, but is there good reason we do not have en.mediawiki.org etc? The Translate tag has always seemed like a hack that I've never quite understood. The translate tag causes lots of issues on mobile (impacting usability and performance) due to not playing well with the rest of the language ecosystem.
On Sun, Apr 3, 2016 at 11:00 AM, Isarra Yos zhorishna@gmail.com wrote:
Is there any way to just flag pages to not get translated? These tags are basically the main reason I've given up on documenting any of my extensions/skins.
On 01/04/16 16:31, Brion Vibber wrote:
Lots of pages on mediawiki.org are pretty much uneditable because they're strewn with <translate> spanning multiple paragraphs that make VisualEditor completely unusable and the source editor very difficult to use.
I'm trying to update documentation, and I'm seriously thinking of regexing out all the <translate> stuff so I can actually edit the wiki pages.
This is probably not a good thing.
https://phabricator.wikimedia.org/T131516
-- brion _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Apr 3, 2016 1:30 AM, "Jon Robson" jdlrobson@gmail.com wrote:
The Translate tag has always seemed like a hack that I've never quite
understood.
+1. Couldn't we use Parsoid data tags to identify paragraphs? It seems like that would lend itself to an incremental migration.
-Adam
On Sun, Apr 3, 2016 at 4:13 PM, Adam Wight awight@wikimedia.org wrote:
On Apr 3, 2016 1:30 AM, "Jon Robson" jdlrobson@gmail.com wrote:
The Translate tag has always seemed like a hack that I've never quite
understood.
+1. Couldn't we use Parsoid data tags to identify paragraphs? It seems like that would lend itself to an incremental migration.
i tried it once, at https://meta.wikimedia.org/wiki/Grants:IdeaLab/Inspire/de. if you look at the page its a mix of english and german. it shows 96% translated. if you look at https://meta.wikimedia.org/w/index.php?title=Special:Translate&group=pag... it seems complete. and if you want to edit the original page i feel the user experience nightmarish. the reusability factor to translate articles from de.wikipedia to en.wikipedia or the other way round is zero. if this mess goes away i'd be happy :)
rupert
2016-04-03 11:29 GMT+03:00 Jon Robson jdlrobson@gmail.com:
The Translate tag has always seemed like a hack that I've never quite understood.
I am happy to direct to our documentation [1] anyone who asks, or explain if the documentation is not sufficient.
The word hack can have both positive and negative meanings and it is unclear what do you mean with it here.
The translate tag causes lots of issues on mobile (impacting usability and performance) due to not playing well with the rest of the language ecosystem.
I was under the impression that mobile does not work with the wikitext directly. Translate tags never appear in the parsed wikitext output. Can you please point me to the tasks in Phabricator where these are explained? I am especially curious about the part about "rest of the language ecosystem" because having worked on many parts of it, I don't feel the same way.
[1] https://www.mediawiki.org/wiki/Help:Extension:Translate
-Niklas
On Sun, Apr 3, 2016 at 9:48 PM, Niklas Laxström niklas.laxstrom@gmail.com wrote:
2016-04-03 11:29 GMT+03:00 Jon Robson jdlrobson@gmail.com:
The Translate tag has always seemed like a hack that I've never quite understood.
I am happy to direct to our documentation [1] anyone who asks, or explain if the documentation is not sufficient.
The word hack can have both positive and negative meanings and it is unclear what do you mean with it here.
lol - negative :) but i did not make that connection to translatewiki. how many approaches to "translate" exist now? one for translating articles? a translate extension? something in wikidata? makes 3?
rupert
On Sun, Apr 3, 2016 at 10:48 PM, Niklas Laxström niklas.laxstrom@gmail.com wrote:
2016-04-03 11:29 GMT+03:00 Jon Robson jdlrobson@gmail.com:
The Translate tag has always seemed like a hack that I've never quite understood.
I am happy to direct to our documentation [1] anyone who asks, or explain if the documentation is not sufficient.
The word hack can have both positive and negative meanings and it is unclear what do you mean with it here.
FWIW I didn't mean this in a negative way. Hack is always a neutral word in my vocabulary :-). Possibly a more descriptive term would have been "oddity". I've personally had to push changes in MobileFrontend that felt wrong, but were necessary due to the environmental constraints I worked in. (Side note: In return have to put up with insulting comments such as "MobileFrontend is an abonimation" from people that haven't taken the time to understand why things are done the way they are). I've ignored those people and still consider the work I've done as useful and important, just as I value the work everyone has put into our language support.
The least I can do is elaborate on my brevity earlier (sent via mobile during hackathon :-)) I think Subbu has covered very well my feelings around this so I don't have much to add to what he's said.
My original concern however was not around the documentation, but how we have 2 mechanisms for doing languages - create a separate site or use the translate tag. It wasn't clear to me which predates the other and why that decision was made. The advantage of a separate site is that it is the simplest possible way to translate.. through no effort whatsoever an editor can translate an article on a mobile device for example by navigating to their project and clicking the edit button. On the other hand Special:Translate doesn't work [1] and the current plan is to make it redirect to desktop which is disappointing and I'd guess loses us lots of potential editors (myself included).
The translate tag causes lots of issues on mobile (impacting usability
and
performance) due to not playing well with the rest of the language ecosystem.
I was under the impression that mobile does not work with the wikitext directly. Translate tags never appear in the parsed wikitext output. Can you please point me to the tasks in Phabricator where these are explained? I am especially curious about the part about "rest of the language ecosystem" because having worked on many parts of it, I don't feel the same way.
When we experimented with the first versions of the editor, we found that edits significantly increased when we loaded sections rather than the entire article - so less wikitext leads to greater number of edits. The translate tag makes the wikitext more confusing and bloats it when translations exist, but I've not investigated so this would need more investigation.
To take a more concrete example of impact on mobile - we've made the mobile skin play nicely with language interwiki links (we've even dedicated this entire quarter to improving language switching on mobile web [2] ). On the other hand, the languages tag does not work the same way as an interwiki link. It does it's own thing which is sadly suffering from usability issues on mobile [3].
The point being that when we don't consolidate systems that do similar things, we are losing opportunities to benefit from things elsewhere or waste engineer time fixing 2 identical problems.
The main issues I have are more from an architecture perspective. I would expect a function along the lines of ArticlePage->getLanguages() to exist and be agnostic to where languages come from - translate tag or interwiki link for consumers such as skins and apps.
I hope my views on this are a little clearer to you now and apologies for putting you on the defensive if I did.
I'd love to see our new language switcher compatible with the output of the translate tag and the translation mechanisms available on mobile phones, so readers can view translations and edit seamlessly around our projects.
-Niklas
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[1] https://phabricator.wikimedia.org/T102922 [2] https://phabricator.wikimedia.org/T121919 [3] https://phabricator.wikimedia.org/T106361
2016-04-05 8:51 GMT+03:00 Jon Robson jrobson@wikimedia.org:
Special:Translate doesn't work [1] and the current plan is to make it redirect to desktop which is disappointing and I'd guess loses us lots of potential editors (myself included).
When we developed the new interface for Special:Translate (aka TUX) we did some testing that it works on tablets. Is there way to mark it suitable for tablets alone because we have not designed it for smart phones?
And what can we do for the rest? Let them too access TUX acknowledging it will be heavy and clunky? Would it make sense to generate minimal non-JavaScript version for all the rest using which they can get the job done if they are desperate but without all the advanced features of the regular TUX UI?
Or in short, I am wondering whether "mobile support" is all or nothing, or whether there is some middle way where we can have some quick wins if the alternative is to have no support at all?
To take a more concrete example of impact on mobile - we've made the mobile skin play nicely with language interwiki links (we've even dedicated this entire quarter to improving language switching on mobile web [2] ). On the other hand, the languages tag does not work the same way as an interwiki link. It does it's own thing which is sadly suffering from usability issues on mobile [3].
This is technical debt. After over a year long push on Content Translation, the Language team is now dedicating some time to address high priority bugs in Translate and Universal Language Selector. There is some related work happening such as the "compact interlanguage links out of beta" and a soon 2 year old patch in Translate to improve the display of the language list [1]. It would be useful to look at the big picture here but there likely isn't enough time beyond fixing the most obvious for now.
[1] https://gerrit.wikimedia.org/r/#/c/149585/
I hope my views on this are a little clearer to you now and apologies for putting you on the defensive if I did.
Thanks for the explanation, because it was not obvious to me what are the pain points from your point of view. For the usability issues you listed, they do affect desktop users as well, but I do appreciate that they are more severe on mobile. For performance, is that only about the unsuitable output from <languages> tag or is there more to it?
I'd love to see our new language switcher compatible with the output of the translate tag and the translation mechanisms available on mobile phones, so readers can view translations and edit seamlessly around our projects.
This is helpful and I will consider it when prioritizing work. I am not well aware of your current priorities [2], but if you think this is important, would you consider helping us on this issue?
[2] I am having hard time navigating through outdated pages on mediawiki.org. Is this the place to look: https://www.mediawiki.org/wiki/Reading/Web/Projects ?
-Niklas
On Mon, Apr 11, 2016 at 4:29 PM, Niklas Laxström niklas.laxstrom@gmail.com wrote:
2016-04-05 8:51 GMT+03:00 Jon Robson jrobson@wikimedia.org:
Special:Translate doesn't work [1] and the current plan is to make it redirect to desktop which is disappointing and I'd guess loses us lots of potential editors (myself included).
When we developed the new interface for Special:Translate (aka TUX) we did some testing that it works on tablets. Is there way to mark it suitable for tablets alone because we have not designed it for smart phones?
It depends on what libraries it uses and whether those are mobile friendly. Best thing to do is create a Phabricator task and tag "reading web" and we can start exploring that. Definitely happy to help (https://phabricator.wikimedia.org/T102922#1466929).
Personally, I'd love the reading web team to collaborate with the languages team more - maybe that's something we can try to convince the powers that be to set aside time for as a future quarterly goal.
And what can we do for the rest? Let them too access TUX acknowledging it will be heavy and clunky? Would it make sense to generate minimal non-JavaScript version for all the rest using which they can get the job done if they are desperate but without all the advanced features of the regular TUX UI?
Or in short, I am wondering whether "mobile support" is all or nothing, or whether there is some middle way where we can have some quick wins if the alternative is to have no support at all?
All good questions and I'm not sure of the answer. Amir did a super interesting hack using Whatsapp and translate wiki at the hackathon in Jerusalem. If porting the existing translate interface to mobile proves too cumbersome we might want to explore and test some small lightweight translating tools.
I'm happy to have a 1-on-1 on you or brainstorm some ideas on a wiki page or spending some time hacking on something with you from the mobile angle... whatever makes sense.
To take a more concrete example of impact on mobile - we've made the mobile skin play nicely with language interwiki links (we've even dedicated this entire quarter to improving language switching on mobile web [2] ). On the other hand, the languages tag does not work the same way as an interwiki link. It does it's own thing which is sadly suffering from usability issues on mobile [3].
This is technical debt. After over a year long push on Content Translation, the Language team is now dedicating some time to address high priority bugs in Translate and Universal Language Selector.
Awesome! Let me know how I can support you with this.
There is some related work happening such as the "compact interlanguage links out of beta" and a soon 2 year old patch in Translate to improve the display of the language list [1]. It would be useful to look at the big picture here but there likely isn't enough time beyond fixing the most obvious for now.
[1] https://gerrit.wikimedia.org/r/#/c/149585/
I hope my views on this are a little clearer to you now and apologies for putting you on the defensive if I did.
Thanks for the explanation, because it was not obvious to me what are the pain points from your point of view. For the usability issues you listed, they do affect desktop users as well, but I do appreciate that they are more severe on mobile. For performance, is that only about the unsuitable output from <languages> tag or is there more to it?
Yup the performance angle is secondary but that's pretty much it. I'm also concerned about future performance issues with having to load additional JS code to support it.
I'd love to see our new language switcher compatible with the output of the translate tag and the translation mechanisms available on mobile phones, so readers can view translations and edit seamlessly around our projects.
This is helpful and I will consider it when prioritizing work. I am not well aware of your current priorities [2], but if you think this is important, would you consider helping us on this issue?
Of course. It would be interesting to explore if there's ways we can link the new language overlay with content translation. I'm fairly confident that there are people that would be willing to translate content on mobile!
[2] I am having hard time navigating through outdated pages on mediawiki.org. Is this the place to look: https://www.mediawiki.org/wiki/Reading/Web/Projects ?
Yup https://www.mediawiki.org/wiki/Reading/Web
-Niklas
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
-- Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי http://aharoni.wordpress.com “We're living in pieces, I want to live in peace.” – T. Moore
2016-04-12 23:31 GMT+03:00 Jon Robson jrobson@wikimedia.org:
On Mon, Apr 11, 2016 at 4:29 PM, Niklas Laxström niklas.laxstrom@gmail.com wrote:
2016-04-05 8:51 GMT+03:00 Jon Robson jrobson@wikimedia.org:
Special:Translate doesn't work [1] and the current plan is to make it redirect to desktop which is disappointing and I'd guess loses us lots
of
potential editors (myself included).
When we developed the new interface for Special:Translate (aka TUX) we did some testing that it works on tablets. Is there way to mark it suitable for tablets alone because we have not designed it for smart phones?
It depends on what libraries it uses and whether those are mobile friendly. Best thing to do is create a Phabricator task and tag "reading web" and we can start exploring that. Definitely happy to help (https://phabricator.wikimedia.org/T102922#1466929).
Personally, I'd love the reading web team to collaborate with the languages team more - maybe that's something we can try to convince the powers that be to set aside time for as a future quarterly goal.
And what can we do for the rest? Let them too access TUX acknowledging it will be heavy and clunky? Would it make sense to generate minimal non-JavaScript version for all the rest using which they can get the job done if they are desperate but without all the advanced features of the regular TUX UI?
Or in short, I am wondering whether "mobile support" is all or nothing, or whether there is some middle way where we can have some quick wins if the alternative is to have no support at all?
All good questions and I'm not sure of the answer. Amir did a super interesting hack using Whatsapp and translate wiki at the hackathon in Jerusalem. If porting the existing translate interface to mobile proves too cumbersome we might want to explore and test some small lightweight translating tools.
Telegram, actually, but yeah, thanks a lot for the shout-out :) The code is at https://github.com/amire80/mediawiki-telegram-bot/ ; it was good enough for hackathon demo, but too buggy and insecure to run as a production service, although I plan to make it live Some Time Soon.
The main thing that made building this hack easy is the fine-grained division of the text to small translatable chunks—precisely the thing that the otherwise rightly dreaded <translate> tag provides. Making this chunking more automatic would be one possible strategy, but major development effort will be required for this.
Translation on mobile devices is desperately needed for a whole lot of reasons, and I hope we will be in a completely different situation with regards to this in a couple of years. Four years ago one could barely edit Wikipedia on mobile, and look where we are now, so I'm optimistic.
To Brion and other people who think the page translation markup is annoying and a usability issue: As the (then volunteer) developer who created it, I can only agree.
The way page translations currently works, which is extensively documented at [0], is the result of lots of experimenting with what works and what does not. When developing this feature, I got some first hand experience working with the MediaWiki parser, which was not always easy.
Yes, the wikipage translation feature can and should be improved as technology progresses: we now have a visual editor which did not exist when this feature was developed. However, as shown by statistics [1], these issues do not prevent the feature from being used to translate thousands of pages, including the weekly tech news. The markup issue is not a reason to stop using this feature.
For this quarter the Language team is going to address high priority issues in Translate that can prevent proper use of the page translation feature [2]. Our team is small and has a huge scope of work. Only with help of others it is possible to proceed faster and cover more ground.
To remind us about our visions: we should "make efforts to support the translation of key documents into multiple languages" [3] and we should "provide the essential infrastructure for the support and development of multilingual wiki projects" [4].
I do not wish to spend my time arguing repeatedly for these goals. Instead I want to work on making them happen. My request is that we (especially us English speakers and developers) accept some inconvenience when necessary to support multilingualism. [5]
[0] https://www.mediawiki.org/wiki/Help:Extension:Translate [1] https://www.mediawiki.org/wiki/Wikimedia_Language_engineering/Reports/2016-M... [2] https://phabricator.wikimedia.org/project/profile/1854/ [3] https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Guiding_Principles [4] https://wikimediafoundation.org/wiki/Mission_statement [5] Not limited to this thread, see https://gerrit.wikimedia.org/r/#/c/214893/ and https://phabricator.wikimedia.org/T39797 for examples which are stuck for a long time.
-Niklas
Niklas and the language team: thanks for your efforts in enabling translation features. They are truly important and necessary.
As for the topic of hacks, I feel wikitext's history has been one where people have stepped in to address critical issues / needs that existed at the time with whatever resources were available at that time to get stuff done. They have not necessarily been the most elegant solutions always. So, I think wikitext's history is one of hacks in the best possible sense of the term demonstrating resourcefulness to get things done even while, when looked at from a technical lens, some of those have also been hacks in the negative sense. I don't think there is anything controversial saying that.
As a member of the parsing team, I have been interested in making incremental progress in addressing this technical debt in wikitext markup. We have been slowly trying to nudge wikitext towards semantics that are more structural rather than being purely string-based. We have been making RFC proposals towards this end. Now that Parsoid's utility has been established, we have energy and mental space freed up to think beyond the immediate problem of supporting visual editing. I believe we have a very powerful change-management tool in Parsoid and we should try to leverage it as such.
Anyway, let us think through what might be good ways of solving this translation problem. Some high-level questions in that direction:
1. Given the success of CX, and the increasing use of VE, is explicit wikitext markup still necessary for enabling translation?
2. Identifying document fragments for translation is another instance of the same problem of associating metadata with document fragments *across edits*. Citations, content-translation, comments-as-documentation, authorship information, maintaining-association-between-translated-fragments, etc. are all different instances of this problem. Translate extension seems to be using comments in wikitext markup as one way to maintain this cross-edit-association. Maybe there is value in thinking about this problem broadly.
3. Are there incremental steps we can take to start deprecating the existing <translate> extension solution and move towards some structural metadata solution? Given that translate extension is not used on *all wikis*, looking at the wikis where translation is enabled, is there a possibility of making this a VE-only / CX-only feature? CX, for example, is already doing work to maintain association between translated fragments across document edits.
Subbu.
On 04/04/2016 07:13 AM, Niklas Laxström wrote:
To Brion and other people who think the page translation markup is annoying and a usability issue: As the (then volunteer) developer who created it, I can only agree.
The way page translations currently works, which is extensively documented at [0], is the result of lots of experimenting with what works and what does not. When developing this feature, I got some first hand experience working with the MediaWiki parser, which was not always easy.
Yes, the wikipage translation feature can and should be improved as technology progresses: we now have a visual editor which did not exist when this feature was developed. However, as shown by statistics [1], these issues do not prevent the feature from being used to translate thousands of pages, including the weekly tech news. The markup issue is not a reason to stop using this feature.
For this quarter the Language team is going to address high priority issues in Translate that can prevent proper use of the page translation feature [2]. Our team is small and has a huge scope of work. Only with help of others it is possible to proceed faster and cover more ground.
To remind us about our visions: we should "make efforts to support the translation of key documents into multiple languages" [3] and we should "provide the essential infrastructure for the support and development of multilingual wiki projects" [4].
I do not wish to spend my time arguing repeatedly for these goals. Instead I want to work on making them happen. My request is that we (especially us English speakers and developers) accept some inconvenience when necessary to support multilingualism. [5]
[0] https://www.mediawiki.org/wiki/Help:Extension:Translate [1] https://www.mediawiki.org/wiki/Wikimedia_Language_engineering/Reports/2016-M... [2] https://phabricator.wikimedia.org/project/profile/1854/ [3] https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Guiding_Principles [4] https://wikimediafoundation.org/wiki/Mission_statement [5] Not limited to this thread, see https://gerrit.wikimedia.org/r/#/c/214893/ and https://phabricator.wikimedia.org/T39797 for examples which are stuck for a long time.
-Niklas
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
2016-04-04 16:00 GMT+03:00 Subramanya Sastry ssastry@wikimedia.org:
Niklas and the language team: thanks for your efforts in enabling translation features. They are truly important and necessary.
And I want to thank you for your positive and constructive approach for solving this issue.
- Given the success of CX, and the increasing use of VE, is explicit
wikitext markup still necessary for enabling translation?
I am actually eager to try out non mark-up approach for Translate's page translation feature. I am not aware of any fundamental reason for not being able to work without additional wikitext mark-up. But I would like to clarify some things about the relation of Translate (specifically its page translation feature) and Content Translation (CX).
The use case for CX is very different from Translate's page translation. Former supports one-time "free form" translation from one language to one language. The latter supports maintaining content-preserving translations from one language to hundreds of languages so that translations can be kept in sync when the source text changes.
Along the same line the occasional suggestions about replacing either extension with the other one is not practical. My opinion is that the way to bring these closer together is to take the best parts of both (and also others like VE) and combine them to produce tools which are tailored for each use case and which are consistent in appearance and functionality. Translate does so much more that it is easier to add a new translation interface to Translate than it is to re-implement all the backend tracking functionality in CX.
- Identifying document fragments for translation is another instance of the
same problem of associating metadata with document fragments *across edits*. Citations, content-translation, comments-as-documentation, authorship information, maintaining-association-between-translated-fragments, etc. are all different instances of this problem. Translate extension seems to be using comments in wikitext markup as one way to maintain this cross-edit-association. Maybe there is value in thinking about this problem broadly.
Definitely. As we have discussed earlier, the definition of what is a "breaking change" does vary between the different use cases, and in fact is left to a humans (translation admins) to decide in the page translation feature. But the other approach of each tool implementing it from scratch is not ideal either.
One thing to remember here is that there are two goals with the current mark-up. The association of content across edits is only one of them (these are the T-comment you refer to). The other goal is to specify what is and what is not translatable. For example, one frequent use case is to mark for translation only the image captions but not rest of the image wikitext mark-up. I am hopeful that some heuristics with additional tools for manual correction would reach good results for this goal.
Yet another thing related to this is that we will want to support WYSIWYG/HTML translation in Translate. As you might know already, Special:Translate has two modes: the list view and the page view. Page view should be replaced with interface not much unlike CX, but we also need some support for the list view.
- Are there incremental steps we can take to start deprecating the existing
<translate> extension solution and move towards some structural metadata solution? Given that translate extension is not used on *all wikis*, looking at the wikis where translation is enabled, is there a possibility of making this a VE-only / CX-only feature? CX, for example, is already doing work to maintain association between translated fragments across document edits.
What to do with the translate tags is a delicate question and whether to stop supporting them altogether is a question that has lots of dependencies such as how long there will exist third party wikis (where Translate is also used a lot) that prefer to use wikitext instead of VE and parsoid or alternatives. See also my thoughts to your first question.
Right now my answer is no: we cannot make this implementation of the feature VE or CX only. My preference is to start developing a parallel mark-up-less system and to provide migration tools. This new system can depend on VE and parsoid and other modern functionality. Its implementation will require cross team planning and collaboration.
-Niklas
On Mon, Apr 4, 2016 at 5:21 PM, Niklas Laxström niklas.laxstrom@gmail.com wrote:
The use case for CX is very different from Translate's page translation. Former supports one-time "free form" translation from one language to one language. The latter supports maintaining content-preserving translations from one language to hundreds of languages so that translations can be kept in sync when the source text changes.
*nod* one big difficulty with doing a CX-like model for documentation pages is in synchronizing updates...
I think that in combination with good diff/changeset displays, the CX model can be extended to support updating translations for edited pages when we're translating the way we do documentation: with one master copy, usually English, that has translated 'clones' in other languages. It would likely be much harder to support back-and-forth edits for multilingual pages that aren't required to be kept in sync with a master, as with Wikipedia articles in different languages.
It would be a little different from how <translate> blocks work in that when new text is added and hasn't been translated yet, you don't see the updated English text on your localized page. But we can automate a marker that says the English page has been updated and can link you back to it.
- Identifying document fragments for translation is another instance of
the
same problem of associating metadata with document fragments *across
edits*.
Citations, content-translation, comments-as-documentation, authorship information, maintaining-association-between-translated-fragments, etc.
are
all different instances of this problem. Translate extension seems to be using comments in wikitext markup as one way to maintain this cross-edit-association. Maybe there is value in thinking about this
problem
broadly.
Definitely. As we have discussed earlier, the definition of what is a "breaking change" does vary between the different use cases, and in fact is left to a humans (translation admins) to decide in the page translation feature. But the other approach of each tool implementing it from scratch is not ideal either.
*nod* with a master->translation model, this is something that can be human-assisted; note that good heuristics in diff detection can already do things like detect moved paragraphs, which should help.
One thing to remember here is that there are two goals with the current mark-up. The association of content across edits is only one of them (these are the T-comment you refer to). The other goal is to specify what is and what is not translatable. For example, one frequent use case is to mark for translation only the image captions but not rest of the image wikitext mark-up. I am hopeful that some heuristics with additional tools for manual correction would reach good results for this goal.
Yet another thing related to this is that we will want to support WYSIWYG/HTML translation in Translate. As you might know already, Special:Translate has two modes: the list view and the page view. Page view should be replaced with interface not much unlike CX, but we also need some support for the list view.
+1 on all this. :)
For marking untranslatable bits, I suspect it may be better to default all text to translatable except where code blocks are explicitly marked. We can perhaps provide a markup like <nowiki> for <notranslate> as well, which should be something that can be integrated in both source and visual editors.
- Are there incremental steps we can take to start deprecating the
existing
<translate> extension solution and move towards some structural metadata solution? Given that translate extension is not used on *all wikis*,
looking
at the wikis where translation is enabled, is there a possibility of
making
this a VE-only / CX-only feature? CX, for example, is already doing work
to
maintain association between translated fragments across document edits.
What to do with the translate tags is a delicate question and whether to stop supporting them altogether is a question that has lots of dependencies such as how long there will exist third party wikis (where Translate is also used a lot) that prefer to use wikitext instead of VE and parsoid or alternatives. See also my thoughts to your first question.
Right now my answer is no: we cannot make this implementation of the feature VE or CX only. My preference is to start developing a parallel mark-up-less system and to provide migration tools. This new system can depend on VE and parsoid and other modern functionality. Its implementation will require cross team planning and collaboration.
+1
-- brion
- Identifying document fragments for translation is another instance of the
same problem of associating metadata with document fragments *across edits*. Citations, content-translation, comments-as-documentation, authorship information, maintaining-association-between-translated-fragments, etc. are all different instances of this problem. Translate extension seems to be using comments in wikitext markup as one way to maintain this cross-edit-association. Maybe there is value in thinking about this problem broadly.
Definitely. As we have discussed earlier, the definition of what is a "breaking change" does vary between the different use cases, and in fact is left to a humans (translation admins) to decide in the page translation feature. But the other approach of each tool implementing it from scratch is not ideal either.
Slightly, but not entirely tangentially, here are some old notes of mine about metadata markup. Don't get hung up on the specific form of the syntax. And, I am also not proposing that we change cite to use that new markup ... I am mostly noting that cite is not "that special" in terms of this model of metadata annotations on dom nodes.
https://www.mediawiki.org/wiki/User:SSastry_%28WMF%29/Notes/Wikitext#Core_id...
I have not updated these notes for over a year now ... so, it might be dusty in there ..
Subbu.
wikitech-l@lists.wikimedia.org