This looks particularly interesting if language variants are on your radar.
---------- Forwarded message ----------
From: *Daniel Kinzler* <daniel(a)brightbyte.de>
Date: Friday, February 5, 2016
Subject: [Wikitech-l] Per-language URLs for multilingual wikis (RFC T114662)
To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
Hi all!
During next Wednesday's RFC meeting on IRC, we are planning to discuss
introducing "Per-language URLs for pages of multilingual wikis". This is an
exploratory discussion, with no expectation of a final decision. We are
looking
for concerns and suggestions.
If you care about improved support for multi-lingual wikis, please visit
<https://phabricator.wikimedia.org/T114662> and comment before next week's
"live" meeting on IRC. And of yourse, join that meeting, if you like!
Thanks,
Daniel
PS: Don't want to click the link? Let me copy&paste that for you... but
please
comment on phab rather than here.
Context:
Some wikis show page content depending on the user's interface language (for
instance wikidata.org and commons.wikimedia.org, or other wikis using the
Translate extension). All language versions (renderings) of a page are
served
from the same URL, which is problematic for web caching. This is the reason
we
currently do not allow anonymous users to change their interface language.
This
makes it hard for use multilingual wikis without logging in.
See also for context:
* {T114640}
Proposal:
* For multilingual wikis, the user language is part of the request path:
e.g. we
would use `/wiki-de/` instead of `/wiki/`.
* The plain `/wiki/` path would act as a 302 redirect to the language
specific
path (based on the user's language, or a best guess or cookie as
implemented by ULS)
* When viewing a page via a language-specific path, all links on the page
(both
content and skin) point to pathes specific to that language. Both content
and
skin are shown in the user's ui language (as far as possible, using whatever
mechanism for content translation or internationalization is available)
* When viewing a page in a language different from the user's preferred
language
(according to user preferences or the cookie set by ULS), a warnign bar is
shown
at the top, giving the user the option to
## switch to the version in their own language (according to user
preference)
## change their user language to the current page's language
## hide the bar for a while (a day, or the browser session, or so).
Challanges:
* Make the Linker class aware of the target language (probably needs a
complete
refactoring), so it generates links to the right path.
* Make all code that generates links in the skin use the Linker class
(directly,
or via the Parser), so the path is consistent.
* Allow efficient purging of the entire "bundle" of all the renderings of a
given page when the page's content changes.
* Should translated names for namespaces and special pages be supported on
the
language specific pathes? (would be nice, but tricky)
* Provide a way to explicitly link to a specific language rendering from
wikitext, e.g. `{{#link|Foo|lang=de-ch}}`
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org <javascript:;>
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
The Android team[0] is pleased to announce a new Wikipedia Android app
beta release, v2.1.140-beta-2016-02-04[1]. This revision contains the
following new functionality[2]:
- Support for animated images throughout the app (gallery, search,
history, etc).
- Media saved in the gallery are now downloaded in their original form
and resolution.
- Improved app memory consumption.
- Resize wide images to fit without scrolling.
- Fix transparent image canvases in galleries.
- Fix copying large amounts of text to clipboard.
- Fix account creation without an email address.
- Bug fix to keep the toolbar hidden when scrolling down quickly.
Included in this version are patches from repeat contributor Dan
Garry[3] and first time contributor, Justin Du[4]. Great work, devs!
You too can help make it better! Read our getting started guide[5]. We
can't wait for your contributions!
-The WMF Android team
[0] https://www.mediawiki.org/wiki/Wikimedia_Apps/Team#Android_App
[1] Rolling out at
https://play.google.com/store/apps/details?id=org.wikipedia.beta
[2] A complete list of changes is available at
https://phabricator.wikimedia.org/diffusion/APAW/history/master/;beta/2.1.1…
[3] https://twitter.com/danjgarry, IRC: Deskana
[4] IRC: MtDu
[5]
https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/Wikipedia_Android_app_ha…
A security vulnerability has been discovered in MediaWiki setups which
use both Wikibase and MobileFrontend.
All projects in the Wikimedia cluster have been since patched but if
you use these two extensions please be sure to apply the fix.
Patch file and issue are documented on https://phabricator.wikimedia.org/T125684
By the way, a meeting has been scheduled for 1920 UTC on Thursday,
4-February-2016 to go into technical specifics on PageImages.
Email me off list if you'd like to be added to the meeting.
-Adam
---------- Forwarded message ----------
From: Adam Baso <abaso(a)wikimedia.org>
Date: Sat, Jan 30, 2016 at 8:11 AM
Subject: Re: [WikimediaMobile] Similar articles feature performance in
CirrusSearch for apps and mobile web
To: Erik Bernhardson <ebernhardson(a)wikimedia.org>
Cc: Joaquin Oltra Hernandez <jhernandez(a)wikimedia.org>, mobile-l <
mobile-l(a)lists.wikimedia.org>
Okay. As per https://phabricator.wikimedia.org/T124225#1984080 I think if
we're doing near term experimentation with a controlled A/B test the
Android app is the only logical place to start. Dmitry, can that work for
you? It's not required, but I think it would be neat to see if we can move
the needle even more. Of course your quarterly goals take top
priority...but what do you think?
On Sat, Jan 23, 2016 at 5:58 AM, Adam Baso <abaso(a)wikimedia.org> wrote:
> Hey all, am planning to look at Phabricator tasks and provide a reply
> during the upcoming weekdays. Just wanted to acknowledge I saw your replies!
>
>
> On Friday, January 22, 2016, Erik Bernhardson <ebernhardson(a)wikimedia.org>
> wrote:
>
>> On Thu, Jan 21, 2016 at 1:29 AM, Joaquin Oltra Hernandez <
>> jhernandez(a)wikimedia.org> wrote:
>>
>>> Regarding the caching, we would need to agree between apps and web about
>>> the url and smaxage parameter as Adam noted so that the urls are
>>> *exactly* the same to not bloat varnish and reuse the same cached
>>> objects across platforms.
>>>
>>> It is an extremely adhoc and brittle solution but seems like it would be
>>> the greatest win.
>>>
>>> 20% of the traffic from searches by being only in android and web beta
>>> seems a lot to me, and we should work on reducing it, otherwise when it
>>> hits web stable we're going to crush the servers, so caching seems the
>>> highest priority.
>>>
>>> To clarify its 20% of the load, as opposed to 20% of the traffic. But
>> same difference :)
>>
>>
>>> Let's chime in https://phabricator.wikimedia.org/T124216 and continue
>>> the cache discussion there.
>>>
>>> Regarding the validity of results with opening text only, how should we
>>> proceed? Adam?
>>>
>>> I've put together https://phabricator.wikimedia.org/T124258 to track
>> putting together an AB test that measures the difference in click through
>> rates for the two approaches.
>>
>>
>>
>>> On Wed, Jan 20, 2016 at 9:34 PM, David Causse <dcausse(a)wikimedia.org>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Yes we can combine many factors, from templates (quality but also
>>>> disambiguation/stubs), size and others.
>>>> Today cirrus uses mostly the number of incoming links which (imho) is
>>>> not very good for morelike.
>>>> On enwiki results will also be scored according the weights defined in
>>>> https://en.wikipedia.org/wiki/MediaWiki:Cirrussearch-boost-templates.
>>>>
>>>> I wrote a small bash to compare results :
>>>> https://gist.github.com/nomoa/93c5097e3c3cb3b6ebad
>>>> Here is some random results from the list (Semetimes better, sometimes
>>>> worse) :
>>>>
>>>> $ sh morelike.sh Revolution_Muslim
>>>> Defaults
>>>> "title": "Chess",
>>>> "title": "Suicide attack",
>>>> "title": "Zachary Adam Chesser",
>>>> =======
>>>> Opening text no boost links
>>>> "title": "Hungarian Revolution of 1956",
>>>> "title": "Muslims for America",
>>>> "title": "Salafist Front",
>>>>
>>>> $ sh morelike.sh Chesser
>>>> Defaults
>>>> "title": "Chess",
>>>> "title": "Edinburgh",
>>>> "title": "Edinburgh Corn Exchange",
>>>> =======
>>>> Opening text no boost links
>>>> "title": "Dreghorn Barracks",
>>>> "title": "Edinburgh Chess Club",
>>>> "title": "Threipmuir Reservoir",
>>>>
>>>> $ sh morelike.sh Time_%28disambiguation%29
>>>> Defaults
>>>> "title": "Atlantis: The Lost Empire",
>>>> "title": "Stargate",
>>>> "title": "Stargate SG-1",
>>>> =======
>>>> Opening text no boost links
>>>> "title": "Father Time (disambiguation)",
>>>> "title": "The Last Time",
>>>> "title": "Time After Time",
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Le 20/01/2016 19:34, Jon Robson a écrit :
>>>>
>>>>> I'm actually interested to see whether this yields better results in
>>>>> certain examples where the algorithm is lacking [1]. If it's done as
>>>>> an A/B test we could even measure things such as click throughs in the
>>>>> related article feature (whether they go up or not)
>>>>>
>>>>> Out of interest is it also possible to take article size and type into
>>>>> account and not returning any morelike results for things like
>>>>> disambiguation pages and stubs?
>>>>>
>>>>> [1] https://www.mediawiki.org/wiki/Topic:Swsjajvdll3pf8ya
>>>>>
>>>>>
>>>>> On Wed, Jan 20, 2016 at 9:47 AM, Adam Baso <abaso(a)wikimedia.org>
>>>>> wrote:
>>>>>
>>>>>> One thing we could do regarding the quality of the output is check
>>>>>> results
>>>>>> against a random sample of popular articles (example approach to find
>>>>>> some
>>>>>> articles) on mdot Wikipedia. Presuming that improves the quality of
>>>>>> the
>>>>>> recommendations or at least does not degrade them, we should consider
>>>>>> adding
>>>>>> the enhancement task to a future sprint, with further instrumentation
>>>>>> and
>>>>>> A/B testing / timeboxed beta test, etc.
>>>>>>
>>>>>> Joaquin, smaxage (e.g., 24 hour cached responses) does seem a good
>>>>>> fix for
>>>>>> now for further reduction of client perceived wait, at least for
>>>>>> non-cold
>>>>>> cache requests, even if we stop beating up the backend. Does anyone
>>>>>> know of
>>>>>> a compelling reason to not do that for the time being? The main thing
>>>>>> that
>>>>>> comes to mind as always is growing the Varnish cache object pool -
>>>>>> probably
>>>>>> not a huge deal while the thing is only in beta, but on the stable
>>>>>> channel
>>>>>> maybe noteworthy because it would run on probably most pages (but
>>>>>> that's
>>>>>> what edge caches are for, after all).
>>>>>>
>>>>>> Erik, from your perspective does use of smaxage relieve the backend
>>>>>> sufficiently?
>>>>>>
>>>>>> If we do smaxage, then Web, Android, iOS should standardize their
>>>>>> URLs so we
>>>>>> get more cache hits at the edge across all clients. Here's the URL I
>>>>>> see
>>>>>> being used on the web today from mobile web beta:
>>>>>>
>>>>>>
>>>>>> https://en.m.wikipedia.org/w/api.php?action=query&format=json&formatversion…
>>>>>>
>>>>>>
>>>>>> -Adam
>>>>>>
>>>>>> On Wed, Jan 20, 2016 at 7:45 AM, Joaquin Oltra Hernandez
>>>>>> <jhernandez(a)wikimedia.org> wrote:
>>>>>>
>>>>>>> I'd be up to it if we manage to cram it up in a following sprint and
>>>>>>> it is
>>>>>>> worth it.
>>>>>>>
>>>>>>> We could run a controlled test against production with a long batch
>>>>>>> of
>>>>>>> articles and check median/percentiles response time with repeated
>>>>>>> runs and
>>>>>>> highlight the different results for human inspection regarding
>>>>>>> quality.
>>>>>>>
>>>>>>> It's been noted previously that the results are far from ideal
>>>>>>> (which they
>>>>>>> are because it is just morelike), and I think it would be a great
>>>>>>> idea to
>>>>>>> change the endpoint to a specific one that is smarter and has some
>>>>>>> cache (we
>>>>>>> could do much more to get relevant results besides text similarity,
>>>>>>> take
>>>>>>> into account links, or see also links if there are, etc...).
>>>>>>>
>>>>>>> As a note, in mobile web the related articles extension allows
>>>>>>> editors to
>>>>>>> specify articles to show in the section, which would avoid queries to
>>>>>>> cirrussearch if it was more used (once rolled into stable I guess).
>>>>>>>
>>>>>>> I remember that the performance related task was closed as resolved
>>>>>>> (https://phabricator.wikimedia.org/T121254#1907192), should we
>>>>>>> reopen it or
>>>>>>> create a new one?
>>>>>>>
>>>>>>> I'm not sure if we ended up adding the smaxage parameter (I think we
>>>>>>> didn't), should we? To me it seems a no-brainer that we should be
>>>>>>> caching
>>>>>>> this results in varnish since they don't need to be completely up to
>>>>>>> date
>>>>>>> for this use case.
>>>>>>>
>>>>>>> On Tue, Jan 19, 2016 at 11:54 PM, Erik Bernhardson
>>>>>>> <ebernhardson(a)wikimedia.org> wrote:
>>>>>>>
>>>>>>>> Both mobile apps and web are using CirrusSearch's morelike: feature
>>>>>>>> which
>>>>>>>> is showing some performance issues on our end. We would like to
>>>>>>>> make a
>>>>>>>> performance optimization to it, but before we would prefer to run
>>>>>>>> an A/B
>>>>>>>> test to see if the results are still "about as good" as they are
>>>>>>>> currently.
>>>>>>>>
>>>>>>>> The optimization is basically: Currently more like this takes the
>>>>>>>> entire
>>>>>>>> article into account, we would like to change this to take only the
>>>>>>>> opening
>>>>>>>> text of an article into account. This should reduce the amount of
>>>>>>>> work we
>>>>>>>> have to do on the backend saving both server load and latency the
>>>>>>>> user sees
>>>>>>>> running the query.
>>>>>>>>
>>>>>>>> This can be triggered by adding these two query parameters to the
>>>>>>>> search
>>>>>>>> api request that is being performed:
>>>>>>>>
>>>>>>>> cirrusMltUseFields=yes&cirrusMltFields=opening_text
>>>>>>>>
>>>>>>>>
>>>>>>>> The API will give a warning that these parameters do not exist, but
>>>>>>>> they
>>>>>>>> are safe to ignore. Would any of you be willing to run this test?
>>>>>>>> We would
>>>>>>>> basically want to look at user perceived latency along with click
>>>>>>>> through
>>>>>>>> rates for the current default setup along with the restricted setup
>>>>>>>> using
>>>>>>>> only opening_text.
>>>>>>>>
>>>>>>>> Erik B.
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Mobile-l mailing list
>>>>>>>> Mobile-l(a)lists.wikimedia.org
>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>>>>>>>
>>>>>>>>
>>>>>> _______________________________________________
>>>>>> Mobile-l mailing list
>>>>>> Mobile-l(a)lists.wikimedia.org
>>>>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>>>>>
>>>>>> _______________________________________________
>>>>> Mobile-l mailing list
>>>>> Mobile-l(a)lists.wikimedia.org
>>>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Mobile-l mailing list
>>>> Mobile-l(a)lists.wikimedia.org
>>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>>>
>>>
>>>
>>> _______________________________________________
>>> Mobile-l mailing list
>>> Mobile-l(a)lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>>
>>>
>>
I'm enjoying the "nearby" feature now that its been restored to the
Android app; and especially looking forward to using it on an upcoming
overseas trip.
However, I don't seem to be able too use it, while at home, to look at
other parts of the world. Is that possible?
It could either be by scrolling or typing in the name of a far-flung
place, or by tapping on the coordinates in a geo-tagged article.
--
Andy Mabbett
@pigsonthewing
http://pigsonthewing.org.uk