Hello and welcome to the latest edition of the WMF Engineering Roadmap
and Deployment update.
The full log of planned deployments next week can be found at:
<https://wikitech.wikimedia.org/wiki/Deployments#Week_of_May_19th>
A quick list of notable items...
== Monday ==
* Enable CirrusSearch as primary search backend on zh-yue wikipedia.
** <http://www.mediawiki.org/wiki/Search>
* Enable anonymous editor acquisition experiment in GettingStarted on
enwiki, dewiki, frwiki, itwiki and test wikis.
== Tuesday ==
* MediaWiki deploy
** group1 to 1.24wmf5: All non-Wikipedia sites (Wiktionary, Wikisource,
Wikinews, Wikibooks, Wikiquote, Wikiversity, and a few other sites)
** <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf5>
* Enable Flow at mw:Talk:Wikibase/Beta Features/Other projects sidebar,
mw:Talk:Design, and mw:Talk:Phabricator/Help
** <https://www.mediawiki.org/wiki/Flow>
== Wednesday ==
* Swapping GeoData to use Elasticsearch as its backend everywhere.
** <http://www.mediawiki.org/wiki/Extension:GeoData>
== Thursday ==
* MediaWiki deploy
** group2 to 1.24wmf4 (all Wikipedias)
** group0 to 1.24wmf5 (test/test2/testwikidata/mediawiki)
* MediaViewer
** English, German, Italian and Russian Wikipedias, and all Wikisources, in
stages.
** <
https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan#Timeline
>
Thanks and as always, questions and comments welcome,
Greg
Hi All,
Please excuse me for sending an HTML attachment to this list. There is
a need to display some part of it with color.
[1]Bug 13462 - Enhance line matching in diffs; An [2]example diff
similar to what I specifically have a problem with on one of Wikimedia
projects for several months: contributors often add and remove line
breaks making paragraph edits hard to analyse.
Effectively I got this (in a table, but tables are hard to read in
emails):
1) -
2) - foobar lin3
1) + foobar line
2)
But I would find this more useful:
1) -
2) - foobar lin3
1)
2) + foobar line
Please participate by providing insight on potential fixes or
workarounds (don't expect me to be able to read the backend codebase of
this software). Thanks.
Gryllida.
References
1. https://bugzilla.wikimedia.org/show_bug.cgi?id=13462
2. https://test.wikipedia.org/w/index.php?diff=199552&oldid=199551
On Thu, May 15, 2014 at 11:53 PM, Andrew Gray <andrew.gray(a)dunelm.org.uk>wrote:
> Definitely agree that we needed something like this. There's a lot of
> confusion about what Wikidata is for, and what is and isn't appropriate for
> it - both from outsiders and from within the Wikimedia community. I've seen
> vague "it's data, it'll go on Wikidata" a few times, which is a bit like
> saying "it's text, it'll go on Wikipedia" ;-)
>
Actually during the Hackathon somebody said: "We need a new project... a
sort of Wiki...Data...Source?" :D And it is great that you make the analogy
of Wikipedia-and-Text because when you think about it, Wikipedia went
through exactly the same process until "Project Sourceberg" was
characterized ;-)
Micru
Sorry for the short notice. Today at 2100 UTC, instead of the regular
RfC discussion, we'll talk about the performance guidelines draft in
#wikimedia-office . We discussed this some in Zurich but I'd love a
chance to ask some followup questions to firm everything up. I'd also
welcome the chance to explain the two similar documents I'm working on:
architecture and security guidelines.
https://www.mediawiki.org/wiki/Architecture_meetings/Performance_guidelines…
Time:
http://www.worldtimebuddy.com/?qm=1&lid=2950159,5128581,2147714,100&h=51285…
11pm Berlin
5pm NYC
2pm San Francisco
7am Sydney
Next week it'll be an RfC chat. :) (I welcome volunteers!)
--
Sumana Harihareswara
Senior Technical Writer
Wikimedia Foundation
I am trying to figure out how thumbnail retrieval & caching works right
now - with Swift, and the frontline & secondary ("frontend" and
"backend") Varnishes. (I am working on the caching-related bit of the
performance guidelines, and want to understand and help push forward on
https://www.mediawiki.org/wiki/Requests_for_comment/Simplify_thumbnail_cache
.) I looked for docs but didn't find anything that had been updated this
year.
Here's how I think it works, assuming you are a MediaWiki developer
who's written, e.g., a page that includes a thumbnail of an image:
First, your code must get the metadata about the image, which might come
from the local database, or memcached, or Commons. Then, you need to get
a thumbnail of the image at the dimensions your page requires. Rather
than create the thumbnail immediately on demand via parsing the filename
and dimensions, Wikimedia's MediaWiki is configured to use the "404
handler." (see [[Manual:Thumb_handler.php]]) Your page first receives a
URL indicating the eventual location of the thumbnail, then the browser
asks for that URL. If it hasn't been created yet, the web server
initially gets an internal 404 error; the 404 handler then kicks off the
thumbnailer to create the thumbnail, and the response gets sent to the
client.
As it is sent to the client, each thumbnail is stored in a Swift store
and stored in our frontline and secondary Varnish caches.
(The Varnish caches cache entire HTTP responses, including thumbnails of
images, frequently-requested pages, ResourceLoader modules, and similar
items that can be retrieved by URL. The frontline Varnishes keep these
in memory. (A weighted-random load balancer (LVS) distributes web
requests to the front-end Varnishes.) But if a frontline Varnish doesn't
have a response cached, it passes the request to the secondary Varnishes
via hash-based load balancing (on the hash of the URL). The secondary
Varnishes hold more responses, storing them ondisk. Every URL is on at
most one secondary Varnish.)
So, at the end of this whole process, any given thumbnail is in:
* the Swift thumbnail store (and will persist until the canonical image
changes, or is deleted, or we run out of space and flush Swift)
* the frontline and secondary Varnishes (and will persist until the
canonical image changes, or is deleted, or we restart the frontline
Varnishes or we evict data from the hard disks of the secondary Varnishes)
Is this right?
--
Sumana Harihareswara
Senior Technical Writer
Wikimedia Foundation
Hi,
During the Zürich Hackathon I met several people that looked for solutions
about how to integrate external open datasets into our projects (mainly
Wikipedia, Wikidata). Since Wikidata is not the right tool to manage them
(reasons explained in the RFC as discussed during the Wikidata session), I
have felt convenient to centralize the discussion about potential
requirements, needs, and how to approach this new changing landscape that
didn't exist a few years ago.
You will find more details here
https://meta.wikimedia.org/wiki/Requests_for_comment/How_to_deal_with_open_…
Your comments, thoughts and ideas are appreciated!
Cheers,
Micru
Hello everyone,
It's a testament to either how awesome our people are… or just how notorious I am for not announcing things promptly that I noticed this sitting in my Google Docs the other day with the note from the VisualEditor Team: "terry, we and Alex think this is now good for you to post, if you're OK with it."
I'm including the letter as is. Please note the date at the end.
…
Hello all,
It is with great pleasure that I’m announcing that Alex Monk[0] has joined the Wikimedia Foundation as a contractor in Features Engineering.
Alex has been a great, wide-ranging and hugely supporting volunteer developer for 2 years on MediaWiki core and several extensions, including LiquidThreads and Echo, helping us get rid of outdated extensions once and for all, and fixing general site issues.
Alex will be working with the VisualEditor[3] team to help better integrate the editor into MediaWiki, and create and improve the editing tools, as well as fixes and support for MediaWiki core and other extensions as needed.
Alex is currently a student in England, and will be working with us part-time whilst he continues his studies. In his spare time, Alex was recently part of a team at his college competing in this year’s Student Robotics[4] competition, and also enjoys playing online computer games.
We’re delighted that he’s agreed to join us, with his first official day Monday, 10 March.
Please join me in an only-slightly-belated welcome of Alex to the Wikimedia Foundation.
Take care,
Terry
[0]: [[mw:User:Krenair]]
[1]: [[mw:Extension:LiquidThreads]]
[2]: [[mw:Extension:Echo]]
[3]: [[mw:VisualEditor]]
[4]: [[w:en:Student Robotics]]
…
Even if the staff tries to cover up for me, they now know that I'll somehow conspire to forget to do so until two months later. :-D
Please join us in a very-belated welcome of Alex to the Wikimedia Foundation!
Take care,
terry
terry chay 최태리
Director of Features Engineering
Wikimedia Foundation
“Imagine a world in which every single human being can freely share in the sum of all knowledge. That's our commitment.”
p: +1 (415) 839-6885 x6832
m: +1 (408) 480-8902
e: tchay(a)wikimedia.org
i: http://terrychay.com/
w: http://meta.wikimedia.org/wiki/User:Tychay
aim: terrychay
Tim I completely agree. This is something we need to setup.
Patches very much welcomed! :-)
On Wed, May 14, 2014 at 7:51 AM, Tim Alder <tim(a)alder-digital.de> wrote:
> I think the most important feature is to create on serverside a
> thumbnail for each map by using something like http://phantomjs.org/
> This thumbnails should than be in the big WMF caches. The map would
> become interactively only in the case a user click on it.
> This would reduce the numbers of request for loading a page and JS
> overhead and it would increase the stability of the system.
> Without this feature I afraid to never see the extension live in Wikipedia.
>
> Other nice features you can see at umap.openstreetmap.fr:
> *Choosing different backgrounds
> *POIs with interactive descriptions
> *Geometry import from OSM (WIWOSM)
> *different layers
> *...
>
> Greeting Tim alias Kolossos
>
>
> Am 14.05.2014 00:34, schrieb Jon Robson:
>> During the Zurich hackathon, DJ Hartman, Aude and I knocked up a
>> generic maps prototype extension [1]. We have noticed that many maps
>> like extensions keep popping up and believed it was time we
>> standardised on one that all these extensions could use so we share
>> data better.
>>
>> We took a look at all the existing use cases and tried to imagine what
>> such an extension would look like that wouldn't be too tied into a
>> specific use case.
>>
>> The extension we came up with was a map extension that introduces a
>> Map namespace where data for the map is stored in raw GeoJSON and can
>> be edited via a JavaScript map editor interface. It also allows the
>> inclusion of maps in wiki articles via a map template.
>>
>> Dan Andreescu also created a similar visualisation namespace which may
>> want to be folded into this as a map could be seen as a visualisation.
>> I invite Dan to comment on this with further details :-)!
>>
>> I'd be interested in people's thoughts around this extension. In
>> particular I'd be interested in the answer to the question "For my
>> usecase A what would the WikiMaps extension have to support for me to
>> use it".
>>
>> Thanks for your involvement in this discussion. Let's finally get a
>> maps extension up on a wikimedia box!
>> Jon
>>
>> [1] https://github.com/jdlrobson/WikiMaps
>>
>> _______________________________________________
>> Maps-l mailing list
>> Maps-l(a)lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/maps-l
>>
>
>
> _______________________________________________
> Maps-l mailing list
> Maps-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/maps-l
During internal review, an XSS (cross-site scripting) vulnerability was
discovered in MobileFrontend extension.
Due to an unneeded unescaping of already sanitized section titles, HTML
inserted as plaintext into them was injected into DOM.
While on ordinary page views only users who have intentionally enabled
MobileFrontend's beta mode are in danger, it is possible to construct URLs
that enable beta for every user following them. Another requirement for
this vulnerability is screen witdth which must be at least 768 pixels.
Affected versions include MobileFrontend for MediaWiki 1.23 (branch
REL1_23, still in release candidate phase) and 1.24 (master). If you are
running a 1.24 WMF branch earlier than wmf/1.24wmf3, please update to a
later branch.
--
Best regards,
Max Semenik ([[User:MaxSem]])