Hello,
Today at 9am UTC, I started upgrading Zuul to a version that uses
Gearman to trigger jobs (and fix a bunch of issues). The upgrade had to
be cancelled, service got disrupted for an hour.
The Zuul code upgrade itself went well and was completed in half an hour
at 9:30am UTC.
Timo reported that a VisualEditor testing job was failing because it has
been triggered on an incorrect Jenkins slaves. Our setup has several
slaves, each configured for different purposes, so our jobs are tied to
specific slaves by using Jenkins labels.
I thought it might be related to the Gearman plugin in Jenkins hence I
restarted Jenkins. That took a roughly 15 minutes and did not solve the
issue.
At 10:05am UTC, Zuul was downgraded and restarted.
At 10:14am UTC, Jenkins completed restart and service got resumed.
The root cause is in Jenkins Gearman plugin which trigger jobs on any
registered slaves even if they have been set to only run jobs tied to
them explicitly.
The upgrade itself went well since I tested it out several times in labs
and documented all the steps on the wiki:
https://www.mediawiki.org/wiki/Continuous_integration/Zuul/gearman_upgrade
I should have tested the multiple slaves setup we are using in labs. The
labs only has one runner though :/
Related server admin log:
https://wikitech.wikimedia.org/w/index.php?title=Server_Admin_Log&diff=8993…
I will report the issue to upstream and find out whether they can fix
that issue for our setup. Meanwhile, I am postponing Zuul upgrade
indefinitely :-(
--
Antoine "hashar" Musso
Dear all,
I'm prod to announce that MathJaX 2.3 has been released this Monday
and successfully integrated to the Wikimedia Math extension by the
MathJax developer Frédéric Wang.
In the past month Gabriel Wicke, Frédéric Wang and me have been
working on a new version of the Math extension that displays either
MathML or high quality SVG images dependent on the users browser.
The conversion from the mediawiki tex syntax to MathML and SVG happens
at the server-side.
In order to preserve full backwards comparability parts of the old
texvc code have been reused. That way latex which was formerly called
by texvc and matjax that is called by now get exactly the same valid
tex input string.
So it is expected that they produce the same output but better quality.
Pleas be informed that the mediawiki input tex inside the <math>
element is no valid tex input. e.g. $\C$ was transformed to
$\mathbb{C}$ by texvc before $\mathbb{C}$ was passed to latex.
I did a detailed analysis of the texvc code, and I see a lot of
potential for improvement. However, I think it is a bad idea to change
the rendering engine and the input language at the same time.
As a result I recommend to keep some parts of the texvc code for now
and remove it, once we demonstrated that the switch from png images to
MathML with SVG fallback has been performed smoothly.
OK. But what's the problem?
The problem is that nobody reviews the parts of the texvc that should
remain for the moment. If someone is around, who wants to review the
little changes I have made to the texvc code this would help me a lot.
Please be aware that texvc that is written in ocaml. The only change I
made is to extract the latex output and pass it to mathjax rather than
latex.
Furthermore, I already verified that the code does what he should do
with all formulae occurring enwiki.
Best regards
physikerwelt
Mit freundlichen Grüßen
Moritz Schubotz
Telefon (Büro): +49 30 314 22784
Telefon (Privat):+49 30 488 27330
E-Mail: schubotz(a)itp.physik.tu-berlin.de
Web: http://www.physikerwelt.de
Skype: Schubi87
ICQ: 200302764
Msn: Moritz(a)Schubotz.de
Hi everyone,
I'm thrilled to announce that Aaron Arcos has agreed to volunteer for
the Wikimedia Foundation as a developer with our Multimedia team
through May. Aaron most recently worked at Google Switzerland (from
2005 until August 2013), where he was a frontend software engineer and
UX designer on several projects such as AdWords Template Center,
SafeSearchLock, GMail Themes and the recent GCalendar redesign. He
also served as a process and release master, in charge of defining,
improving and enforcing their development, testing and release
processes, and taught classes at Google about project management, user
experience and Google culture.
He decided to leave Google, and to give back to the community in a
number of ways, working for free for projects that do good for
humanity (like us!) His first stop was at nairobits.com, a project
that provides the economically-disadvantaged youth in Nairobi with web
design and programming skills, teaching PHP and Javascript. He plans
to return there in May. After that, he has other organizations that
he'd like to work with.
Prior to Google, Aaron worked at IMTF.com in Switzerland, Techasi
Ingénierie in Paris, Yahoo! and Electronics for Imaging in the Bay
Area.
He earned his Masters in Computer Engineering at University of
Washington and has a BS in Computer Engineering from UNAM in Mexico
City.
Aaron will be visiting the San Francisco office while he's
volunteering for us. He's "aarcos" on Freenode (and will often be on
#wikimedia-multimedia), and his email address is
aarcos.wiki(a)gmail.com. Welcome Aaron!
Rob
Hello,
As discussed in the last RFC review meeting, I've put together a
proposal for an on-wiki configuration scheme:
https://www.mediawiki.org/wiki/Requests_for_comment/Configuration_database_2.
Comments and feedback would be appreciated!
I've also added this to the list for tomorrow's RFC review meeting.
Thanks,
-- Kunal Mehta (Legoktm)
Thanks for responding, Aran, Tyler, and Daniel. I appreciate your
thoughtful advice.
I am planning to use SSE. It's for some slow server-side operations
that typically take 1-3 minutes or so, where I want to give the user
near-real-time updates on what's happening. I don't need full duplex
or a long-term connection, and I don't want to make wiki admins install
extra server software. I could just do it with Ajax polling, but I
think SSE will give a more responsive feel.
I have an early version of this working now, that watches a file and
spools it out in an SSE event stream (by violently hijacking the
ApiBase logic and seizing low-level control of the HTTP response - I'm
sure it could be integrated into the output options in the Api class
hierarchy if there's interest). This seems to work pretty well so far.
For slow operations with side effects, I'm going to do it in two parts:
client calls api.php to request the operation; it sends back a single
SSE event containing a key, and goes on with its business, writing
information about its progress to a file; client calls api.php a second
time, using the key, to watch that file. This way it can robustly
handle a bad connection or timeout by reconnecting without causing any
unwanted side effects, like restarting the long operation. The stream
of output could just as well go in memcache or the database as in a
file, but for my project it makes more sense to use the file system,
because some of it will come from redirecting the output of unix commands.
LW
> From: Aran <aran(a)organicdesign.co.nz>
>
> Wouldn't WebSocket be the better choice for a full duplex channel?
>
> On 14/11/13 21:12, Lee Worden wrote:
>> In the MW extension development I'm doing, I'm thinking of writing
>> some operations that use [[Comet_(programming)]] to deliver continuous
>> updates to the client, rather than the Ajax pattern of one request,
>> one response.
>>
>> Has anyone messed with this? Any code I should crib from, or advice
>> or cautionary tales? Also, if it develops into something useful, I
>> could split it out for others to use.
>>
>> Thanks,
>> LW
> From: Tyler Romeo <tylerromeo(a)gmail.com>
>
> I have not messed with it personally, but I think it is a good idea. You
> should also know that the HTML5 standard has standardized the Comet model
> into server-sent events (SSE). [1] Mozilla also provides a nice tutorial on
> how to use it. [2] However, one big catch is that this is not currently
> implemented in Internet Explorer or mobile browsers. [3] So you'd have to
> have your own custom pure-JavaScript implementation for IE support.
>
> WebSocket, as another mentioned, is also an approach you could use.
> However, WebSockets are meant for full duplex communication, meaning the
> client is also talking back to the server, which may or may not be what you
> want. Also using WebSockets means the internals of what is sent over the
> socket and what it means is left to you to design, rather than being
> standardized. Not to mention the fact that you have to implement WebSockets
> in PHP or find a reliable library that will do it for you. And even then,
> WebSockets are only supported in IE 10 and later, so you're still a bit
> screwed in terms of backwards compatibility.
>
> [1] http://www.w3.org/TR/eventsource/
> [2]
> https://developer.mozilla.org/en-US/docs/Server-sent_events/Using_server-se…
> [3] http://caniuse.com/#feat=eventsource
> From: Daniel Friesen <daniel(a)nadir-seen-fire.com>
>
> Rather than natively using WebSockets you could use a higher-level library.
> There are two of these in existence, Socket.IO[1] and SockJS[2].
> Socket.IO is the popular implementation.
> However when I looked into it I found SockJS to be the superior
> implementation IMHO.
>
> These libraries use WebSockets when available and fall back to a variety
> of other methods when WebSockets aren't available.
> So they support practically every browser.
>
> However you're probably going to have to give up on PHP.
>
> Neither Socket.IO nor SockJS have a PHP server implementation.
> The only thing that shows up for Socket.IO is Elephant.IO[3] which is a
> Socket.IO *client*.
>
> If you go down to raw WebSockets there is Ratchet[4]. However that
> brings up a number of issues that accumulate up to losing every single
> point you could possibly have to run it using PHP.
> Ratchet needs to spawn its own server. This means it needs a dedicated
> port, domain, or a reverse proxy in front of it.
> It also means that even though it's written in PHP you can no longer run
> it on any form of shared hosting.
> PHP is also not designed to run long running daemons. It can do it, but
> you will have to watch this very carefully and be prepared to fix any
> bugs that show up.
> You'd expect that since it's in PHP you at least have the one remaining
> advantage that you can directly interact with MediaWiki.
> However all of MediaWiki's code is synchronous. This means that if you
> directly use MW every time it needs to do something with the database,
> memcached, filesystem, other storage, or a slow MW code path your
> WebSocket server will seize up for a moment for ALL clients connected to
> it. If this happens to be a parser cache miss waiting for a parser run
> that takes 15s to complete. Your "real-time" application will be
> completely unresponsive for those 15s.
> The only way to avoid that would be to run the MW code in processes
> separate from the WebSocket server.
> But at that point there's absolutely no remaining advantage to the
> server being in PHP instead of Node.JS, Python, Ruby, etc...
>
> The situation is different but similarly hopeless[citation needed] for
> EventSource / Server-sent events.
> SSE's protocol is simple and "can" be implemented simply in PHP.
> However in PHP every SSE connection will have it's own open connection
> in the webserver and it's own dedicated PHP process.
> This means that as people connect to your server you will quickly reach
> the webserver's limit of maximum open connections (maybe even ram with
> all that overhead).
> You cannot avoid that fatal flaw without doing a bunch of strange and
> hacky things that ending up with the same faults to using Ratchet.
>
> -- summary --
> Basically in the end your best bet is probably to just use something
> other than PHP for this.
> The native implementations for both Socket.IO's and SockJS' servers are
> in Node.JS.
> So that's probably the best bet for doing this.
> You can communicate with something in PHP running MW stuff from the
> Node.JS server using another protocol.
> Such as STOMP, ZeroMQ, Thrift, D-Bus, etc... or even simply HTTP calls
> to the webserver/API or execution of PHP processes from the Node.JS server.
>
>
> [1] http://socket.io/
> [2] http://sockjs.org/
> [3] http://elephant.io/
> [4] http://socketo.me/
>
> ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]
Following the mediawiki-l
discussion<http://lists.wikimedia.org/pipermail/mediawiki-l/2013-November/042038.html>about
$wgNoFollowLinks and various other discussions, in which some
discontent was expressed with the current two options of either applying or
not applying nofollow to all external links, I wanted to see what support
there might be for applying nofollow only to external links added in
revisions that are still unpatrolled (bug
42599<https://bugzilla.wikimedia.org/show_bug.cgi?id=42599>
).
How common do you think it would be for a use case to arise in which one
could be confident that a revision's being patrolled means that the
external links added in that revision have been adequately reviewed for
spamminess? Nemo had mentioned "sysadmins would be interested in this only
if their wiki has a strict definition of what's patrollable which matches
the assumptions here." In my experience, spam is pretty easy to spot
because the bots aren't very subtle about it.
I would think that if someone went around marking such obviously spammy
edits as patrolled, that if there were any bureaucrats around who cared
about keeping spam off the wiki, his patrol rights would end up getting
taken away. Spam is a form of vandalism, so it would fall under the duties
of patrollers. At Wikipedia, RecentChanges patrollers are expected to be on
the lookout for spam.
https://en.wikipedia.org/wiki/Wikipedia:Recent_changes_patrol#Spam
--
Nathan Larson <https://mediawiki.org/wiki/User:Leucosticte>
Hi,
If you're using Sublime Text as your text editor[1], I'd recommend checking out
the DocBlockr plugin[2]. It makes it easier to produce documentation comments.
It helps you through various features such as:
* Autocomplete various @-tags
* Auto-create blocks[3]
* Detect function params and property value types
* And lots more..
It avoids mistakes and speeds up the creation of conformant comments so that
Jenkins/JSDuck won't shout at you.
Though it provides all this by default, I recommend you fine tune it to your liking
(and to the specifics of JSDuck).
To deal with the variety in how different projects write documentation comments,
it has various configuration options[4] (e.g. @return vs @returns, @var, vs @property,
type {Boolean}, {boolean} or {bool} etc.). I've published my configuration on GitHub.
It might be useful to you to get started[5].
You can install DocBlockr in Subllime the "easy" way using Package Control[6],
or check out the plugin page[2] for manual installation.
-- Krinkle
[1] http://www.sublimetext.com/
[2] https://github.com/spadgos/sublime-jsdocs
[3] Type "/**" followed by tab to insert the template, then tab trough placeholders to fill them in
[4] https://github.com/spadgos/sublime-jsdocs/blob/2.11.7/Base%20File.sublime-s…
[4] https://github.com/Krinkle/dotfiles/blob/b2088da/hosts/KrinkleMac/SublimePr…
[5] https://sublime.wbond.net/
Hello,
We made some change to Echo api recently. The api ApiEchoNotifications
mixed both read and write actions, we have migrated the 'markasread' action
to its own api module - ApiEchoMarkRead. The 'markasread' function still
works in ApiEchoNotifications but it will be removed once all external api
calls have been migrated to the new API.
Example about how to use the new API is in here:
https://git.wikimedia.org/blob/mediawiki%2Fextensions%2FEcho.git/40ea204cdc…
Thanks,
> On 2013-11-18 1:09 AM, "Timothy Pearson" <kb9vqf(a)pearsoncomputing.net>
> wrote:
>>
>> > "wm-license-information-description" will probably be comming from
>> E:WikimediaMessages, You will also want a dump containing the locally
>> one done on the wiki's MediaWiki: namespace.
>>
>> Thank you for your response. Is there a dump file containing this
>> information, and if not, is there any way to extract it (sanely) from
>> the
>> running Wikipedia instance?
>>
>> I am learning as I go, so my apologies if I am overlooking something
>> simple. :-)
>>
>> Thanks!
>>
>> Tim
>>
>>
>>
>>
>> _______________________________________________
>> Wikitech-l mailing list
>> Wikitech-l(a)lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> The db dumps that are listed as containing content from all namespaces
> should have it. (Custom messages are pages in the mediawiki namespace)
>
> Also you will probably want to install most (if not all) extensions listed
> at http://en.wikipedia.org/wiki/Special:version
>
> -bawolff
>
I loaded the MediaWiki instance with the contents of
enwiki-latest-pages-articles.xml; does that dump not contain all content
minus history? Downloading the enormous multi-part dumps just to obtain
the mediawiki namespace contents would not be feasible due to equipment
limitations on this end.
I have all the applicable extensions installed; thanks for the heads up
anyway.
Tim