Thanks for responding, Aran, Tyler, and Daniel. I appreciate your
thoughtful advice.
I am planning to use SSE. It's for some slow server-side operations
that typically take 1-3 minutes or so, where I want to give the user
near-real-time updates on what's happening. I don't need full duplex
or a long-term connection, and I don't want to make wiki admins install
extra server software. I could just do it with Ajax polling, but I
think SSE will give a more responsive feel.
I have an early version of this working now, that watches a file and
spools it out in an SSE event stream (by violently hijacking the
ApiBase logic and seizing low-level control of the HTTP response - I'm
sure it could be integrated into the output options in the Api class
hierarchy if there's interest). This seems to work pretty well so far.
For slow operations with side effects, I'm going to do it in two parts:
client calls api.php to request the operation; it sends back a single
SSE event containing a key, and goes on with its business, writing
information about its progress to a file; client calls api.php a second
time, using the key, to watch that file. This way it can robustly
handle a bad connection or timeout by reconnecting without causing any
unwanted side effects, like restarting the long operation. The stream
of output could just as well go in memcache or the database as in a
file, but for my project it makes more sense to use the file system,
because some of it will come from redirecting the output of unix commands.
LW
From: Aran <aran(a)organicdesign.co.nz>
Wouldn't WebSocket be the better choice for a full duplex channel?
On 14/11/13 21:12, Lee Worden wrote:
> In the MW extension development I'm doing, I'm thinking of writing
> some operations that use [[Comet_(programming)]] to deliver continuous
> updates to the client, rather than the Ajax pattern of one request,
> one response.
>
> Has anyone messed with this? Any code I should crib from, or advice
> or cautionary tales? Also, if it develops into something useful, I
> could split it out for others to use.
>
> Thanks,
> LW
From: Tyler Romeo <tylerromeo(a)gmail.com>
I have not messed with it personally, but I think it is a good idea. You
should also know that the HTML5 standard has standardized the Comet model
into server-sent events (SSE). [1] Mozilla also provides a nice tutorial on
how to use it. [2] However, one big catch is that this is not currently
implemented in Internet Explorer or mobile browsers. [3] So you'd have to
have your own custom pure-JavaScript implementation for IE support.
WebSocket, as another mentioned, is also an approach you could use.
However, WebSockets are meant for full duplex communication, meaning the
client is also talking back to the server, which may or may not be what you
want. Also using WebSockets means the internals of what is sent over the
socket and what it means is left to you to design, rather than being
standardized. Not to mention the fact that you have to implement WebSockets
in PHP or find a reliable library that will do it for you. And even then,
WebSockets are only supported in IE 10 and later, so you're still a bit
screwed in terms of backwards compatibility.
[1]
http://www.w3.org/TR/eventsource/
[2]
https://developer.mozilla.org/en-US/docs/Server-sent_events/Using_server-se…
[3]
http://caniuse.com/#feat=eventsource
From: Daniel Friesen
<daniel(a)nadir-seen-fire.com>
Rather than natively using WebSockets you could use a higher-level library.
There are two of these in existence, Socket.IO[1] and SockJS[2].
Socket.IO is the popular implementation.
However when I looked into it I found SockJS to be the superior
implementation IMHO.
These libraries use WebSockets when available and fall back to a variety
of other methods when WebSockets aren't available.
So they support practically every browser.
However you're probably going to have to give up on PHP.
Neither Socket.IO nor SockJS have a PHP server implementation.
The only thing that shows up for Socket.IO is Elephant.IO[3] which is a
Socket.IO *client*.
If you go down to raw WebSockets there is Ratchet[4]. However that
brings up a number of issues that accumulate up to losing every single
point you could possibly have to run it using PHP.
Ratchet needs to spawn its own server. This means it needs a dedicated
port, domain, or a reverse proxy in front of it.
It also means that even though it's written in PHP you can no longer run
it on any form of shared hosting.
PHP is also not designed to run long running daemons. It can do it, but
you will have to watch this very carefully and be prepared to fix any
bugs that show up.
You'd expect that since it's in PHP you at least have the one remaining
advantage that you can directly interact with MediaWiki.
However all of MediaWiki's code is synchronous. This means that if you
directly use MW every time it needs to do something with the database,
memcached, filesystem, other storage, or a slow MW code path your
WebSocket server will seize up for a moment for ALL clients connected to
it. If this happens to be a parser cache miss waiting for a parser run
that takes 15s to complete. Your "real-time" application will be
completely unresponsive for those 15s.
The only way to avoid that would be to run the MW code in processes
separate from the WebSocket server.
But at that point there's absolutely no remaining advantage to the
server being in PHP instead of Node.JS, Python, Ruby, etc...
The situation is different but similarly hopeless[citation needed] for
EventSource / Server-sent events.
SSE's protocol is simple and "can" be implemented simply in PHP.
However in PHP every SSE connection will have it's own open connection
in the webserver and it's own dedicated PHP process.
This means that as people connect to your server you will quickly reach
the webserver's limit of maximum open connections (maybe even ram with
all that overhead).
You cannot avoid that fatal flaw without doing a bunch of strange and
hacky things that ending up with the same faults to using Ratchet.
-- summary --
Basically in the end your best bet is probably to just use something
other than PHP for this.
The native implementations for both Socket.IO's and SockJS' servers are
in Node.JS.
So that's probably the best bet for doing this.
You can communicate with something in PHP running MW stuff from the
Node.JS server using another protocol.
Such as STOMP, ZeroMQ, Thrift, D-Bus, etc... or even simply HTTP calls
to the webserver/API or execution of PHP processes from the Node.JS server.
[1]
http://socket.io/
[2]
http://sockjs.org/
[3]
http://elephant.io/
[4]
http://socketo.me/
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [
http://danielfriesen.name/]