Thanks for responding, Aran, Tyler, and Daniel. I appreciate your thoughtful advice.
I am planning to use SSE. It's for some slow server-side operations that typically take 1-3 minutes or so, where I want to give the user near-real-time updates on what's happening. I don't need full duplex or a long-term connection, and I don't want to make wiki admins install extra server software. I could just do it with Ajax polling, but I think SSE will give a more responsive feel.
I have an early version of this working now, that watches a file and spools it out in an SSE event stream (by violently hijacking the ApiBase logic and seizing low-level control of the HTTP response - I'm sure it could be integrated into the output options in the Api class hierarchy if there's interest). This seems to work pretty well so far.
For slow operations with side effects, I'm going to do it in two parts: client calls api.php to request the operation; it sends back a single SSE event containing a key, and goes on with its business, writing information about its progress to a file; client calls api.php a second time, using the key, to watch that file. This way it can robustly handle a bad connection or timeout by reconnecting without causing any unwanted side effects, like restarting the long operation. The stream of output could just as well go in memcache or the database as in a file, but for my project it makes more sense to use the file system, because some of it will come from redirecting the output of unix commands.
LW
From: Aran aran@organicdesign.co.nz
Wouldn't WebSocket be the better choice for a full duplex channel?
On 14/11/13 21:12, Lee Worden wrote:
In the MW extension development I'm doing, I'm thinking of writing some operations that use [[Comet_(programming)]] to deliver continuous updates to the client, rather than the Ajax pattern of one request, one response.
Has anyone messed with this? Any code I should crib from, or advice or cautionary tales? Also, if it develops into something useful, I could split it out for others to use.
Thanks, LW
From: Tyler Romeo tylerromeo@gmail.com
I have not messed with it personally, but I think it is a good idea. You should also know that the HTML5 standard has standardized the Comet model into server-sent events (SSE). [1] Mozilla also provides a nice tutorial on how to use it. [2] However, one big catch is that this is not currently implemented in Internet Explorer or mobile browsers. [3] So you'd have to have your own custom pure-JavaScript implementation for IE support.
WebSocket, as another mentioned, is also an approach you could use. However, WebSockets are meant for full duplex communication, meaning the client is also talking back to the server, which may or may not be what you want. Also using WebSockets means the internals of what is sent over the socket and what it means is left to you to design, rather than being standardized. Not to mention the fact that you have to implement WebSockets in PHP or find a reliable library that will do it for you. And even then, WebSockets are only supported in IE 10 and later, so you're still a bit screwed in terms of backwards compatibility.
[1] http://www.w3.org/TR/eventsource/ [2] https://developer.mozilla.org/en-US/docs/Server-sent_events/Using_server-sen... [3] http://caniuse.com/#feat=eventsource
From: Daniel Friesen daniel@nadir-seen-fire.com
Rather than natively using WebSockets you could use a higher-level library. There are two of these in existence, Socket.IO[1] and SockJS[2]. Socket.IO is the popular implementation. However when I looked into it I found SockJS to be the superior implementation IMHO.
These libraries use WebSockets when available and fall back to a variety of other methods when WebSockets aren't available. So they support practically every browser.
However you're probably going to have to give up on PHP.
Neither Socket.IO nor SockJS have a PHP server implementation. The only thing that shows up for Socket.IO is Elephant.IO[3] which is a Socket.IO *client*.
If you go down to raw WebSockets there is Ratchet[4]. However that brings up a number of issues that accumulate up to losing every single point you could possibly have to run it using PHP. Ratchet needs to spawn its own server. This means it needs a dedicated port, domain, or a reverse proxy in front of it. It also means that even though it's written in PHP you can no longer run it on any form of shared hosting. PHP is also not designed to run long running daemons. It can do it, but you will have to watch this very carefully and be prepared to fix any bugs that show up. You'd expect that since it's in PHP you at least have the one remaining advantage that you can directly interact with MediaWiki. However all of MediaWiki's code is synchronous. This means that if you directly use MW every time it needs to do something with the database, memcached, filesystem, other storage, or a slow MW code path your WebSocket server will seize up for a moment for ALL clients connected to it. If this happens to be a parser cache miss waiting for a parser run that takes 15s to complete. Your "real-time" application will be completely unresponsive for those 15s. The only way to avoid that would be to run the MW code in processes separate from the WebSocket server. But at that point there's absolutely no remaining advantage to the server being in PHP instead of Node.JS, Python, Ruby, etc...
The situation is different but similarly hopeless[citation needed] for EventSource / Server-sent events. SSE's protocol is simple and "can" be implemented simply in PHP. However in PHP every SSE connection will have it's own open connection in the webserver and it's own dedicated PHP process. This means that as people connect to your server you will quickly reach the webserver's limit of maximum open connections (maybe even ram with all that overhead). You cannot avoid that fatal flaw without doing a bunch of strange and hacky things that ending up with the same faults to using Ratchet.
-- summary -- Basically in the end your best bet is probably to just use something other than PHP for this. The native implementations for both Socket.IO's and SockJS' servers are in Node.JS. So that's probably the best bet for doing this. You can communicate with something in PHP running MW stuff from the Node.JS server using another protocol. Such as STOMP, ZeroMQ, Thrift, D-Bus, etc... or even simply HTTP calls to the webserver/API or execution of PHP processes from the Node.JS server.
[1] http://socket.io/ [2] http://sockjs.org/ [3] http://elephant.io/ [4] http://socketo.me/
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]