The few times I've tried to look at the existing rcstream service, I've quickly been stymied by not finding any documentation of the actual protocol involved.
Whatever solution is chosen, it would be very nice if there would be easy-to-find documentation that a skilled developer could use to consume the service starting with the ability to make an SSL connection to a server, instead of starting from "use python or nodejs, then require ' socket.io'".
On Fri, Sep 23, 2016 at 5:15 PM, Andrew Otto otto@wikimedia.org wrote:
Hi all,
We’ve been busy working on building a replacement for RCStream. This new service would expose recentchanges as a stream as usual, but also other types of event streams that we can make public.
But we’re having a bit of an existential crisis! We had originally chosen to implement this using an up to date socket.io server, as RCStream also uses socket.io. We’re mostly finished with this, but now we are taking a step back and wondering if socket.io/websockets are the best technology to use to expose stream data these days.
The alternative is to just use ‘streaming’ HTTP chunked transfer encoding. That is, the client makes a HTTP request for a stream, and the server declares that it will be sending back data indefinitely in the response body. Clients just read (and parse) events out of the HTTP response body. There is some event tooling built on top of this (namely SSE / EventSource), but the basic idea is a never ending streamed HTTP response body.
So, I’m reaching out to to gather some input to help inform a decision. What will be easier for you users of RCStream in the future? Would you prefer to keep using socket.io (newer version), or would you prefer to work directly with HTTP? There seem to be good clients for socket.io and for SSE/EventSource in many languages.
https://phabricator.wikimedia.org/T130651 has more context, but don’t worry about reading it; it is getting a little long. Feel free to chime in there or on this thread.
Thanks! -Andrew Otto _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l