The few times I've tried to look at the existing rcstream service, I've
quickly been stymied by not finding any documentation of the actual
protocol involved.
Whatever solution is chosen, it would be very nice if there would be
easy-to-find documentation that a skilled developer could use to consume
the service starting with the ability to make an SSL connection to a
server, instead of starting from "use python or nodejs, then require '
socket.io'".
On Fri, Sep 23, 2016 at 5:15 PM, Andrew Otto <otto(a)wikimedia.org> wrote:
Hi all,
We’ve been busy working on building a replacement for RCStream. This new
service would expose recentchanges as a stream as usual, but also other
types of event streams that we can make public.
But we’re having a bit of an existential crisis! We had originally chosen
to implement this using an up to date socket.io server, as RCStream also
uses socket.io. We’re mostly finished with this, but now we are taking a
step back and wondering if socket.io/websockets are the best technology to
use to expose stream data these days.
The alternative is to just use ‘streaming’ HTTP chunked transfer encoding.
That is, the client makes a HTTP request for a stream, and the server
declares that it will be sending back data indefinitely in the response
body. Clients just read (and parse) events out of the HTTP response body.
There is some event tooling built on top of this (namely SSE /
EventSource), but the basic idea is a never ending streamed HTTP response
body.
So, I’m reaching out to to gather some input to help inform a decision.
What will be easier for you users of RCStream in the future? Would you
prefer to keep using socket.io (newer version), or would you prefer to
work
directly with HTTP? There seem to be good clients for socket.io and for
SSE/EventSource in many languages.
https://phabricator.wikimedia.org/T130651 has more context, but don’t
worry
about reading it; it is getting a little long. Feel free to chime in there
or on this thread.
Thanks!
-Andrew Otto
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation