On Mon, Sep 26, 2016 at 5:57 AM, Andrew Otto <otto(a)wikimedia.org> wrote:
A public resumable stream of Wikimedia events would
allow folks
outside of WMF networks to build realtime stream processing tooling on top
of our data. Folks with their own Spark or Flink or Storm clusters (in
Amazon or labs or wherever) could consume this and perform complex stream
processing (e.g. machine learning algorithms (like ORES), windowed trending
aggregations, etc.).
I recall WMDE trying something similar a year ago (via PubSubHubbub) and
getting vetoed by ops. If they are not aware yet, might be worth contacting
them and asking if the new streaming service would cover their use cases
(it was about Wikidata change invalidation on third-party wikis, I think).