On Mon, Sep 26, 2016 at 5:57 AM, Andrew Otto otto@wikimedia.org wrote:
A public resumable stream of Wikimedia events would allow folks outside of WMF networks to build realtime stream processing tooling on top of our data. Folks with their own Spark or Flink or Storm clusters (in Amazon or labs or wherever) could consume this and perform complex stream processing (e.g. machine learning algorithms (like ORES), windowed trending aggregations, etc.).
I recall WMDE trying something similar a year ago (via PubSubHubbub) and getting vetoed by ops. If they are not aware yet, might be worth contacting them and asking if the new streaming service would cover their use cases (it was about Wikidata change invalidation on third-party wikis, I think).