You can seek back on EventBus events, but not permanently (by default, only
up to 1 week). If you want to respond to changes in an event stream, you
should consume the full event stream realtime and react to the events as
they come in. A proper Stream Processing system (like Flink or Spark
Streaming) could help with this, but we don’t have that right now. But, I
think for your use case, you don’t need a big stream processing system, as
this stream will be relatively small, and you don’t need fancy features
like time based windowing. You just need to update something based on an
event, right?
The change-propagation service that the Services team is building can help
you with this. It allows you to consume events, and specify matching rules
and actions to take based on those rules.
https://www.mediawiki.org/wiki/Change_propagation
On Fri, Sep 23, 2016 at 2:55 PM, Stas Malyshev <smalyshev(a)wikimedia.org>
wrote:
Hi!
Could we emit a page/properties-change event to
EventBus when page props
are updated? Similar to how we emit an event for revision visibility
changes:
This, however, still is missing a part because, as I understand,
EventBus is not seekable. I.e., if I have data up-to-date to timepoint
T, and I am now at timepoint N, I can scan recent changes list from T to
N and know if certain item X has changed or not. However, since recent
changes list has no entries for page props, and events on EventBus past
N are lost to me, I have no idea if page props for X changed between T
and N. To know that, I need permanent seekable record of changes. Or
some flag that says when it was last updated, at least.
Unless of course I'm missing the part where you can seek back on
EventBus events, then please point me to the API that allows to do so.
--
Stas Malyshev
smalyshev(a)wikimedia.org
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l