tl;dr
Ignore this email if you do not use MediaWiki event streams.
On Monday December 11 2023, all MediaWiki related event streams will have artificial canary events https://wikitech.wikimedia.org/wiki/Event_Platform/EventStreams#Canary_Events injected into them. If you use any of these streams, you should discard thesee canary events.
Add code to your consumers that discards events where meta.domain == "canary". Canary Events
At WMF, we use artificial 'canary' AKA 'heartbeat' events https://wikitech.wikimedia.org/wiki/Event_Platform/Stream_Configuration#canary_events_enabled to differentiate between a broken event stream and an empty event stream. Canary events should be produced at least once an hour. If there are no events in a stream for an hour, then something is likely broken with that stream.
These artificial canary events can be identified by the fact that their meta.domain field is set to "canary". If you use any of the streams listed below, you will need to add code that discards any events where meta.domain == "canary".
Back in 2020, we began producing canary events into all new streams, but we never got around to enabling these for streams that already existed. We needed to ensure that all consumers of these streams filtered out the canary events. We're just finally getting around to enabling canary events for all streams.
We will enable canary event production https://phabricator.wikimedia.org/T266798 for the following streams on Monday, December 11th, 2023:
- mediawiki.recentchange
- mediawiki.page-create
- mediawiki.page-delete
- mediawiki.page-links-change
- mediawiki.page-move
- mediawiki.page-properties-change
- mediawiki.page-restrictions-change
- mediawiki.page-suppress
- mediawiki.page-undelete
- mediawiki.revision-create
- mediawiki.revision-visibility-change
- mediawiki.user-blocks-change
- mediawiki.centralnotice.campaign-change
- mediawiki.centralnotice.campaign-create
- mediawiki.centralnotice.campaign-delete
If you consume any of these streams, either external to WMF networks using EventStreams, or internally using Kafka, please ensure that your consumer logic discards events where meta.domain == "canary" before this date. (Note that not all of these streams are exposed publicly at stream.wikimedia.org https://stream.wikimedia.org/?doc#/streams.)
Thank you,
-Andrew Otto & the WMF Data Engineering team https://wikitech.wikimedia.org/wiki/Data_Engineering
References
- T266798 - Enable canary events for all MediaWiki streams https://phabricator.wikimedia.org/T266798
- T251609 - Automate ingestion and refinement into Hive of event data from Kafka using stream configs and canary/heartbeat events https://phabricator.wikimedia.org/T251609
Hi everyone, I just enabled canary events for the mentioned streams.
Please comment here or on the task if you encounter any issues.
Thank you! -Andrew Otto & the WMF Data Engineering team https://wikitech.wikimedia.org/wiki/Data_Engineering
On Tue, Nov 7, 2023 at 1:57 PM Andrew Otto otto@wikimedia.org wrote:
tl;dr
Ignore this email if you do not use MediaWiki event streams.
On Monday December 11 2023, all MediaWiki related event streams will have artificial canary events https://wikitech.wikimedia.org/wiki/Event_Platform/EventStreams#Canary_Events injected into them. If you use any of these streams, you should discard thesee canary events.
Add code to your consumers that discards events where meta.domain == "canary". Canary Events
At WMF, we use artificial 'canary' AKA 'heartbeat' events https://wikitech.wikimedia.org/wiki/Event_Platform/Stream_Configuration#canary_events_enabled to differentiate between a broken event stream and an empty event stream. Canary events should be produced at least once an hour. If there are no events in a stream for an hour, then something is likely broken with that stream.
These artificial canary events can be identified by the fact that their meta.domain field is set to "canary". If you use any of the streams listed below, you will need to add code that discards any events where meta.domain == "canary".
Back in 2020, we began producing canary events into all new streams, but we never got around to enabling these for streams that already existed. We needed to ensure that all consumers of these streams filtered out the canary events. We're just finally getting around to enabling canary events for all streams.
We will enable canary event production https://phabricator.wikimedia.org/T266798 for the following streams on Monday, December 11th, 2023:
- mediawiki.recentchange - mediawiki.page-create - mediawiki.page-delete - mediawiki.page-links-change - mediawiki.page-move - mediawiki.page-properties-change - mediawiki.page-restrictions-change - mediawiki.page-suppress - mediawiki.page-undelete - mediawiki.revision-create - mediawiki.revision-visibility-change - mediawiki.user-blocks-change - mediawiki.centralnotice.campaign-change - mediawiki.centralnotice.campaign-create - mediawiki.centralnotice.campaign-deleteIf you consume any of these streams, either external to WMF networks using EventStreams, or internally using Kafka, please ensure that your consumer logic discards events where meta.domain == "canary" before this date. (Note that not all of these streams are exposed publicly at stream.wikimedia.org https://stream.wikimedia.org/?doc#/streams.)
Thank you,
-Andrew Otto & the WMF Data Engineering team https://wikitech.wikimedia.org/wiki/Data_Engineering
References
- T266798 - Enable canary events for all MediaWiki streams
https://phabricator.wikimedia.org/T266798
- T251609 - Automate ingestion and refinement into Hive of event data
from Kafka using stream configs and canary/heartbeat events https://phabricator.wikimedia.org/T251609
analytics-announce@lists.wikimedia.org