Hi everyone, I just enabled canary events for the mentioned streams.
Please comment here or on the task if you encounter any issues.
Thank you!
-Andrew Otto & the WMF Data Engineering team
<https://wikitech.wikimedia.org/wiki/Data_Engineering>
On Tue, Nov 7, 2023 at 1:57 PM Andrew Otto <otto(a)wikimedia.org> wrote:
> tl;dr
>
> Ignore this email if you do not use MediaWiki event streams.
>
> On Monday December 11 2023, all MediaWiki related event streams will have artificial
> canary events
> <https://wikitech.wikimedia.org/wiki/Event_Platform/EventStreams#Canary_Even…>
> injected into them. If you use any of these streams, you should discard
> thesee canary events.
>
> Add code to your consumers that discards events where meta.domain ==
> "canary".
> Canary Events
>
> At WMF, we use artificial 'canary' AKA 'heartbeat' events
> <https://wikitech.wikimedia.org/wiki/Event_Platform/Stream_Configuration#can…>
> to differentiate between a broken event stream and an empty event stream.
> Canary events should be produced at least once an hour. If there are no
> events in a stream for an hour, then something is likely broken with that
> stream.
>
> These artificial canary events can be identified by the fact that their
> meta.domain field is set to "canary". If you use any of the streams
> listed below, you will need to add code that discards any events where meta.domain
> == "canary".
>
> Back in 2020, we began producing canary events into all new streams, but
> we never got around to enabling these for streams that already existed. We
> needed to ensure that all consumers of these streams filtered out the
> canary events. We're just finally getting around to enabling canary events
> for all streams.
>
> We will enable canary event production
> <https://phabricator.wikimedia.org/T266798> for the following streams on
> Monday, December 11th, 2023:
>
> - mediawiki.recentchange
>
> - mediawiki.page-create
>
> - mediawiki.page-delete
>
> - mediawiki.page-links-change
>
> - mediawiki.page-move
>
> - mediawiki.page-properties-change
>
> - mediawiki.page-restrictions-change
>
> - mediawiki.page-suppress
>
> - mediawiki.page-undelete
>
> - mediawiki.revision-create
>
> - mediawiki.revision-visibility-change
>
> - mediawiki.user-blocks-change
>
> - mediawiki.centralnotice.campaign-change
>
> - mediawiki.centralnotice.campaign-create
>
> - mediawiki.centralnotice.campaign-delete
>
> If you consume any of these streams, either external to WMF networks using
> EventStreams, or internally using Kafka, please ensure that your consumer
> logic discards events where meta.domain == "canary" before this date.
> (Note that not all of these streams are exposed publicly at
> stream.wikimedia.org <https://stream.wikimedia.org/?doc#/streams>.)
>
> Thank you,
>
> -Andrew Otto & the WMF Data Engineering team
> <https://wikitech.wikimedia.org/wiki/Data_Engineering>
>
> References
>
> - T266798 - Enable canary events for all MediaWiki streams
> <https://phabricator.wikimedia.org/T266798>
>
> - T251609 - Automate ingestion and refinement into Hive of event data
> from Kafka using stream configs and canary/heartbeat events
> <https://phabricator.wikimedia.org/T251609>
>
>
Dear MediaWiki community,
we are happy to announce that Markus Krötzsch will be joining us in Paderborn for the keynote talk at SMWCon 2023!
In this talk, Markus will provide a personal perspective on the origins and principles of semantic wikis, and some of the key challenges that lie ahead in managing knowledge in the age of A!
Don't miss it - you can still attend in person, or follow the conference via YouTube or Zoom.
https://www.semantic-mediawiki.org/wiki/SMWCon_Fall_2023
I you have not already participated in our __community survey__, please do so until this Wednesday. It is important for us to learn more about your SMW usage. Results will be presented next Tuesday.
We thank this year's conference sponsors:
ArchiXL: http://www.archixl.nl/
Specialists in enterprise architecture, knowledge management, and semantics
Hallo Welt!: https://bluespice.com/
The company behind BlueSpice, the open-source enterprise wiki software
MyWikis Europe: https://mywikis.eu/
GDPR compliant (Semantic) MediaWiki hosting from the heart of Europe.
Wikibase Solutions: https://wikibase-solutions.com/
Specialist in business solutions with MediaWiki
As well as the conference organizers
MediaWiki Stakeholders' Group: https://mwstake.org/
Advocating the needs of MediaWiki users outside the Wikimedia Foundation
KM-A Knowledge Management Associates: https://km-a.net/
KM-A educates and advises Knowledge Managers and connects the KM Community in Austria and the world.
Paderborn University: https://www.uni-paderborn.de/en/
While always keeping society's needs in mind, scientists at Paderborn University are working on the technologies of the future.
Juggel: https://www.juggel.com/
AI supported Knowledge Management based on MediaWiki and Semantic MediaWiki
Best regards,
Bernhard, Ad and Tobias
Hey! actually I am finding it a bit difficult to solve issues and make pull
requests. so just wanted to ask if wikimedia has a system of providing
personal mentors? It will be helpful and from whom I can ask questions in
general?
On Sat, Nov 18, 2023 at 11:58 PM Mailtrack Notification <
notification(a)mailtrack.io> wrote:
> 🔥 Hot conversation: wikitech-l(a)lists.wikimedia.org opened it many times
> in a short period or forwarded it. View all 315 opens
> <https://mailtrack.io/reauth?url=https://mailtrack.io/en/dashboard/show/2903…>
> | turn off hot conversations
> <https://mailtrack.io/en/dashboard/hot-notification/disable>
>
Hello!
After our initial announcement of the Grid Engine shutdown timeline[0],
some of you raised concerns about losing your tools.
We want to address those apprehensions while hopefully providing
reassurance. No tools will be deleted until the grid engine shutdown date
on 14 February 2023. However, for tools with unreachable maintainers, an
outage will happen starting on 14 December 2023[1]. This is intended to
raise awareness for users or maintainers who have not otherwise been
reached. A list of these tools can be found here[2]. If you are a
maintainer or a user of a tool in this list, comment on the associated
phabricator ticket with migration plans or a request for more support. The
goal is to have a plan for all tools running on the grid. We want all
actively used tools to be migrated, and will help support users of critical
tools without a maintainer. Thanks for your help in identifying and
migrating those tools you maintain and depend on.
We acknowledge that the timeline might seem tight, and we want to clarify
that our approach is to make this process as seamless as possible. We have
been actively engaging with tool maintainers over the past year, and we
genuinely appreciate the efforts many of you have already made to migrate
your tools to Kubernetes.
We will continue to work closely with maintainers who might need additional
time or assistance.
If for any reason you have not received a phabricator ticket for your tool,
please reach out.
The phabricator ticket is a good place to communicate your needs and plans
for any remaining tools or jobs.
This will help us further organize and plan this process.
Our primary goal is to support you through this transition. If you have
further concerns about the deadline or if you need assistance with the
migration process, please don't hesitate to reach out to us. We are
available on IRC, Telegram, Phabricator[3], and through our other support
channels[4].
Do you still have concerns or questions? Please let us know. We want to do
this together with you, in a way which makes sense to everyone. We’re very
grateful for all the hard work you do, and our only goal here is to secure
the future of tools in the Wikimedia sphere, not to make your lives more
difficult.
Thank you!
[0]:
https://lists.wikimedia.org/hyperkitty/list/cloud-announce@lists.wikimedia.…
[1]:
https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation#…
[2]:
https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation/…
[3]: https://phabricator.wikimedia.org/project/board/6135/
[4]:
https://wikitech.wikimedia.org/wiki/Portal:Toolforge/About_Toolforge#Commun…
--
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services
Recently I completed initial work to enable commit-message-validator
[0] to work with GitLab CI and GitLab merge requests. For those who
are unaware, commit-message-validator is a linter tool designed to
enforce Wikimedia's commit message guidelines. [1]
Soon after the feature was available, James Forrester added it to the
test suite for abstract-wiki/wikifunctions/function-orchestrator [2]
and found the first issue with the integration that I had not
anticipated which he helpfully filed as T351253. [3]
In my estimation, the problem comes down to a question of whether we
should prioritize reading commit message footer information nicely in
GitLab's merge request interface where they are rendered as GitLab
flavored markdown data or not. James' team has developed a convention
of appending a backslash (\) after footer lines so that they render as
individual lines when processed as markdown. This in turn leads to
commit-message-validator rejecting some footers, most obviously "Bug:
Tnnnn" footers, for having unwanted characters (the trailing " \").
Reasonable people can disagree on the "best" solution here, but I
think it is likely that as a group we can reach consensus on what the
proper behavior of the commit-message-validator tool should be. The
most obvious options are:
* Change nothing in commit-message-validator and suggest folks live
with markdown rendering artifacts in GitLab merge request
descriptions.
* Change commit-message-validator to allow trailing " \" data for
commit message footers in GitLab repos.
* Change commit-message-validator to allow users (typically a CI
process) to configure allow/disallow of trailing " \" data for commit
message footers
Adding support for per-repo rules configuration would be a first for
commit-message-validator. Until now it has provided a single
opinionated ruleset based on interpretation of the commit message
guidelines. [2]
Folks who actually care about this minutia (how git commit message
footers look in and out of GitLab markdown rendering) are encouraged
to think carefully and provide their opinions and supporting data on
T351253. [3]
[0]: https://www.mediawiki.org/wiki/Commit-message-validator
[1]: https://www.mediawiki.org/wiki/Gerrit/Commit_message_guidelines
[2]: https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-orc…
[3]: https://phabricator.wikimedia.org/T351253
Bryan
--
Bryan Davis Wikimedia Foundation
Principal Software Engineer Boise, ID USA
[[m:User:BDavis_(WMF)]] irc: bd808
Hello all!
The Search Platform Team usually holds an open meeting on the first
Wednesday of each month. Come talk to us about anything related to
Wikimedia search, Wikidata Query Service (WDQS), Wikimedia Commons Query
Service (WCQS), etc.!
Feel free to add your items to the Etherpad Agenda for the next meeting.
Details for our next meeting:
Date: Wednesday, December 6, 2023
Time: 16:00-17:00 UTC / 08:00 PDT / 11:00 EDT / 17:00 CEST
Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours
Google Meet link: https://meet.google.com/vgj-bbeb-uyi
Join by phone: https://tel.meet/vgj-bbeb-uyi?pin=8118110806927
Have fun and see you soon!
Guillaume
--
*Guillaume Lederrey* (he/him)
Engineering Manager
Wikimedia Foundation <https://wikimediafoundation.org/>
Hi all,
I have asked for +2 access to the PageTriage extension
<https://mediawiki.org/wiki/Extension:PageTriage> to be able to participate
in reviewing code contributions made to the repository :) Please share any
comments at https://phabricator.wikimedia.org/T351972
Regards,
Sohom Datta
---
Open-source contributor @Wikimedia, (and sometimes @Chromium)