Hello Hello Everyone,
Welcome to the 2nd edition of the Wikimedia Technical Community Newsletter!
We're excited to bring you a roundup of highlights, news, and information
of interest from and about the Wikimedia Technical Community. The
newsletter is compiled by staff(s) working on developer outreach.
Read the full April’24 newsletter edition here:
🔗https://www.mediawiki.org/wiki/Technical_Community_Newsletter/2024/April
🥁 Here’s a sneak peek of what to expect from this newsletter edition
📢 Action Items
-
Send in your Nominations for the 2024 Coolest Tool Award 🛠
The nomination for the 2024 Coolest Tool Award is open! Please recommend
tools until Friday May 10th, 2024. You can nominate as many tools as you
want by filling out the survey multiple times. Submitting multiple entries
is encouraged! Visit -
https://meta.wikimedia.org/wiki/Coolest_Tool_Award#Coolest_Tool_Award_2024
to nominate a tool and for more information on the award & nomination
process.
-
Are you passionate about the intersection of wiki-based collaboration
and cutting-edge research? Do you have insights, ideas, or findings to
share with the wiki community? We invite you to submit your proposals
<https://wikiworkshop.org/call-for-hall> for presentations, workshops,
and papers to the upcoming Wiki Workshop! Additionally the team will be
holding office hours to answer any questions about this new track and the
submission process. We invite you to schedule some time on Tuesday, April
16 or Thursday, April 25 in the following link-
https://calendar.app.google/igcEC3fqTAF8MRDE7. You can also email us at
wikiworkshop(a)googlegroups.com with any questions about the Hall or the
Workshop in general.
🚀 Project Spotlights: With the 2024 Wikimedia Hackathon just around the
corner, discover everything you need to know about the event format,
program details, social activities, and more! Plus, catch up on the success
stories from the recently concluded Outreachy Internship Program and gain
fascinating insights from the Wikimedia Wishaton.
The Wikimedia Technical Community is large and diverse - it's hard to
capture everything. We would love to hear your ideas. Have a topic you'd
like us to cover or a story to share? Ideas for future newsletter editions?
Please add your suggestion to the talk page
<https://www.mediawiki.org/wiki/Talk:Technical_Community_Newsletter>. We'd
love to hear from you!
If you'd like to keep up with updates and information, subscribe to the
Technical Community Newsletter:
https://www.mediawiki.org/wiki/Newsletter:Technical_Community_Newsletter
Until the next edition,
--
*Onyinyechi Onifade *
Technical Community Program Manager
Wikimedia Foundation <https://wikimediafoundation.org/>
(If you don’t work with pagelinks table, feel free to ignore this message)
Hello,
Here is an update and reminder on the previous announcement
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
regarding normalization of links tables that was sent around a year ago.
As part of that work, soon the pl_namespace and pl_title columns of
pagelinks table will be dropped and you will need to use pl_target_id
joining with the linktarget table instead. This is basically identical to
the templatelinks normalization that happened a year ago.
Currently, MediaWiki writes to both data schemes of pagelinks for new rows
in all wikis except English Wikipedia and Wikimedia Commons (we will start
writing to these two wikis next week). We have started to backfill the data
with the new schema but it will take weeks to finish in large wikis.
So if you query this table directly or your tools do, You will need to
update them accordingly. I will write a reminder before dropping the old
columns once the data has been fully backfilled.
You can keep track of the general long-term work in T300222
<https://phabricator.wikimedia.org/T300222> and the specific work for
pagelinks in T299947 <https://phabricator.wikimedia.org/T299947>. You can
also read more on the reasoning in T222224
<https://phabricator.wikimedia.org/T222224> or the previous announcement
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
.
Thank you,
--
*Amir Sarabadani (he/him)*
Staff Database Architect
Wikimedia Foundation <https://wikimediafoundation.org/>
I am contributing to wikimedia-commons android app and we encountered a case where we need to show CAPTCHA within the app, is there any end-points available for this ? To Complete the CAPTCHA verification when it's required ?
Hi Wikitech,
I have over 5 years of professional experience as a software developer, I
would like to join this awesome team and make meaningful contributions . I
have made progress as far as following the participate instructions
<https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hacker> goes, I
have the code setup locally and would love some help in using phabricator
and making and pushing changes to the codebase.
Is there someone who has experience and is willing to mentor me into doing
this? I would highly appreciate and cannot wait to be part of the team.
Kind regards.
Newton
Hi everyone,
We invite your contributions to the Wiki Workshop Hall, a new track as part
of Wiki Workshop 2024 <https://wikiworkshop.org/> which will take place
virtually as a standalone event on June 20, 2024 (tentatively 12:00-19:00
UTC).
The Hall will be a novel space for Wikimedia researchers and Wikimedia
movement members to connect with each other. Through this new track, we aim
to provide a dedicated space for learning, exchange of ideas, the spark of
curiosity, and community building.
We welcome proposals that align with the interactive and collaborative
spirit of the Wiki Workshop Hall and look forward to a wide variety of
content: experiences and learnings, knowledge pieces, how-tos, open
questions, pain points, etc. During the Hall, a breakout room will be set
up for each accepted proposal, so that Wiki Workshop attendees can move
between rooms to interact with their hosts.
*Learn more about the Wiki Workshop Hall at *
*https://wikiworkshop.org/2024/call-for-hall*
<https://wikiworkshop.org/2024/call-for-hall.html>* and submit your
contributions by **April 29, 2024 (23:59 AoE)*
<https://www.timeanddate.com/worldclock/converter.html?iso=20240430T115900&p…>
*. *
If you have questions about the workshop or about Wiki Workshop Hall,
please email wikiworkshop(a)googlegroups.com with a [Wiki Workshop Hall] tag
in the subject of your email or comment on this post.
Looking forward to seeing many of you in this year's edition.
The Wiki Workshop Hall chairs,
Pablo Aragón, Wikimedia Foundation
Kinneret Gordon, Wikimedia Foundation
Hey all,
This is a quick note to highlight that in five weeks' time, the REL1_42
branch will be created for MediaWiki core and each of the extensions and
skins in Wikimedia git, with some (the 'tarball') included as sub-modules
of MediaWiki itself[0]. This is the first step in the release process for
MediaWiki 1.42, which should be out in May 2024, approximately six months
after MediaWiki 1.41.
The branches will reflect the code as of the last 'alpha' branch for the
release, 1.42.0-wmf.26, which will be deployed to Wikimedia wikis in the
week beginning 8 April 2024 for MediaWiki itself and those extensions
and skins available there.
After that point, patches that land in the main development branch of
MediaWiki and its bundled extensions and skins will be instead be slated
for the MediaWiki 1.43 release unless specifically backported[1].
If you are working on a new feature that you wish to land for the release,
you now have a few days to finish your work and land it in the development
branch; feature changes should not be backported except in an urgent case.
If your work might not be complete in time, and yet should block release
for everyone else, please file a task against the `mw-1.42-release` project
on Phabricator.[2]
If you have tickets that are already tagged for `mw-1.42-release`, please
finish them, untag them, or reach out to get them resolved in the next few
weeks.
We hope to issue the first release candidate, 1.42.0-rc.0, two weeks after
the branch point, and if all goes well, to release MediaWiki 1.42.0 a few
weeks after that.
[0]: https://www.mediawiki.org/wiki/Bundled_extensions_and_skins
[1]: https://www.mediawiki.org/wiki/Backporting_fixes
[2]: https://phabricator.wikimedia.org/tag/mw-1.42-release/
Yours,
--
James D. Forrester (he/him or they/themself)
Wikimedia Foundation
Hi All, welcome to the monthly MediaWiki Insights email!
We’re starting this email again by celebrating volunteer contributors who
got their first patch merged in MW core, WMF deployed extensions or
services over the past month:
Many thanks and congrats to Nemoralis, RockingPenny4, Theprotonade, S8321414,
Philip and Aram!
Enable more people to know MediaWiki and contribute effectively
We’ve officially hit the baseline
<https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Contributor_reten…>
we’ve set to measure the growth in number of people who have submitted
patches to MediaWiki core: During the period July 2022 - June 2023 (= last
WMF fiscal year), 70 people have submitted more than 5 patches to MW core.
This year, we reached this number already in March and are currently
observing a 14.5% increase in the number of contributors who have submitted
more than 5 patches between July 2023 and March 2024 compared to July 2022
- March 2023.
To achieve the goal of a 20% increase, we will continue with consultancy
and code review for teams and volunteers as part of our regular work. We
are also planning a dedicated MediaWiki focus area for the upcoming Wikimedia
Hackathon <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2024> to help
people contribute to MediaWiki. If you are attending the Hackathon and are
interested in joining the fun to help others onboard in MediaWiki, want to
get started with MediaWiki development or already have a project you’d like
to get help on, please monitor the Hackathon channels and workboard over
the coming weeks for more communication around this!
Project Snapshots: Sustainability and evolution of the platform requires
the efforts of many: Deprecated wfGetDB(), migration to Prometheus and
supporting page content in core REST API
The theme of many initiatives that ensure sustainability and evolution of
the MediaWiki platform is that it requires the collaborative effort of
many. In some cases this may be work done primarily by specific teams, in
other cases this is a collective effort of staff across teams and
volunteers.
One great example for the collective effort of volunteers and staff is
that wfGetDB()
- the old global function to get database connections - is now
hard-deprecated <https://phabricator.wikimedia.org/T273239>, thanks to many
people who did the migration across many extensions!
<https://phabricator.wikimedia.org/T273239>
With wfGetDB() out of the picture, we no longer rely on global state when
accessing database connections. This means that code that wants to access
another wiki’s database may not end up accidentally mixing information from
two wikis. Besides, the new IConnectionProvider interface -
getReplicaDatabase() is hopefully more readable than wfGetDB(DB_REPLICA). This
is part of our wider work to modernize MediaWiki's platform so that we can
inject services for scale and testing.
Supporting page content in core REST API for fetching HTML
We already talked about RESTBase deprecation and Parser Unification
milestones in the last edition
<https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Reports/February_…>
of the MediaWiki Insights email. Oftentimes, work on one initiative
benefits other initiatives and vice versa. This is very true for the
following achievement, which is closely related to Parser Unification and
RESTBase deprecation:
The MediaWiki REST endpoints for fetching page HTML
<https://www.mediawiki.org/wiki/API:REST_API/Reference#Get_HTML> now
support all kinds of content. This provides an easy way for bots and
scripts to get the content of any wiki page just like it would be shown to
users when they view an article in the browser (*).
This may seem like a simple and obvious thing, but we did not have the
capability until recently: From introduction of these endpoints in 2020 up
until the completion of the work on T359426
<https://phabricator.wikimedia.org/T359426>, they were limited to wikitext
pages and would fail on pages containing JavaScript or Lua or Wikidata
items.
A lot of changes were necessary “under the hood” to get to a point where
the REST API could provide rendered HTML for all kinds of content, while
using the Parsoid rendering of wiki pages. Much of the necessary work
overlaps with the efforts of sunsetting RESTbase
<https://www.mediawiki.org/wiki/RESTBase/deprecation> and implementing
support for Parsoid page views <https://phabricator.wikimedia.org/T55784>.
Some key aspects were:
- Integrating Parsoid with ContentHandler (T311648
<https://phabricator.wikimedia.org/T311648>)
- Populating the cache with Parsoid output when pages are edited (T320534
<https://phabricator.wikimedia.org/T320534>)
- Removing the caching layer for Parsoid output in RESTbase (T344945
<https://phabricator.wikimedia.org/T344945>)
- Implementing variant conversion for the API endpoint (T317019
<https://phabricator.wikimedia.org/T317019>)
Having support for all kinds of page content in the REST API while using
Parsoid for wikitext pages is a major milestone towards supporting Parsoid
page views. It demonstrates that Parsoid has been fully integrated with the
content rendering and caching infrastructure in MediaWiki. This is the
culmination of the efforts of several teams over multiple years. Many
thanks to everyone involved!
Note that, while the REST endpoints now support all kinds of content, they
are not quite yet ready for prime time: they still lack proper integration
with the edge caches <https://www.mediawiki.org/wiki/Manual:Varnish_caching>
that will ensure good performance when a lot of clients start using these
APIs. To address this issue and to improve version management for API
endpoints, we are considering changing the canonical URLs of the endpoints.
(*) We still use the old parser of page views though. See the last
MediaWiki Insights email
<https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Reports/February_…>
for where we’re at on the roadmap.
MediaWiki metrics to Prometheus migration: Status and call to action
The Observability team is currently working on migrating MediaWiki metrics
to Prometheus <https://prometheus.io/docs/introduction/overview/>,
utilizing StatsLib <https://www.mediawiki.org/wiki/Manual:Stats>, an
internally developed, Prometheus-capable metrics interface. We have been
using Prometheus in Wikimedia production for several years as it offers
several benefits over Graphite
<https://prometheus.io/docs/introduction/comparison/>. Migrating ensures we
stay ahead with a supported, scalable metrics platform for a more
effective, multidimensional metrics analysis and storage engine.
We are closing in on about 8% of total metrics emitted to graphite migrated
over to Prometheus
<https://grafana.wikimedia.org/d/nCxX65cSk/mediawiki-statslib-migration?orgI…>
and are now ready to invite more people in to help contribute to this
effort! Your expertise can help drive the success of this migration and support
in migrating your component’s metrics to StatsLib (T350592)
<https://phabricator.wikimedia.org/T350592>:
- Look up your component, extension, or module and follow the
examples/docs in the task to migrate your metrics to the new metrics
interface.
- Help deprecate and clean up/remove outdated metrics not in use (or
graphed in dashboards).
- Collaboration in testing and feedback for a seamless transition.
A more detailed announcement and call to action will follow.
Many thanks to the Observability team for their leadership on this
initiative, Derrick, Timo and Larissa from the MediaWiki Platform team for
their consultancy and help in converting MediaWiki metrics so far; Kavitha,
Giuseppe, Janis, and Clement from Service Ops for infrastructure support;
and the Search Platform team for their recent involvement!
Annual Plan 2024/25: Key Result drafts published
The Wikimedia Foundation has recently published the draft “key results”
<https://en.wikipedia.org/wiki/Objectives_and_key_results> by the Product &
Technology department for the upcoming annual plan. While this is not yet
at the project/initiatives level (“hypotheses”), the draft KRs give
insights in focus areas for the next year. The most relevant objectives for
MediaWiki platform work and services for developers are WE5 (“Knowledge
Platform I”) and WE6 (“Knowledge Platform II”). Input and questions on
these drafts
<https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2024-2025/…>
are welcome!
Upcoming: MW 1.42 release
MW 1.42 <https://mediawiki.org/wiki/MediaWiki_1.42> release is coming! The
tentative target was set as May 2024
<https://phabricator.wikimedia.org/T359833> and in the next few weeks, it
will be the time to polish and prepare for the release. Stay tuned for
updates and use Phabricator
<https://phabricator.wikimedia.org/project/view/6601/> to engage with us
and raise potential blockers if you haven’t done yet.
Thanks all for reading,
Birgit
--
Birgit Müller (she/her)
Director of Product, MediaWiki and Developer Experiences
Wikimedia Foundation <https://wikimediafoundation.org/>
Hello all!
The Search Platform Team usually holds an open meeting on the first
Wednesday of each month. Come talk to us about anything related to
Wikimedia search, Wikidata Query Service (WDQS), Wikimedia Commons Query
Service (WCQS), etc.!
Feel free to add your items to the Etherpad Agenda for the next meeting.
Details for our next meeting:
Date: Wednesday, April 3, 2024
Time: 15:00-16:00 UTC / 08:00 PDT / 11:00 EDT / 17:00 CEST
Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours
Google Meet link: https://meet.google.com/vgj-bbeb-uyi
Join by phone: https://tel.meet/vgj-bbeb-uyi?pin=8118110806927
Have fun and see you soon!
Guillaume
--
*Guillaume Lederrey* (he/him)
Engineering Manager
Wikimedia Foundation <https://wikimediafoundation.org/>
Dear all,
The MathML rendering option was enabled in all WMF-deployed wikis
on March 12 for logged-in users [1].
Based on the feedback from the German-speaking community, many
advanced Math users prefer the LaTeX / MathJax style rendering, which
uses a bigger font size and provides an extra bold rendering experience.
Client-side MathJax is perceived to make it easier to differentiate
text and formulae. Consequently, a client-side MathJax rendering
option will be added, allowing power users to have the same experience
as many other websites that rely on client-side MathJax for math rendering [2].
This eventually means that we no longer rely on RestBASE or Mathoid
for math rendering in MediaWiki and can use math support even without
connections to external services. This goes beyond what was promised
after the technical conference in 2019 when the shift away from the
restbase cache was announced [3,4].
MathML rendering makes use of MediaWikis' built-in cache abstraction.
For more details on the new rendering option, see
https://arxiv.org/pdf/2401.16786.pdf.
In the optimal case, we can discontinue restbase and mathoid math
rendering from version 1.42 and backport the changes to LTS versions.
Please let me know if you rely on mathoid in one way or another.
All the best
Moritz (physikerwelt)
[1]: https://phabricator.wikimedia.org/T358803
[2]: https://phabricator.wikimedia.org/T354136
[3]: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…
[4]: https://www.mediawiki.org/w/index.php?title=RESTBase/deprecation&oldid=6045…
Hi All,
I hope this email finds you well. We are thrilled to announce the
successful conclusion of the Outreachy Round 27 Internship program! Over
the past few months, our interns have showcased immense dedication,
enthusiasm, and talent, contributing significantly to our community's
growth and development.
We would like to extend our heartfelt gratitude to our mentors [Sage Ross,
Amine Hassou, Anmol Wassan, Parthiv Menon, Satdeep Gill, Sam Wilson, Mike
Raish, Andi Inácio, Éder Porto, Jan-Bart de Vreede, Sailesh Patnaik, Nat
Hernández Clavijo, Luca Martinelli, Éder Porto, Mike Peel, Alberto Leoncio]
who played a pivotal role in guiding and nurturing our interns throughout
their journey. Your invaluable support, expertise, and mentorship have
undoubtedly helped shape the future of these budding professionals. Your
commitment to their development and success is truly commendable.
We encourage each of you to take a moment to read about our interns'
amazing journey. Their experiences, achievements, and insights are not just
inspiring but also show how mentorship and hands-on learning can truly make
a difference.
You can find more about their journey and accomplishments in the following
links:
-
Outreachy 27 Mediawiki: https://www.mediawiki.org/wiki/Outreachy/Round_27
-
Outreachy 27 Internship Report: From Interns to Innovators: The Impact
of Outreachy Round 27 Internship Program
<https://diff.wikimedia.org/2024/04/01/from-interns-to-innovators-the-impact…>
[1]
-
Reimagining Wikimedia’s Socio Technical Infrastructure: Contributions
from Brazil, Nigeria, and Uganda in Outreachy Round 27
<https://diff.wikimedia.org/2024/03/06/reimagining-wikimedias-sociotechnical…>
[2]
Once again, thank you to our mentors for your unwavering support, and
congratulations to our interns on reaching this significant milestone. We
look forward to witnessing your continued growth and success.
If you're interested in becoming a part of our Outreach Programs, be sure
to subscribe to Wikitech-I mailing list
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/>
[3] and Technical Community Newsletter
<https://www.mediawiki.org/wiki/Newsletter:Technical_Community_Newsletter>
[4] to stay updated on upcoming internship & mentorship opportunities and
other exciting initiatives.
Best regards,
Wikimedia Org Admins (Onyinyechi Onifade & Sheila Wangari)
[1]
https://diff.wikimedia.org/2024/04/01/from-interns-to-innovators-the-impact…
[2]
https://diff.wikimedia.org/2024/03/06/reimagining-wikimedias-sociotechnical…
[3]
https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/
[4] https://www.mediawiki.org/wiki/Newsletter:Technical_Community_Newsletter