Yesterday there was a conversation about code review on irc and among other
things, how sometimes patches can get "stuck".
I had an idea for a way to improve things. I'm not sure if it is a good
idea, but there's only one way to find out.
So without further ado, announcing the Code Review Patch Board:
https://www.mediawiki.org/wiki/Code_review/patch_board
In short - each person is allowed to list one of their patches on the board
that they would really like to see reviewed. You can only list one patch at
a time, and it should be a patch that you have been unable to get review
for for at least a week through normal means. See the page for the full
list of guidelines.
I encourage people to give it a try. Add a patch you wrote that you cannot
get a review for. Or if you have +2 rights, try giving some love to these
underloved patches.
I would also love to hear feedback on the general idea as well as the
current guidelines.
To repeat, the url is:
https://www.mediawiki.org/wiki/Code_review/patch_board
Thanks,
bawolff
Hello all!
The Search Platform Team usually holds an open meeting on the first
Wednesday of each month. Come talk to us about anything related to
Wikimedia search, Wikidata Query Service (WDQS), Wikimedia Commons Query
Service (WCQS), etc.!
Feel free to add your items to the Etherpad Agenda for the next meeting.
Details for our next meeting:
Date: Wednesday, October 4, 2023
Time: 15:00-16:00 UTC / 08:00 PT / 11:00 EDT / 17:00 CET
Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours
Google Meet link: https://meet.google.com/vgj-bbeb-uyi
Join by phone: https://tel.meet/vgj-bbeb-uyi?pin=8118110806927
Have fun and see you soon!
Guillaume
--
*Guillaume Lederrey* (he/him)
Engineering Manager
Wikimedia Foundation <https://wikimediafoundation.org/>
Hi all! This is an announcement for a new developer feature in MediaWiki.
If you don’t develop MediaWiki core, extensions or skins, you can stop
reading :)
MediaWiki interface messages are generally “safe” to edit: when they
contain markup, it is either parsed (as wikitext), sanitized, or fully
HTML-escaped. For this reason, administrators are allowed to edit normal
messages on-wiki in the MediaWiki: namespace, while editing JS code (which
is more dangerous) is restricted to interface administrators. (A few
exceptions, messages that are not escaped and which can only be edited by
interface administrators, are configured in $wgRawHtmlMessages
<https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:$wgRawHtmlMessages>.)
Occasionally, a bug in the software means that a message isn’t properly
escaped, which can in theory be abused by administrators to effectively
gain interface administrator powers (by editing a MediaWiki: page for a
message to contain <script> tags, or onclick="" attributes, or whatever).
Such bugs are usually considered low-severity security issues; some of them
are tracked in T2212 <https://phabricator.wikimedia.org/T2212>. (The
general issue is known as cross-site scripting
<https://www.mediawiki.org/wiki/Special:MyLanguage/Cross-site_scripting>
and can be much more severe when it’s not limited to interface messages.)
Previously, checking for these issues as a developer was tedious: if you
suspected that a message was vulnerable to HTML injection, you had to
create a page for it in the MediaWiki: namespace, or edit the corresponding
en.json file on disk (and potentially rebuild the localisation cache). The
recently merged “xss language code” feature simplifies this process. If the
developer setting $wgUseXssLanguage is set to true, then an “x-xss”
language code becomes available and can be selected with *?uselang=x-xss*
in the URL. When using this language code, all messages become “malicious”:
every message is replaced by a snippet of HTML that tries to run alert('
*message-key*'). If everything is implemented correctly, all of those HTML
snippets should be escaped, and no alerts should fire, although the wiki
will look quite strange:
If you see any alert, then that means that a message has not been escaped
correctly; use the message key shown in the alert to hunt down the buggy
code (or add the message key to $wgRawHtmlMessages). This feature is
intended to be especially useful during code review: check out the change,
load a page with ?uselang=x-xss, and see if any alerts come up.
Miscellaneous notes:
- This is a developer-only feature. I strongly recommend against
setting $wgUseXssLanguage
= true; in any production setting. (It will be added to
DevelopmentSettings.php soon.)
- Above, I focused on the possibility to abuse unescaped messages via
the MediaWiki: namespace. You might also be thinking about the potential
for translatewiki.net contributors to inject malicious HTML into message
translations; however, the translation exports from translatewiki.net to
the JSON files automatically check for any HTML in translations, and flag
suspicious cases for human review. Therefore, it’s much harder to exploit
an unescaped message via translatewiki.net than via the MediaWiki:
namespace.
- Finally, I should mention that we already found several
vulnerabilities using this feature, which will be fixed with the upcoming
security release
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>.
If you try out this feature now, and find a vulnerable message, I suggest
you wait until then, and check whether it’s still vulnerable, before
reporting it.
Cheers,
Lucas
--
Lucas Werkmeister (he/er)
Software Engineer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0)30-577 11 62-0
https://wikimedia.de
Imagine a world in which every single human being can freely share in the
sum of all knowledge. Help us to achieve our vision!
https://spenden.wikimedia.de
Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Charlottenburg, VR 23855 B.
Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin,
Steuernummer 27/029/42207.
Hello folks!
For some years now, I've been the main or only point of contact for the
Wiki project sql/xml dumps semimonthly, as well as for a number of
miscellaneous weekly datasets.
This work is now passing to Data Platform Engineering (DPE), and your new
points of contact, starting right away, will be Will Doran (email:wdoran)
and Virginia Poundstone (email:vpoundstone). I'll still be lending a hand
in the background for a little while but by the end of the month I'll have
transitioned into a new role at the Wikimedia Foundation, working more
directly on MediaWiki itself.
The Data Products team, a subteam of DPE, will be managing the current
dumps day-to-day, as well as working on a new dumps system intended to
replace and greatly improve the current one. What formats will it produce,
and what content, and in what bundles? These are all great questions, and
you have a chance to help decide on the answers. The team is gathering
feedback right now; follow this link [
https://docs.google.com/forms/d/e/1FAIpQLScp2KzkcTF7kE8gilCeSogzpeoVN-8yp_S…]
to give your input!
If you want to follow along on work being done on the new dumps system, you
can check the phabricator workboard at
https://phabricator.wikimedia.org/project/board/6630/ and look for items
with the "Dumps 2.0" tag.
Members of the Data Products team are already stepping up to manage the
xmldatadumps-l mailing list, so you should not notice any changes as far as
that goes.
And as always, for dumps-related questions people on this list cannot
answer, and which are not covered in the docs at
https://meta.wikimedia.org/wiki/Data_dumps or
https://wikitech.wikimedia.org/wiki/Dumps you can always email ops-dumps
(at) wikimedia.org.
See you on the wikis!
Ariel Glenn
ariel(a)wikimedia.org
Hello All,
Welcome to the second edition of the monthly MediaWiki Insights
<https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Reports> email!
Since the beginning of July, the Foundation has dedicated MediaWiki product
leadership and a new MediaWiki engineering group
<https://www.mediawiki.org/wiki/MediaWiki_Engineering_Group>.
In August
<https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Reports/August_20…>,
we’ve shared a broad overview on the focus for the next few quarters:
1. Building up the new MW Engineering group and MW Product function
2. Developing a strategy for MediaWiki - by June 30th, 2024 [WMF Annual
Plan, WE3
<https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2023-2024/…>
]
3. Reaching a 20% increase of authors to selected MediaWiki repositories
deployed in Wikimedia production - by June 30th, 2024 [WE3.2]
4. Investing in developer experiences and reduce fragmentation of
developer workflows [WE 3.1] - continuous work with specific deliverables
in 2023/24
5. Exploring and resolving a set of questions around stewardship and
Open Source strategy (goes beyond MediaWiki) [WE3.3]
We’re still in the process of “settling in” (1.), but also made progress on
a few things that we wanted to start tackling early:
Stewardship
We’ve had conversations within the MediaWiki Engineering group
<https://www.mediawiki.org/wiki/MediaWiki_Engineering_Group> on which
components we should prioritize initially/own directly and what the things
are that we’d primarily provide guidance on. While this is work in progress
and also touches on bigger questions, one notable decision is that MW
Engineering takes on stewardship for the authentication-related components
(in MW core and extensions), with support from the Security team. We’ve
resolved outstanding code stewardship requests for the CentralAuth
<https://phabricator.wikimedia.org/T252244> and Oauth
<https://phabricator.wikimedia.org/T224919> extensions as a consequence of
this decision. These changes and other updates are reflected on the developers
and maintainers page <https://www.mediawiki.org/wiki/Developers/Maintainers>
on MW.org.
MediaWiki within Wikimedia’s ecosystem: Update on interviews
So far we’ve interviewed about 40 people on their experiences with
MediaWiki within Wikimedia’s ecosystem and plan a few more interviews. We
expect to be able to wrap up this first round of research in October and to
share the outcome and conclusions in November. These conversations have
been incredibly helpful - many thanks to all the people who took the time
to share their thoughts or still will do so <3
You can find a tentative timeline and overview on research planned
throughout the next few quarters on this page
<https://www.mediawiki.org/wiki/MediaWiki_Product_Insights#Research_pillars_…>
.
Project snapshot: Source Maps and top-level autologin
Over the past few weeks, the MediaWiki Engineering group has been working
on a mix of onboarding tasks (i.e. ResourceLoader, ActionAPI, CentralAuth),
production errors, long term initiatives (Parsoid Read Views, RESTBase
deprecation), consultancy, code review for staff and volunteers’ patches,
and completed projects that had been in the making for a while. A few
snapshots:
Source Maps <https://web.dev/source-maps/> aim to make debugging in web
development easier. It’s a technique for mapping combined and minified
JavaScript back to the original files. Support for source maps is now
implemented in ResourceLoader
<https://www.mediawiki.org/wiki/ResourceLoader>, to aid with debugging
ResourceLoader in production. You can learn more about this work in this
ticket. <https://phabricator.wikimedia.org/T47514> <3 to Tim, Timo and
others for their work on this!
Browsers increasingly roll out anti-tracking measures and limitations on
third-party cookie use. An unfortunate side effect of this is that it also
impacts CentralAuth autologin. One way to mitigate the effects and to allow
auto-login when the browser blocks third-party cookies is to attempt
central auto-login via top-level navigation. This has been enabled in
September. You can learn more about this work in this ticket
<https://phabricator.wikimedia.org/T326281>. <3 to Gergö and others for the
work on this!
Onboarding, among other means, has continued via the weekly Code Mob
sessions: Check out the recordings on this page
<https://meta.wikimedia.org/wiki/User:BPirkle_(WMF)/Code_Mobs_2023> if you
want to follow along.
Next: Enable more people to know MediaWiki and contribute effectively
A key question this year is how we can grow the number of people willing
and able to contribute to MediaWiki. So far we’ve explored approaches and
focus areas, turned some aspects of this already into active practice
through code review and consultancy for teams whose projects touch
MediaWiki core; and came up with first ideas that may help new MediaWiki
contributors (example <https://phabricator.wikimedia.org/T347347>).
We’ll be sharing more about this work and possible initiatives in October,
which is when we “officially” start with working towards an increase of
authors across a specific set of MW repositories that are deployed to
production (WMF Annual Plan, WE3.2
<https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2023-2024/…>
).
Thanks all for reading! - Have a great weekend,
Birgit
--
Birgit Müller (she/her)
Director of Product, MediaWiki and Developer Experiences
Wikimedia Foundation <https://wikimediafoundation.org/>
>
> If the developer setting $wgUseXssLanguage is set to true, then an “x-xss”
> language code becomes available and can be selected with *?uselang=x-xss*
> in the URL. When using this language code, all messages become “malicious”:
> every message is replaced by a snippet of HTML that tries to run alert('
> *message-key*').
Clever feature - this will be great for testing. Thank you!
-Yaron
--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
Hi all,
On Thursday we will be issuing a security and maintenance release to all
supported branches of MediaWiki.
The new releases will be:
- 1.35.12
- 1.39.5
- 1.40.1
This will resolve four security issues in MediaWiki core, two in a bundled
skin, along with bug fixes included for maintenance reasons. This includes
various patches for PHP 8.0, PHP 8.1 and PHP 8.2 support.
One issue in a bundled skin only affects MediaWiki 1.40 and master, the
other bundled skin issue affects MediaWiki 1.39, 1.40 and master.
A partial fix for one of the skin issues is already merged into the
relevant release branch.
One more minor security fix was merged in public after the releases of
1.35.11/1.38.7/1.39.4/1.40.0.
We will make the fixes available in the respective release branches and
master in git. Tarballs will be available for the above mentioned point
releases as well.
A summary of some of the security fixes that have gone into non-bundled
MediaWiki extensions will also follow later.
As a reminder, when 1.35 was released, it was originally due to become end
of life (EOL) at the end of September 2023. Due to 1.39 being released late
(November 2022), and to honor the commitment to the 1 year overlap of
MediaWiki LTS releases, this formal EOL process is being delayed till at
least the end of November 2023.
In practice, this may become sometime in December 2023, to coincide with
the security and maintenance release for that quarter. A formal EOL
announcement for 1.35 will come in advance of that point.
It is therefore expected that 1.35.13 in December 2023 will become the
final release for the 1.35 branch.
It is noted that support and CI for 1.35 is becoming more limited;
backports are becoming best-effort. Browser testing has been dropped for
1.35 in Wikimedia CI, due to the difficulties to support this.
It is strongly recommended to upgrade to 1.39 (the next LTS after 1.35),
which will be supported until November 2025, or 1.40, which will be
supported until June 2024.
[1] https://www.mediawiki.org/wiki/Version_lifecycle
Hi everybody,
TL;DR We would like users of ORES models to migrate to our new open source
ML infrastructure, Lift Wing, within the next five months. We are available
to help you do that, from advice to making code commits. It is important to
note: All ML models currently accessible on ORES are also currently
accessible on Lift Wing.
As part of the Machine Learning Modernization Project (
https://www.mediawiki.org/wiki/Machine_Learning/Modernization), the Machine
Learning team has deployed a Wikimedia’s new machine learning inference
infrastructure, called Lift Wing (
https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing). Lift Wing
brings a lot of new features such as support for GPU-based models, open
source LLM hosting, auto-scaling, stability, and ability to host a larger
number of models.
With the creation of Lift Wing, the team is turning its attention to
deprecating the current machine learning infrastructure, ORES. ORES served
us really well over the years, it was a successful project but it came
before radical changes in technology like Docker, Kubernetes and more
recently MLOps. The servers that run ORES are at the end of their planned
lifespan and so to save cost we are going to shut them down in early 2024.
We have outlined a deprecation path on Wikitech (
https://wikitech.wikimedia.org/wiki/ORES), please read the page if you are
a maintainer of a tool or code that uses the ORES endpoint
https://ores.wikimedia.org/). If you have any doubt or if you need
assistance in migrating to Lift Wing, feel free to contact the ML team via:
- Email: ml(a)wikimedia.org
- Phabricator: #Machine-Learning-Team tag
- IRC (Libera): #wikimedia-ml
The Machine Learning team is available to help projects migrate, from
offering advice to making code commits. We want to make this as easy as
possible for folks.
High Level timeline:
**By September 30th 2023: *Infrastructure powering the ORES API endpoint
will be migrated from ORES to Lift Wing. For users, the API endpoint will
remain the same, and most users won’t notice any change. Rather just the
backend services powering the endpoint will change.
Details: We'd like to add a DNS CNAME that points ores.wikimedia.org to
ores-legacy.wikimedia.org, a new endpoint that offers a almost complete
replacement of the ORES API calling Lift Wing behind the scenes. In an
ideal world we'd migrate all tools to Lift Wing before decommissioning the
infrastructure behind ores.wikimedia.org, but it turned out to be really
challenging so to avoid disrupting users we chose to implement a transition
layer/API.
To summarize, if you don't have time to migrate before September to Lift
Wing, your code/tool should work just fine on ores-legacy.wikimedia.org and
you'll not have to change a line in your code thanks to the DNS CNAME. The
ores-legacy endpoint is not a 100% replacement for ores, we removed some
very old and not used features, so we highly recommend at least test the
new endpoint for your use case to avoid surprises when we'll make the
switch. In case you find anything weird, please report it to us using the
aforementioned channels.
**September to January: *We will be reaching out to every user of ORES we
can identify and working with them to make the migration process as easy as
possible.
**By January 2024: *If all goes well, we would like zero traffic on the
ORES API endpoint so we can turn off the ores-legacy API.
If you want more information about Lift Wing, please check
https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
Thanks in advance for the patience and the help!
Regards,
The Machine Learning Team