Hello,
I have got issue with ListFiles page in mediawiki 1.35.1
Filtering worked not very good, was case-sensitive and not always got
text in middle of file name.
I looked in DB and saw that img_name column is varbinary, but
pagers/ImageListPager.php tries to do case-insensitive select with
LOWERing both sides of strings. But LOWER does not work for varbinary
So I think that following change will be reasonable:
--- ImageListPager.php.orig 2021-10-14 16:31:52.000000000 +0300
+++ ImageListPager.php 2021-10-14 16:00:10.127694733 +0300
@@ -90,9 +90,10 @@
if ( $nt ) {
$dbr = wfGetDB( DB_REPLICA );
- $this->mQueryConds[] = 'LOWER(img_name)'
.
+ $this->mQueryConds[] =
'LOWER(CONVERT(img_name USING utf8))' .
$dbr->buildLike(
$dbr->anyString(),
- strtolower(
$nt->getDBkey() ), $dbr->anyString() );
+ mb_strtolower(
$nt->getDBkey() ), $dbr->anyString() );
+
}
}
@@ -161,9 +162,9 @@
$nt = Title::newFromText( $this->mSearch );
if ( $nt ) {
$dbr = wfGetDB( DB_REPLICA );
- $conds[] = 'LOWER(' . $prefix . '_name)'
.
+ $conds[] = 'LOWER(CONVERT(' . $prefix .
'_name USING utf8))' .
$dbr->buildLike(
$dbr->anyString(),
- strtolower(
$nt->getDBkey() ), $dbr->anyString() );
+ mb_strtolower(
$nt->getDBkey() ), $dbr->anyString() );
}
}
--
Sergey
A long time coming! The interface looks amazing and seems very well thought
out. It's nice to have an official, single go-to for all things tools, user
scripts, gadgets, etc. Huge thanks and kudos to everyone involved!
~ MA
On Thu, Oct 14, 2021 at 3:37 PM Galder Gonzalez Larrañaga <
galder158(a)hotmail.com> wrote:
> Thanks, Brigit, for this hub, it is great to have it! I have tried and
> can't find any way to look for tools that are not nominated as "Coolest
> Tool Award" besides looking for name. Is there a way for searching by
> categories?
>
> Thanks
> Galder
> ------------------------------
> *From:* Birgit Müller <bmueller(a)wikimedia.org>
> *Sent:* Thursday, October 14, 2021 4:58 PM
> *To:* wikitech-l <wikitech-l(a)lists.wikimedia.org>;
> wikimedia-l(a)lists.wikimedia.org <wikimedia-l(a)lists.wikimedia.org>;
> Wikimedia Cloud Services general discussion and support <
> cloud(a)lists.wikimedia.org>; wikidata(a)lists.wikimedia.org <
> wikidata(a)lists.wikimedia.org>; wikitech-ambassadors(a)lists.wikimedia.org <
> wikitech-ambassadors(a)lists.wikimedia.org>
> *Subject:* [Wikimedia-l] Toolhub 1.0 is launched! Discover software tools
> used at Wikimedia
>
>
> Hi All,
>
> We are happy to announce the launch of Toolhub
> <https://toolhub.wikimedia.org/> – a community-authored catalogue that
> aims to make software tools
> <https://meta.wikimedia.org/wiki/Toolhub#What_is_a_%22tool%22?> used in
> the Wikimedia movement discoverable to everyone.
>
> Community developed tools – including web applications, bots, gadgets,
> user scripts, lua modules, and more – play a significant role in the
> Wikimedia projects. These software applications address a wide range of use
> cases including finding bad faith edits and other content curation, bulk
> editing, collecting statistical information, creating special citations,
> and much more. About ⅓ of all edits are made by bots and tools. In
> addition, semi-automated edits are helped by user scripts, gadgets, and
> other editing assistance tools that run from the user's local computer or
> directly inside the wikis. There are thousands of tools available, but how
> can you find them?
>
> With Toolhub, you can document and find tools
> <https://meta.wikimedia.org/wiki/Toolhub>, promote their use in your wiki
> community, and help improve them by contributing data. You can create and
> share lists of tools relevant to your work - for example, for GLAM tools,
> or for wiki projects such as Women in Red.
>
> This first release provides a core set of functionalities
> <https://meta.wikimedia.org/wiki/Toolhub/Roadmap>, and contains an
> initial data set of about 1500 tools. Most of the initial tools in the
> catalog are imported from the same data files developers have created for Hay's
> Directory <https://hay.toolforge.org/directory/> which has been a major
> inspiration for Toolhub.
>
> Toolhub serves developers and users of tools alike. It is part of our
> efforts to improve the infrastructure and services for technical
> contributors, captured under one of Technology’s top level objectives in
> the FY 2020-2021 and 2021-2022 annual plans: Tech Community Building
> <https://www.mediawiki.org/wiki/Wikimedia_Technology/Annual_Plans/ERF_OKR:_T…>.
> We hope to continue conversations with developers and users of tools, plan
> to improve Toolhub, and to further expand the functionality.
>
> A collaborative system and open developer platform
>
> Toolhub is built as an API driven platform that makes it possible to
> extend and remix the catalogue, and to make collecting and reusing
> information about tools as open and collaborative as we can. Everything
> that can be done interactively with the Toolhub website can also be done
> remotely through the API. We would love to hear from technical
> contributors interested in using the Toolhub API
> <https://meta.wikimedia.org/wiki/Talk:Toolhub#API-use> to build new tools
> that make new ways to add or consume information from Toolhub's catalog.
>
> Our decision record
> <https://meta.wikimedia.org/wiki/Toolhub/Decision_record> and weekly
> progress reports
> <https://meta.wikimedia.org/wiki/Toolhub/Progress_reports> on Meta
> provide more insights in technical implementation details and decisions
> made throughout the development process. The Toolhub/About page
> <https://meta.wikimedia.org/wiki/Toolhub/About> provides information on
> project origin, research, use cases, data model, and roadmap. This recording
> from a lightning talk at ‘21 Wikimania
> <https://www.youtube.com/watch?v=J2hNm7bKjDo> gives an overview of the
> main aspects in 10 minutes.
>
> Thank you <3
>
> This project wouldn’t have been possible without the support, knowledge,
> ideas and prior work of many. One of the nicest side-effects of a release
> is that it’s a great opportunity to thank folks for their time and
> contributions :-)
>
>
> -
>
> Husky <https://meta.wikimedia.org/wiki/User:Husky>, whose Hay's
> Directory <https://hay.toolforge.org/directory/> provided the
> foundation for the data model used by Toolhub and inspired some of its
> features.
> -
>
> Harej <https://meta.wikimedia.org/wiki/User:Harej>, for his invaluable
> contributions in the early stages of the Toolhub project.
> -
>
> Our 'advisory board' - Giuseppe
> <https://meta.wikimedia.org/wiki/User:GLavagetto_(WMF)> (SRE), Risker
> <https://meta.wikimedia.org/wiki/User:Risker> (editor, admin), Reedy
> <https://meta.wikimedia.org/wiki/User:Reedy_(WMF)> (Security), Keegan
> <https://meta.wikimedia.org/wiki/User:Keegan_(WMF)> (Community
> Relations), and Eran
> <https://meta.wikimedia.org/wiki/User:%D7%A2%D7%A8%D7%9F> (volunteer
> developer & RTL expert) for providing their perspectives on key questions
> throughout the development process.
> -
>
> Giuseppe, Kunal <https://meta.wikimedia.org/wiki/User:Legoktm>, Manuel
> <https://meta.wikimedia.org/wiki/User:MArostegui_(WMF)>, Effie
> <https://meta.wikimedia.org/wiki/User:EMouzeli_(WMF)>, Cole
> <https://meta.wikimedia.org/wiki/User:CWhite_(WMF)> and Emanuele
> <https://meta.wikimedia.org/wiki/User:ERocca_(WMF)> from SRE and
> Majavah <https://en.wikipedia.org/wiki/User:Majavah> for their help on
> finding and resolving deployment issues.
> -
>
> Dan <https://www.mediawiki.org/wiki/User:Dzduvall> and Jeena
> <https://www.mediawiki.org/wiki/User:JHuneidi_(WMF)> from Release
> Engineering for help with build tooling.
> -
>
> Guillaume <https://www.mediawiki.org/wiki/User:GLederrey_(WMF)> and
> the rest of the Search Platform team for supporting our search index needs.
> -
>
> Manuel for supporting our database needs.
> -
>
> Niklas <https://meta.wikimedia.org/wiki/User:Nlaxstrom-WMF> and the
> whole translatewiki.net community for help with localization and
> internationalization.
> -
>
> Rita <https://meta.wikimedia.org/wiki/User:RHo_(WMF)>, Olga
> <https://meta.wikimedia.org/wiki/User:OTichonova_(WMF)>, Alex
> <https://meta.wikimedia.org/wiki/User:AHollender_(WMF)>, and Matthew
> <https://meta.wikimedia.org/wiki/User:MWilliams_(WMF)> from the Product
> Design <https://www.mediawiki.org/wiki/Design> team for their feedback
> on the Toolhub user interface.
> -
>
> Scott <https://www.mediawiki.org/wiki/User:SBassett_(WMF)> from the
> Security team for our security readiness review.
> -
>
> Amire <https://en.wikipedia.org/wiki/User:Amire80>, Kunal, Eran,
> Reedy, and Dan for contributing code to the project.
> -
>
> Ricordisamoa <https://meta.wikimedia.org/wiki/User:Ricordisamoa>, Quim
> <https://meta.wikimedia.org/wiki/User:Qgil-WMF>, and the people
> participating in conversations on wikitech-l
> <https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
> for T115650 <https://phabricator.wikimedia.org/T115650> which inspired
> this whole project.
> -
>
> Finally, a huge thanks to all the folks who gave input and feedback on
> the talk page, in Phabricator, and at sessions - this is really
> appreciated!
>
>
> We hope that this new resource will be fun to explore, inspire you with
> new ideas, and ultimately be useful for your work.
>
> Feedback, bug reports, ideas and questions are more than welcome on the talk
> page of the project <https://meta.wikimedia.org/wiki/Talk:Toolhub>, or in
> Phabricator <https://phabricator.wikimedia.org/project/board/3224/>. Bryan
> (tech lead) & Seve (our new Product Manager) will be there to chat with
> interested folks and help with any questions. We are looking forward to
> evolving this project step-by-step and jointly with everyone!
>
> Birgit – on behalf of Technical Engagement & our Toolhub project team
>
>
> --
> Birgit Müller (she/her)
> Director of Technical Engagement
>
> Wikimedia Foundation <https://wikimediafoundation.org/>
> _______________________________________________
> Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines
> at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l
> Public archives at
> https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org…
> To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org
Courtesy of Tyler's new https://trainbow.toolforge.org/ - this email is
a belated summary of last week's Wikimedia production deployment of
1.38.0-wmf.3:
Conductor: Brennen Bearnes
Backup Conductor: Jeena Huneidi
Blocker Task: https://phabricator.wikimedia.org/T281167
Current Status: https://versions.toolforge.org/
📊 By the Numbers
Sparklines comparing with the last 5 trains.
261 Patches ▄▇██▁
4 Rollbacks ▁▃▁▃█
3 Days of delay ▁▃▁▁█
7 Blockers ▃▃▁█▆
🌈 Traintastic Folks 🎉
Thanks to folks who reported, resolved, or triaged blockers, and
otherwise assisted mightily with this train:
Timo Tijhof
Umherirrender
Zabe
DannyS712
Taavi Väänänen
James Forrester
Legoktm
Daniel Kinzler
Tim Starling
Pchelolo
GeoffreyT2000
AntiCompositeNumber
Pigsonthewing
Nikerabbit
(We only automated generating the e-mail, not the train. Automating the
train out of existence is a longer-term goal.)
--
Brennen Bearnes
Release Engineering
Wikimedia Foundation
Hey all,
A quick update from me about the work of the Asynchronous Content Fragments
working group.
*TL;DR*
We're meeting each week to talk about adding async content support to MW.
This week we discussed possible upper-level use cases, with initial
thoughts documented on this page
<https://www.mediawiki.org/wiki/Abstract_Wikipedia_team/Asynchronous_content…>.
Comments welcome!
*Context*
The overall goal <https://phabricator.wikimedia.org/T282585> is to explore,
decide on, and build a way to include asynchronously-available content
fragments in MediaWiki pages, to allow new forms of content (like
Wikifunctions) and a faster, less tightly-coupled design for MediaWiki
overall. The working group (Subbu and C. Scott from Content Transformation,
Tim from Platform, Moriel from Architecture, and me from Abstract
Wikipedia) exists to turn the Decision Statement Overview
<https://docs.google.com/document/d/1kU7a5V-exubbXcyyMA0jKlqioQkoRptexgoWrZo…>
agreed by the Technical Decision Forum into a set of options for
implementation
<https://www.mediawiki.org/wiki/Technical_Decision_Forum/Technical_Decision_…>
(as will be finally agreed in a Decision Record).
*Work this week*
This week we discussed some use cases that I proposed. There was a lot of
talk around the differing needs in what readers, most API consumers, search
engines, *etc.* will want vs. what editors (and other logged-in users) will
need to be effective.
In particular, we considered the need for anti-abuse features,
Notifications, Recent Changes and Watchlist entries to all trigger
immediately (as is current behaviour) despite not having the full result
yet, and then needing the final renders to update wherever they're stored.
This will be complicated, and vary by specific use case. For example the
product need for immediacy will be very high and second results not wanted
(e.g. talk page notification e-mails should be sent immediately and not
wait, but also not be duplicated later); in others, it will be lower and so
things can wait a little bit for some things (e.g. link notifications can
wait a few minutes).
We also talked about the need for having default values, placeholders,
timeouts, and handling errors, and whether that should be controlled by
MediaWiki centrally or if each fragment provider could be synchronously
called to provide default/placeholders as needed.
Much more of this will be discussed next week.
Hope this is of interest. If you have thoughts or comments, please do let
us know on the discussion page
<https://www.mediawiki.org/wiki/Talk:Abstract_Wikipedia_team/Asynchronous_co…>
or directly.
Yours,
--
*James D. Forrester* (he/him <http://pronoun.is/he> or they/themself
<http://pronoun.is/they/.../themself>)
Wikimedia Foundation <https://wikimediafoundation.org/>
Last week, I spoke to a few of my Wikimedia Foundation colleagues about how
we deploy code—I completely botched it.
At the end of the conversation, I was pretty sure I'd only succeeded in
making a complex process more opaque. I decided to write a blog to redeem
myself: How We Deploy Code
<https://phabricator.wikimedia.org/phame/post/view/253/how_we_deploy_code/>
My goal was to write a very high-level overview of the process we use to
deploy code to Wikimedia production.
Hopefully, this is helpful.
<3
– Tyler Cipriani (he/him)
Engineering Manager, Release Engineering
Wikimedia Foundation
Hi everyone,
The Search Platform Team
<https://www.mediawiki.org/wiki/Wikimedia_Search_Platform> usually holds
office hours the first Wednesday of each month—but this month it's the
second Wednesday. Come talk to us about anything related to Wikimedia
search, Wikidata Query Service, Wikimedia Commons Query Service, etc.!
Feel free to add your items to the Etherpad Agenda for the next meeting.
Details for our next meeting:
Date: Wednesday, October 13th, 2021
Time: 15:00-16:00 GMT / 08:00-09:00 PDT / 11:00-12:00 EDT / 17:00-18:00 CEST
Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours
Google Meet link: https://meet.google.com/vgj-bbeb-uyi
Join by phone: https://tel.meet/vgj-bbeb-uyi?pin=8118110806927
*NOTE: We have a new Google Meet link as of August 2021, which offers
international calling options.*
Hope to talk to you next week!
—Trey
Trey Jones
Sr. Computational Linguist, Search Platform
Wikimedia Foundation
UTC–4 / EDT
Dear all,
It’s time for our third edition of the Coolest Tool Award!
Tools play an essential role at Wikimedia, and so do the many volunteer
developers who experiment with new ideas, develop & maintain local &
global solutions and enhance the experience for Wikimedia communities.
We’d like to invite you all to nominate your favorite & most used tools
and help us celebrate the people who create them!
As no one can possibly know all the cool tools out there, we’re looking
for some help and inspiration: please point us to the tools that you
think are great - for any reason you can think of!
Please go to https://meta.wikimedia.org/wiki/Coolest_Tool_Award
to recommend tools by October 27, 2021. You can nominate as many tools
as you want by filling out the form multiple times.
Thank you very much for your ideas & recommendation(s)!
The award is organized & selected by the Coolest Tool Academy 2021. We
plan to recognize the greatest tools in a variety of categories (for
examples, see last year’s categories). The award ceremony will take
place virtually again this year and we will provide more details soon
about the specific logistics and dates.
We will continue to spread the word over the next week, but if you get
the chance, please feel welcome to share this information with others
too!
Thanks :-)
Andre, for the Coolest Tool Academy 2021
--
Andre Klapper (he/him) | Bugwrangler / Developer Advocate
https://blogs.gnome.org/aklapper/
Hi everyone,
tl;dr: External shell outs are now run via Shellbox. Any deployed code
needs to use Shellbox/BoxedCommand, and documentation is available to
help migrate.
To safely re-enable Score (LilyPond) on Wikimedia wikis, we developed
Shellbox, a way to run shell commands in a remote, isolated container.
This is (hopefully) a stronger level of isolation than we previously had
with firejail, since it's relying on Linux containers and Kubernetes to
do the isolation. At the same time, this helps us in moving towards
running MediaWiki on Kubernetes, as we don't want to include all these
external commands inside the MediaWiki container. For the most part, any
new shelling out to external commands needs to be done via Shellbox.
A lot of the design and rationale behind Shellbox is captured in the
RfC: <https://phabricator.wikimedia.org/T260330>.
In Wikimedia production, so far Score, Timeline, SyntaxHighlight and
Wikidata constraint regex checking are all using Shellbox. Details about
that and links to dashboards are available at
<https://wikitech.wikimedia.org/wiki/Shellbox>. The main things that are
left are media-handling code that extracts metadata: DjVu, PdfHandler
and PagedTiffHandler, which is tracked at
<https://phabricator.wikimedia.org/T289228>, and videoscaling
(TimedMediaHandler).
Some work has to be done in MediaWiki to make code compatible with
Shellbox, specifically switching to "BoxedCommand", which now has its
own documentation page:
<https://www.mediawiki.org/wiki/Manual:BoxedCommand>. BoxedCommand works
transparently whether you have a separate Shellbox service set up or
not. This is the preferred way to write new shellouts going forward,
though Shell::command() isn't officially deprecated yet. So far all
shellouts that are used in Wikimedia production have already been
converted except for TimedMediaHandler.
Looking forward, I think this also gives us a lot of flexibility in
using more external commands in the future. First, we're less tied to
whatever OS version MediaWiki is running on, as long as it can be
built/shipped in a container, we can use it. And secondly, it's probably
OK if external commands aren't super well behaved (e.g. use too much
memory) since they're no longer sharing the same resources as an
appserver (this shouldn't be interpreted as a free pass for super
inefficient stuff of course).
I tried to keep this summary short, and am intending to write a longer
blog post that explains some more history in detail. But if you have any
questions or something isn't clear, please ask!
-- Kunal