TLDR: Invoking maintenance scripts directly will be deprecated in MW 1.40, use maintenance/run.php instead. This affects anyone managing a MediaWiki installation, for development, testing, or production use.
Until now, MediaWiki maintenance scripts have been handled standalonePHP scripts - for instance, to run the script that outputs the MediaWiki version, you would use:
php maintenance/version.php
Starting with MediaWiki 1.40, this is deprecated. The preferred way to run maintenance scripts is now by name, using the maintenance runner:
php maintenance/run.php version
Similarly, the preferred way to run the updater is now:
php maintenance/run.php update
The script to run cal also be specified using the full path of the script file, or the full PHP class name of a subclass of the Maintenance class. For more details, run
php maintenance/run.php --help
Rationale and History:
Treating maintenance scripts as standalone PHP scripts requires some boilerplate code to be present at the top and at the bottom of every file. This is error prone and makes it difficult to update the maintenance framework. But more importantly, for this boilerplate to work, the location of the MediaWiki installation has to be known relative to the maintenance script, which is not reliably possible for scripts defined in extensions.
A similar problem arises if the maintenance script needs a base class other than the default Maintenance class: since the class is loaded before MediaWiki is initialized, the autoloader is not yet in place, and the file containing the base class needs to be included explicitly.
These and similar issues can be avoided by creating a wrapper script that loads and executes the actual maintenance class. This way, the maintenance wrapper can initialize MediaWiki before passing control to the script.
I propose creating such a wrapper as an RFC in 2018 (T99268)[^1], which was approved in 2019. However, implementing the proposal proved challenging, and soon stalled. I picked it up again as a side project after working on overhauling the configuration and bootstrapping code in early 2022: With the introduction of SettingsBuilder, it became much simpler to create a MaintenanceRunner class, because it was no longer necessary to juggle global variables.
Several bits and pieces got reviewed and merged over the course of 2022 (shout out to Amir, Tim, Timo, and everyone who contributed). Now the runner is ready, and we should stop calling maintenance scripts directly.
For now, existing maintenance scripts will function both ways[^2] : when called using the runner, or directly. However, newly created maintenance scripts should not be required to be callable as standalone scripts. So it's best to change all callers to use the wrapper.
This should now work for nearly all[^2] cases, though there are still a couple of rough edges to be smoothed out. If you are running MediaWiki 1.40, please try the new mechanism, and report any isses on Phabricator.
Thanks, Daniel
[^1] https://phabricator.wikimedia.org/T99268 [^2] with the exception over very old-school scripts that do not use the Maintenance base class and rely on CommandLineInc.php instead.
Hey Daniel,
Thanks for the email. Is any refactoring of the maintenance scripts themselves suggested yet? e.g. removing the top and bottom boilerplate?
Thanks,
Novem Linguae
From: Daniel Kinzler dkinzler@wikimedia.org Sent: Monday, January 9, 2023 12:30 PM To: Wikimedia developers wikitech-l@lists.wikimedia.org Subject: [Wikitech-l] Deprecation: directly invoking maintenance scripts
TLDR: Invoking maintenance scripts directly will be deprecated in MW 1.40, use maintenance/run.php instead. This affects anyone managing a MediaWiki installation, for development, testing, or production use.
Until now, MediaWiki maintenance scripts have been handled standalone PHP scripts - for instance, to run the script that outputs the MediaWiki version, you would use:
php maintenance/version.php
Starting with MediaWiki 1.40, this is deprecated. The preferred way to run maintenance scripts is now by name, using the maintenance runner:
php maintenance/run.php version
Similarly, the preferred way to run the updater is now:
php maintenance/run.php update
The script to run cal also be specified using the full path of the script file, or the full PHP class name of a subclass of the Maintenance class. For more details, run
php maintenance/run.php --help
Rationale and History:
Treating maintenance scripts as standalone PHP scripts requires some boilerplate code to be present at the top and at the bottom of every file. This is error prone and makes it difficult to update the maintenance framework. But more importantly,
for this boilerplate to work, the location of the MediaWiki installation has to be known relative to the maintenance script, which is not reliably possible for scripts defined in extensions.
A similar problem arises if the maintenance script needs a base class other than the default Maintenance class: since the class is loaded before MediaWiki is initialized, the autoloader is not yet in place, and the file containing the base class needs to be included explicitly.
These and similar issues can be avoided by creating a wrapper script that loads and executes the actual maintenance class. This way, the maintenance wrapper can initialize MediaWiki before passing control to the script.
I propose creating such a wrapper as an RFC in 2018 (T99268)[^1], which was approved in 2019. However, implementing the proposal proved challenging, and soon stalled. I picked it up again as a side project after working on overhauling the configuration and bootstrapping code in early 2022: With the introduction of SettingsBuilder, it became much simpler to create a MaintenanceRunner class, because it was no longer necessary to juggle global variables.
Several bits and pieces got reviewed and merged over the course of 2022 (shout out to Amir, Tim, Timo, and everyone who contributed). Now the runner is ready, and we should stop calling maintenance scripts directly.
For now, existing maintenance scripts will function both ways[^2] : when called using the runner, or directly. However, newly created maintenance scripts should not be required to be callable as standalone scripts. So it's best to change all callers to use the wrapper.
This should now work for nearly all[^2] cases, though there are still a couple of rough edges to be smoothed out. If you are running MediaWiki 1.40, please try the new mechanism, and report any isses on Phabricator.
Thanks,
Daniel
[^1] https://phabricator.wikimedia.org/T99268
[^2] with the exception over very old-school scripts that do not use the Maintenance base class and rely on CommandLineInc.php instead.
No, backwards compatibility should not be broken yet. In general, there should be a major release between deprecating code and removing it. While https://www.mediawiki.org/wiki/Stable_interface_policy#Removal doesn't explicitly apply to the command line interface, it's still a good guide.
AntiCompositeNumber (he/him)
On Mon, Jan 9, 2023 at 6:31 PM novemlinguae@gmail.com wrote:
Hey Daniel,
Thanks for the email. Is any refactoring of the maintenance scripts themselves suggested yet? e.g. removing the top and bottom boilerplate?
Thanks,
Novem Linguae
From: Daniel Kinzler dkinzler@wikimedia.org Sent: Monday, January 9, 2023 12:30 PM To: Wikimedia developers wikitech-l@lists.wikimedia.org Subject: [Wikitech-l] Deprecation: directly invoking maintenance scripts
TLDR: Invoking maintenance scripts directly will be deprecated in MW 1.40, use maintenance/run.php instead. This affects anyone managing a MediaWiki installation, for development, testing, or production use.
Until now, MediaWiki maintenance scripts have been handled standalone PHP scripts - for instance, to run the script that outputs the MediaWiki version, you would use:
php maintenance/version.php
Starting with MediaWiki 1.40, this is deprecated. The preferred way to run maintenance scripts is now by name, using the maintenance runner:
php maintenance/run.php version
Similarly, the preferred way to run the updater is now:
php maintenance/run.php update
The script to run cal also be specified using the full path of the script file, or the full PHP class name of a subclass of the Maintenance class. For more details, run
php maintenance/run.php --help
Rationale and History:
Treating maintenance scripts as standalone PHP scripts requires some boilerplate code to be present at the top and at the bottom of every file. This is error prone and makes it difficult to update the maintenance framework. But more importantly,
for this boilerplate to work, the location of the MediaWiki installation has to be known relative to the maintenance script, which is not reliably possible for scripts defined in extensions.
A similar problem arises if the maintenance script needs a base class other than the default Maintenance class: since the class is loaded before MediaWiki is initialized, the autoloader is not yet in place, and the file containing the base class needs to be included explicitly.
These and similar issues can be avoided by creating a wrapper script that loads and executes the actual maintenance class. This way, the maintenance wrapper can initialize MediaWiki before passing control to the script.
I propose creating such a wrapper as an RFC in 2018 (T99268)[^1], which was approved in 2019. However, implementing the proposal proved challenging, and soon stalled. I picked it up again as a side project after working on overhauling the configuration and bootstrapping code in early 2022: With the introduction of SettingsBuilder, it became much simpler to create a MaintenanceRunner class, because it was no longer necessary to juggle global variables.
Several bits and pieces got reviewed and merged over the course of 2022 (shout out to Amir, Tim, Timo, and everyone who contributed). Now the runner is ready, and we should stop calling maintenance scripts directly.
For now, existing maintenance scripts will function both ways[^2] : when called using the runner, or directly. However, newly created maintenance scripts should not be required to be callable as standalone scripts. So it's best to change all callers to use the wrapper.
This should now work for nearly all[^2] cases, though there are still a couple of rough edges to be smoothed out. If you are running MediaWiki 1.40, please try the new mechanism, and report any isses on Phabricator.
Thanks,
Daniel
[^1] https://phabricator.wikimedia.org/T99268
[^2] with the exception over very old-school scripts that do not use the Maintenance base class and rely on CommandLineInc.php instead.
--
Daniel Kinzler
Principal Software Engineer, Platform Engineering
Wikimedia Foundation
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
On 9. Jan 2023, at 21:29, Daniel Kinzler dkinzler@wikimedia.org wrote:
I propose creating such a wrapper as an RFC in 2018 (T99268)[^1], which was approved in 2019. However, implementing the proposal proved challenging, and soon stalled. I picked it up again as a side project after working on overhauling the configuration and bootstrapping code in early 2022: With the introduction of SettingsBuilder, it became much simpler to create a MaintenanceRunner class, because it was no longer necessary to juggle global variables.
Thank you for working on this, and for seeing it through!
Kosta
Hello! Where might I find documentation on the new maintenance runner system? I can't find any examples in the Phabricator task or the linked RFC, and searching for "MaintenanceRunner" or "run.php" yields no results on mediawiki.org. Specifically, I was expecting Manual:run.php https://www.mediawiki.org/wiki/Manual:Run.php to exist with some info on how it works.
If given a good example patch, I'm happy to help write the docs :)
Best,
~ MusikAnimal
On Tue, Jan 10, 2023 at 4:10 AM Kosta Harlan kharlan@wikimedia.org wrote:
On 9. Jan 2023, at 21:29, Daniel Kinzler dkinzler@wikimedia.org wrote:
I propose creating such a wrapper as an RFC in 2018 (T99268)[^1], which
was approved in 2019. However, implementing the proposal proved challenging, and soon stalled. I picked it up again as a side project after working on overhauling the configuration and bootstrapping code in early 2022: With the introduction of SettingsBuilder, it became much simpler to create a MaintenanceRunner class, because it was no longer necessary to juggle global variables.
Thank you for working on this, and for seeing it through!
Kosta _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Le 28/02/2023 à 21:42, MusikAnimal a écrit :
Hello! Where might I find documentation on the new maintenance runner system? I can't find any examples in the Phabricator task or the linked RFC, and searching for "MaintenanceRunner" or "run.php" yields no results on mediawiki.org http://mediawiki.org. Specifically, I was expecting Manual:run.php https://www.mediawiki.org/wiki/Manual:Run.php to exist with some info on how it works.
If given a good example patch, I'm happy to help write the docs :)
Hello,
run.php is more or less a wrapper so that instead of invoking:
php maintenance/parse.php
One should use:
maintenance/run parse
That will be released in MediaWiki 1.40 and all maintenance scripts will emit a warning to the console when invoked directly.
Here are some references to assist in writing the documentation:
* invocation examples in the commit message of https://gerrit.wikimedia.org/r/c/mediawiki/core/+/798983 * the warning being added https://gerrit.wikimedia.org/r/c/mediawiki/core/+/874922 * T99268 - RfC: Create a proper command-line runner for MediaWiki maintenance tasks https://phabricator.wikimedia.org/T99268
And as an extra, I have implemented a back compat layer in Quibble https://doc.wikimedia.org/quibble/ (which is written in Python): https://gerrit.wikimedia.org/r/c/integration/quibble/+/875981/6/quibble/medi...
Antoine "hashar" Musso Wikimedia Release Engineering
Thanks! This works great for Core maintenance scripts, but it's not working for extensions on my end. I'm probably doing something stupid:
maintenance/run extensions/MyExt/maintenance/someScript.php php maintenance/run.php extensions/MyExt/maintenance/someScript.php ...(and the other variants using the class name)
for all, I get the error:
Script 'extensions/MyExt/maintenance/someScript.php' not found (tried path '/var/www/html/w/maintenance/extensions/MyExt/maintenance/someScript.php' and class 'extensions/MyExt/maintenance/someScript\php').
I tried several extensions. I of course am pointing to actual scripts and not the example MyExt::SomeScript :) MW and the extensions I tried on my machine are up-to-date with master. Any ideas? This error is what led me to believe that perhaps the scripts weren't built to support the new maintenance runner, but from what you're saying it sounds like no changes should be needed.
As far as docs go, is it too soon to start adding MW >=1.40 info at Manual:Writing maintenance scripts https://www.mediawiki.org/wiki/Manual:Writing_maintenance_scripts and Manual:Maintenance scripts/Running the scripts https://www.mediawiki.org/wiki/Manual:Maintenance_scripts/Running_the_scripts, etc.?
~ MA
On Wed, Mar 1, 2023 at 3:40 AM Antoine Musso hashar@free.fr wrote:
Le 28/02/2023 à 21:42, MusikAnimal a écrit :
Hello! Where might I find documentation on the new maintenance runner system? I can't find any examples in the Phabricator task or the linked RFC, and searching for "MaintenanceRunner" or "run.php" yields no results on mediawiki.org. Specifically, I was expecting Manual:run.php https://www.mediawiki.org/wiki/Manual:Run.php to exist with some info on how it works.
If given a good example patch, I'm happy to help write the docs :)
Hello,
run.php is more or less a wrapper so that instead of invoking:
php maintenance/parse.php
One should use:
maintenance/run parse
That will be released in MediaWiki 1.40 and all maintenance scripts will emit a warning to the console when invoked directly.
Here are some references to assist in writing the documentation:
- invocation examples in the commit message of
https://gerrit.wikimedia.org/r/c/mediawiki/core/+/798983
- the warning being added
https://gerrit.wikimedia.org/r/c/mediawiki/core/+/874922
- T99268 - RfC: Create a proper command-line runner for MediaWiki
maintenance tasks https://phabricator.wikimedia.org/T99268
And as an extra, I have implemented a back compat layer in Quibble https://doc.wikimedia.org/quibble/ (which is written in Python): https://gerrit.wikimedia.org/r/c/integration/quibble/+/875981/6/quibble/medi...
Antoine "hashar" Musso Wikimedia Release Engineering
Ah... we need relative paths!
maintenance run ./extensions/MyExt/maintenance/someScript.php
That works great, but the class variants still don't work for me. Examples:
maintenance/run
MediaWiki.Extension.GlobalBlocking.FixGlobalBlockWhitelist Script 'MediaWiki.Extension.GlobalBlocking.FixGlobalBlockWhitelist' not found (tried path '/var/www/html/w/maintenance/MediaWiki.Extension.GlobalBlocking.FixGlobalBlockWhitelist.php' and class 'MediaWiki\Extension\GlobalBlocking\FixGlobalBlockWhitelist').
maintenance/run GlobalBlocking:FixGlobalBlockWhitelist
Script 'GlobalBlocking:FixGlobalBlockWhitelist' not found (tried path '/var/www/html/w/extensions/GlobalBlocking/maintenance/FixGlobalBlockWhitelist.php' and class 'MediaWiki\Extension\GlobalBlocking\Maintenance\FixGlobalBlockWhitelist').
~ MA
On Thu, Mar 2, 2023 at 7:26 PM MusikAnimal musikanimal@gmail.com wrote:
Thanks! This works great for Core maintenance scripts, but it's not working for extensions on my end. I'm probably doing something stupid:
maintenance/run extensions/MyExt/maintenance/someScript.php php maintenance/run.php extensions/MyExt/maintenance/someScript.php ...(and the other variants using the class name)
for all, I get the error:
Script 'extensions/MyExt/maintenance/someScript.php' not found (tried path '/var/www/html/w/maintenance/extensions/MyExt/maintenance/someScript.php' and class 'extensions/MyExt/maintenance/someScript\php').
I tried several extensions. I of course am pointing to actual scripts and not the example MyExt::SomeScript :) MW and the extensions I tried on my machine are up-to-date with master. Any ideas? This error is what led me to believe that perhaps the scripts weren't built to support the new maintenance runner, but from what you're saying it sounds like no changes should be needed.
As far as docs go, is it too soon to start adding MW >=1.40 info at Manual:Writing maintenance scripts https://www.mediawiki.org/wiki/Manual:Writing_maintenance_scripts and Manual:Maintenance scripts/Running the scripts https://www.mediawiki.org/wiki/Manual:Maintenance_scripts/Running_the_scripts, etc.?
~ MA
On Wed, Mar 1, 2023 at 3:40 AM Antoine Musso hashar@free.fr wrote:
Le 28/02/2023 à 21:42, MusikAnimal a écrit :
Hello! Where might I find documentation on the new maintenance runner system? I can't find any examples in the Phabricator task or the linked RFC, and searching for "MaintenanceRunner" or "run.php" yields no results on mediawiki.org. Specifically, I was expecting Manual:run.php https://www.mediawiki.org/wiki/Manual:Run.php to exist with some info on how it works.
If given a good example patch, I'm happy to help write the docs :)
Hello,
run.php is more or less a wrapper so that instead of invoking:
php maintenance/parse.php
One should use:
maintenance/run parse
That will be released in MediaWiki 1.40 and all maintenance scripts will emit a warning to the console when invoked directly.
Here are some references to assist in writing the documentation:
- invocation examples in the commit message of
https://gerrit.wikimedia.org/r/c/mediawiki/core/+/798983
- the warning being added
https://gerrit.wikimedia.org/r/c/mediawiki/core/+/874922
- T99268 - RfC: Create a proper command-line runner for MediaWiki
maintenance tasks https://phabricator.wikimedia.org/T99268
And as an extra, I have implemented a back compat layer in Quibble https://doc.wikimedia.org/quibble/ (which is written in Python): https://gerrit.wikimedia.org/r/c/integration/quibble/+/875981/6/quibble/medi...
Antoine "hashar" Musso Wikimedia Release Engineering
Hi MusikAnimal!
I looked into this issue. Running FixGlobalBlockWhitelist by class name doesn't work for two reasons:
1) FixGlobalBlockWhitelist isn't namespaced. fixGlobalBlockWhitelist.php doesn't contain a namespace declaration. If you try it with PopulateCentralId, it works.
2) The file name doesn't match the class name ("fix" vs "Fix"). So even if the namespace declaration is added, the autoloader won't find the file to load.
Pro tip: there is a shorthand notation for running maintenance scripts in extensions, if the conventions for path and namespace are followed:
* GlobalBlocking:fixGlobalBlockWhitelist will work (it will find the file in the maintenance directory)
* GlobalBlocking:FixGlobalBlockWhitelist would work once the namespace and file name is fixed
If you want an auto-loadable file and a nice script name, make a file with a nice name as a stub. E.g. maintance/fixWhitelist.php, which contains nothing but this:
require_once __DIR__ . '/FixGlobalBlockWhitelist .php';
The you can run the script using GlobalBlocking:fixWhitelist as well.
HTH, Daniel
Am 03.03.2023 um 01:49 schrieb MusikAnimal:
Ah... we need relative paths!
maintenance run ./extensions/MyExt/maintenance/someScript.php
That works great, but the class variants still don't work for me. Examples:
> maintenance/run MediaWiki.Extension.GlobalBlocking.FixGlobalBlockWhitelist Script 'MediaWiki.Extension.GlobalBlocking.FixGlobalBlockWhitelist' not found (tried path '/var/www/html/w/maintenance/MediaWiki.Extension.GlobalBlocking.FixGlobalBlockWhitelist.php' and class 'MediaWiki\Extension\GlobalBlocking\FixGlobalBlockWhitelist').
maintenance/run GlobalBlocking:FixGlobalBlockWhitelist
Script 'GlobalBlocking:FixGlobalBlockWhitelist' not found (tried path '/var/www/html/w/extensions/GlobalBlocking/maintenance/FixGlobalBlockWhitelist.php' and class 'MediaWiki\Extension\GlobalBlocking\Maintenance\FixGlobalBlockWhitelist').
~ MA
On Thu, Mar 2, 2023 at 7:26 PM MusikAnimal musikanimal@gmail.com wrote:
Thanks! This works great for Core maintenance scripts, but it's not working for extensions on my end. I'm probably doing something stupid:   maintenance/run extensions/MyExt/maintenance/someScript.php   php maintenance/run.php extensions/MyExt/maintenance/someScript.php   ...(and the other variants using the class name) for all, I get the error:  Script 'extensions/MyExt/maintenance/someScript.php' not found (tried path '/var/www/html/w/maintenance/extensions/MyExt/maintenance/someScript.php' and class 'extensions/MyExt/maintenance/someScript\php'). I tried several extensions. I of course am pointing to actual scripts and not the example MyExt::SomeScript :) MW and the extensions I tried on my machine are up-to-date with master. Any ideas? This error is what led me to believe that perhaps the scripts weren't built to support the new maintenance runner, but from what you're saying it sounds like no changes should be needed. As far as docs go, is it too soon to start adding MW >=1.40 info at Manual:Writing maintenance scripts <https://www.mediawiki.org/wiki/Manual:Writing_maintenance_scripts> and Manual:Maintenance scripts/Running the scripts <https://www.mediawiki.org/wiki/Manual:Maintenance_scripts/Running_the_scripts>, etc.? ~ MA On Wed, Mar 1, 2023 at 3:40 AM Antoine Musso <hashar@free.fr> wrote: Le 28/02/2023 à 21:42, MusikAnimal a écrit :Hello! Where might I find documentation on the new maintenance runner system? I can't find any examples in the Phabricator task or the linked RFC, and searching for "MaintenanceRunner" or "run.php" yields no results on mediawiki.org <http://mediawiki.org>. Specifically, I was expecting Manual:run.php <https://www.mediawiki.org/wiki/Manual:Run.php> to exist with some info on how it works. If given a good example patch, I'm happy to help write the docs :)Hello, run.php is more or less a wrapper so that instead of invoking:  php maintenance/parse.php One should use:  maintenance/run parse That will be released in MediaWiki 1.40 and all maintenance scripts will emit a warning to the console when invoked directly. Here are some references to assist in writing the documentation: * invocation examples in the commit message of https://gerrit.wikimedia.org/r/c/mediawiki/core/+/798983 * the warning being added https://gerrit.wikimedia.org/r/c/mediawiki/core/+/874922 * T99268 - RfC: Create a proper command-line runner for MediaWiki maintenance tasks <https://phabricator.wikimedia.org/T99268> And as an extra, I have implemented a back compat layer in Quibble <https://doc.wikimedia.org/quibble/> (which is written in Python): https://gerrit.wikimedia.org/r/c/integration/quibble/+/875981/6/quibble/mediawiki/maintenance.py Antoine "hashar" Musso Wikimedia Release Engineering
Wikitech-l mailing list --wikitech-l@lists.wikimedia.org To unsubscribe send an email towikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
wikitech-l@lists.wikimedia.org