Sorry, forgot to CC this list.
---------- Forwarded message --------- From: James Forrester jforrester@wikimedia.org Date: Tue, 26 Nov 2019 at 12:50 Subject: Production config dblists are now auto-generated at merge time from YAML To: Wikimedia developers wikitech-l@lists.wikimedia.org
Hey all,
[Unless you're writing Wikimedia production MediaWiki-land configuration changes, you can ignore this note.]
As part of my work on moving variant configuration in static files[0], I've just merged and deployed a change to how dblists work. They are now auto-generated at merge time[1] (except for the very few lists which are live-computed in production, which remain). The lists are calculated from the per-wiki YAML files in wmf-config/config (and if you try to change the dblist files manually, CI will error at you).
Changes to dblist memberships are quite rare, and this will enforce that we don't make mistakes. (As part of this deployment, I removed three duplicate entries and noted several irregularities we probably didn't intend.)
This means that if you're writing a change that previously would have meant manually editing the dblist files, you now have to touch the YAML file instead (and run composer build locally, or use CI to tell you what to change).
This mostly affects the creation of new wikis (which should now be slightly simpler), plus a few major features currently enabled through dblists. For an example, if you were to remove FlaggedRevisions from Chechen Wikipedia, you'd now need to edit cewiki.yaml[2] to remove the " - flaggedrevs" line, as well as changes to InitialiseSettings.php.
This work unblocks some of the future re-architecting of config, which I've mentioned before, as part of the long-term plan to move Wikimedia production into containers. Configuration traits related to FR will inherit from flaggedrevs.yaml, and static-like code in CommonSettings.php (and in this case, flaggedrevs.php) can be significantly shrunk.
If anything has broken or has become, if you have any questions, or want to talk about things, I'd be happy to discuss, here, on IRC, or on Phabricator.
Thanks!
[0] - https://phabricator.wikimedia.org/T223602 [1] - https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/545411 [2] - https://gerrit.wikimedia.org/r/plugins/gitiles/operations/mediawiki-config/+...
Yours,
Hello,
to ensure my understanding is correct, is the correct way to do dblists changes nowadays to first change yaml and then run composer build? From first reading the email, I thought it's automated by CI, but it's clearly not <https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/553449
.
Best, Martin
út 26. 11. 2019 v 21:51 odesílatel James Forrester jforrester@wikimedia.org napsal:
Sorry, forgot to CC this list.
---------- Forwarded message --------- From: James Forrester jforrester@wikimedia.org Date: Tue, 26 Nov 2019 at 12:50 Subject: Production config dblists are now auto-generated at merge time from YAML To: Wikimedia developers wikitech-l@lists.wikimedia.org
Hey all,
[Unless you're writing Wikimedia production MediaWiki-land configuration changes, you can ignore this note.]
As part of my work on moving variant configuration in static files[0], I've just merged and deployed a change to how dblists work. They are now auto-generated at merge time[1] (except for the very few lists which are live-computed in production, which remain). The lists are calculated from the per-wiki YAML files in wmf-config/config (and if you try to change the dblist files manually, CI will error at you).
Changes to dblist memberships are quite rare, and this will enforce that we don't make mistakes. (As part of this deployment, I removed three duplicate entries and noted several irregularities we probably didn't intend.)
This means that if you're writing a change that previously would have meant manually editing the dblist files, you now have to touch the YAML file instead (and run composer build locally, or use CI to tell you what to change).
This mostly affects the creation of new wikis (which should now be slightly simpler), plus a few major features currently enabled through dblists. For an example, if you were to remove FlaggedRevisions from Chechen Wikipedia, you'd now need to edit cewiki.yaml[2] to remove the " - flaggedrevs" line, as well as changes to InitialiseSettings.php.
This work unblocks some of the future re-architecting of config, which I've mentioned before, as part of the long-term plan to move Wikimedia production into containers. Configuration traits related to FR will inherit from flaggedrevs.yaml, and static-like code in CommonSettings.php (and in this case, flaggedrevs.php) can be significantly shrunk.
If anything has broken or has become, if you have any questions, or want to talk about things, I'd be happy to discuss, here, on IRC, or on Phabricator.
Thanks!
[0] - https://phabricator.wikimedia.org/T223602 [1] - https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/545411 [2] -
https://gerrit.wikimedia.org/r/plugins/gitiles/operations/mediawiki-config/+...
Yours,
*James D. Forrester* (he/him http://pronoun.is/he or they/themself http://pronoun.is/they/.../themself) Wikimedia Foundation https://wikimediafoundation.org/
-- *James D. Forrester* (he/him http://pronoun.is/he or they/themself http://pronoun.is/they/.../themself) Wikimedia Foundation https://wikimediafoundation.org/
-- To unsubscribe from this group and stop receiving emails from it, send an email to sitereq-l+unsubscribe@wikimedia.org.
On Wed, 27 Nov 2019 at 23:44, Martin Urbanec martin.urbanec@wikimedia.cz wrote:
Hello,
to ensure my understanding is correct, is the correct way to do dblists changes nowadays to first change yaml and then run composer build?
Yes.
J.
Hello,
Sorry to resurrect this old thread but I came across some issues while attempting to upload the initial configuration for a new chapter wiki[1].
The new composer commands do look for an entry in wikiversions.json no matter what, apparently; and refuse to work if they find none. I am not sure if we want that behaviour, considering that it takes weeks after the initial configuration gets uploaded for a wiki to be created, and wikiversions.json changes thrice per week due to the mediawiki trains.
I used to upload configuration patches without that entry on purpose so it was easier to rebase once the configuration patch was about to be merged. I did the trick locally adding a wikiversion.json entry, running the composer commands but not committing the wikiversions file later; however that also causes Jenkins to fail as it detects wikis in all.dblists in all but not in wikiversions.json.
`composer test` also failed due to some weird errors[2] which I'm not sure are relevant for local testing: TimelineTest::testTimelineFontFileEexists.
Also, on the Wikitech guide[3] there's no mention of this new procedure. We should be updating it I guess.
Best regards, M.
[1]: https://gerrit.wikimedia.org/r/574184/ [2]: https://phabricator.wikimedia.org/P10477 [3]: https://wikitech.wikimedia.org/wiki/Add_a_wiki#MediaWiki_configuration
El lun., 2 dic. 2019 a las 17:19, James Forrester (jforrester@wikimedia.org) escribió:
On Wed, 27 Nov 2019 at 23:44, Martin Urbanec martin.urbanec@wikimedia.cz wrote:
Hello,
to ensure my understanding is correct, is the correct way to do dblists changes nowadays to first change yaml and then run composer build?
Yes.
J.
James D. Forrester (he/him or they/themself) Wikimedia Foundation
-- To unsubscribe from this group and stop receiving emails from it, send an email to sitereq-l+unsubscribe@wikimedia.org.
On Sat, 22 Feb 2020 at 10:15, Marco A. strigiwm@gmail.com wrote:
Hello,
Sorry to resurrect this old thread but I came across some issues while attempting to upload the initial configuration for a new chapter wiki[1].
The new composer commands do look for an entry in wikiversions.json no matter what, apparently; and refuse to work if they find none. I am not sure if we want that behaviour, considering that it takes weeks after the initial configuration gets uploaded for a wiki to be created, and wikiversions.json changes thrice per week due to the mediawiki trains.
Why does it take weeks?
I used to upload configuration patches without that entry on purpose so it was easier to rebase once the configuration patch was about to be merged. I did the trick locally adding a wikiversion.json entry, running the composer commands but not committing the wikiversions file later; however that also causes Jenkins to fail as it detects wikis in all.dblists in all but not in wikiversions.json.
Just add a wikiversions.json line. Rebasing this is pretty trivial whenever someone is actually making the wiki.
In the future, assuming we do go ahead and dispose of InitialiseSettings.php entirely to only have the YAML files, it won't be much hardship to have to rebase for wikiversions.json only.
`composer test` also failed due to some weird errors[2] which I'm not sure are relevant for local testing: TimelineTest::testTimelineFontFileEexists.
That's because you've not `git submodule update --init`'ed.
Also, on the Wikitech guide[3] there's no mention of this new procedure. We should be updating it I guess.
Updated today. Hope that helps, but I probably missed something, so feel free to shout.
J.