*tl;dr*: use `scap backport <gerrit url>` to deploy MediaWiki backports ____
There’s now a *single step* to deploy changes to Wikimedia’s production MediaWiki.
🤯 An 85% reduction in command remembering!
On the deployment host run:
scap backport <gerrit url>
This works for any change to a live branch for mediawiki/core, extensions, skins, or operations/mediawiki-config. ____
More details (and a demo) inside Jeena Huneidi’s excellent write up https://phabricator.wikimedia.org/phame/post/view/297/scap_backport_makes_deployments_easy/ .
<3
– Tyler Cipriani (on behalf of the RelEngers https://releng.team who really do make dreams come true) Engineering Manager, Release Engineering Wikimedia Foundation
Hi,
Thanks for all the work on this project! I have some thoughts.
I often +2 all backports at the start of the window (or even prior the window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Also, I'd like to mention that many ordinary deployments currently require a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Martin
Ăşt 27. 9. 2022 v 23:05 odesĂlatel Tyler Cipriani tcipriani@wikimedia.org napsal:
*tl;dr*: use `scap backport <gerrit url>` to deploy MediaWiki backports ____
There’s now a *single step* to deploy changes to Wikimedia’s production MediaWiki.
🤯 An 85% reduction in command remembering!
On the deployment host run:
scap backport <gerrit url>This works for any change to a live branch for mediawiki/core, extensions, skins, or operations/mediawiki-config. ____
More details (and a demo) inside Jeena Huneidi’s excellent write up https://phabricator.wikimedia.org/phame/post/view/297/scap_backport_makes_deployments_easy/ .
<3
– Tyler Cipriani (on behalf of the RelEngers https://releng.team who really do make dreams come true) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Hi Martin, thanks for the feedback!
I often +2 all backports at the start of the window (or even prior the
window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Yes, you can do a +2 ahead of time and still use scap backport to do the remainder of the process, just be aware that if you do this and a patch ends up getting merged before the patch you are in the process of backporting with scap backport gets merged, then scap backport would end up syncing both patches at the same time (you will be given a warning and asked if you want to proceed).
Also, I'd like to mention that many ordinary deployments currently require
a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Good question...Do you mean the confirmation after syncing to debug servers? I assume you'd need to run the scripts right after pulling the changes to the deploy server. If so we would need to add an additional step.
Jeena
On Tue, Sep 27, 2022 at 2:23 PM Martin Urbanec martin.urbanec@wikimedia.cz wrote:
Hi,
Thanks for all the work on this project! I have some thoughts.
I often +2 all backports at the start of the window (or even prior the window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Also, I'd like to mention that many ordinary deployments currently require a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Martin
Ăşt 27. 9. 2022 v 23:05 odesĂlatel Tyler Cipriani tcipriani@wikimedia.org napsal:
*tl;dr*: use `scap backport <gerrit url>` to deploy MediaWiki backports ____
There’s now a *single step* to deploy changes to Wikimedia’s production MediaWiki.
🤯 An 85% reduction in command remembering!
On the deployment host run:
scap backport <gerrit url>This works for any change to a live branch for mediawiki/core, extensions, skins, or operations/mediawiki-config. ____
More details (and a demo) inside Jeena Huneidi’s excellent write up https://phabricator.wikimedia.org/phame/post/view/297/scap_backport_makes_deployments_easy/ .
<3
– Tyler Cipriani (on behalf of the RelEngers https://releng.team who really do make dreams come true) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
I also have a question. The majority of my deployments are config changes, not backports – can scap backport deploy those as well? I wouldn’t assume so based on the name, but Tyler’s email mentions operations/mediawiki-config…
Am Di., 27. Sept. 2022 um 23:51 Uhr schrieb Jeena Huneidi < jhuneidi@wikimedia.org>:
Hi Martin, thanks for the feedback!
I often +2 all backports at the start of the window (or even prior the
window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Yes, you can do a +2 ahead of time and still use scap backport to do the remainder of the process, just be aware that if you do this and a patch ends up getting merged before the patch you are in the process of backporting with scap backport gets merged, then scap backport would end up syncing both patches at the same time (you will be given a warning and asked if you want to proceed).
Also, I'd like to mention that many ordinary deployments currently require
a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Good question...Do you mean the confirmation after syncing to debug servers? I assume you'd need to run the scripts right after pulling the changes to the deploy server. If so we would need to add an additional step.
Jeena
On Tue, Sep 27, 2022 at 2:23 PM Martin Urbanec < martin.urbanec@wikimedia.cz> wrote:
Hi,
Thanks for all the work on this project! I have some thoughts.
I often +2 all backports at the start of the window (or even prior the window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Also, I'd like to mention that many ordinary deployments currently require a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Martin
Ăşt 27. 9. 2022 v 23:05 odesĂlatel Tyler Cipriani tcipriani@wikimedia.org napsal:
*tl;dr*: use `scap backport <gerrit url>` to deploy MediaWiki backports ____
There’s now a *single step* to deploy changes to Wikimedia’s production MediaWiki.
🤯 An 85% reduction in command remembering!
On the deployment host run:
scap backport <gerrit url>This works for any change to a live branch for mediawiki/core, extensions, skins, or operations/mediawiki-config. ____
More details (and a demo) inside Jeena Huneidi’s excellent write up https://phabricator.wikimedia.org/phame/post/view/297/scap_backport_makes_deployments_easy/ .
<3
– Tyler Cipriani (on behalf of the RelEngers https://releng.team who really do make dreams come true) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- Jeena Huneidi Software Engineer, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
The majority of my deployments are config changes, not backports – can scap backport deploy those as well?
Yes, I've primarily been using it for those :)
Also, just want to echo the thanks above — the script is awesome!
Kind regards,
*Sammy*
*(They/Them)*
User:TheresNoTime https://meta.wikimedia.org/wiki/User:TheresNoTime
*Wikimedia Steward*
*A/CU/OS*
*Unless otherwise stated, all statements from this account are made in a volunteer capacity, and may not reflect the views of the Wikimedia Foundation.*
*This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email.*
On Wed, 28 Sept 2022 at 09:49, Lucas Werkmeister < lucas.werkmeister@wikimedia.de> wrote:
I also have a question. The majority of my deployments are config changes, not backports – can scap backport deploy those as well? I wouldn’t assume so based on the name, but Tyler’s email mentions operations/mediawiki-config…
Am Di., 27. Sept. 2022 um 23:51 Uhr schrieb Jeena Huneidi < jhuneidi@wikimedia.org>:
Hi Martin, thanks for the feedback!
I often +2 all backports at the start of the window (or even prior the
window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Yes, you can do a +2 ahead of time and still use scap backport to do the remainder of the process, just be aware that if you do this and a patch ends up getting merged before the patch you are in the process of backporting with scap backport gets merged, then scap backport would end up syncing both patches at the same time (you will be given a warning and asked if you want to proceed).
Also, I'd like to mention that many ordinary deployments currently
require a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Good question...Do you mean the confirmation after syncing to debug servers? I assume you'd need to run the scripts right after pulling the changes to the deploy server. If so we would need to add an additional step.
Jeena
On Tue, Sep 27, 2022 at 2:23 PM Martin Urbanec < martin.urbanec@wikimedia.cz> wrote:
Hi,
Thanks for all the work on this project! I have some thoughts.
I often +2 all backports at the start of the window (or even prior the window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Also, I'd like to mention that many ordinary deployments currently require a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Martin
Ăşt 27. 9. 2022 v 23:05 odesĂlatel Tyler Cipriani < tcipriani@wikimedia.org> napsal:
*tl;dr*: use `scap backport <gerrit url>` to deploy MediaWiki backports ____
There’s now a *single step* to deploy changes to Wikimedia’s production MediaWiki.
🤯 An 85% reduction in command remembering!
On the deployment host run:
scap backport <gerrit url>This works for any change to a live branch for mediawiki/core, extensions, skins, or operations/mediawiki-config. ____
More details (and a demo) inside Jeena Huneidi’s excellent write up https://phabricator.wikimedia.org/phame/post/view/297/scap_backport_makes_deployments_easy/ .
<3
– Tyler Cipriani (on behalf of the RelEngers https://releng.team who really do make dreams come true) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- Jeena Huneidi Software Engineer, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- Lucas Werkmeister (he/er) Software Engineer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0)30-577 11 62-0 https://wikimedia.de
Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision! https://spenden.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
On Wed, Sep 28, 2022 at 7:37 AM Sammy Tarling sam@theresnotime.co.uk wrote:
The majority of my deployments are config changes, not backports – can scap backport deploy those as well?
Yes, I've primarily been using it for those :)
Yep! Think `scap backport(-window)`—it should be able to deploy everything we deploy in the backport window (yes, even the strange cases of cross-file dependencies).
This is the new canonical way to deploy all the things.
And if you've got weird edgecases—file a task[0]; we'd love to hear about them :)
Thanks! – Tyler
[0]: https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=releng-team,scap
Also, just want to echo the thanks above — the script is awesome!
Kind regards,
Sammy
(They/Them)
User:TheresNoTime
Wikimedia Steward
A/CU/OS
Unless otherwise stated, all statements from this account are made in a volunteer capacity, and may not reflect the views of the Wikimedia Foundation.
This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email.
On Wed, 28 Sept 2022 at 09:49, Lucas Werkmeister lucas.werkmeister@wikimedia.de wrote:
I also have a question. The majority of my deployments are config changes, not backports – can scap backport deploy those as well? I wouldn’t assume so based on the name, but Tyler’s email mentions operations/mediawiki-config…
Am Di., 27. Sept. 2022 um 23:51 Uhr schrieb Jeena Huneidi jhuneidi@wikimedia.org:
Hi Martin, thanks for the feedback!
I often +2 all backports at the start of the window (or even prior the window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Yes, you can do a +2 ahead of time and still use scap backport to do the remainder of the process, just be aware that if you do this and a patch ends up getting merged before the patch you are in the process of backporting with scap backport gets merged, then scap backport would end up syncing both patches at the same time (you will be given a warning and asked if you want to proceed).
Also, I'd like to mention that many ordinary deployments currently require a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Good question...Do you mean the confirmation after syncing to debug servers? I assume you'd need to run the scripts right after pulling the changes to the deploy server. If so we would need to add an additional step.
Jeena
On Tue, Sep 27, 2022 at 2:23 PM Martin Urbanec martin.urbanec@wikimedia.cz wrote:
Hi,
Thanks for all the work on this project! I have some thoughts.
I often +2 all backports at the start of the window (or even prior the window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Also, I'd like to mention that many ordinary deployments currently require a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Martin
Ăşt 27. 9. 2022 v 23:05 odesĂlatel Tyler Cipriani tcipriani@wikimedia.org napsal:
tl;dr: use `scap backport <gerrit url>` to deploy MediaWiki backports ____
There’s now a single step to deploy changes to Wikimedia’s production MediaWiki.
🤯 An 85% reduction in command remembering!
On the deployment host run:
scap backport <gerrit url>This works for any change to a live branch for mediawiki/core, extensions, skins, or operations/mediawiki-config. ____
More details (and a demo) inside Jeena Huneidi’s excellent write up.
<3
– Tyler Cipriani (on behalf of the RelEngers who really do make dreams come true) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- Jeena Huneidi Software Engineer, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- Lucas Werkmeister (he/er) Software Engineer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0)30-577 11 62-0 https://wikimedia.de
Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision! https://spenden.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Hi Jeena, Tyler and others,
Thanks for the reply!
I also have one other question: Will scap sync-file stay with us? There are also complex deployments handled outside of backport windows that I think might benefit from it, as it offers fine-grained control on what is happening.
Yes, you can do a +2 ahead of time and still use scap backport to do the
remainder of the process, just be aware that if you do this and a patch ends up getting merged before the patch you are in the process of backporting with scap backport gets merged, then scap backport would end up syncing both patches at the same time (you will be given a warning and asked if you want to proceed).
Good to know, thanks! I tend to +2 all backports at once, and decide on the deployment order during the window. This means patches getting merged in the "wrong" order will likely be an issue. Are backports in different repositories still an issue in that regard? For example, if I have three patches for three extensions, +2 them at once, and then use scap backport, will it sync everything, or will it only fetch submodules
Good question...Do you mean the confirmation after syncing to debug
servers? I assume you'd need to run the scripts right after pulling the changes to the deploy server. If so we would need to add an additional step.
I meant the "Would you like to see the diff" step, but there are many scripts, and the "when to run the script" differs. Some have to be executed as the very first step (createExtensionTables.php, for example). Other scripts need to be executed as the very last step (purgeList.php). There are also scripts that are not much sensitive to when they run, they just have to run at some point.
Martin
Ăşt 27. 9. 2022 v 23:51 odesĂlatel Jeena Huneidi jhuneidi@wikimedia.org napsal:
Hi Martin, thanks for the feedback!
I often +2 all backports at the start of the window (or even prior the
window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Yes, you can do a +2 ahead of time and still use scap backport to do the remainder of the process, just be aware that if you do this and a patch ends up getting merged before the patch you are in the process of backporting with scap backport gets merged, then scap backport would end up syncing both patches at the same time (you will be given a warning and asked if you want to proceed).
Also, I'd like to mention that many ordinary deployments currently require
a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Good question...Do you mean the confirmation after syncing to debug servers? I assume you'd need to run the scripts right after pulling the changes to the deploy server. If so we would need to add an additional step.
Jeena
On Tue, Sep 27, 2022 at 2:23 PM Martin Urbanec < martin.urbanec@wikimedia.cz> wrote:
Hi,
Thanks for all the work on this project! I have some thoughts.
I often +2 all backports at the start of the window (or even prior the window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Also, I'd like to mention that many ordinary deployments currently require a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Martin
Ăşt 27. 9. 2022 v 23:05 odesĂlatel Tyler Cipriani tcipriani@wikimedia.org napsal:
*tl;dr*: use `scap backport <gerrit url>` to deploy MediaWiki backports ____
There’s now a *single step* to deploy changes to Wikimedia’s production MediaWiki.
🤯 An 85% reduction in command remembering!
On the deployment host run:
scap backport <gerrit url>This works for any change to a live branch for mediawiki/core, extensions, skins, or operations/mediawiki-config. ____
More details (and a demo) inside Jeena Huneidi’s excellent write up https://phabricator.wikimedia.org/phame/post/view/297/scap_backport_makes_deployments_easy/ .
<3
– Tyler Cipriani (on behalf of the RelEngers https://releng.team who really do make dreams come true) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- Jeena Huneidi Software Engineer, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Hey Martin
On Wed, Sep 28, 2022 at 4:18 PM Martin Urbanec martin.urbanec@wikimedia.cz wrote:
I also have one other question: Will scap sync-file stay with us? There are also complex deployments handled outside of backport windows that I think might benefit from it, as it offers fine-grained control on what is happening.
scap sync-file still exists.
But as we move towards Kubernetes we'll need to start thinking on the patch-level rather than the file-per-patch level. That is, what's merged is what will be deployed.
Apart from the dependencies between files within a patch, do you have specific complex scenarios in mind? (Feel free to reply here or in a task or on IRC—whatever's easiest for you).
Yes, you can do a +2 ahead of time and still use scap backport to do the remainder of the process, just be aware that if you do this and a patch ends up getting merged before the patch you are in the process of backporting with scap backport gets merged, then scap backport would end up syncing both patches at the same time (you will be given a warning and asked if you want to proceed).
Good to know, thanks! I tend to +2 all backports at once, and decide on the deployment order during the window. This means patches getting merged in the "wrong" order will likely be an issue. Are backports in different repositories still an issue in that regard? For example, if I have three patches for three extensions, +2 them at once, and then use scap backport, will it sync everything, or will it only fetch submodules
If you merge two changes, and then run `scap backport <one-change>`?
In that case, `scap backport <one-change>` will warn you that other changes have merged while you were trying to sync out `<one-change>`, and ask if you want to proceed with the sync—both changes will go out at the same time.
Now that we no longer have PHP revalidate files in the opcache and instead perform a restart: these changes should be atomic (mostly).
scap backport will always sync everything—not just submodules (this is fast now, I swear :)).
Good question...Do you mean the confirmation after syncing to debug servers? I assume you'd need to run the scripts right after pulling the changes to the deploy server. If so we would need to add an additional step.
I meant the "Would you like to see the diff" step, but there are many scripts, and the "when to run the script" differs. Some have to be executed as the very first step (createExtensionTables.php, for example). Other scripts need to be executed as the very last step (purgeList.php). There are also scripts that are not much sensitive to when they run, they just have to run at some point.
For createExtensionTables.php, you could +2 fetch down and run that: same as now—scap backport can do the sync when you're ready to proceed. We'll need to think about how new extensions are deployed in our Kubernetes future.
For things where you would like to run a check on migration changes, could you use the pause on mwdebug servers to do so? For these, would it be helpful to have a step to go to mwmaint servers, too?
For others, you can run them at the end—same as now.
Would that cover all the scenarios we have currently?
Thanks, as always, for being an amazing human and deployer
<3
– Tyler
Martin
Ăşt 27. 9. 2022 v 23:51 odesĂlatel Jeena Huneidi jhuneidi@wikimedia.org napsal:
Hi Martin, thanks for the feedback!
I often +2 all backports at the start of the window (or even prior the window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Yes, you can do a +2 ahead of time and still use scap backport to do the remainder of the process, just be aware that if you do this and a patch ends up getting merged before the patch you are in the process of backporting with scap backport gets merged, then scap backport would end up syncing both patches at the same time (you will be given a warning and asked if you want to proceed).
Also, I'd like to mention that many ordinary deployments currently require a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Good question...Do you mean the confirmation after syncing to debug servers? I assume you'd need to run the scripts right after pulling the changes to the deploy server. If so we would need to add an additional step.
Jeena
On Tue, Sep 27, 2022 at 2:23 PM Martin Urbanec martin.urbanec@wikimedia.cz wrote:
Hi,
Thanks for all the work on this project! I have some thoughts.
I often +2 all backports at the start of the window (or even prior the window), because I want to save time. It enables me to deploy more patches than I would be able to deploy with scap backport +2'ing it each patch separately. I also parallelize some of the steps (such as, +2'ing a config patch during the sync step), which also saves a bit of time. How would I do this with scap backport? By +2'ing the patch manually, and then letting scap backport figure out its magic?
Also, I'd like to mention that many ordinary deployments currently require a script to run. How would that work in scap backport world? When it asks me for confirmation, I will wait, run the script, and then finish it, to ensure the script runs at the right time
Martin
Ăşt 27. 9. 2022 v 23:05 odesĂlatel Tyler Cipriani tcipriani@wikimedia.org napsal:
tl;dr: use `scap backport <gerrit url>` to deploy MediaWiki backports ____
There’s now a single step to deploy changes to Wikimedia’s production MediaWiki.
🤯 An 85% reduction in command remembering!
On the deployment host run:
scap backport <gerrit url>This works for any change to a live branch for mediawiki/core, extensions, skins, or operations/mediawiki-config. ____
More details (and a demo) inside Jeena Huneidi’s excellent write up.
<3
– Tyler Cipriani (on behalf of the RelEngers who really do make dreams come true) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- Jeena Huneidi Software Engineer, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
wikitech-l@lists.wikimedia.org