Last week, I spoke to a few of my Wikimedia Foundation colleagues about how we deploy code—I completely botched it.
At the end of the conversation, I was pretty sure I'd only succeeded in making a complex process more opaque. I decided to write a blog to redeem myself: How We Deploy Code https://phabricator.wikimedia.org/phame/post/view/253/how_we_deploy_code/
My goal was to write a very high-level overview of the process we use to deploy code to Wikimedia production.
Hopefully, this is helpful.
<3 – Tyler Cipriani (he/him) Engineering Manager, Release Engineering Wikimedia Foundation
This is great, thanks Tyler! \o/
On Mon, Sep 27, 2021 at 3:25 PM Tyler Cipriani tcipriani@wikimedia.org wrote:
Last week, I spoke to a few of my Wikimedia Foundation colleagues about how we deploy code—I completely botched it.
At the end of the conversation, I was pretty sure I'd only succeeded in making a complex process more opaque. I decided to write a blog to redeem myself: How We Deploy Code https://phabricator.wikimedia.org/phame/post/view/253/how_we_deploy_code/
My goal was to write a very high-level overview of the process we use to deploy code to Wikimedia production.
Hopefully, this is helpful.
<3 – Tyler Cipriani (he/him) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
So succinct! Great read.
It reads like the Inverted pyramid[1]:
- The gist - Important details - The hairy details that you couldn't understand without the gist and important details
1. https://en.wikipedia.org/wiki/Inverted_pyramid_(journalism)
On Mon, Sep 27, 2021 at 4:04 PM Greg Grossmeier greg@wikimedia.org wrote:
This is great, thanks Tyler! \o/
On Mon, Sep 27, 2021 at 3:25 PM Tyler Cipriani tcipriani@wikimedia.org wrote:
Last week, I spoke to a few of my Wikimedia Foundation colleagues about how we deploy code—I completely botched it.
At the end of the conversation, I was pretty sure I'd only succeeded in making a complex process more opaque. I decided to write a blog to redeem myself: How We Deploy Code https://phabricator.wikimedia.org/phame/post/view/253/how_we_deploy_code/
My goal was to write a very high-level overview of the process we use to deploy code to Wikimedia production.
Hopefully, this is helpful.
<3 – Tyler Cipriani (he/him) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Dir. Engineering Productivity A18D 1138 8E47 FAC8 1C7D | _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Great, really nice! I like. <3
I wonder if it makes sense to sneak in the aspect of canary servers. Whenever I talk to other devs about our deployment processes they are most amazed by that concept.
Am Di., 28. Sept. 2021 um 01:13 Uhr schrieb Aaron Halfaker < aaron.halfaker@gmail.com>:
So succinct! Great read.
It reads like the Inverted pyramid[1]:
- The gist
- Important details
- The hairy details that you couldn't understand without the gist and
important details
On Mon, Sep 27, 2021 at 4:04 PM Greg Grossmeier greg@wikimedia.org wrote:
This is great, thanks Tyler! \o/
On Mon, Sep 27, 2021 at 3:25 PM Tyler Cipriani tcipriani@wikimedia.org wrote:
Last week, I spoke to a few of my Wikimedia Foundation colleagues about how we deploy code—I completely botched it.
At the end of the conversation, I was pretty sure I'd only succeeded in making a complex process more opaque. I decided to write a blog to redeem myself: How We Deploy Code https://phabricator.wikimedia.org/phame/post/view/253/how_we_deploy_code/
My goal was to write a very high-level overview of the process we use to deploy code to Wikimedia production.
Hopefully, this is helpful.
<3 – Tyler Cipriani (he/him) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Dir. Engineering Productivity A18D 1138 8E47 FAC8 1C7D | _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
On Thu, Sep 30, 2021 at 9:38 AM Christoph Jauera < christoph.jauera@wikimedia.de> wrote:
I wonder if it makes sense to sneak in the aspect of canary servers. Whenever I talk to other devs about our deployment processes they are most amazed by that concept.
That's a great thought! During deployment training[0] I always forget to mention how many safeguards there are on deployments. It's (thankfully) much harder to break production than it used to be.
For future iterations of this series, I think :)
[0]: https://wikitech.wikimedia.org/wiki/Deployments/Training
Am Di., 28. Sept. 2021 um 01:13 Uhr schrieb Aaron Halfaker <
aaron.halfaker@gmail.com>:
So succinct! Great read.
It reads like the Inverted pyramid[1]:
- The gist
- Important details
- The hairy details that you couldn't understand without the gist and
important details
On Mon, Sep 27, 2021 at 4:04 PM Greg Grossmeier greg@wikimedia.org wrote:
This is great, thanks Tyler! \o/
On Mon, Sep 27, 2021 at 3:25 PM Tyler Cipriani tcipriani@wikimedia.org wrote:
Last week, I spoke to a few of my Wikimedia Foundation colleagues about how we deploy code—I completely botched it.
At the end of the conversation, I was pretty sure I'd only succeeded in making a complex process more opaque. I decided to write a blog to redeem myself: How We Deploy Code https://phabricator.wikimedia.org/phame/post/view/253/how_we_deploy_code/
My goal was to write a very high-level overview of the process we use to deploy code to Wikimedia production.
Hopefully, this is helpful.
<3 – Tyler Cipriani (he/him) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Dir. Engineering Productivity A18D 1138 8E47 FAC8 1C7D | _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- Christoph Jauera Software Developer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0)30 219 158 26-0 https://wikimedia.de
Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision! https://spenden.wikimedia.de
Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Great read! Thanks!
As someone on a team that mostly deploys non-Mediawiki code, I appreciate the mention that this code also exists and does not follow the same process!
Big thanks!
MrG
On Thu, 30 Sept 2021 at 23:06, Tyler Cipriani tcipriani@wikimedia.org wrote:
On Thu, Sep 30, 2021 at 9:38 AM Christoph Jauera < christoph.jauera@wikimedia.de> wrote:
I wonder if it makes sense to sneak in the aspect of canary servers. Whenever I talk to other devs about our deployment processes they are most amazed by that concept.
That's a great thought! During deployment training[0] I always forget to mention how many safeguards there are on deployments. It's (thankfully) much harder to break production than it used to be.
For future iterations of this series, I think :)
Am Di., 28. Sept. 2021 um 01:13 Uhr schrieb Aaron Halfaker <
aaron.halfaker@gmail.com>:
So succinct! Great read.
It reads like the Inverted pyramid[1]:
- The gist
- Important details
- The hairy details that you couldn't understand without the gist
and important details
On Mon, Sep 27, 2021 at 4:04 PM Greg Grossmeier greg@wikimedia.org wrote:
This is great, thanks Tyler! \o/
On Mon, Sep 27, 2021 at 3:25 PM Tyler Cipriani tcipriani@wikimedia.org wrote:
Last week, I spoke to a few of my Wikimedia Foundation colleagues about how we deploy code—I completely botched it.
At the end of the conversation, I was pretty sure I'd only succeeded in making a complex process more opaque. I decided to write a blog to redeem myself: How We Deploy Code https://phabricator.wikimedia.org/phame/post/view/253/how_we_deploy_code/
My goal was to write a very high-level overview of the process we use to deploy code to Wikimedia production.
Hopefully, this is helpful.
<3 – Tyler Cipriani (he/him) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Dir. Engineering Productivity A18D 1138 8E47 FAC8 1C7D | _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
-- Christoph Jauera Software Developer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0)30 219 158 26-0 https://wikimedia.de
Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision! https://spenden.wikimedia.de
Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Tyler, thanks for the process document, very helpful:
Here is another type of release challenge: Database table alteration is a whole additional minefield I am dealing with. Preservation of existing data and migration of existing data into the added fields, dealing with 3 database engines, ensuring existing code can function pre and post alteration if needed and ensuring that a feature needing the table alteration is released in phases. Tack on doing this as an extension not core code. Interesting dependencies are how 3rd party deployments will be affected, how replicated databases are affected, how to test the non manually updated table alteration scripts and the one time executing data migration code and how the UI will work during the various release phases. Security risks need assessment, performance of the new code needs assessment, adding appropriate indexes to reduce database load of new queries needs assessment. I think I am close to having figured out the path and consulted with Sam and Manuel, and others have chimed in such as Alex and Subbu. Best, Shannon
On Mon, Sep 27, 2021 at 3:25 PM Tyler Cipriani tcipriani@wikimedia.org wrote:
Last week, I spoke to a few of my Wikimedia Foundation colleagues about how we deploy code—I completely botched it.
At the end of the conversation, I was pretty sure I'd only succeeded in making a complex process more opaque. I decided to write a blog to redeem myself: How We Deploy Code https://phabricator.wikimedia.org/phame/post/view/253/how_we_deploy_code/
My goal was to write a very high-level overview of the process we use to deploy code to Wikimedia production.
Hopefully, this is helpful.
<3 – Tyler Cipriani (he/him) Engineering Manager, Release Engineering Wikimedia Foundation _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
wikitech-l@lists.wikimedia.org