Hello folks,
*What:*
We'd like to schedule a maintenance window for all the stat machines:
stat1004.eqiad.wmnet stat1005.eqiad.wmnet stat1006.eqiad.wmnet stat1007.eqiad.wmnet stat1008.eqiad.wmnet
*In this maintenance window, we will:* * Upgrade the conda-analytics debian package. This package is how we deploy Spark3. The upgrade will allow us to run JupyterHub and JupyterLab on top of it. * Deploy new puppet configurations that will switch from running Jupyter on top of anaconda-wmf to running on top of conda-analytics. * This effectively upgrades our Jupyter deployment as follows: * Upgrades JupyterHub from 1.1.0 to 1.5.0. * Upgrades JupyterLab from 3.2.9 to 3.4.8. * Upgrades Spark on newly created conda environments from 2.4.4 to 3.1.2. * Upgrades wmfdata on newly created conda environments to 2.0.0. This version of wmfdata includes breaking changes.
*IMPORTANT*: After this maintenance window, you should expect the following: *** JupyterHub and any existing JupyterLab processes including any running kernels, will be shut down, and will have to be restarted manually by the respective owners. *** *** JupyterLab UI will have minor changes. *** *** New conda environments created via JupyterHub will now be based off of conda-analytics and will utilize Spark3. *** *** Existing conda environments based off of anaconda-wmf will continue to work, and continue to utilize Spark2. ***
*Why: *
As part of our effort to deprecate Spark2 and make Spark3 widely available, we are deprecating the use of anaconda-wmf and Spark2, in favor of conda-analytics and Spark3. Note this is just deprecation, you will still be able to use your existing conda environments running on top of anaconda-wmf and Spark2.
*When:*
We are proposing the following window for these changes:
Wednesday 30 Nov 2022 12:30 to 13:30 UTC. (7:30 AM ET / 4:30 AM PT)
*More info:*
Spark3 upgrade: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Spark/Migratio... The old anaconda-wmf base conda environment: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Conda The new conda-analytics base conda environment: https://wikitech.wikimedia.org/wiki/Analytics/Systems/conda-analytics wmfdata changes: https://github.com/wikimedia/wmfdata-python/blob/main/CHANGELOG.md
Please let us know if you have any questions or objections.
This is happening in 15 minutes.
On Mon, Nov 28, 2022 at 3:25 PM Xabriel Collazo Mojica < xcollazo@wikimedia.org> wrote:
Hello folks,
*What:*
We'd like to schedule a maintenance window for all the stat machines:
stat1004.eqiad.wmnet stat1005.eqiad.wmnet stat1006.eqiad.wmnet stat1007.eqiad.wmnet stat1008.eqiad.wmnet
*In this maintenance window, we will:*
- Upgrade the conda-analytics debian package. This package is how we
deploy Spark3. The upgrade will allow us to run JupyterHub and JupyterLab on top of it.
- Deploy new puppet configurations that will switch from running Jupyter
on top of anaconda-wmf to running on top of conda-analytics.
- This effectively upgrades our Jupyter deployment as follows:
- Upgrades JupyterHub from 1.1.0 to 1.5.0.
- Upgrades JupyterLab from 3.2.9 to 3.4.8.
- Upgrades Spark on newly created conda environments from 2.4.4 to
3.1.2. * Upgrades wmfdata on newly created conda environments to 2.0.0. This version of wmfdata includes breaking changes.
*IMPORTANT*: After this maintenance window, you should expect the following: *** JupyterHub and any existing JupyterLab processes including any running kernels, will be shut down, and will have to be restarted manually by the respective owners. *** *** JupyterLab UI will have minor changes. *** *** New conda environments created via JupyterHub will now be based off of conda-analytics and will utilize Spark3. *** *** Existing conda environments based off of anaconda-wmf will continue to work, and continue to utilize Spark2. ***
*Why: *
As part of our effort to deprecate Spark2 and make Spark3 widely available, we are deprecating the use of anaconda-wmf and Spark2, in favor of conda-analytics and Spark3. Note this is just deprecation, you will still be able to use your existing conda environments running on top of anaconda-wmf and Spark2.
*When:*
We are proposing the following window for these changes:
Wednesday 30 Nov 2022 12:30 to 13:30 UTC. (7:30 AM ET / 4:30 AM PT)
*More info:*
Spark3 upgrade: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Spark/Migratio... The old anaconda-wmf base conda environment: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Conda The new conda-analytics base conda environment: https://wikitech.wikimedia.org/wiki/Analytics/Systems/conda-analytics wmfdata changes: https://github.com/wikimedia/wmfdata-python/blob/main/CHANGELOG.md
Please let us know if you have any questions or objections.
-- Xabriel J. Collazo Mojica (he/him, pronunciation https://commons.wikimedia.org/wiki/File:Xabriel_Collazo_Mojica_-_pronunciation.ogg ) Sr Software Engineer Wikimedia Foundation
Successful deployment, details at https://phabricator.wikimedia.org/T321088.
On Wed, Nov 30, 2022 at 7:19 AM Xabriel Collazo Mojica < xcollazo@wikimedia.org> wrote:
This is happening in 15 minutes.
On Mon, Nov 28, 2022 at 3:25 PM Xabriel Collazo Mojica < xcollazo@wikimedia.org> wrote:
Hello folks,
*What:*
We'd like to schedule a maintenance window for all the stat machines:
stat1004.eqiad.wmnet stat1005.eqiad.wmnet stat1006.eqiad.wmnet stat1007.eqiad.wmnet stat1008.eqiad.wmnet
*In this maintenance window, we will:*
- Upgrade the conda-analytics debian package. This package is how we
deploy Spark3. The upgrade will allow us to run JupyterHub and JupyterLab on top of it.
- Deploy new puppet configurations that will switch from running Jupyter
on top of anaconda-wmf to running on top of conda-analytics.
- This effectively upgrades our Jupyter deployment as follows:
- Upgrades JupyterHub from 1.1.0 to 1.5.0.
- Upgrades JupyterLab from 3.2.9 to 3.4.8.
- Upgrades Spark on newly created conda environments from 2.4.4 to
3.1.2. * Upgrades wmfdata on newly created conda environments to 2.0.0. This version of wmfdata includes breaking changes.
*IMPORTANT*: After this maintenance window, you should expect the following: *** JupyterHub and any existing JupyterLab processes including any running kernels, will be shut down, and will have to be restarted manually by the respective owners. *** *** JupyterLab UI will have minor changes. *** *** New conda environments created via JupyterHub will now be based off of conda-analytics and will utilize Spark3. *** *** Existing conda environments based off of anaconda-wmf will continue to work, and continue to utilize Spark2. ***
*Why: *
As part of our effort to deprecate Spark2 and make Spark3 widely available, we are deprecating the use of anaconda-wmf and Spark2, in favor of conda-analytics and Spark3. Note this is just deprecation, you will still be able to use your existing conda environments running on top of anaconda-wmf and Spark2.
*When:*
We are proposing the following window for these changes:
Wednesday 30 Nov 2022 12:30 to 13:30 UTC. (7:30 AM ET / 4:30 AM PT)
*More info:*
Spark3 upgrade: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Spark/Migratio... The old anaconda-wmf base conda environment: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Conda The new conda-analytics base conda environment: https://wikitech.wikimedia.org/wiki/Analytics/Systems/conda-analytics wmfdata changes: https://github.com/wikimedia/wmfdata-python/blob/main/CHANGELOG.md
Please let us know if you have any questions or objections.
-- Xabriel J. Collazo Mojica (he/him, pronunciation https://commons.wikimedia.org/wiki/File:Xabriel_Collazo_Mojica_-_pronunciation.ogg ) Sr Software Engineer Wikimedia Foundation
-- -xabriel
analytics-announce@lists.wikimedia.org