Hello!
Next Wednesday 2019-10-23 at 09:00 UTC we will be doing another maintenance operation on some of our cloudvirts servers (the hypervisor servers) that involves rebooting both the physical servers and the virtual machines running on them. The reasons is that we ned to update the running linux kernel version they have.
In this window we will reboot 3 hypervisors: * cloudvirt1014 * cloudvirt1025 * cloudvirt1026
The procedure will be to reboot a server, wait for it to come back online (could take up to 5 minutes) and wait for all the VMs to come back online. Then move to the next server.
Toolforge users may see their tools and webservices briefly disrupted due to several components of the Toolforge infrastructure being rebooted in this operation.
If nothing changes (reallocated or new virtual machine, etc) this is the list of affected VM instances in each hypervisor:
* cloudvirt1014:
VM: globalcu-app-01 PROJECT: globalcu VM: xtools-dev05 PROJECT: xtools VM: ores-web-05 PROJECT: ores VM: roebling PROJECT: wikispore VM: ores-web-04 PROJECT: ores VM: canary1014-01 PROJECT: testlabs VM: cloudinfra-db02 PROJECT: cloudinfra VM: Krypton PROJECT: codereview VM: dashiki-staging-01 PROJECT: dashiki VM: discovery-production-02 PROJECT: shiny-r VM: logparse01 PROJECT: security-tools VM: tools-sgegrid-shadow PROJECT: tools VM: maps-tiles1 PROJECT: maps VM: wikimetrics-01 PROJECT: wikimetrics VM: tools-sgecron-01 PROJECT: tools VM: wikitextexp-base-1002 PROJECT: wikitextexp VM: kask-client PROJECT: services VM: clm-web-01 PROJECT: community-labs-monitoring VM: accounts-appserver4 PROJECT: account-creation-assistance VM: taxonbota PROJECT: dwl VM: tofawiki02 PROJECT: fa-wp VM: federated-commons PROJECT: wikidata-federation VM: puppetmaster PROJECT: thumbor VM: lizenzhinweisgenerator PROJECT: lizenzhinweisgenerator VM: hound-puppet-02 PROJECT: hound VM: packagist-mirror1 PROJECT: packagist-mirror VM: deployment-elastic06 PROJECT: deployment-prep VM: deployment-changeprop PROJECT: deployment-prep VM: deployment-restbase02 PROJECT: deployment-prep VM: deployment-imagescaler01 PROJECT: deployment-prep VM: deployment-zookeeper02 PROJECT: deployment-prep VM: deployment-kafka-jumbo-1 PROJECT: deployment-prep VM: deployment-memc07 PROJECT: deployment-prep VM: deployment-eventlog05 PROJECT: deployment-prep VM: deployment-cpjobqueue PROJECT: deployment-prep VM: deployment-mediawiki-07 PROJECT: deployment-prep VM: deployment-chromium01 PROJECT: deployment-prep VM: deployment-puppetdb02 PROJECT: deployment-prep VM: deployment-puppetmaster03 PROJECT: deployment-prep VM: deployment-cache-text05 PROJECT: deployment-prep VM: gfg01 PROJECT: video VM: encoding01 PROJECT: video VM: whgi PROJECT: wikidumpparse VM: wikilabels PROJECT: wmf-research-tools VM: gitservices PROJECT: getstarted VM: design-research-methods PROJECT: design VM: wikilabels-backups PROJECT: wikilabels VM: wikilabels-02 PROJECT: wikilabels VM: dumpgrepper PROJECT: visualeditor VM: compiler1001 PROJECT: puppet-diffs VM: af-puppetdb02 PROJECT: automation-framework VM: clickmodel PROJECT: search VM: tool PROJECT: recommendation-api VM: missing-sections PROJECT: recommendation-api VM: ores-lb-03 PROJECT: ores VM: puppet-ema-2 PROJECT: puppet VM: matrix-synapse-01 PROJECT: matrix VM: wikibrain-embeddings-01 PROJECT: wikibrain VM: qube-node2 PROJECT: k8splay VM: captcha-tf-43 PROJECT: privpol-captcha VM: k4-2 PROJECT: analytics
* cloudvirt1025:
VM: integration-agent-docker-1006 PROJECT: integration VM: striker-deploy04 PROJECT: striker VM: rec-wiki-2 PROJECT: recommendation-api VM: deployment-ms-fe03 PROJECT: deployment-prep VM: deployment-poolcounter05 PROJECT: deployment-prep VM: deployment-ms-be05 PROJECT: deployment-prep VM: readers-web-stephen PROJECT: reading-web-staging VM: traffic-upload-stretch PROJECT: traffic VM: traffic-recdns-anycast PROJECT: traffic VM: deployment-maps05 PROJECT: deployment-prep VM: gerrit-sizzle PROJECT: security-tools VM: tools-sgewebgrid-generic-0901 PROJECT: tools VM: shinken-puppetmaster-01 PROJECT: shinken VM: osmit-due PROJECT: osmit VM: deployment-acme-chief03 PROJECT: deployment-prep VM: meza-cindy PROJECT: pluggableauth VM: accounts-db4 PROJECT: account-creation-assistance VM: krenair-clientpackages-py3-jessie PROJECT: testlabs VM: deployment-sessionstore01 PROJECT: deployment-prep VM: paws-worker-04 PROJECT: paws VM: paws-ext-lb-02 PROJECT: paws VM: paws-int-lb-01 PROJECT: paws VM: paws-master-03 PROJECT: paws VM: paws-master-01 PROJECT: paws VM: language-readership PROJECT: language VM: wmde-wikidiff2-patched-stretch PROJECT: wikidiff2-wmde-dev VM: tools-sgebastion-08 PROJECT: tools VM: compiler1002 PROJECT: puppet-diffs VM: phragile-db PROJECT: phragile VM: cloud-puppetmaster-01 PROJECT: cloudinfra VM: chicotest-cappy01 PROJECT: chicotestproject VM: visualeditor-prototype2 PROJECT: visualeditor VM: programs-and-events-dashboard PROJECT: globaleducation VM: osmit-uno PROJECT: osmit VM: tools-sgewebgrid-lighttpd-0904 PROJECT: tools VM: canary1025-01 PROJECT: testlabs VM: mathosphere PROJECT: math VM: social-tools3 PROJECT: social-tools VM: togetherjs PROJECT: visualeditor VM: language-mleb-legacy PROJECT: language VM: women-in-red PROJECT: globaleducation VM: ntp-01 PROJECT: cloudinfra VM: mc-clusterA-1 PROJECT: test-twemproxy VM: wikifarm PROJECT: pluggableauth VM: login-test PROJECT: catgraph VM: puppenmeister PROJECT: planet
* cloudvirt1026:
VM: integration-agent-docker-1016 PROJECT: integration VM: wikidata-new-wbterm PROJECT: wikidata-dev VM: incubator-test PROJECT: incubator VM: cloudinfra-internal-puppetmaster01 PROJECT: cloudinfra VM: cloudinfra-db01 PROJECT: cloudinfra VM: tools-checker-03 PROJECT: tools VM: tools-static-13 PROJECT: tools VM: wp1 PROJECT: mwoffliner VM: pk8s PROJECT: planet VM: arturo-k8s-test-4-1 PROJECT: openstack VM: banner PROJECT: wikidumpparse VM: packager01 PROJECT: packaging VM: tools-package-builder-02 PROJECT: tools VM: canary1026-02 PROJECT: testlabs VM: security-checker1 PROJECT: packagist-mirror VM: logstack02 PROJECT: security-tools VM: logstack01 PROJECT: security-tools VM: mediawiki2latex-large PROJECT: collection-alt-renderer VM: tools-sge-services-03 PROJECT: tools VM: tools-sgewebgrid-lighttpd-0928 PROJECT: tools VM: tools-sgewebgrid-lighttpd-0921 PROJECT: tools VM: tools-sgewebgrid-generic-0903 PROJECT: tools VM: tools-sgeexec-0938 PROJECT: tools VM: tools-sgeexec-0936 PROJECT: tools VM: tools-sgeexec-0935 PROJECT: tools VM: tools-sgeexec-0919 PROJECT: tools VM: tools-sgeexec-0917 PROJECT: tools VM: tools-sgeexec-0916 PROJECT: tools VM: tools-sgeexec-0915 PROJECT: tools VM: tools-sgeexec-0914 PROJECT: tools VM: tools-paws-worker-1010 PROJECT: tools VM: tools-paws-worker-1019 PROJECT: tools VM: openstack-puppetmaster-01 PROJECT: openstack VM: web1 PROJECT: graphql VM: etytree-b PROJECT: etytree VM: canary1026-01 PROJECT: testlabs VM: db-instance PROJECT: videowiki VM: tools-sgeexec-0906 PROJECT: tools VM: mwoffliner5 PROJECT: mwoffliner
regards
Follow-up:
We just discovered cloudvirt1014 doesn't require reboot, so this operation is only for cloudvirt1025 and cloudvirt1026.
regards.
On 10/16/19 1:10 PM, Arturo Borrero Gonzalez wrote:
Hello!
Next Wednesday 2019-10-23 at 09:00 UTC we will be doing another maintenance operation on some of our cloudvirts servers (the hypervisor servers) that involves rebooting both the physical servers and the virtual machines running on them. The reasons is that we ned to update the running linux kernel version they have.
In this window we will reboot 2 hypervisors:
- cloudvirt1025
- cloudvirt1026
The procedure will be to reboot a server, wait for it to come back online (could take up to 5 minutes) and wait for all the VMs to come back online. Then move to the next server.
Toolforge users may see their tools and webservices briefly disrupted due to several components of the Toolforge infrastructure being rebooted in this operation.
If nothing changes (reallocated or new virtual machine, etc) this is the list of affected VM instances in each hypervisor:
- cloudvirt1025:
VM: integration-agent-docker-1006 PROJECT: integration VM: striker-deploy04 PROJECT: striker VM: rec-wiki-2 PROJECT: recommendation-api VM: deployment-ms-fe03 PROJECT: deployment-prep VM: deployment-poolcounter05 PROJECT: deployment-prep VM: deployment-ms-be05 PROJECT: deployment-prep VM: readers-web-stephen PROJECT: reading-web-staging VM: traffic-upload-stretch PROJECT: traffic VM: traffic-recdns-anycast PROJECT: traffic VM: deployment-maps05 PROJECT: deployment-prep VM: gerrit-sizzle PROJECT: security-tools VM: tools-sgewebgrid-generic-0901 PROJECT: tools VM: shinken-puppetmaster-01 PROJECT: shinken VM: osmit-due PROJECT: osmit VM: deployment-acme-chief03 PROJECT: deployment-prep VM: meza-cindy PROJECT: pluggableauth VM: accounts-db4 PROJECT: account-creation-assistance VM: krenair-clientpackages-py3-jessie PROJECT: testlabs VM: deployment-sessionstore01 PROJECT: deployment-prep VM: paws-worker-04 PROJECT: paws VM: paws-ext-lb-02 PROJECT: paws VM: paws-int-lb-01 PROJECT: paws VM: paws-master-03 PROJECT: paws VM: paws-master-01 PROJECT: paws VM: language-readership PROJECT: language VM: wmde-wikidiff2-patched-stretch PROJECT: wikidiff2-wmde-dev VM: tools-sgebastion-08 PROJECT: tools VM: compiler1002 PROJECT: puppet-diffs VM: phragile-db PROJECT: phragile VM: cloud-puppetmaster-01 PROJECT: cloudinfra VM: chicotest-cappy01 PROJECT: chicotestproject VM: visualeditor-prototype2 PROJECT: visualeditor VM: programs-and-events-dashboard PROJECT: globaleducation VM: osmit-uno PROJECT: osmit VM: tools-sgewebgrid-lighttpd-0904 PROJECT: tools VM: canary1025-01 PROJECT: testlabs VM: mathosphere PROJECT: math VM: social-tools3 PROJECT: social-tools VM: togetherjs PROJECT: visualeditor VM: language-mleb-legacy PROJECT: language VM: women-in-red PROJECT: globaleducation VM: ntp-01 PROJECT: cloudinfra VM: mc-clusterA-1 PROJECT: test-twemproxy VM: wikifarm PROJECT: pluggableauth VM: login-test PROJECT: catgraph VM: puppenmeister PROJECT: planet
- cloudvirt1026:
VM: integration-agent-docker-1016 PROJECT: integration VM: wikidata-new-wbterm PROJECT: wikidata-dev VM: incubator-test PROJECT: incubator VM: cloudinfra-internal-puppetmaster01 PROJECT: cloudinfra VM: cloudinfra-db01 PROJECT: cloudinfra VM: tools-checker-03 PROJECT: tools VM: tools-static-13 PROJECT: tools VM: wp1 PROJECT: mwoffliner VM: pk8s PROJECT: planet VM: arturo-k8s-test-4-1 PROJECT: openstack VM: banner PROJECT: wikidumpparse VM: packager01 PROJECT: packaging VM: tools-package-builder-02 PROJECT: tools VM: canary1026-02 PROJECT: testlabs VM: security-checker1 PROJECT: packagist-mirror VM: logstack02 PROJECT: security-tools VM: logstack01 PROJECT: security-tools VM: mediawiki2latex-large PROJECT: collection-alt-renderer VM: tools-sge-services-03 PROJECT: tools VM: tools-sgewebgrid-lighttpd-0928 PROJECT: tools VM: tools-sgewebgrid-lighttpd-0921 PROJECT: tools VM: tools-sgewebgrid-generic-0903 PROJECT: tools VM: tools-sgeexec-0938 PROJECT: tools VM: tools-sgeexec-0936 PROJECT: tools VM: tools-sgeexec-0935 PROJECT: tools VM: tools-sgeexec-0919 PROJECT: tools VM: tools-sgeexec-0917 PROJECT: tools VM: tools-sgeexec-0916 PROJECT: tools VM: tools-sgeexec-0915 PROJECT: tools VM: tools-sgeexec-0914 PROJECT: tools VM: tools-paws-worker-1010 PROJECT: tools VM: tools-paws-worker-1019 PROJECT: tools VM: openstack-puppetmaster-01 PROJECT: openstack VM: web1 PROJECT: graphql VM: etytree-b PROJECT: etytree VM: canary1026-01 PROJECT: testlabs VM: db-instance PROJECT: videowiki VM: tools-sgeexec-0906 PROJECT: tools VM: mwoffliner5 PROJECT: mwoffliner
regards
On 10/16/19 3:34 PM, Arturo Borrero Gonzalez wrote:
Follow-up:
We just discovered cloudvirt1014 doesn't require reboot, so this operation is only for cloudvirt1025 and cloudvirt1026.
Reminder:
this is happening in a few minutes!
regards
cloud-announce@lists.wikimedia.org