We
just finished moving the first 100 or so VMs to Ceph[0] and
despite a couple of wrinkles things are working well. Assuming
that next week is fairly peaceful, we'll resume the migration to
Ceph in earnest the following week, converting another nine
hypervisors to Ceph-enabled hosts. On the day specified below you
can expect each VM to be shut down for 10-20 minutes and then
restarted. The total window of migration may be as long as 5-6
hours, which means you should plan for your VM to be moved at some
point between 14:00 and 20:00 UTC on the indicated day.
No action is needed on your part other than planning for brief downtime. Toolforge should be largely unaffected, although certain services may be briefly unreliable.
After
this round of migration there will be another pause of a few days
to assess capacity and performance before the last group of VMs is
moved in mid-October.
[0] https://techblog.wikimedia.org/2020/08/24/ceph-distributed-vm-storage-coming-to-cloud-services/
Monday, 2020-09-28, 1400-20:00 UTC:
== cloudvirt1012 ==Tuesday, 2020-09-29, 1400-20:00 UTC:
== cloudvirt1016 ==
Wednesday, 2020-09-30, 1400-20:00 UTC:
Thursday, 2020-10-01, 1400-20:00 UTC:
== cloudvirt1018 ==
Friday, 2020-10-02, 1400-20:00 UTC:
== cloudvirt1021 ==
atdt.analytics.eqiad1.wikimedia.cloud
(11937864-e396-4270-b34f-28930a84aea0)
chat-mattermost02.chat.eqiad1.wikimedia.cloud
(bafd608c-ab17-403c-8cec-5100d55fe44e)
cloudstore-client-1.cloudstore.eqiad1.wikimedia.cloud
(24d82270-157b-45d6-b273-d5d1485e7534)
codesearch6.codesearch.eqiad1.wikimedia.cloud
(6097e7ab-ec52-45a6-b006-5cad01f160b2)
deployment-puppetdb03.deployment-prep.eqiad1.wikimedia.cloud
(05f311e9-1ef4-4acd-b467-adb59f6c2f93)
deployment-eventgate-3.deployment-prep.eqiad1.wikimedia.cloud
(21480702-050f-4324-8bbe-4716ef3a29ca)
deployment-wdqs01.deployment-prep.eqiad1.wikimedia.cloud
(75bca43a-92ec-40ca-ae8c-6e7b18b75ec5)
deployment-ircd.deployment-prep.eqiad1.wikimedia.cloud
(162c71a3-2d84-4a78-8f23-b2531fabd9d7)
deployment-snapshot01.deployment-prep.eqiad1.wikimedia.cloud
(898fe007-ec78-465e-b298-d9c57aee7aad)
deployment-mwmaint01.deployment-prep.eqiad1.wikimedia.cloud
(708b39a9-c00f-4643-b64b-f8135d06a9c0)
phabricator-prod-1001.devtools.eqiad1.wikimedia.cloud
(b5c89bd3-adde-48ac-98e4-4673e66e492d)
extdist-04.extdist.eqiad1.wikimedia.cloud
(ae99983c-1d5c-4b84-a426-9e37b4264de8)
finding-glams-monumental.finding-glams.eqiad1.wikimedia.cloud
(cf6d5a71-ae3d-401b-b767-89fd5e151e7f)
integration-agent-docker-1007.integration.eqiad1.wikimedia.cloud
(a7d626dc-3324-4fb1-8ac6-7e1e442db9fa)
integration-agent-docker-1010.integration.eqiad1.wikimedia.cloud
(ee15d4af-18ce-4f2d-8c29-72fd8c35016e)
integration-agent-docker-1009.integration.eqiad1.wikimedia.cloud
(d78a771d-d787-4ff8-99b5-f23b1ce9c337)
integration-castor03.integration.eqiad1.wikimedia.cloud
(8911a89e-4247-48d0-9cb4-7377244b37bb)
mwv-puppetmaster.mediawiki-vagrant.eqiad1.wikimedia.cloud
(0c1f419b-47a1-419e-b121-d2a252e298bf)
apps2.mobile.eqiad1.wikimedia.cloud
(80dfbd61-0ba8-4aca-addc-0b07fc655489)
pontoon-icinga-01.monitoring.eqiad1.wikimedia.cloud
(2d017f78-828e-47b0-972d-e9c4a3674427)
cloud-bootstrapvz-stretch.openstack.eqiad1.wikimedia.cloud
(0f34b1cb-0757-44f3-82af-21f66cfdad5b)
petscan4.petscan.eqiad1.wikimedia.cloud
(8b5a3f65-1080-432b-8e7e-8843388222f3)
compiler1002.puppet-diffs.eqiad1.wikimedia.cloud
(40695cd9-2ac2-46ab-9703-cdfd4cfc212f)
compiler1003.puppet-diffs.eqiad1.wikimedia.cloud
(317e0362-398a-4163-b385-42bc821c69cb)
nehpets.reading-web-staging.eqiad1.wikimedia.cloud
(e14c03c7-9346-43b4-8c08-a77154e87674)
trendingbuster.reading-web-staging.eqiad1.wikimedia.cloud
(8cd52077-4b5a-4bd7-a601-4da6a70b5341)
phamhi-puppetmaster.testlabs.eqiad1.wikimedia.cloud
(bbf9d46a-572b-42ee-9874-f57e17faf17f)
tools-puppetmaster-02.tools.eqiad1.wikimedia.cloud
(e4a34e65-c079-4942-a306-6e13cc2a226c)
tools-prometheus-03.tools.eqiad1.wikimedia.cloud
(b24e29d7-a468-4882-9652-9863c8acfb88)
tools-k8s-haproxy-1.tools.eqiad1.wikimedia.cloud
(9c1cecee-ff23-4f68-a140-3c29d0aa6f63)
tools-proxy-05.tools.eqiad1.wikimedia.cloud
(8bc8989c-3cf2-432d-a6c7-8a743fc1445e)
toolsbeta-test-k8s-ingress-2.toolsbeta.eqiad1.wikimedia.cloud
(03321e69-da5c-4ac7-a050-d01ef57124d6)
toolsbeta-test-k8s-haproxy-1.toolsbeta.eqiad1.wikimedia.cloud
(ba228903-c8a6-41d2-bec0-c43aec863ad0)
toolsbeta-test-k8s-worker-1.toolsbeta.eqiad1.wikimedia.cloud
(051fb107-db81-4aab-a79a-fbb5f52c1a61)
toolsbeta-test-k8s-worker-2.toolsbeta.eqiad1.wikimedia.cloud
(e2897372-557d-4276-96d5-e3bcdc686b8e)
toolsbeta-test-k8s-control-3.toolsbeta.eqiad1.wikimedia.cloud
(9279c426-dff5-4417-a44e-6ff1a191753c)
toolsbeta-test-k8s-etcd-3.toolsbeta.eqiad1.wikimedia.cloud
(12f027d0-9568-421f-91d5-336e337c14a7)
toolsbeta-sgecron-01.toolsbeta.eqiad1.wikimedia.cloud
(739d6134-7968-4c5e-a21c-168d13e3e244)
gfg.video.eqiad1.wikimedia.cloud
(c4483452-d485-4f29-a5fb-79c5d6460ea5)
wikidata-icinga.wikidata-dev.eqiad1.wikimedia.cloud
(77f2605f-566e-49df-b406-cabc5ced42fb)
pst.wikidata-primary-sources-tool.eqiad1.wikimedia.cloud
(8beacc89-d604-4fa3-af56-2e1bbc704854)
search-loader-01.wikidata-query.eqiad1.wikimedia.cloud
(687160cf-d96f-4e97-a0e0-7f69892967ca)
wikiloop3.wikiloop.eqiad1.wikimedia.cloud
(65efa1c7-ba72-43ec-82cc-ac99cea36e0f)
wikispore-prod.wikispore.eqiad1.wikimedia.cloud
(4b432cfd-e972-43d7-b1c8-a8c2037361ae)
wm-bot.wm-bot.eqiad1.wikimedia.cloud
(35bae355-1cc8-4837-ae1e-3cfe40af9acc)
Monday, 2020-10-05, 1400-20:00 UTC:
== cloudvirt1022 ==
accounts-dbreplica.account-creation-assistance.eqiad1.wikimedia.cloud
(23e7622f-acb7-4a71-8b01-c4785d5f023f)
accounts-db5.account-creation-assistance.eqiad1.wikimedia.cloud
(4ec54524-1e3c-4141-8859-c987028244ad)
k4-2.analytics.eqiad1.wikimedia.cloud
(fdb7710a-de83-40c5-beae-98c0ab8ea7a8)
af-puppetdb02.automation-framework.eqiad1.wikimedia.cloud
(d94c37a2-b71c-4ea4-91ae-a3a236166392)
chat-database01.chat.eqiad1.wikimedia.cloud
(3d103b70-a2a4-47b7-8e91-83b12ede2e5f)
cloud-puppetmaster-03.cloudinfra.eqiad1.wikimedia.cloud
(5f96add9-ccc8-4308-955d-ec1ce4b32cc8)
cloudinfra-db02.cloudinfra.eqiad1.wikimedia.cloud
(fc5e2197-5de3-4ff2-8ae3-9656a6398c2d)
Krypton.codereview.eqiad1.wikimedia.cloud
(64d0dc6a-058e-48ef-bcd6-d5b53e678c18)
commtech-wikiwho.commtech.eqiad1.wikimedia.cloud
(155ea43c-4850-4c44-8462-9b8e291a0b16)
deployment-elastic06.deployment-prep.eqiad1.wikimedia.cloud
(6f0423b6-b3b9-44f3-b7e8-d51dfa9c6744)
deployment-restbase02.deployment-prep.eqiad1.wikimedia.cloud
(63ae9220-64ec-440c-b46b-52c8fc1c44c2)
deployment-imagescaler01.deployment-prep.eqiad1.wikimedia.cloud
(d0220309-5972-4710-a617-d57532a47a05)
deployment-kafka-jumbo-1.deployment-prep.eqiad1.wikimedia.cloud
(f04b0bdd-852d-4ff5-a2a0-4006e064c51b)
deployment-memc07.deployment-prep.eqiad1.wikimedia.cloud
(06c8f1f1-2070-4d05-b97b-386c6bf5636b)
deployment-eventlog05.deployment-prep.eqiad1.wikimedia.cloud
(09b3a1c6-4a8d-43a4-9730-c4a2e62229d9)
deployment-mediawiki-07.deployment-prep.eqiad1.wikimedia.cloud
(4bff32b7-316a-40d4-9de4-3a13a7ebf62a)
tofawiki02.fa-wp.eqiad1.wikimedia.cloud
(d8062efc-59df-42ee-bed0-011cb6a14b16)
gitservices.getstarted.eqiad1.wikimedia.cloud
(9cc4397d-5efb-40fa-9282-9338afc19453)
mw01.globalcu.eqiad1.wikimedia.cloud
(528eda19-d5f3-4ceb-a75f-81d03dc2be8a)
grantreview-04.grantreview.eqiad1.wikimedia.cloud
(4e7b37bf-2037-4179-9b03-63ea0f032345)
graphql-kube1.graphql.eqiad1.wikimedia.cloud
(4d31be52-c3b3-441b-a69a-21e98d3b7cb6)
integration-puppetmaster-02.integration.eqiad1.wikimedia.cloud
(f856c5fb-28b4-412a-b92c-e5578fbd8fc9)
opusmt.language.eqiad1.wikimedia.cloud
(f0c8cf87-e0be-4b77-a5d4-535782bdf05e)
mailman-puppetmaster.mailman.eqiad1.wikimedia.cloud
(d85d549a-e7a5-4ffa-91d7-0894f7eba8d4)
maps-tiles1.maps.eqiad1.wikimedia.cloud
(ce430b95-cf87-4e5f-a9fe-4f4018945665)
puppetdb.mariadb104-test.eqiad1.wikimedia.cloud
(1a149053-416c-453a-8594-469cd8aceb6c)
matrix-synapse-01.matrix.eqiad1.wikimedia.cloud
(deb637c2-cfce-4bcc-aa10-49f0f8241bdd)
mobileapps-ios-experiments.mobile.eqiad1.wikimedia.cloud
(40a61ec3-aed4-49a8-aab4-fb2f4538ef33)
pontoon-kafka-01.monitoring.eqiad1.wikimedia.cloud
(e5288040-eded-46e2-8b7b-6ab3f161e764)
pontoon-grafana-01.monitoring.eqiad1.wikimedia.cloud
(5a8aad7b-a975-467c-97a5-50581ca87901)
pontoon-puppetdb-01.monitoring.eqiad1.wikimedia.cloud
(7829eaa2-2370-4d6f-848f-35399b5e0e02)
pontoon-log-01.monitoring.eqiad1.wikimedia.cloud
(24a93a3d-cd07-4479-992d-16b1e07d8b56)
mwoffliner2.mwoffliner.eqiad1.wikimedia.cloud
(7d49fc73-8c37-4d58-be26-c457aa59b941)
ores-web-05.ores.eqiad1.wikimedia.cloud
(817e4922-bd49-4a07-8476-ab4e544420ce)
ores-web-04.ores.eqiad1.wikimedia.cloud
(e5840383-05ea-4721-9c0b-d198d1f3b0ad)
ores-lb-03.ores.eqiad1.wikimedia.cloud
(c4a27ce0-5a56-4bf1-b914-8a158fd52e72)
packagist-mirror1.packagist-mirror.eqiad1.wikimedia.cloud
(770397da-6555-46e1-b79c-7f6321ee02e8)
paws-acme-chief-01.paws.eqiad1.wikimedia.cloud
(d5065ba1-a7f5-4e0c-add8-22aa24db7803)
wikifarm-dev.pluggableauth.eqiad1.wikimedia.cloud
(46b56e40-ea0c-4b64-85be-86cb857dbef6)
wikifarm-prod.pluggableauth.eqiad1.wikimedia.cloud
(116e6331-792b-432b-aade-95c7a5870f71)
jmm-debm-01.puppet.eqiad1.wikimedia.cloud
(7ed021f3-ee41-4bd9-bf3a-bd43ce2aaa59)
reading-web-staging.reading-web-staging.eqiad1.wikimedia.cloud
(d82c19d0-74fa-4129-98fb-bb6f75156655)
idp.sso.eqiad1.wikimedia.cloud
(9260ba33-a0d4-41cd-8696-753ee1ba44f9)
bstorm-testk8s-control-2.testlabs.eqiad1.wikimedia.cloud
(4dcf2daf-c4a1-4977-be69-bef54faef10f)
tools-sgecron-01.tools.eqiad1.wikimedia.cloud
(0ab42fcf-f405-4159-90b9-e61a973f3bd3)
toolsbeta-test-k8s-haproxy-2.toolsbeta.eqiad1.wikimedia.cloud
(95f23fb7-3b84-4f2b-9802-11b385247c1e)
toolsbeta-test-k8s-control-1.toolsbeta.eqiad1.wikimedia.cloud
(68065e80-8b4d-461b-bd34-f027d079330b)
toolsbeta-test-k8s-etcd-2.toolsbeta.eqiad1.wikimedia.cloud
(59922260-bd39-4bdc-b233-feb78ddb2665)
toolsbeta-test-k8s-etcd-1.toolsbeta.eqiad1.wikimedia.cloud
(7637ae11-247d-47ad-aa6d-f7e1b31c836e)
traffic-cache-atsupload-buster.traffic.eqiad1.wikimedia.cloud
(6211c49f-3d21-4e19-abbd-df6dbe4e7432)
staging.twl.eqiad1.wikimedia.cloud
(55831fc4-7a34-4d61-a9fb-f2d060b5bd7b)
video-cut-tool-worker-3.videocuttool.eqiad1.wikimedia.cloud
(bcf17475-4b73-4a02-921a-4ae070657375)
video-cut-tool-worker-1.videocuttool.eqiad1.wikimedia.cloud
(bbfeb703-aa0d-4d51-98dd-41a5565043e3)
wikidata-federated-properties.wikidata-dev.eqiad1.wikimedia.cloud
(00c523ef-8792-4362-a540-1954bcee0973)
sdc-test-runner-2.wikidata-federation.eqiad1.wikimedia.cloud
(b3723688-8b63-4926-b5c4-f0d59ddff9c5)
whgi.wikidumpparse.eqiad1.wikimedia.cloud
(7eaa6920-558c-48ba-8711-d0332d0408e9)
wikilabels-02.wikilabels.eqiad1.wikimedia.cloud
(7bfb8119-7d9a-4abe-8647-50879a023a3f)
wikitextexp-base-1002.wikitextexp.eqiad1.wikimedia.cloud
(cbf687b4-f6aa-4202-a3f4-ff6ce59793a4)
mediawiki1004.wmde-templates-alpha.eqiad1.wikimedia.cloud
(77c99917-daad-48e0-bc2a-fb9cc928dc57)
covid-data.wmf-research-tools.eqiad1.wikimedia.cloud
(8574e5f0-d53a-433a-8f40-58b75ee005aa)
xtools-dev05.xtools.eqiad1.wikimedia.cloud
(594b4b48-0bbb-4296-bb52-ef7861036ef6)
Tuesday, 2020-10-06, 1400-20:00 UTC:
== cloudvirt1023 ==
zk1-1.analytics.eqiad1.wikimedia.cloud
(05d2539e-332e-4dd2-b044-c433460a8a8c)
commonsarchive-mwtest.commonsarchive.eqiad1.wikimedia.cloud
(f1a23993-ccc0-431b-8c70-acf9c2df5dd3)
deployment-puppetdb04.deployment-prep.eqiad1.wikimedia.cloud
(df82a7d8-4311-4dea-b81f-e96ea40b42f2)
deployment-docker-mobileapps01.deployment-prep.eqiad1.wikimedia.cloud
(fdc4054c-d176-4202-b466-4478a0a1b2ba)
deployment-docker-cpjobqueue01.deployment-prep.eqiad1.wikimedia.cloud
(df18863a-2da7-4951-aa69-936b3d889592)
deployment-mailman01.deployment-prep.eqiad1.wikimedia.cloud
(327c7d60-4b4c-48ee-9f00-fae074e5635c)
deployment-push-notifications01.deployment-prep.eqiad1.wikimedia.cloud
(08e7b7ab-2bc5-4a32-9487-3adedd27c0db)
deployment-docker-changeprop01.deployment-prep.eqiad1.wikimedia.cloud
(39317231-7710-4973-8cb6-50c05bf6ac79)
deployment-echostore01.deployment-prep.eqiad1.wikimedia.cloud
(c4fb73cb-adfd-485d-a1fd-336f9152cf87)
deployment-schema-2.deployment-prep.eqiad1.wikimedia.cloud
(84f75359-8e4b-44a8-9b58-cc566b1e1125)
gitlab.gitlab-test.eqiad1.wikimedia.cloud
(cda50b0e-3ad2-4e8c-bb1c-e7ff541065dd)
incubator-mw.incubator.eqiad1.wikimedia.cloud
(87eff33d-ee5a-48da-b15f-5f09fb074a59)
kubespray2.k8splay.eqiad1.wikimedia.cloud
(8380f229-79f0-4ab9-bbd6-f3fc3079574f)
upgrader-07.library-upgrader.eqiad1.wikimedia.cloud
(d3202418-6fd6-4c86-985b-0cbc5336bf74)
libup-diff01.library-upgrader.eqiad1.wikimedia.cloud
(6752bcc6-ff27-4848-aad5-e860e2194489)
visionoid.machine-vision.eqiad1.wikimedia.cloud
(10a164fd-4586-4c54-8cf2-f10d36806329)
mailman-mailman01.mailman.eqiad1.wikimedia.cloud
(18c4117a-b50c-4bf2-a830-f4b6b54822a7)
zarcillo1.mariadb104-test.eqiad1.wikimedia.cloud
(2f01bc27-ebaf-424a-99e9-ce4ddc2c6aef)
cumin.mariadb104-test.eqiad1.wikimedia.cloud
(22ef2c0e-0980-467d-a2da-3097b3ae3b3c)
jitsi02.meet.eqiad1.wikimedia.cloud
(3afd2468-6194-4a58-9610-0224f7189025)
mwoffliner3.mwoffliner.eqiad1.wikimedia.cloud
(e3a739ce-4e53-4563-b4d9-89b56463e611)
mwoffliner1.mwoffliner.eqiad1.wikimedia.cloud
(b44713ab-aeb6-4b40-bd61-7f861439c686)
media-streaming.ogvjs-integration.eqiad1.wikimedia.cloud
(0188d2b5-1e4d-4dcc-959f-44ea9a4c5565)
paws-k8s-worker-5.paws.eqiad1.wikimedia.cloud
(00110cef-288b-485e-b023-6cb8c8da58a2)
paws-k8s-worker-3.paws.eqiad1.wikimedia.cloud
(19eba07b-3f79-4ada-91a0-8ed817910b26)
paws-k8s-worker-4.paws.eqiad1.wikimedia.cloud
(446f5ae0-0a3e-4980-99cf-5745278a5a92)
paws-k8s-haproxy-1.paws.eqiad1.wikimedia.cloud
(52141b29-d533-4815-a82e-644630ccfe64)
paws-k8s-control-1.paws.eqiad1.wikimedia.cloud
(28a0c3ec-ec6f-4cad-b17d-bf652cbb7b4b)
project-proxy-acme-chief-01.project-proxy.eqiad1.wikimedia.cloud
(ae49f1f9-d091-4dfd-afe3-eac0ac62189c)
project-proxy-puppetmaster-01.project-proxy.eqiad1.wikimedia.cloud
(0c1fd3a1-600a-46d6-af58-36a8cfb94336)
maps-proxy-02.project-proxy.eqiad1.wikimedia.cloud
(6ecf3986-e383-49f1-b991-c8a50c381078)
maps-proxy-01.project-proxy.eqiad1.wikimedia.cloud
(ff9373de-bde3-4f09-8424-9364a066fff5)
wikipedia-country.recommendation-api.eqiad1.wikimedia.cloud
(c510a811-b721-406d-be1e-532f8702de49)
rec-wiki.recommendation-api.eqiad1.wikimedia.cloud
(87535381-8575-41cf-a61a-fa9c30144917)
redwarn-production.redwarn.eqiad1.wikimedia.cloud
(e529d804-a509-474d-89f0-8b8f4a424f8d)
sccache-01.sccache.eqiad1.wikimedia.cloud
(c71eb8b0-3a03-4a3b-8f12-34db1e51412c)
echo-test-store-buster.services.eqiad1.wikimedia.cloud
(540cdcb9-3978-4fc6-88f8-a37ab5899457)
Esther-outreachy-intern.services.eqiad1.wikimedia.cloud
(39bd77b0-7520-4569-8794-4168780517e2)
krenair-t252721-test.testlabs.eqiad1.wikimedia.cloud
(b5379463-43b5-4820-ada4-3b6906aa75ba)
tools-legacy-redirector.tools.eqiad1.wikimedia.cloud
(fe59ebda-bff6-4703-ace9-f785d50282d6)
toolsbeta-puppetmaster-04.toolsbeta.eqiad1.wikimedia.cloud
(d14ccabe-9728-4936-a099-5f3f63c12397)
toolsbeta-acme-chief-01.toolsbeta.eqiad1.wikimedia.cloud
(dbcb2067-4219-45ae-a828-7f49ce4be9fd)
toolsbeta-sgewebgrid-generic-0901.toolsbeta.eqiad1.wikimedia.cloud
(28bcc47b-4640-400a-948d-1018cc1e9020)
wcdo.wcdo.eqiad1.wikimedia.cloud
(4a87c9dd-d42c-42ff-ad5f-8b59125c49e0)
wikidata-autodesc.wikidata-autodesc.eqiad1.wikimedia.cloud
(384734b3-4094-4587-8908-70998c56b940)
queripulator.wikidata-dev.eqiad1.wikimedia.cloud
(69eb45e5-bc73-40d3-b9e8-115203c02428)
fedprops-euspecies.wikidata-dev.eqiad1.wikimedia.cloud
(c03f0bdf-cf77-4d9d-83df-171273b9eee5)
ldf-test.wikidata-query.eqiad1.wikimedia.cloud
(327dc67b-a1b1-4617-8b5c-c693214fcb0f)
wikilink-prod-swarm.wikilink.eqiad1.wikimedia.cloud
(5fa885dc-8737-463d-a361-8b8a62512741)
wikilink-prod.wikilink.eqiad1.wikimedia.cloud
(e76afce9-60d0-4168-b5f2-d282e4837156)
reader.wmf-research-tools.eqiad1.wikimedia.cloud
(8ce3626b-fea1-43e8-b794-c0b25cab679b)
wikipedia-topic.wmf-research-tools.eqiad1.wikimedia.cloud
(e5e1f622-fadb-426e-981a-10d3adfffc3f)