The issue that resulted in partial VM reboots last week[0] (see "VM reboots coming Tuesday, 2024-01-20") turns out to be more widespread, affecting virtually all instances. The primary symptom is that it restricts our ability to drain and maintain WMCS hardware.
In order to resolve the issue, all that's needed is a hard reboot of each VM. Note that a simple in-place reboot (for instance, issued at in the shell of the VM) does NOT resolve the issue. Hard reboots must be performed via Horizon, by selecting 'Hard Reboot Instance' on the Instances panel.
So, please, at your convenience, log into Horizon and hard reboot any of your VMs that appear on the list below. For informal coordination I have also populated an etherpad[1] with the list of affected VMs.
Anything that remains in need of a reboot by late next week (Thursday, February 6th) I will reboot for you. So, if you don't care when/if your VM is rebooted you can ignore this message :)
Thank you! And, sorry for the inconvenience. We have a plan in place[2] which should prevent this issue from re-appearing in the future.
-Andrew
[0] https://phabricator.wikimedia.org/T383583
[1] https://etherpad.wikimedia.org/p/rmrebootsforcephmons
[2] https://phabricator.wikimedia.org/T385288
--
[] account-creation-assistance: accounts-appserver6 9d7288ae-ad02-40a6-8976-495d63c040a0 [] account-creation-assistance: accounts-db7 8fcade67-7226-469a-8fac-dda6a08f24d0 [] account-creation-assistance: accounts-oauth-g 3d107f3e-1200-416f-9190-94b0cd9541b0 [] adiutor: testbed ed652064-c031-495b-bb4b-6ace51e2b01a [] adiutor: testbed-vagrant 671aae8e-3a57-47f6-bb44-9275b7b4516b [] analytics: gitlab-docker-runner 2122f5ae-3354-41fa-96e2-eed9efb25a4d [] analytics: growthbook 8e26bc28-39f0-49f8-8cc4-bd6b61411bd3 [] analytics: kdc 32a69b77-1dfc-4e30-a522-883abcaf43f0 [] appservers: j-tst-puppet-01 0c315541-2483-42bf-84f0-5ab8ee592359 [] auditlogging: syslog-client03 c0645c2a-e8d9-48cd-898b-4c466f1be8dc [] auditlogging: syslog-client04 2d6ab6a0-cb97-4ee7-bd8f-e7e8408c5db3 [] auditlogging: syslog-client05 d8ccc10d-3c77-4b30-8179-7498ac8cc17a [] auditlogging: syslog-server03 07e9058c-62c6-4a81-97a1-f36b11603689 [] auditlogging: syslog-server05 8e75ce49-694f-4c68-9d1a-10115e7f791b [] automation-framework: diffscan02 dba77a95-d482-43af-853a-7b72b309dd9e [] bastion: bastion-eqiad1-03 15700b39-5773-4fec-9349-47f16bc457bd [] bastion: bastion-eqiad1-04 20d097f7-3866-4f06-b3d6-cd7d37eb33b5 [] bastion: bastion-restricted-eqiad1-3 47b0da1d-e50a-42c1-8cd9-dc255cc2f1a3 [] bub2: bub2-instance 6cc04f18-9f59-41bb-9bcd-9aaa8bcc2d96 [] capacity-exchange: production d747c697-0895-43a0-93ee-2b5db340bb22 [] catalyst-dev: k3s-node01 6799679b-ac3f-4e07-8bd4-8c23640a5886 [] catalyst-qte: k3s-main 1ad1581e-24e0-4903-bda0-ef74fe5e1b61 [] catalyst-qte: k3s-node01 886df770-6c2a-4f38-8045-34fa5e695a69 [] citefix: citefix 709122ba-c8b5-48ba-9694-fbc89b1cf5f7 [] civicrm-prototype: puppetserver-01 d1a2fc36-1a82-46c1-bd01-d1382e54feee [] cloudinfra: enc-1 e94378df-7a48-4b11-b44a-bf69aaf132bd [] cloudinfra: etcd-discovery-1 926dc59c-3e85-4ae0-a65f-0140fb202a2a [] cloudinfra: mx-out05 a9c57526-aaed-4b93-9cbe-e992c6fc25e5 [] cloudinfra: ntp-03 dd448d31-6a7c-4202-afeb-88ba838c2d85 [] cloudinfra: ntp-04 7773ba13-e882-4780-ae0d-da8a39f26a3a [] cloudinfra: syslog-server-audit01 be0eb3db-cd9a-482a-a31b-40c0babf66ae [] codesearch: codesearch9 683ef4dd-491d-401b-b60e-88e2945e54ea [] collection-alt-renderer: mediawiki2latex 5001eac3-74c2-49a6-b081-195f68464c25 [] commtech: commtech-bot02 a2466526-104b-4f96-ada1-66d28aff7572 [] commtech: commtech-nsfw03 8db5b1a2-506a-4c90-924c-030005488308 [] commtech: pageviews01 0e4a8b46-a264-457b-8711-0bee2345e871 [] copypatrol: copypatrol-backend-prod-01 50d3e893-aee7-4591-9f2b-630107c6a036 [] copypatrol: dev01 df58d6c5-6dae-4b56-8507-dc2e00679834 [] copypatrol: prod01 a870b45e-e184-4833-9117-d49de7263d69 [] cvn: cvn-apache10 f5c20388-c623-4ad4-914c-db565654bf9f [] cvn: cvn-app10 1c7097fe-f721-46ef-928f-2d975d96afb6 [] cyberbot: cyberbot-exec-iabot-01 58bff802-41bc-4ae6-b832-30636d25e0e8 [] cyberbot: cyberbot-exec-iabot-02 eb1784d4-f013-43d6-97d4-2a4ae5f92af3 [] cyberbot: cyberbot-web-iabot-01 af3129d7-3d4a-43b9-8156-53eb636f27a5 [] dashiki: dashiki-03 595b3e7d-13cc-4695-9f60-a86618541cc8 [] dashiki: dashiki-staging-03 525bc74f-950c-427b-90b8-1db821f8d68b [] deployment-prep: deployment-acme-chief05 9bc3b4dd-bea1-47c4-8f54-d4f9c918c19f [] deployment-prep: deployment-acme-chief06 89296f92-ecfe-4fdd-bec9-10ff30b27f7d [] deployment-prep: deployment-bastion bdde8967-2caa-4dd1-ae27-829e808eb64a [] deployment-prep: deployment-cache-text08 7102217c-5d7f-4758-8936-fd72b578f3e8 [] deployment-prep: deployment-cache-upload08 11bc7c03-20a9-4488-86c7-ce4b7faa9283 [] deployment-prep: deployment-changeprop-1 0cedcc02-56cd-42aa-9017-add8a4291793 [] deployment-prep: deployment-cumin-3 2d02fdb4-8068-4d75-9be2-ac2ea715b8c4 [] deployment-prep: deployment-db11 369dfb8f-6311-4f99-85c8-cd9aa293d630 [] deployment-prep: deployment-deploy04 60461772-4536-4055-ab8e-9180c719cf25 [] deployment-prep: deployment-docker-charts01 d6b21175-b9f9-454a-afdc-6e647c40cd02 [] deployment-prep: deployment-docker-citoid02 a092d208-885b-40e4-9d27-998669e9ac11 [] deployment-prep: deployment-docker-cxserver02 b0ef7806-2ed0-4bbc-afbb-175a8ec1d85a [] deployment-prep: deployment-docker-mathoid02 52009c1a-5a0b-4d47-abf0-11efc090e692 [] deployment-prep: deployment-docker-mobileapps02 c6221ce4-48ac-4129-a984-eb7db83b0adb [] deployment-prep: deployment-docker-proton02 918c1510-917c-43f7-9829-d6f669a700c4 [] deployment-prep: deployment-docker-wikifunctions01 52cca4f5-9aeb-4a4f-90d0-b0f2d5e07cb7 [] deployment-prep: deployment-elastic09 11a817d0-3184-47be-a846-316ceb21221b [] deployment-prep: deployment-elastic10 5ef1f5ce-b1d9-48c4-9f85-c727f7d12851 [] deployment-prep: deployment-elastic11 9e94e6d5-a094-4ba3-8594-0cc1d3c4e315 [] deployment-prep: deployment-etcd02 c27abddd-8330-43db-80a9-dafebc76fc79 [] deployment-prep: deployment-etcd05 9e5e712f-eb60-4136-a66d-eb23a56d3349 [] deployment-prep: deployment-eventgate-4 2d94e48b-4e1b-4cf9-8b11-9d0d4bbc8efb [] deployment-prep: deployment-eventstreams-2 beed7e0d-4e7f-446f-a73c-60dce7ecff4f [] deployment-prep: deployment-ircd03 ea50001a-ca9b-44be-833f-2563d13e9779 [] deployment-prep: deployment-kafka-jumbo-5 3ceeda11-82e7-4fd6-9321-24713686318f [] deployment-prep: deployment-kafka-jumbo-8 1b4812a5-db65-4aa2-8adb-6af7333ae950 [] deployment-prep: deployment-kafka-jumbo-9 8c77502b-f887-4974-99b1-387c776605af [] deployment-prep: deployment-kafka-logging01 4f08a0ac-5810-47fd-941c-dbc883811f43 [] deployment-prep: deployment-kafka-main-5 b2eeb153-d31b-4c92-9c06-c64242bda3a0 [] deployment-prep: deployment-kafka-main-6 9178db63-751e-4727-8b9c-4b58f44cd214 [] deployment-prep: deployment-maps-master02 e170e0e7-226b-476e-9df0-4a48a53f4366 [] deployment-prep: deployment-memc11 0776b1c1-956f-46f6-a909-dca50a3d2f28 [] deployment-prep: deployment-memc12 f019dd93-b279-419d-a9fd-36e1256150fc [] deployment-prep: deployment-memc13 77aaac97-8184-4ab1-a8a4-947fca686420 [] deployment-prep: deployment-ms-be07 643b1616-eede-4859-8917-e10b4057126b [] deployment-prep: deployment-ms-be08 20f22f9d-5734-49b9-8a19-9908e588c92f [] deployment-prep: deployment-ms-fe04 7e6b3fec-b97a-4a32-b5c6-279244f78739 [] deployment-prep: deployment-mwlog02 56bc21a6-4fd4-4ea8-9cb1-2c9d9f170c15 [] deployment-prep: deployment-mwmaint03 69da7d16-942e-45d7-afbd-f01eb1d88d9f [] deployment-prep: deployment-mx03 8eb272dc-8006-4e93-a966-5035809324d9 [] deployment-prep: deployment-puppetdb05 7bd079eb-82f6-453a-9bf4-ddcf68057de3 [] deployment-prep: deployment-puppetserver-1 3192a8d0-0fda-4d24-a1b7-2e2dc20fbe0e [] deployment-prep: deployment-push-notifications02 c1994a14-51da-434a-941b-69e14dbcdaf9 [] deployment-prep: deployment-rdb01 0df5294e-2b0c-4b32-aa01-9774da28b01f [] deployment-prep: deployment-restbase05 0f8ff3ed-24ad-4dc8-ac30-6423901a66f1 [] deployment-prep: deployment-schema-3 b4da0c29-b7bb-40a8-9fbf-d54dc72d8257 [] deployment-prep: deployment-sessionstore06 716ce882-8593-4129-90d7-16adb99ae3d1 [] deployment-prep: deployment-shellbox01 1c094211-c3c9-43ef-8bd9-8f2255d85e5a [] deployment-prep: deployment-shellbox-video 6789abfe-3a7f-4d3a-bf32-0b1030c8be4a [] deployment-prep: deployment-urldownloader04 1d187238-9307-440f-b63f-3c3b6a34e239 [] deployment-prep: deployment-webperf21 ba5ee9c6-08e0-474f-8049-d31ec8bed047 [] deployment-prep: deployment-webperf22 a678753b-570c-47d9-86cf-bad88ab20bc6 [] deployment-prep: deployment-wikifeeds02 36458df2-70be-4666-aa09-eafef6b38fc2 [] deployment-prep: deployment-zookeeper-3 5794b8a9-a29f-46e3-8058-52188bade672 [] devportal: devportal-demo01 3fa16445-55ae-458c-a229-f3dd019de2af [] devtools: codesearch 5fca62da-0a71-4856-b58d-9566bdb52d14 [] devtools: deploy-1006 6c97c169-54f8-419b-a747-cd4e3b47f868 [] devtools: etherpad-bookworm 604af9b2-65f1-45bc-943a-52bed0012491 [] devtools: gitlab-runner-1002 7af5196c-795c-4d1a-bb11-15b3506cd16f [] devtools: gitlab-runner-1005 9b170998-ef69-4a3c-aa0f-35855107f691 [] devtools: phabricator-bullseye c2b2b2a1-140b-402f-be7a-50749c78bfc7 [] devtools: puppetmaster-1003 b7f828a5-8269-40f4-87bf-a78b3ce5f147 [] devtools: vrts-1003 a77ff416-995a-424f-a8bf-c7261e93b162 [] discourse: discuss-space 46e675f2-9115-4443-a91f-5c50b7850417 [] discourse: discuss-space-new ca3a1b1f-2a36-4e6f-83ad-d8bbb49b5862 [] download: download-03 02bcf69a-f8ae-4a31-b2ec-cd8e53193e65 [] duct: eb 23a8d36e-d5c5-4264-9d7b-a87d86bb99ac [] dumps: dumps-nfs-1 4761bc85-b4c9-4a88-ac13-8205a79f43d9 [] dwl: irc-bullseye 596d0df6-714c-4bc2-9049-1a459c1360ac [] dwl: taxonbota3 85578a0d-caad-40b4-8b5e-2126c220105b [] dwl: taxonkatbot3 60eb9364-03ca-4b2e-adef-3a155e03445c [] entity-detection: maps-workspace 7e758ab0-b51e-47b5-92ba-606bf19771ec [] etytree: etytree-b acfe0102-959d-46d5-bde9-3e03f8814df6 [] eventmetrics: prod03 5ebd0672-64c9-4dba-a687-b029767433ab [] extdist: extdist-06 472c3694-e68a-4c36-bce8-6517ab3bcc73 [] fastcci: fastcci-master 61e2acff-66b0-4f0d-a0cd-ce737feca69a [] fastcci: fastcci-worker1 b9352e93-fb87-4e32-b30c-55e475ade73a [] fastcci: fastcci-worker2 c3a88537-8f70-44e3-8f0e-9c0fe592429e [] fa-wp: chemistoid 38c1a830-be37-4472-8310-45f5ad5b277d [] fa-wp: tofawiki03 feaa04b8-27e0-4f78-abd9-a74a18fdecfd [] fr-tech-dev: payments e759a967-e322-4e3f-93c3-f546381a7b8d [] gitlab-runners: runner-1021 7ee0fcc7-43ea-4321-878e-a6922ac1236d [] gitlab-runners: runner-1022 cb4a87e4-264e-4c8f-8197-3efff54346de [] gitlab-runners: runner-1023 56e55a31-8d32-455e-b650-b7194e71d2fd [] gitlab-runners: runner-1024 921ddea1-0288-467f-8458-56860f383b93 [] gitlab-runners: runner-1025 6b8137a7-243e-4319-9e80-735095b8c30c [] gitlab-runners: runner-1026 329fc8c9-c51a-406d-9bad-9d88013b3e4b [] gitlab-runners: runner-1027 c3bdc48c-9ecf-48b3-bad9-67b6fc26cc77 [] gitlab-runners: runner-1028 b5b34987-d4ce-4f2b-a96b-b042d6e65623 [] gitlab-runners: runner-1029 39693d07-dad5-4ad0-a49c-e6fa468ae53b [] gitlab-runners: runner-1030 a9902e35-173f-4dbd-bce3-076e552bcd65 [] glams: glams-testing f9d47b22-bf9b-491c-b48d-dbbb56c487a9 [] glamwikidashboard: serviceapp 7aa37f8a-9e13-4ef2-8362-af49656c7e2d [] glamwikidashboard: webapp e78526c2-32c0-49f9-9b03-1ec49fff26be [] globaleducation: impact-visualizer-tool 7009df57-c7ec-45a7-a65e-da69eb018392 [] globaleducation: p-and-e-dashboard-redis b3c07919-318d-45a8-9c29-88cb3a59f56f [] globaleducation: peony-sidekiq-3 2962a54e-eaa3-4a46-a5ba-dc4773bf2fdd [] globaleducation: peony-sidekiq a78c5ea3-0c0e-4589-8ac9-8e24023a314f [] google-api-proxy: google-api-proxy-04 6bf5e226-050f-4307-8550-a3ef7e960ea3 [] hoiscript: Peony 6b32f902-7d6f-4ff3-a0ac-20f264d0c5de [] huggle: huggle-nfs-1 b74e3f99-5aee-4702-b501-7cc93cd8f4fc [] huggle: xmlrcs2 e840cdbd-349e-42f8-bf83-12507be6ebeb [] huma: web01 583ad29e-f9b3-47e7-8b83-0440bf72d7fa [] huma: worker01 cb20f298-f78e-4ae6-8752-9c32fb79a7cf [] iiab: iiab-med 79f31d08-d21f-4ef6-9cc0-7bf495cf28ed [] imagebulk: imagebulk 21cf9a6d-6abd-44e5-b12d-08b611643466 [] impactvisualizer: impactvisualizer 207f3ae3-9627-408a-8c07-bd6c78a56c71 [] impactvisualizer: replica-revision-tools 133429b1-2a3d-477a-ba3a-006939ba138f [] incubator: incubator-1 d7c826ce-1b23-4b63-bf2c-280b9396644b [] integration: integration-agent-docker-1040 42d3d9a4-5d2d-433f-8b3f-c8fdb2a24a3c [] integration: integration-agent-docker-1041 db4ae352-bbc3-450e-b4bb-1bb95f186072 [] integration: integration-agent-docker-1042 426d1a55-e8ce-4332-9e3c-3a52f47b36af [] integration: integration-agent-docker-1043 4c87de46-8ea5-476f-812f-4e627dbd6671 [] integration: integration-agent-docker-1044 88c795b4-72ee-4b82-b399-04d3457f2ce1 [] integration: integration-agent-docker-1045 d0e8fb1b-ec89-4934-86cd-6118db1219bc [] integration: integration-agent-docker-1046 270b6533-dc99-4e5d-a642-c61138b11891 [] integration: integration-agent-docker-1047 4c9a5ffd-f322-4b14-8f71-a87a18f54b09 [] integration: integration-agent-docker-1048 ab9080c3-fbcd-4e5f-90f4-d9669b9658de [] integration: integration-agent-docker-1049 80924e5c-f001-49f4-8043-6623af917151 [] integration: integration-agent-docker-1050 cfdbd9d9-6cbe-4516-89dc-b19f0e210f2d [] integration: integration-agent-docker-1051 ecbc72ca-601a-4ee5-b590-10606d64886a [] integration: integration-agent-docker-1052 7f7fe96a-c25f-4116-a79a-c77f1a88ca33 [] integration: integration-agent-docker-1053 14789ac1-bc06-4677-9bb0-66c16c887427 [] integration: integration-agent-docker-1054 702b9925-099e-4ecc-9d67-35865affa64d [] integration: integration-agent-docker-1055 6616fcbf-a49e-4e03-b735-84d31b535c08 [] integration: integration-agent-docker-1056 cc9e0883-0282-4f94-8e39-9c5ebe5ef805 [] integration: integration-agent-docker-1057 5c67ffe7-9c5b-4790-92b8-ca8299613ac0 [] integration: integration-agent-pkgbuilder-1003 a08a80f5-1b45-4aba-8945-cfc080a4c940 [] integration: integration-agent-pkgbuilder-1004 277f302a-b2fa-4635-8c88-0f8d6e9971f4 [] integration: integration-agent-puppet-docker-1003 96cf9ddc-daa3-4c9f-8c21-cdd58e95973e [] integration: integration-agent-qemu-1003 aa7075c2-f321-4817-a53a-ce553ecb6904 [] integration: integration-cumin 80f8822f-c0d9-427a-b743-b9185f58c0eb [] integration: integration-puppetserver-01 a2ce4233-268a-45ac-b542-ab6fc4e660a6 [] ipoidopensearch: ipoidopensearch 41b83b6d-344e-4b13-b0d7-a8a7e8b72dfe [] isa: m2c f8b9e023-248d-49a7-bf38-2dd448c48d33 [] k8splay: abogott-control-1 e470f1de-2f84-4068-981e-4af29685f04a [] k8splay: abogott-worker-2 e28a75a0-daf2-453b-8efc-fe88ae819470 [] k8splay: dzahn-bookworm 5ec028f1-3fcb-4774-9984-b3bc3f7a163d [] language: cxserver 9e0f89e0-a177-4a2a-a677-32e888b6e37f [] language: language-cx 9a857e61-052c-4025-a4f7-e42c31edd3a3 [] language: language-mleb-legacy a44b9b31-a11f-4594-b9f2-170ba2bfb4eb [] language: language-mleb-master 1c0ac8ab-1c86-4ad6-a65f-7802984c5ae4 [] language: language-sx2 a63a8815-e397-4b8d-9ec6-98fc3f380ef7 [] language: mleb-stable 78fae2d3-dc53-4242-9e18-3edacc02786f [] language: recommend 3e3fac3d-be77-4f50-8b18-9d7616c8fa58 [] language: translate 15ed27d5-cc31-462a-97b5-68601d3e88d7 [] ldap-dev: dcl-dev1 80bac244-e231-4390-9de9-79dc71ac0827 [] library-upgrader: libup-runner08 9a99163d-35a6-4596-a559-9014e72e0379 [] library-upgrader: libup-web02 53f8c1f4-0c85-407d-806f-47b585852cd7 [] linkwatcher: coibot-2 e08f00f6-5a4c-4981-8ee1-75e773204742 [] linkwatcher: linkwatcher-db 8ff173eb-163e-47bf-a429-a114d7affce8 [] linkwatcher: liwa3-2 ce9426cb-923e-4af7-a703-f81d9b3e6149 [] linkwatcher: xlinkbot-2 168548e6-4a9f-4faa-a109-17c90b6cb421 [] logging: logging-alert-01 a1141332-0400-4a87-ad7e-24564676e48a [] logging: logging-logstash-02 15a048f8-ab89-43ff-80be-63cf5b5691fa [] logging: logging-logstash-03 14abc0a9-ef08-4e73-a54b-8d91c1bbc752 [] logging: logging-opensearch-ssd-03 f502126e-1aa7-4f02-a047-bed58688e30c [] lta-tracker: bots2 27acf162-300a-4d92-a293-5b2d34e5c4b5 [] lta-tracker: prod 2b56c575-08a5-4def-87cb-bee5bd43e4f9 [] mailman: mailman1 187649b3-0313-46d4-bbdf-34b08d5c3d48 [] maps: maps-warper4 d3521b71-1096-4fb5-8a34-33ae6191965c [] mariadbtest: cumin-1 3776d295-b63e-4766-ac60-a7027821d11a [] mariadbtest: db-1 bafb528e-0490-4ce6-9505-a0526f95782f [] mariadbtest: db-2 68aa2f4b-a14d-4d82-b1f4-e6a0f1ec2d1e [] mariadbtest: mariadbcompiler e50dcf0c-9bdb-4a45-b337-9e46e3ba2e42 [] mariadbtest: test-with bfd02baa-da18-41c0-b06c-f6028bb5440e [] mariadbtest: test-wo 1cd86198-ce89-4c46-8218-20313d467f4b [] masz: web02 b43ae478-d62c-43c1-ad8f-eb09ca6f9f86 [] math: math24 0235bb05-e38c-44f6-93ac-13532c32ffff [] math: math-docker-2 372adf1a-3379-4fac-b2f3-e05c48ba869e [] matrix: synapse01 1fa9f0d9-42e8-4273-bdb1-a7d49998c13f [] mdwikioffline: mdwiki 60dff349-faea-4de9-b4d0-b776f43f2e5a [] media-streaming: media-streaming de2f0e88-ca81-4345-b445-9036de1740fb [] metricsinfra: metricsinfra-grafana-1 ae59a32d-c913-4b75-bf08-e598a43a2892 [] metricsinfra: metricsinfra-haproxy-2 46dfccb9-f74b-4b99-934d-f724b1c6a63b [] metricsinfra: metricsinfra-prometheus-2 7cb371bb-a53a-4e65-a1cf-f1a8264a9166 [] metricsinfra: metricsinfra-thanos-fe-1 b67ceb26-97c1-402e-9c7a-7c6d52b2f119 [] mwcli: gitlab-runner-addshore-1016 37e23659-4516-4bd8-a9be-4cc55def5560 [] mwstake: mwstake 77e0ad73-61dd-4a9d-a0c8-fc89e30a858b [] netops-clab: netops-clab de3c8ae8-9e09-4ece-9c19-07c39365b239 [] o11y: deployment-server-phi 5899163e-bbfb-474e-8a9a-2dae5006de22 [] o11y: phi-alert-01 98e6d07d-dd64-4083-8fab-f180e11dc402 [] o11y: phi-arclamp-01 cd686956-e65a-491f-9fc0-7bb216c61fa9 [] o11y: phi-grafana-01 8d5dba20-52d6-47f2-b26e-73531401d813 [] o11y: phi-kafka-01 e0878f27-bcff-48b9-a9c3-8af797e92622 [] o11y: phi-kafkamon-01 1b7de1cd-0857-425a-84d2-a4af29f6e982 [] o11y: phi-lb-01 562d3627-c869-4255-8818-90cc6d059ad4 [] o11y: phi-logs-be-01 940e0318-604f-4a88-ba11-e0f8ee0e0cdc [] o11y: phi-logs-fe-01 0c57ca62-098d-4f99-900a-d6cc3d88cb6d [] o11y: phi-mwlog-01 1802b911-6aa9-4142-b54d-a5b542f504e8 [] o11y: phi-pki-01 6383bd75-63cf-4703-b3a7-acb84a4cb93d [] o11y: phi-prometheus-01 d7ec787a-55f4-42a0-8d03-00efbccc8d21 [] o11y: phi-prometheus-02 871a7169-2748-41ec-a3e9-6b13a81ec526 [] o11y: phi-puppet-01 e964291c-48d8-4d44-89dd-24e4d49d7f97 [] o11y: phi-puppetdb-01 64149a5f-eeea-4978-9921-17de05c88dd6 [] o11y: phi-syslog-01 5393926d-b5cb-4132-9bf1-06f0a89d1d9c [] o11y: phi-titan-01 2d729afd-8290-4744-ab1d-57297ca6bbf6 [] o11y: phi-webperf-01 e5ed77fc-1077-4129-80b3-120c570c8edd [] ocrtoy: ocrtoy-web-bullseye 347d3b08-cd71-46f2-8028-ec5581719114 [] onfire: klaxon-keith 44ca15de-5737-4daf-8418-bb0068e106ed [] openvas: openvasv1 a236086b-56a7-42f5-ad04-a99dd10cafaa [] osmit: osmit-estratti 457f53c3-fbce-4b3d-ab72-bbb08c2716b7 [] osmit: osmit-umap2 1e476394-4ed9-439e-aed4-053486653df7 [] owidm: owidm-dev-1 c1f67eec-fff5-4ea1-9c1a-450b2702c206 [] owidm: owidm-prod-1 93eb7e45-7bd5-461b-93b1-e0637e947de0 [] packaging: cas-build ebfaa640-88ba-4bdc-a7d6-f35cd80a14fc [] packaging: packager-etherpad01 05a87de9-51e4-441c-bed8-03372ac90231 [] paws: bastion 9c7625e2-2dd0-4289-a934-f617fbb7999b [] paws: paws-nfs-1 1b001a01-0a37-4d81-9dd6-1339b6391f37 [] pixel: beta 2f8314c8-6d73-4462-ae6b-1d8ea7fc5e68 [] pixel: production 7af72145-4bdb-4a19-b960-f428eedfcb7b [] pki: pki-db 3788dd45-8e96-471f-a4f5-4c788c12601b [] pki: pki-intermediate f56e0ae1-b26f-43d8-8e9e-2e7fb424ce0b [] pki: pki-pm 22e7671e-c144-4508-afe0-f0e24603fa69 [] pki: pki-proxy 7081de14-d7f5-44ca-b82e-6b99af667696 [] pki: pki-puppetserver-1 1e528d51-9710-4047-ba0b-a58e9e53ae35 [] pki: pki-root edfc4cbd-dd02-41f2-8732-c48ef55e9c67 [] pki: pki-test01 3e6c31d7-b4db-4a5f-a610-a74d0013f631 [] pluggableauth: survey 8cf4a071-f4c1-4312-b635-926e0742ae5c [] pluggableauth: wikifarm 9f2a71c3-5683-4c30-b28d-617396fc3e3d [] procbot: bastion 102ecf69-5699-49bc-92ec-17b4823cbcb0 [] project-proxy: maps-proxy-03 cbb613c3-9480-4e96-a83e-90560205657f [] project-proxy: maps-proxy-04 d63dd9dc-a2f9-434e-b315-5d9d1d4f7c9a [] project-proxy: project-proxy-acme-chief-02 fd43cf27-3b19-4035-8047-7529c69b3e53 [] project-proxy: project-proxy-puppetserver-1 e8a89ddc-9129-42c4-bac5-1034de4ad90d [] project-proxy: proxy-04 24219963-e446-4c89-8ad7-30e8c4c69168 [] puppet-diffs: pcc-worker1004 a0d371be-dafb-4228-89c0-fc25b8d7430e [] puppet-diffs: pcc-worker1006 f9c9f01d-b1fd-4e64-a521-522fc767e604 [] puppet-diffs: pcc-worker1009 b02c4866-e0da-4029-b7ca-73252b5492c3 [] puppet-diffs: pcc-worker1014 72986a86-4401-4d7c-9730-5a71d7d22624 [] puppet-diffs: pcc-worker1015 a4871202-b93d-47e5-8b5d-eba47b274c5f [] puppet-diffs: pcc-worker1016 7589c350-58c4-4064-ad91-418e44ef5432 [] qrank: webserver 6efd77f9-2725-4a22-a898-625e3123bb0b [] quarry: k8s-nfs c18ab12b-fa18-4a29-a074-40535930f6b0 [] quarry: quarry-127a-g4ndvpkr5sro-node-0 8b68b650-19d3-421c-975f-7f98a47b5d2c [] quarry: quarry-127a-g4ndvpkr5sro-node-1 c86b3288-aff1-40a5-b619-887bdcefca44 [] quarry: quarry-bastion bec16bd4-988d-46b9-8b73-d044a4392a92 [] reading-web-staging: simple-summary-server 42121024-526f-497f-9001-12925c59fb1e [] reading-web-staging: skin24 3196c742-86ba-4308-b5a1-04d4bace5691 [] reading-web-staging: tarsier d99bd61e-094e-4a97-b856-96cad37fa905 [] reading-web-staging: wikitalky a43d6054-daf8-4049-b445-4952c98ddced [] recommendation-api: article-desc-deprecation 7bbd7cb5-8784-4091-aa0b-3058303695c1 [] recommendation-api: gapfinder-deprecation 1fdcc8e1-2274-4232-87d3-924a60800744 [] recommendation-api: search-referrals 24d2da5a-36b2-46c7-9217-9b5dc0d2a022 [] redirects: redirects-nginx03 dfbc6968-d803-4885-a34e-ba53cda922e7 [] research-collaborations-api: citationq b038356d-53f2-456d-9654-564261db0395 [] research-collaborations-api: edit-types 5f26c230-c05a-4f3c-ad4c-d4aae19f1c5e [] research-collaborations-api: fact-checkign-new f2f0790b-948d-4523-bfa5-a332da588aec [] research-collaborations-api: kitools 7da824f3-c9f3-4460-b9b2-2a894268a7ca [] research-collaborations-api: wikinav-bookworm 774a256b-2b8f-4384-93e7-53d368ff9a96 [] search: airflow 69825ca0-1fe2-4960-9c1f-24b1425ae85d [] search: cirrus-integ03 28e597a1-769e-48df-be12-73dcf53ea658 [] search: opensearch01 68a499c0-54a6-445c-af12-8e5265b10bee [] security-tools: security-dashboard 4bd8f0ba-26e3-440f-a972-f3218f2be97f [] signwriting: signwriting-swis-2022 e4bb9712-5ac3-4e57-8aa7-58754aa5216c [] signwriting: signwriting-swserver-2022 c778195a-b439-4c32-843b-40db2c6404b1 [] soweego: soweego b958511a-10cb-4e62-bdbb-6da5013dd62f [] spacemedia: glitchtip 3717d09b-ca52-4b92-8126-3720f4930251 [] spi-tools: bullseye ffae23a6-47bf-4c76-bfd2-d6b7d2cf495b [] spi-tools: opensearch2 e56b810b-6a74-41cf-8aa0-5a5fddcb2014 [] statanalyser: statanalyser f78be803-6d05-43cc-9b40-a8748ff56204 [] striker: striker-docker-02 5acbe429-549d-4d2a-889a-ed1bb841ee43 [] suggestbot: suggestbot-02 f5ff0e3c-82ea-428d-a263-9044a0419249 [] superset: superset-bastion 05a0a9b8-d56d-4117-bd40-f747869397f0 [] swift: dp-cssndr-puppet-01 2848e015-24ef-4061-b7a2-41401cffbf0a [] terraform: tf-puppetserver-1 99c2a647-36ef-40d5-9c21-8a20f437daff [] terraform: tf-registry-2 7946a9ea-136a-4223-8996-2e3a0334e3d8 [] testlabs: abogott-puppetclient1 6d344a91-80e8-48b0-b334-9bb77b153116 [] testlabs: abogott-puppetserver 74bca50a-fafc-483f-ad29-a68b3960d73a [] testlabs: kolla-test-3 92c89dd1-0cd3-4ca7-a7e3-09d4c3673814 [] testlabs: rook-tf 3ceb9b8d-a544-48f8-86ae-986ea938d7ed [] testlabs: slavina-k8s-workshop 8fc0eed6-8f20-4916-a81d-fb82b2d704cf [] testlabs: testlabs-nfs-1 14695d95-e47d-4b27-9023-20a38f07d11f [] text-to-speech: monitor 35d1d34c-7bc7-443b-a0c6-922de7368549 [] teyora: teyora 68b3e7b9-11ff-4223-a20b-dd19664b88f5 [] teyora: ty-analytics 4ae3bca1-7ea7-489c-9433-d0ebc2652e18 [] tofuinfratest: tf-bastion 3c385a09-9d18-42cc-965b-5d6cee9d005c [] toolsbeta: toolsbeta-bastion-6 33e430fb-f679-41ce-8ee6-4db1e812454c [] toolsbeta: toolsbeta-cumin-1 868d3dca-3e5c-4089-89a9-2c7e756c3e31 [] toolsbeta: toolsbeta-docker-imagebuilder-2 c07c4b31-fa95-4f18-927b-a17cf1003ae5 [] toolsbeta: toolsbeta-harbor-1 caf923d3-a42b-49b8-b861-f20996ab4ab9 [] toolsbeta: toolsbeta-legacy-redirector-2 ca99f9d1-a806-4eee-ab96-e4712abc611b [] toolsbeta: toolsbeta-mail-2 1cd98b82-6c77-465d-8dcd-f0c20e6d7789 [] toolsbeta: toolsbeta-prometheus-1 d110734e-aa20-404a-9f1e-a58c266f9fc5 [] toolsbeta: toolsbeta-proxy-5 1870a647-224b-4b60-b16f-64714e15cf7b [] toolsbeta: toolsbeta-proxy-6 64e17166-af72-4aa6-a0b7-544a12569655 [] toolsbeta: toolsbeta-puppetdb-03 56691327-d836-4f78-a5fd-93bdef49bfa9 [] toolsbeta: toolsbeta-redis-4 44007ae1-c81b-4cc8-b733-536daf5c14e6 [] toolsbeta: toolsbeta-redis-5 f794f6fd-59fa-4fd1-9179-f348e2a0c9ab [] toolsbeta: toolsbeta-redis-6 019519e7-2169-4615-8985-424af8d0e0d7 [] toolsbeta: toolsbeta-static-2 13f3646e-000e-4cbb-a643-54f73eecae4e [] toolsbeta: toolsbeta-test-k8s-control-10 7d35bbd7-cc41-48b3-87a5-35604980c40d [] toolsbeta: toolsbeta-test-k8s-control-11 44513a1b-807b-47cb-9f47-8ae343fe5f82 [] toolsbeta: toolsbeta-test-k8s-control-12 652f13e1-1c7f-46cd-9ca1-d7bebdc5abba [] toolsbeta: toolsbeta-test-k8s-haproxy-5 b65d430f-8873-48b4-9a08-681ab0552739 [] toolsbeta: toolsbeta-test-k8s-haproxy-6 8fbe70cd-a471-4f8b-b3e8-0136c20e8a3e [] toolsbeta: toolsbeta-test-k8s-ingress-10 5d4584f9-559c-4d3e-9af5-6542200b5158 [] toolsbeta: toolsbeta-test-k8s-ingress-11 5130e84c-f8fc-4e02-8c93-9412dab3f630 [] toolsbeta: toolsbeta-test-k8s-ingress-9 72561491-3b1e-437c-878d-e1db110728f3 [] toolsbeta: toolsbeta-test-k8s-worker-12 e5f37100-ebe9-4238-98ae-4869237baf36 [] toolsbeta: toolsbeta-test-k8s-worker-nfs-10 8a0ca656-8179-4544-9154-7d11b8941899 [] toolsbeta: toolsbeta-test-k8s-worker-nfs-5 60e69fe7-f6dd-48d4-85f6-3c13a946c2da [] toolsbeta: toolsbeta-test-k8s-worker-nfs-7 cd12b691-19cb-4e28-a1f1-433e53378f8e [] toolsbeta: toolsbeta-test-k8s-worker-nfs-8 93848139-6952-4edc-a927-9f207fab6b35 [] tools: tools-acme-chief-3 452dc8d3-6ee0-412c-90ba-66f21f9d09c1 [] tools: tools-acme-chief-4 2cfcfc76-8443-4b04-b129-ba381e6fbf22 [] tools: tools-checker-5 f0936015-b1ef-4b8c-ac6e-74766ed882ce [] tools: tools-cumin-1 5c8b0a31-4074-4629-a996-b4c156cdaac6 [] tools: tools-db-4 230c30f1-37a1-42a7-a829-4369448b73f5 [] tools: tools-db-5 51a560c1-d6dd-42b9-bec9-bd2b2504f2d8 [] tools: tools-harbor-1 243f3a30-eb5f-4726-9862-bcb1a35e267b [] tools: tools-k8s-control-7 67dbc73c-a7f8-40e4-96ef-ee838766ecff [] tools: tools-k8s-control-8 1d0839d5-0a11-4ed2-83b8-b0c15c62a5ae [] tools: tools-k8s-control-9 c195fc23-61a1-4093-9c9c-02f72e793380 [] tools: tools-k8s-haproxy-5 1de296d9-e469-47ab-a1ce-eb08c38d05b5 [] tools: tools-k8s-haproxy-6 469f0822-42df-4268-9e75-32e6973bf86e [] tools: tools-k8s-ingress-7 a96f04aa-9200-4b1a-86a8-3bfb836843bc [] tools: tools-k8s-ingress-8 54a3d128-ddfc-4346-bf84-57df94409fd7 [] tools: tools-k8s-ingress-9 959a47f2-87b1-43f3-bce5-f302fa3d1a3a [] tools: tools-k8s-worker-102 1bac7184-6f29-43b2-a686-1eba60ffbfed [] tools: tools-k8s-worker-103 1f64b6ab-4318-48e5-b885-c80151d4ab8f [] tools: tools-k8s-worker-105 0a3ebade-abf5-496e-88d5-ca29ceb52965 [] tools: tools-k8s-worker-106 eec8ebe0-64bf-40c1-b470-8e017b8e20e9 [] tools: tools-k8s-worker-107 da629018-4983-441c-b83d-7ecf90b6dd48 [] tools: tools-k8s-worker-108 3019cf57-1301-46f0-97f7-234b006e8be6 [] tools: tools-k8s-worker-nfs-10 7a497017-e462-473b-bdf5-5798f08b3179 [] tools: tools-k8s-worker-nfs-11 91080d53-f5a6-4d96-b622-722405de8982 [] tools: tools-k8s-worker-nfs-12 81a33d19-55cf-407c-b888-5b06438a095f [] tools: tools-k8s-worker-nfs-13 8155a7ef-5889-421a-a1fe-d43c00d60a0e [] tools: tools-k8s-worker-nfs-14 524da4e8-c4f6-4ed1-b18a-5bbcc51900ac [] tools: tools-k8s-worker-nfs-17 00aaf7e9-b8bb-489f-965a-bb99ecc050e2 [] tools: tools-k8s-worker-nfs-1 7b355092-f5e4-4be0-af94-b1307983c8e9 [] tools: tools-k8s-worker-nfs-24 77140f83-1a12-43b7-9e47-0e779503a525 [] tools: tools-k8s-worker-nfs-32 cab36feb-ef28-4f5f-88db-83270d181138 [] tools: tools-k8s-worker-nfs-3 5312e2f8-035c-464d-903e-79670197ab8f [] tools: tools-k8s-worker-nfs-39 8fb68878-67fe-4118-bb08-2f88b6e5f095 [] tools: tools-k8s-worker-nfs-40 728ae870-9430-4cd9-9fe3-d8138b97169b [] tools: tools-k8s-worker-nfs-43 18dc79d9-df55-4bf6-8bcf-5f40fb22aecd [] tools: tools-k8s-worker-nfs-45 62a59163-64c4-4295-bac0-f7f7aaabadf7 [] tools: tools-k8s-worker-nfs-46 1842d2b7-ca46-4c16-857b-8d588d5df6c6 [] tools: tools-k8s-worker-nfs-47 1423479b-afaa-4e16-b970-f4fb81db76f0 [] tools: tools-k8s-worker-nfs-50 3c99f41d-8a93-4347-8bc1-3ad5c63f55b1 [] tools: tools-k8s-worker-nfs-54 761089d7-d849-4b5d-8f8e-7368738f70dd [] tools: tools-k8s-worker-nfs-66 f9015696-9010-4a1c-a7cf-9bc8e8098df4 [] tools: tools-k8s-worker-nfs-71 e06b79a7-990b-4142-844e-5e826f3d4569 [] tools: tools-k8s-worker-nfs-74 11266010-8581-4750-9375-03fd1efb9e98 [] tools: tools-k8s-worker-nfs-9 33bfa2d5-31c5-4fe0-9a77-aaefd42eef1d [] tools: tools-legacy-redirector-2 7db5fc0e-34d2-495e-8045-8602639fe2bf [] tools: tools-mail-4 1484e4c9-0e93-4476-8882-56a7da42ecaf [] tools: tools-package-builder-04 62045cf9-59ed-44b9-a268-1c9f171b5aae [] tools: tools-proxy-7 31457fd1-ba1b-4231-ac7b-5e1df1d7f458 [] tools: tools-proxy-8 24d99fcf-6a44-4321-a653-00f7e839c479 [] tools: tools-puppetdb-2 94166aa7-59e0-4f8c-9bec-e8b018fda855 [] tools: tools-redis-5 5d256217-56b6-4e10-b4a8-422738b6d244 [] tools: tools-redis-6 10457456-a444-49e9-9468-ffbdf5a73703 [] tools: tools-redis-7 5ee8dd81-2224-4a6a-bdbe-b0a498f101e5 [] traffic: traffic-acmechief01 b49ed32f-fac3-4c19-b031-abb3f3bc2e36 [] traffic: traffic-cache-bookworm 4bca356d-86cc-4e10-a7a6-db574cf706b3 [] traffic: traffic-cache-bullseye ab5877a3-5184-485f-9d5a-afec6fa6b4cf [] traffic: traffic-cache-text-bullseye e634e20e-a0c3-4332-a1dc-0a75812a03a8 [] traffic: traffic-cache-upload-bullseye 3c766c63-f92a-486c-900c-1f752619332b [] trove: maps-test-2 b615abe0-9cc2-4cee-a224-c1f0b90e7957 [] trove: quarry-db-02 4b932658-2b4d-4157-b62e-1e73faaed8aa [] trove: quarry-k8s d277044f-a5db-4939-b1e4-5a142116f9d0 [] twl: glitchtip-twl 3927f0a3-d8f8-488e-a00b-27482202e737 [] twl: prod0 56cd3c88-7fcf-4bfb-a192-a86ba04e44a0 [] twl: staging a04ce916-1310-49bf-ac50-e5aca7738a5e [] twl: twl-nfs-1 64e0fcfb-4f8b-4912-ab79-83984205d016 [] usdtest: primary-app 6356dbde-37f8-40fa-ae3e-26c5913702be [] utrs: utrs-alpha 5aed0a3d-c48d-4c36-b9d5-f216de0dbfc1 [] utrs: utrs-database 5203247c-7191-4bde-88d4-7dea51a31920 [] videocuttool: beta-videocuttool-bookworm 565fe5fd-1b53-46bc-a5ec-410ab5e65537 [] video: encoding02 9a9b2610-e1ac-4dda-8309-2dc816f78dab [] video: encoding04 8ab8234a-5e85-4de6-a42c-031c7f05850d [] video: video-dev-bookworm f17b4b49-c1f0-43aa-8cd4-16a07288f704 [] video: video-nfs-1 6aa18f0e-d275-4c05-ab9f-65da1b7a7c3b [] video: video-redis-bookworm 02961b09-1520-4b10-b6d6-6118133cba20 [] videowiki: db-instance2 79f8e042-3bbd-4777-9d9d-509363b436a1 [] videowiki: videowiki-app-1 577a10c0-dfce-4bf4-8452-db0b1c68b248 [] videowiki: videowiki-db-1 2acb2154-ec20-4649-b7bb-6d619676e4ad [] vuessr: prototype2 cdb5e683-ddc7-4714-9d3d-951cf01af9ef [] wcdo: wcdo02 d28c57b4-2c75-4e24-a900-52bcd9c2e402 [] webperformancetest: crux-metrics 241f3730-4e8d-48c9-90f7-8c67e271d273 [] webperformancetest: queue b54d0173-24e7-426a-97db-7d9ca9489009 [] webperf: wikidiff2-demo 8cd85dcf-5ef9-4f6e-8428-e0312ff0ccb0 [] wikiapiary: scripts d2e8dc6a-0d72-4e84-99f1-ba669b12cf09 [] wikibase-registry: redir1 fb6a81ce-0337-4133-8928-47c2b5bb158d [] wikicite: wikicite-refsdb-proc-1 a11237f6-9436-4ad8-a8f9-0af4e020663c [] wikidata-dev: special-new-lexeme-testing 4f62099c-b9bc-4094-8c4f-db435781fdf7 [] wikidata-dev: wikidata-icinga-2024 504cb696-f783-4806-9e95-b1b6c2d88096 [] wikidata-reconciliation: wikidata-reconciliation 692086ce-c855-4f66-8b92-58426516a880 [] wikidocumentaries: hupu2 c13fc11c-3856-4588-96b9-8473878d752d [] wikidumpparse: wikidumpparse-nfs-1 fea0bea6-8b82-434f-849d-b837f98ff9cc [] wikifunctions: piglet 6143e81e-b683-4b0c-ac84-1ecc7a9b0cf0 [] wikifunctions: porky 11c29cdd-fd9b-455a-90d3-9593f17f44b7 [] wikilink: glitchtip 258ed0d6-41a5-4b96-b057-6ca71037b3f7 [] wikimania-mautic: mautic-web d92e8df1-0e53-4607-ab9c-1e59de1fba97 [] wikipeoplestats: wikipeoplestats-prod01 c2f528c1-6668-4619-91f9-b6ef53d3d8a3 [] wikiqlever: puppetmaster 57d26b02-dcb3-447e-b2eb-ac667852932c [] wikiqlever: qlever1 b2c96f28-03bb-4ff4-ab83-f79cc943d31d [] wikisource: ia-upload-test2 f6e22480-0e2c-4cd3-8d67-08250ab32709 [] wikisource: kraken-ocr 62e585db-7abf-4a0a-b076-595cdc0521d1 [] wikisource: ocr-prod02 22aa1613-9328-43f0-a0a9-f9bb48dcc635 [] wikisource: ocr-test02 d9acbf41-02da-4604-ac9d-93e1fe769b66 [] wikisource: wsexport-test 1328e006-5595-417c-864c-f771f027c4a1 [] wikisp: diana 4a74b89d-185e-41ca-82b3-fd121b477c47 [] wikispeech: demo-wiki b010ef0b-475a-4c6d-aa87-3f33efac317d [] wikispeech: tts-dev 24adb48b-ee87-4399-8036-0d0daefcb99d [] wikispore: owmf-test 0b1e12a4-dfff-42ae-84ca-0754959ffbb3 [] wikispore: wiki-nyc-owmf f6074ec9-28eb-4653-ba84-e7d735e3122c [] wikispore: wikispore-prod-02 5a312470-f9be-45ed-848b-6406211b3e82 [] wikispore: wikispore-test-bullseye 30dd5c6d-0444-4ed0-8386-fe232e0588b5 [] wikistats: wikistats-bookworm cf582bc8-0c2f-4ec5-8064-2cad2110b2ad [] wikiwho: wikiwho01 1b0a01e4-e8cb-4a68-ad6c-306b9869db05 [] wlm-it-visual: wlmitvisual01 1931dcad-42dd-4cdd-8246-0d13bdb38ad1 [] wm-bot: wm-bot-pg2-bookworm 63b82d38-3026-408f-8dcc-0ecbd1a3c870 [] wmcs-uptime: test-01 b96b8d02-164b-4d08-9046-1fd2b467a564 [] wmde-techwishes-survey: beta 664e04b3-34b7-4b87-9296-bfb1cbb5f872 [] wmf-research-tools: cloud-modeling dce8372c-fa22-4f73-a0d5-a855e342810e [] wmf-research-tools: dp-pageview-visualization c5688475-f082-460b-9c3f-81ebf1ff78cb [] wmf-research-tools: geo-provenance 3351eecb-e2ca-4889-a5cf-dad5b40add0f [] wmf-research-tools: knowledge-gap-index-tool 948c8c55-6fa8-45f7-9e3e-f3d923f2172c [] wmf-research-tools: mwparserfromhtml 609f5c13-499a-4fad-8b1d-6bf70e6e3b36 [] wmf-research-tools: outlink-model-embeddings 00d1b8e5-1211-4ec1-b975-0a5955fdcee7 [] wmf-research-tools: person-topic-test 9b9ecd8f-1ff1-4dec-a48b-722c15cf0bde [] wmf-research-tools: quality-article 90239a84-0dd4-4414-a6ea-5f72d98846d5 [] wmf-research-tools: quality-bulk-prototype 128d77c8-8c86-42c9-a3d5-6c94c1b8b860 [] wmf-research-tools: reader-embedding 7e5361b4-f195-4316-ada1-a156da2c8b69 [] wmf-research-tools: research-api-template 3ded4501-74a9-4752-9fdb-7d8f03ee9742 [] wmf-research-tools: section-ranker 507f6cad-3313-49c8-b4aa-a7ff7f8bc3bb [] wmf-research-tools: topic-model-prototype c7354620-5323-4888-9fcf-e5d55d263d4d [] wmf-research-tools: vector-search 3684f788-7edd-4e5d-a7aa-240d54f9c4a2 [] wmf-research-tools: wikidata-quality 58ca6c1c-e6b0-4963-8687-f3d81776fab6 [] wmf-research-tools: wiki-region-test 1f42bdbd-6dae-4387-b81b-f79dda18c9a4 [] xtools: xtools-dev06 1edf4d4e-fced-4815-b945-ba94b20f6011
_______________________________________________ Cloud-announce mailing list -- cloud-announce@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.o...
Hi Andrew,
I just added a @reboot line to the crontab of the mediawiki2latex instance. And everything came up well after I rebooted from the command line. Unfortunately I currently cannot log into horizon at the moment, so I cannot issue a hard reset as you requested. But it is perfectly Ok for me if you do it any time you like. I just think it is a good idea to shutdown the machine normally before you hard-reset it in order to avoid any data corruption.
Thanks a lot for your help.
Yours Dirk
On 1/31/25 16:27, Andrew Bogott wrote:
The issue that resulted in partial VM reboots last week[0] (see "VM reboots coming Tuesday, 2024-01-20") turns out to be more widespread, affecting virtually all instances. The primary symptom is that it restricts our ability to drain and maintain WMCS hardware.
In order to resolve the issue, all that's needed is a hard reboot of each VM. Note that a simple in-place reboot (for instance, issued at in the shell of the VM) does NOT resolve the issue. Hard reboots must be performed via Horizon, by selecting 'Hard Reboot Instance' on the Instances panel.
So, please, at your convenience, log into Horizon and hard reboot any of your VMs that appear on the list below. For informal coordination I have also populated an etherpad[1] with the list of affected VMs.
Anything that remains in need of a reboot by late next week (Thursday, February 6th) I will reboot for you. So, if you don't care when/if your VM is rebooted you can ignore this message :)
Thank you! And, sorry for the inconvenience. We have a plan in place[2] which should prevent this issue from re-appearing in the future.
-Andrew
Hello!
I have done the shutdown and reboot as requested -- please let me know if you see any bad effects.
Regarding the Horizon login... this may be a case of https://phabricator.wikimedia.org/T383370, in which case clearing cookies or trying a different browser would work. However, I've discovered a different Horizon login issue while traveling, which is that the network where I'm staying today blocks high-numbered port access; a complete login to Horizon requires access to https://openstack.codfw1dev.wikimediacloud.org:25000 (as part of authentication). So if you have a way to unblock port 25000 that may also solve the problem.
Please let me know if you sort out the Horizon login issue, I'm concerned that the high-port issue might be a widespread issue that I just haven't heard much about.
Thank you!
-Andrew
On 2/2/25 5:33 AM, Dirk Hünniger via Cloud wrote:
Hi Andrew,
I just added a @reboot line to the crontab of the mediawiki2latex instance. And everything came up well after I rebooted from the command line. Unfortunately I currently cannot log into horizon at the moment, so I cannot issue a hard reset as you requested. But it is perfectly Ok for me if you do it any time you like. I just think it is a good idea to shutdown the machine normally before you hard-reset it in order to avoid any data corruption.
Thanks a lot for your help.
Yours Dirk
On 1/31/25 16:27, Andrew Bogott wrote:
The issue that resulted in partial VM reboots last week[0] (see "VM reboots coming Tuesday, 2024-01-20") turns out to be more widespread, affecting virtually all instances. The primary symptom is that it restricts our ability to drain and maintain WMCS hardware.
In order to resolve the issue, all that's needed is a hard reboot of each VM. Note that a simple in-place reboot (for instance, issued at in the shell of the VM) does NOT resolve the issue. Hard reboots must be performed via Horizon, by selecting 'Hard Reboot Instance' on the Instances panel.
So, please, at your convenience, log into Horizon and hard reboot any of your VMs that appear on the list below. For informal coordination I have also populated an etherpad[1] with the list of affected VMs.
Anything that remains in need of a reboot by late next week (Thursday, February 6th) I will reboot for you. So, if you don't care when/if your VM is rebooted you can ignore this message :)
Thank you! And, sorry for the inconvenience. We have a plan in place[2] which should prevent this issue from re-appearing in the future.
-Andrew
Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
Hi Andrew,
the instance mediawiki2latex is up and running. Everything looks good.
When I click on https://openstack.codfw1dev.wikimediacloud.org:25000
I get the following message:
{"versions": {"values": [{"id": "v3.14", "status": "stable", "updated": "2020-04-07T00:00:00Z", "links": [{"rel": "self", "href": "https://openstack.codfw1dev.wikimediacloud.org:25000/v3/%22%7D], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}]}]}}
So I think the port is open. I already tried different browser and clearing cookies but still could not log in. I guess it could be about the Single Sign On, that was recently established. I am also not promoted for the second factor authentication code that I usually have to copy from my mobile phone.
To me being able to login into horizon is not important right now. Since you have done the reboot of mediawiki2latex already.
Yours Dirk
On 2/2/25 16:41, Andrew Bogott wrote:
Hello!
I have done the shutdown and reboot as requested -- please let me know if you see any bad effects.
Regarding the Horizon login... this may be a case of https://phabricator.wikimedia.org/T383370, in which case clearing cookies or trying a different browser would work. However, I've discovered a different Horizon login issue while traveling, which is that the network where I'm staying today blocks high-numbered port access; a complete login to Horizon requires access to https://openstack.codfw1dev.wikimediacloud.org:25000 (as part of authentication). So if you have a way to unblock port 25000 that may also solve the problem.
Please let me know if you sort out the Horizon login issue, I'm concerned that the high-port issue might be a widespread issue that I just haven't heard much about.
Thank you!
-Andrew
On 2/2/25 5:33 AM, Dirk Hünniger via Cloud wrote:
Hi Andrew,
I just added a @reboot line to the crontab of the mediawiki2latex instance. And everything came up well after I rebooted from the command line. Unfortunately I currently cannot log into horizon at the moment, so I cannot issue a hard reset as you requested. But it is perfectly Ok for me if you do it any time you like. I just think it is a good idea to shutdown the machine normally before you hard-reset it in order to avoid any data corruption.
Thanks a lot for your help.
Yours Dirk
On 1/31/25 16:27, Andrew Bogott wrote:
The issue that resulted in partial VM reboots last week[0] (see "VM reboots coming Tuesday, 2024-01-20") turns out to be more widespread, affecting virtually all instances. The primary symptom is that it restricts our ability to drain and maintain WMCS hardware.
In order to resolve the issue, all that's needed is a hard reboot of each VM. Note that a simple in-place reboot (for instance, issued at in the shell of the VM) does NOT resolve the issue. Hard reboots must be performed via Horizon, by selecting 'Hard Reboot Instance' on the Instances panel.
So, please, at your convenience, log into Horizon and hard reboot any of your VMs that appear on the list below. For informal coordination I have also populated an etherpad[1] with the list of affected VMs.
Anything that remains in need of a reboot by late next week (Thursday, February 6th) I will reboot for you. So, if you don't care when/if your VM is rebooted you can ignore this message :)
Thank you! And, sorry for the inconvenience. We have a plan in place[2] which should prevent this issue from re-appearing in the future.
-Andrew
Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
Reminder: I'll be rebooting the remaining VMs over the course of the day today, starting in a minute or two with the bastions.
-Andrew
On 1/31/25 11:27 AM, Andrew Bogott wrote:
The issue that resulted in partial VM reboots last week[0] (see "VM reboots coming Tuesday, 2024-01-20") turns out to be more widespread, affecting virtually all instances. The primary symptom is that it restricts our ability to drain and maintain WMCS hardware.
In order to resolve the issue, all that's needed is a hard reboot of each VM. Note that a simple in-place reboot (for instance, issued at in the shell of the VM) does NOT resolve the issue. Hard reboots must be performed via Horizon, by selecting 'Hard Reboot Instance' on the Instances panel.
So, please, at your convenience, log into Horizon and hard reboot any of your VMs that appear on the list below. For informal coordination I have also populated an etherpad[1] with the list of affected VMs.
Anything that remains in need of a reboot by late next week (Thursday, February 6th) I will reboot for you. So, if you don't care when/if your VM is rebooted you can ignore this message :)
Thank you! And, sorry for the inconvenience. We have a plan in place[2] which should prevent this issue from re-appearing in the future.
-Andrew
[0] https://phabricator.wikimedia.org/T383583
[1] https://etherpad.wikimedia.org/p/rmrebootsforcephmons
[2] https://phabricator.wikimedia.org/T385288
--
[] account-creation-assistance: accounts-appserver6 9d7288ae-ad02-40a6-8976-495d63c040a0 [] account-creation-assistance: accounts-db7 8fcade67-7226-469a-8fac-dda6a08f24d0 [] account-creation-assistance: accounts-oauth-g 3d107f3e-1200-416f-9190-94b0cd9541b0 [] adiutor: testbed ed652064-c031-495b-bb4b-6ace51e2b01a [] adiutor: testbed-vagrant 671aae8e-3a57-47f6-bb44-9275b7b4516b [] analytics: gitlab-docker-runner 2122f5ae-3354-41fa-96e2-eed9efb25a4d [] analytics: growthbook 8e26bc28-39f0-49f8-8cc4-bd6b61411bd3 [] analytics: kdc 32a69b77-1dfc-4e30-a522-883abcaf43f0 [] appservers: j-tst-puppet-01 0c315541-2483-42bf-84f0-5ab8ee592359 [] auditlogging: syslog-client03 c0645c2a-e8d9-48cd-898b-4c466f1be8dc [] auditlogging: syslog-client04 2d6ab6a0-cb97-4ee7-bd8f-e7e8408c5db3 [] auditlogging: syslog-client05 d8ccc10d-3c77-4b30-8179-7498ac8cc17a [] auditlogging: syslog-server03 07e9058c-62c6-4a81-97a1-f36b11603689 [] auditlogging: syslog-server05 8e75ce49-694f-4c68-9d1a-10115e7f791b [] automation-framework: diffscan02 dba77a95-d482-43af-853a-7b72b309dd9e [] bastion: bastion-eqiad1-03 15700b39-5773-4fec-9349-47f16bc457bd [] bastion: bastion-eqiad1-04 20d097f7-3866-4f06-b3d6-cd7d37eb33b5 [] bastion: bastion-restricted-eqiad1-3 47b0da1d-e50a-42c1-8cd9-dc255cc2f1a3 [] bub2: bub2-instance 6cc04f18-9f59-41bb-9bcd-9aaa8bcc2d96 [] capacity-exchange: production d747c697-0895-43a0-93ee-2b5db340bb22 [] catalyst-dev: k3s-node01 6799679b-ac3f-4e07-8bd4-8c23640a5886 [] catalyst-qte: k3s-main 1ad1581e-24e0-4903-bda0-ef74fe5e1b61 [] catalyst-qte: k3s-node01 886df770-6c2a-4f38-8045-34fa5e695a69 [] citefix: citefix 709122ba-c8b5-48ba-9694-fbc89b1cf5f7 [] civicrm-prototype: puppetserver-01 d1a2fc36-1a82-46c1-bd01-d1382e54feee [] cloudinfra: enc-1 e94378df-7a48-4b11-b44a-bf69aaf132bd [] cloudinfra: etcd-discovery-1 926dc59c-3e85-4ae0-a65f-0140fb202a2a [] cloudinfra: mx-out05 a9c57526-aaed-4b93-9cbe-e992c6fc25e5 [] cloudinfra: ntp-03 dd448d31-6a7c-4202-afeb-88ba838c2d85 [] cloudinfra: ntp-04 7773ba13-e882-4780-ae0d-da8a39f26a3a [] cloudinfra: syslog-server-audit01 be0eb3db-cd9a-482a-a31b-40c0babf66ae [] codesearch: codesearch9 683ef4dd-491d-401b-b60e-88e2945e54ea [] collection-alt-renderer: mediawiki2latex 5001eac3-74c2-49a6-b081-195f68464c25 [] commtech: commtech-bot02 a2466526-104b-4f96-ada1-66d28aff7572 [] commtech: commtech-nsfw03 8db5b1a2-506a-4c90-924c-030005488308 [] commtech: pageviews01 0e4a8b46-a264-457b-8711-0bee2345e871 [] copypatrol: copypatrol-backend-prod-01 50d3e893-aee7-4591-9f2b-630107c6a036 [] copypatrol: dev01 df58d6c5-6dae-4b56-8507-dc2e00679834 [] copypatrol: prod01 a870b45e-e184-4833-9117-d49de7263d69 [] cvn: cvn-apache10 f5c20388-c623-4ad4-914c-db565654bf9f [] cvn: cvn-app10 1c7097fe-f721-46ef-928f-2d975d96afb6 [] cyberbot: cyberbot-exec-iabot-01 58bff802-41bc-4ae6-b832-30636d25e0e8 [] cyberbot: cyberbot-exec-iabot-02 eb1784d4-f013-43d6-97d4-2a4ae5f92af3 [] cyberbot: cyberbot-web-iabot-01 af3129d7-3d4a-43b9-8156-53eb636f27a5 [] dashiki: dashiki-03 595b3e7d-13cc-4695-9f60-a86618541cc8 [] dashiki: dashiki-staging-03 525bc74f-950c-427b-90b8-1db821f8d68b [] deployment-prep: deployment-acme-chief05 9bc3b4dd-bea1-47c4-8f54-d4f9c918c19f [] deployment-prep: deployment-acme-chief06 89296f92-ecfe-4fdd-bec9-10ff30b27f7d [] deployment-prep: deployment-bastion bdde8967-2caa-4dd1-ae27-829e808eb64a [] deployment-prep: deployment-cache-text08 7102217c-5d7f-4758-8936-fd72b578f3e8 [] deployment-prep: deployment-cache-upload08 11bc7c03-20a9-4488-86c7-ce4b7faa9283 [] deployment-prep: deployment-changeprop-1 0cedcc02-56cd-42aa-9017-add8a4291793 [] deployment-prep: deployment-cumin-3 2d02fdb4-8068-4d75-9be2-ac2ea715b8c4 [] deployment-prep: deployment-db11 369dfb8f-6311-4f99-85c8-cd9aa293d630 [] deployment-prep: deployment-deploy04 60461772-4536-4055-ab8e-9180c719cf25 [] deployment-prep: deployment-docker-charts01 d6b21175-b9f9-454a-afdc-6e647c40cd02 [] deployment-prep: deployment-docker-citoid02 a092d208-885b-40e4-9d27-998669e9ac11 [] deployment-prep: deployment-docker-cxserver02 b0ef7806-2ed0-4bbc-afbb-175a8ec1d85a [] deployment-prep: deployment-docker-mathoid02 52009c1a-5a0b-4d47-abf0-11efc090e692 [] deployment-prep: deployment-docker-mobileapps02 c6221ce4-48ac-4129-a984-eb7db83b0adb [] deployment-prep: deployment-docker-proton02 918c1510-917c-43f7-9829-d6f669a700c4 [] deployment-prep: deployment-docker-wikifunctions01 52cca4f5-9aeb-4a4f-90d0-b0f2d5e07cb7 [] deployment-prep: deployment-elastic09 11a817d0-3184-47be-a846-316ceb21221b [] deployment-prep: deployment-elastic10 5ef1f5ce-b1d9-48c4-9f85-c727f7d12851 [] deployment-prep: deployment-elastic11 9e94e6d5-a094-4ba3-8594-0cc1d3c4e315 [] deployment-prep: deployment-etcd02 c27abddd-8330-43db-80a9-dafebc76fc79 [] deployment-prep: deployment-etcd05 9e5e712f-eb60-4136-a66d-eb23a56d3349 [] deployment-prep: deployment-eventgate-4 2d94e48b-4e1b-4cf9-8b11-9d0d4bbc8efb [] deployment-prep: deployment-eventstreams-2 beed7e0d-4e7f-446f-a73c-60dce7ecff4f [] deployment-prep: deployment-ircd03 ea50001a-ca9b-44be-833f-2563d13e9779 [] deployment-prep: deployment-kafka-jumbo-5 3ceeda11-82e7-4fd6-9321-24713686318f [] deployment-prep: deployment-kafka-jumbo-8 1b4812a5-db65-4aa2-8adb-6af7333ae950 [] deployment-prep: deployment-kafka-jumbo-9 8c77502b-f887-4974-99b1-387c776605af [] deployment-prep: deployment-kafka-logging01 4f08a0ac-5810-47fd-941c-dbc883811f43 [] deployment-prep: deployment-kafka-main-5 b2eeb153-d31b-4c92-9c06-c64242bda3a0 [] deployment-prep: deployment-kafka-main-6 9178db63-751e-4727-8b9c-4b58f44cd214 [] deployment-prep: deployment-maps-master02 e170e0e7-226b-476e-9df0-4a48a53f4366 [] deployment-prep: deployment-memc11 0776b1c1-956f-46f6-a909-dca50a3d2f28 [] deployment-prep: deployment-memc12 f019dd93-b279-419d-a9fd-36e1256150fc [] deployment-prep: deployment-memc13 77aaac97-8184-4ab1-a8a4-947fca686420 [] deployment-prep: deployment-ms-be07 643b1616-eede-4859-8917-e10b4057126b [] deployment-prep: deployment-ms-be08 20f22f9d-5734-49b9-8a19-9908e588c92f [] deployment-prep: deployment-ms-fe04 7e6b3fec-b97a-4a32-b5c6-279244f78739 [] deployment-prep: deployment-mwlog02 56bc21a6-4fd4-4ea8-9cb1-2c9d9f170c15 [] deployment-prep: deployment-mwmaint03 69da7d16-942e-45d7-afbd-f01eb1d88d9f [] deployment-prep: deployment-mx03 8eb272dc-8006-4e93-a966-5035809324d9 [] deployment-prep: deployment-puppetdb05 7bd079eb-82f6-453a-9bf4-ddcf68057de3 [] deployment-prep: deployment-puppetserver-1 3192a8d0-0fda-4d24-a1b7-2e2dc20fbe0e [] deployment-prep: deployment-push-notifications02 c1994a14-51da-434a-941b-69e14dbcdaf9 [] deployment-prep: deployment-rdb01 0df5294e-2b0c-4b32-aa01-9774da28b01f [] deployment-prep: deployment-restbase05 0f8ff3ed-24ad-4dc8-ac30-6423901a66f1 [] deployment-prep: deployment-schema-3 b4da0c29-b7bb-40a8-9fbf-d54dc72d8257 [] deployment-prep: deployment-sessionstore06 716ce882-8593-4129-90d7-16adb99ae3d1 [] deployment-prep: deployment-shellbox01 1c094211-c3c9-43ef-8bd9-8f2255d85e5a [] deployment-prep: deployment-shellbox-video 6789abfe-3a7f-4d3a-bf32-0b1030c8be4a [] deployment-prep: deployment-urldownloader04 1d187238-9307-440f-b63f-3c3b6a34e239 [] deployment-prep: deployment-webperf21 ba5ee9c6-08e0-474f-8049-d31ec8bed047 [] deployment-prep: deployment-webperf22 a678753b-570c-47d9-86cf-bad88ab20bc6 [] deployment-prep: deployment-wikifeeds02 36458df2-70be-4666-aa09-eafef6b38fc2 [] deployment-prep: deployment-zookeeper-3 5794b8a9-a29f-46e3-8058-52188bade672 [] devportal: devportal-demo01 3fa16445-55ae-458c-a229-f3dd019de2af [] devtools: codesearch 5fca62da-0a71-4856-b58d-9566bdb52d14 [] devtools: deploy-1006 6c97c169-54f8-419b-a747-cd4e3b47f868 [] devtools: etherpad-bookworm 604af9b2-65f1-45bc-943a-52bed0012491 [] devtools: gitlab-runner-1002 7af5196c-795c-4d1a-bb11-15b3506cd16f [] devtools: gitlab-runner-1005 9b170998-ef69-4a3c-aa0f-35855107f691 [] devtools: phabricator-bullseye c2b2b2a1-140b-402f-be7a-50749c78bfc7 [] devtools: puppetmaster-1003 b7f828a5-8269-40f4-87bf-a78b3ce5f147 [] devtools: vrts-1003 a77ff416-995a-424f-a8bf-c7261e93b162 [] discourse: discuss-space 46e675f2-9115-4443-a91f-5c50b7850417 [] discourse: discuss-space-new ca3a1b1f-2a36-4e6f-83ad-d8bbb49b5862 [] download: download-03 02bcf69a-f8ae-4a31-b2ec-cd8e53193e65 [] duct: eb 23a8d36e-d5c5-4264-9d7b-a87d86bb99ac [] dumps: dumps-nfs-1 4761bc85-b4c9-4a88-ac13-8205a79f43d9 [] dwl: irc-bullseye 596d0df6-714c-4bc2-9049-1a459c1360ac [] dwl: taxonbota3 85578a0d-caad-40b4-8b5e-2126c220105b [] dwl: taxonkatbot3 60eb9364-03ca-4b2e-adef-3a155e03445c [] entity-detection: maps-workspace 7e758ab0-b51e-47b5-92ba-606bf19771ec [] etytree: etytree-b acfe0102-959d-46d5-bde9-3e03f8814df6 [] eventmetrics: prod03 5ebd0672-64c9-4dba-a687-b029767433ab [] extdist: extdist-06 472c3694-e68a-4c36-bce8-6517ab3bcc73 [] fastcci: fastcci-master 61e2acff-66b0-4f0d-a0cd-ce737feca69a [] fastcci: fastcci-worker1 b9352e93-fb87-4e32-b30c-55e475ade73a [] fastcci: fastcci-worker2 c3a88537-8f70-44e3-8f0e-9c0fe592429e [] fa-wp: chemistoid 38c1a830-be37-4472-8310-45f5ad5b277d [] fa-wp: tofawiki03 feaa04b8-27e0-4f78-abd9-a74a18fdecfd [] fr-tech-dev: payments e759a967-e322-4e3f-93c3-f546381a7b8d [] gitlab-runners: runner-1021 7ee0fcc7-43ea-4321-878e-a6922ac1236d [] gitlab-runners: runner-1022 cb4a87e4-264e-4c8f-8197-3efff54346de [] gitlab-runners: runner-1023 56e55a31-8d32-455e-b650-b7194e71d2fd [] gitlab-runners: runner-1024 921ddea1-0288-467f-8458-56860f383b93 [] gitlab-runners: runner-1025 6b8137a7-243e-4319-9e80-735095b8c30c [] gitlab-runners: runner-1026 329fc8c9-c51a-406d-9bad-9d88013b3e4b [] gitlab-runners: runner-1027 c3bdc48c-9ecf-48b3-bad9-67b6fc26cc77 [] gitlab-runners: runner-1028 b5b34987-d4ce-4f2b-a96b-b042d6e65623 [] gitlab-runners: runner-1029 39693d07-dad5-4ad0-a49c-e6fa468ae53b [] gitlab-runners: runner-1030 a9902e35-173f-4dbd-bce3-076e552bcd65 [] glams: glams-testing f9d47b22-bf9b-491c-b48d-dbbb56c487a9 [] glamwikidashboard: serviceapp 7aa37f8a-9e13-4ef2-8362-af49656c7e2d [] glamwikidashboard: webapp e78526c2-32c0-49f9-9b03-1ec49fff26be [] globaleducation: impact-visualizer-tool 7009df57-c7ec-45a7-a65e-da69eb018392 [] globaleducation: p-and-e-dashboard-redis b3c07919-318d-45a8-9c29-88cb3a59f56f [] globaleducation: peony-sidekiq-3 2962a54e-eaa3-4a46-a5ba-dc4773bf2fdd [] globaleducation: peony-sidekiq a78c5ea3-0c0e-4589-8ac9-8e24023a314f [] google-api-proxy: google-api-proxy-04 6bf5e226-050f-4307-8550-a3ef7e960ea3 [] hoiscript: Peony 6b32f902-7d6f-4ff3-a0ac-20f264d0c5de [] huggle: huggle-nfs-1 b74e3f99-5aee-4702-b501-7cc93cd8f4fc [] huggle: xmlrcs2 e840cdbd-349e-42f8-bf83-12507be6ebeb [] huma: web01 583ad29e-f9b3-47e7-8b83-0440bf72d7fa [] huma: worker01 cb20f298-f78e-4ae6-8752-9c32fb79a7cf [] iiab: iiab-med 79f31d08-d21f-4ef6-9cc0-7bf495cf28ed [] imagebulk: imagebulk 21cf9a6d-6abd-44e5-b12d-08b611643466 [] impactvisualizer: impactvisualizer 207f3ae3-9627-408a-8c07-bd6c78a56c71 [] impactvisualizer: replica-revision-tools 133429b1-2a3d-477a-ba3a-006939ba138f [] incubator: incubator-1 d7c826ce-1b23-4b63-bf2c-280b9396644b [] integration: integration-agent-docker-1040 42d3d9a4-5d2d-433f-8b3f-c8fdb2a24a3c [] integration: integration-agent-docker-1041 db4ae352-bbc3-450e-b4bb-1bb95f186072 [] integration: integration-agent-docker-1042 426d1a55-e8ce-4332-9e3c-3a52f47b36af [] integration: integration-agent-docker-1043 4c87de46-8ea5-476f-812f-4e627dbd6671 [] integration: integration-agent-docker-1044 88c795b4-72ee-4b82-b399-04d3457f2ce1 [] integration: integration-agent-docker-1045 d0e8fb1b-ec89-4934-86cd-6118db1219bc [] integration: integration-agent-docker-1046 270b6533-dc99-4e5d-a642-c61138b11891 [] integration: integration-agent-docker-1047 4c9a5ffd-f322-4b14-8f71-a87a18f54b09 [] integration: integration-agent-docker-1048 ab9080c3-fbcd-4e5f-90f4-d9669b9658de [] integration: integration-agent-docker-1049 80924e5c-f001-49f4-8043-6623af917151 [] integration: integration-agent-docker-1050 cfdbd9d9-6cbe-4516-89dc-b19f0e210f2d [] integration: integration-agent-docker-1051 ecbc72ca-601a-4ee5-b590-10606d64886a [] integration: integration-agent-docker-1052 7f7fe96a-c25f-4116-a79a-c77f1a88ca33 [] integration: integration-agent-docker-1053 14789ac1-bc06-4677-9bb0-66c16c887427 [] integration: integration-agent-docker-1054 702b9925-099e-4ecc-9d67-35865affa64d [] integration: integration-agent-docker-1055 6616fcbf-a49e-4e03-b735-84d31b535c08 [] integration: integration-agent-docker-1056 cc9e0883-0282-4f94-8e39-9c5ebe5ef805 [] integration: integration-agent-docker-1057 5c67ffe7-9c5b-4790-92b8-ca8299613ac0 [] integration: integration-agent-pkgbuilder-1003 a08a80f5-1b45-4aba-8945-cfc080a4c940 [] integration: integration-agent-pkgbuilder-1004 277f302a-b2fa-4635-8c88-0f8d6e9971f4 [] integration: integration-agent-puppet-docker-1003 96cf9ddc-daa3-4c9f-8c21-cdd58e95973e [] integration: integration-agent-qemu-1003 aa7075c2-f321-4817-a53a-ce553ecb6904 [] integration: integration-cumin 80f8822f-c0d9-427a-b743-b9185f58c0eb [] integration: integration-puppetserver-01 a2ce4233-268a-45ac-b542-ab6fc4e660a6 [] ipoidopensearch: ipoidopensearch 41b83b6d-344e-4b13-b0d7-a8a7e8b72dfe [] isa: m2c f8b9e023-248d-49a7-bf38-2dd448c48d33 [] k8splay: abogott-control-1 e470f1de-2f84-4068-981e-4af29685f04a [] k8splay: abogott-worker-2 e28a75a0-daf2-453b-8efc-fe88ae819470 [] k8splay: dzahn-bookworm 5ec028f1-3fcb-4774-9984-b3bc3f7a163d [] language: cxserver 9e0f89e0-a177-4a2a-a677-32e888b6e37f [] language: language-cx 9a857e61-052c-4025-a4f7-e42c31edd3a3 [] language: language-mleb-legacy a44b9b31-a11f-4594-b9f2-170ba2bfb4eb [] language: language-mleb-master 1c0ac8ab-1c86-4ad6-a65f-7802984c5ae4 [] language: language-sx2 a63a8815-e397-4b8d-9ec6-98fc3f380ef7 [] language: mleb-stable 78fae2d3-dc53-4242-9e18-3edacc02786f [] language: recommend 3e3fac3d-be77-4f50-8b18-9d7616c8fa58 [] language: translate 15ed27d5-cc31-462a-97b5-68601d3e88d7 [] ldap-dev: dcl-dev1 80bac244-e231-4390-9de9-79dc71ac0827 [] library-upgrader: libup-runner08 9a99163d-35a6-4596-a559-9014e72e0379 [] library-upgrader: libup-web02 53f8c1f4-0c85-407d-806f-47b585852cd7 [] linkwatcher: coibot-2 e08f00f6-5a4c-4981-8ee1-75e773204742 [] linkwatcher: linkwatcher-db 8ff173eb-163e-47bf-a429-a114d7affce8 [] linkwatcher: liwa3-2 ce9426cb-923e-4af7-a703-f81d9b3e6149 [] linkwatcher: xlinkbot-2 168548e6-4a9f-4faa-a109-17c90b6cb421 [] logging: logging-alert-01 a1141332-0400-4a87-ad7e-24564676e48a [] logging: logging-logstash-02 15a048f8-ab89-43ff-80be-63cf5b5691fa [] logging: logging-logstash-03 14abc0a9-ef08-4e73-a54b-8d91c1bbc752 [] logging: logging-opensearch-ssd-03 f502126e-1aa7-4f02-a047-bed58688e30c [] lta-tracker: bots2 27acf162-300a-4d92-a293-5b2d34e5c4b5 [] lta-tracker: prod 2b56c575-08a5-4def-87cb-bee5bd43e4f9 [] mailman: mailman1 187649b3-0313-46d4-bbdf-34b08d5c3d48 [] maps: maps-warper4 d3521b71-1096-4fb5-8a34-33ae6191965c [] mariadbtest: cumin-1 3776d295-b63e-4766-ac60-a7027821d11a [] mariadbtest: db-1 bafb528e-0490-4ce6-9505-a0526f95782f [] mariadbtest: db-2 68aa2f4b-a14d-4d82-b1f4-e6a0f1ec2d1e [] mariadbtest: mariadbcompiler e50dcf0c-9bdb-4a45-b337-9e46e3ba2e42 [] mariadbtest: test-with bfd02baa-da18-41c0-b06c-f6028bb5440e [] mariadbtest: test-wo 1cd86198-ce89-4c46-8218-20313d467f4b [] masz: web02 b43ae478-d62c-43c1-ad8f-eb09ca6f9f86 [] math: math24 0235bb05-e38c-44f6-93ac-13532c32ffff [] math: math-docker-2 372adf1a-3379-4fac-b2f3-e05c48ba869e [] matrix: synapse01 1fa9f0d9-42e8-4273-bdb1-a7d49998c13f [] mdwikioffline: mdwiki 60dff349-faea-4de9-b4d0-b776f43f2e5a [] media-streaming: media-streaming de2f0e88-ca81-4345-b445-9036de1740fb [] metricsinfra: metricsinfra-grafana-1 ae59a32d-c913-4b75-bf08-e598a43a2892 [] metricsinfra: metricsinfra-haproxy-2 46dfccb9-f74b-4b99-934d-f724b1c6a63b [] metricsinfra: metricsinfra-prometheus-2 7cb371bb-a53a-4e65-a1cf-f1a8264a9166 [] metricsinfra: metricsinfra-thanos-fe-1 b67ceb26-97c1-402e-9c7a-7c6d52b2f119 [] mwcli: gitlab-runner-addshore-1016 37e23659-4516-4bd8-a9be-4cc55def5560 [] mwstake: mwstake 77e0ad73-61dd-4a9d-a0c8-fc89e30a858b [] netops-clab: netops-clab de3c8ae8-9e09-4ece-9c19-07c39365b239 [] o11y: deployment-server-phi 5899163e-bbfb-474e-8a9a-2dae5006de22 [] o11y: phi-alert-01 98e6d07d-dd64-4083-8fab-f180e11dc402 [] o11y: phi-arclamp-01 cd686956-e65a-491f-9fc0-7bb216c61fa9 [] o11y: phi-grafana-01 8d5dba20-52d6-47f2-b26e-73531401d813 [] o11y: phi-kafka-01 e0878f27-bcff-48b9-a9c3-8af797e92622 [] o11y: phi-kafkamon-01 1b7de1cd-0857-425a-84d2-a4af29f6e982 [] o11y: phi-lb-01 562d3627-c869-4255-8818-90cc6d059ad4 [] o11y: phi-logs-be-01 940e0318-604f-4a88-ba11-e0f8ee0e0cdc [] o11y: phi-logs-fe-01 0c57ca62-098d-4f99-900a-d6cc3d88cb6d [] o11y: phi-mwlog-01 1802b911-6aa9-4142-b54d-a5b542f504e8 [] o11y: phi-pki-01 6383bd75-63cf-4703-b3a7-acb84a4cb93d [] o11y: phi-prometheus-01 d7ec787a-55f4-42a0-8d03-00efbccc8d21 [] o11y: phi-prometheus-02 871a7169-2748-41ec-a3e9-6b13a81ec526 [] o11y: phi-puppet-01 e964291c-48d8-4d44-89dd-24e4d49d7f97 [] o11y: phi-puppetdb-01 64149a5f-eeea-4978-9921-17de05c88dd6 [] o11y: phi-syslog-01 5393926d-b5cb-4132-9bf1-06f0a89d1d9c [] o11y: phi-titan-01 2d729afd-8290-4744-ab1d-57297ca6bbf6 [] o11y: phi-webperf-01 e5ed77fc-1077-4129-80b3-120c570c8edd [] ocrtoy: ocrtoy-web-bullseye 347d3b08-cd71-46f2-8028-ec5581719114 [] onfire: klaxon-keith 44ca15de-5737-4daf-8418-bb0068e106ed [] openvas: openvasv1 a236086b-56a7-42f5-ad04-a99dd10cafaa [] osmit: osmit-estratti 457f53c3-fbce-4b3d-ab72-bbb08c2716b7 [] osmit: osmit-umap2 1e476394-4ed9-439e-aed4-053486653df7 [] owidm: owidm-dev-1 c1f67eec-fff5-4ea1-9c1a-450b2702c206 [] owidm: owidm-prod-1 93eb7e45-7bd5-461b-93b1-e0637e947de0 [] packaging: cas-build ebfaa640-88ba-4bdc-a7d6-f35cd80a14fc [] packaging: packager-etherpad01 05a87de9-51e4-441c-bed8-03372ac90231 [] paws: bastion 9c7625e2-2dd0-4289-a934-f617fbb7999b [] paws: paws-nfs-1 1b001a01-0a37-4d81-9dd6-1339b6391f37 [] pixel: beta 2f8314c8-6d73-4462-ae6b-1d8ea7fc5e68 [] pixel: production 7af72145-4bdb-4a19-b960-f428eedfcb7b [] pki: pki-db 3788dd45-8e96-471f-a4f5-4c788c12601b [] pki: pki-intermediate f56e0ae1-b26f-43d8-8e9e-2e7fb424ce0b [] pki: pki-pm 22e7671e-c144-4508-afe0-f0e24603fa69 [] pki: pki-proxy 7081de14-d7f5-44ca-b82e-6b99af667696 [] pki: pki-puppetserver-1 1e528d51-9710-4047-ba0b-a58e9e53ae35 [] pki: pki-root edfc4cbd-dd02-41f2-8732-c48ef55e9c67 [] pki: pki-test01 3e6c31d7-b4db-4a5f-a610-a74d0013f631 [] pluggableauth: survey 8cf4a071-f4c1-4312-b635-926e0742ae5c [] pluggableauth: wikifarm 9f2a71c3-5683-4c30-b28d-617396fc3e3d [] procbot: bastion 102ecf69-5699-49bc-92ec-17b4823cbcb0 [] project-proxy: maps-proxy-03 cbb613c3-9480-4e96-a83e-90560205657f [] project-proxy: maps-proxy-04 d63dd9dc-a2f9-434e-b315-5d9d1d4f7c9a [] project-proxy: project-proxy-acme-chief-02 fd43cf27-3b19-4035-8047-7529c69b3e53 [] project-proxy: project-proxy-puppetserver-1 e8a89ddc-9129-42c4-bac5-1034de4ad90d [] project-proxy: proxy-04 24219963-e446-4c89-8ad7-30e8c4c69168 [] puppet-diffs: pcc-worker1004 a0d371be-dafb-4228-89c0-fc25b8d7430e [] puppet-diffs: pcc-worker1006 f9c9f01d-b1fd-4e64-a521-522fc767e604 [] puppet-diffs: pcc-worker1009 b02c4866-e0da-4029-b7ca-73252b5492c3 [] puppet-diffs: pcc-worker1014 72986a86-4401-4d7c-9730-5a71d7d22624 [] puppet-diffs: pcc-worker1015 a4871202-b93d-47e5-8b5d-eba47b274c5f [] puppet-diffs: pcc-worker1016 7589c350-58c4-4064-ad91-418e44ef5432 [] qrank: webserver 6efd77f9-2725-4a22-a898-625e3123bb0b [] quarry: k8s-nfs c18ab12b-fa18-4a29-a074-40535930f6b0 [] quarry: quarry-127a-g4ndvpkr5sro-node-0 8b68b650-19d3-421c-975f-7f98a47b5d2c [] quarry: quarry-127a-g4ndvpkr5sro-node-1 c86b3288-aff1-40a5-b619-887bdcefca44 [] quarry: quarry-bastion bec16bd4-988d-46b9-8b73-d044a4392a92 [] reading-web-staging: simple-summary-server 42121024-526f-497f-9001-12925c59fb1e [] reading-web-staging: skin24 3196c742-86ba-4308-b5a1-04d4bace5691 [] reading-web-staging: tarsier d99bd61e-094e-4a97-b856-96cad37fa905 [] reading-web-staging: wikitalky a43d6054-daf8-4049-b445-4952c98ddced [] recommendation-api: article-desc-deprecation 7bbd7cb5-8784-4091-aa0b-3058303695c1 [] recommendation-api: gapfinder-deprecation 1fdcc8e1-2274-4232-87d3-924a60800744 [] recommendation-api: search-referrals 24d2da5a-36b2-46c7-9217-9b5dc0d2a022 [] redirects: redirects-nginx03 dfbc6968-d803-4885-a34e-ba53cda922e7 [] research-collaborations-api: citationq b038356d-53f2-456d-9654-564261db0395 [] research-collaborations-api: edit-types 5f26c230-c05a-4f3c-ad4c-d4aae19f1c5e [] research-collaborations-api: fact-checkign-new f2f0790b-948d-4523-bfa5-a332da588aec [] research-collaborations-api: kitools 7da824f3-c9f3-4460-b9b2-2a894268a7ca [] research-collaborations-api: wikinav-bookworm 774a256b-2b8f-4384-93e7-53d368ff9a96 [] search: airflow 69825ca0-1fe2-4960-9c1f-24b1425ae85d [] search: cirrus-integ03 28e597a1-769e-48df-be12-73dcf53ea658 [] search: opensearch01 68a499c0-54a6-445c-af12-8e5265b10bee [] security-tools: security-dashboard 4bd8f0ba-26e3-440f-a972-f3218f2be97f [] signwriting: signwriting-swis-2022 e4bb9712-5ac3-4e57-8aa7-58754aa5216c [] signwriting: signwriting-swserver-2022 c778195a-b439-4c32-843b-40db2c6404b1 [] soweego: soweego b958511a-10cb-4e62-bdbb-6da5013dd62f [] spacemedia: glitchtip 3717d09b-ca52-4b92-8126-3720f4930251 [] spi-tools: bullseye ffae23a6-47bf-4c76-bfd2-d6b7d2cf495b [] spi-tools: opensearch2 e56b810b-6a74-41cf-8aa0-5a5fddcb2014 [] statanalyser: statanalyser f78be803-6d05-43cc-9b40-a8748ff56204 [] striker: striker-docker-02 5acbe429-549d-4d2a-889a-ed1bb841ee43 [] suggestbot: suggestbot-02 f5ff0e3c-82ea-428d-a263-9044a0419249 [] superset: superset-bastion 05a0a9b8-d56d-4117-bd40-f747869397f0 [] swift: dp-cssndr-puppet-01 2848e015-24ef-4061-b7a2-41401cffbf0a [] terraform: tf-puppetserver-1 99c2a647-36ef-40d5-9c21-8a20f437daff [] terraform: tf-registry-2 7946a9ea-136a-4223-8996-2e3a0334e3d8 [] testlabs: abogott-puppetclient1 6d344a91-80e8-48b0-b334-9bb77b153116 [] testlabs: abogott-puppetserver 74bca50a-fafc-483f-ad29-a68b3960d73a [] testlabs: kolla-test-3 92c89dd1-0cd3-4ca7-a7e3-09d4c3673814 [] testlabs: rook-tf 3ceb9b8d-a544-48f8-86ae-986ea938d7ed [] testlabs: slavina-k8s-workshop 8fc0eed6-8f20-4916-a81d-fb82b2d704cf [] testlabs: testlabs-nfs-1 14695d95-e47d-4b27-9023-20a38f07d11f [] text-to-speech: monitor 35d1d34c-7bc7-443b-a0c6-922de7368549 [] teyora: teyora 68b3e7b9-11ff-4223-a20b-dd19664b88f5 [] teyora: ty-analytics 4ae3bca1-7ea7-489c-9433-d0ebc2652e18 [] tofuinfratest: tf-bastion 3c385a09-9d18-42cc-965b-5d6cee9d005c [] toolsbeta: toolsbeta-bastion-6 33e430fb-f679-41ce-8ee6-4db1e812454c [] toolsbeta: toolsbeta-cumin-1 868d3dca-3e5c-4089-89a9-2c7e756c3e31 [] toolsbeta: toolsbeta-docker-imagebuilder-2 c07c4b31-fa95-4f18-927b-a17cf1003ae5 [] toolsbeta: toolsbeta-harbor-1 caf923d3-a42b-49b8-b861-f20996ab4ab9 [] toolsbeta: toolsbeta-legacy-redirector-2 ca99f9d1-a806-4eee-ab96-e4712abc611b [] toolsbeta: toolsbeta-mail-2 1cd98b82-6c77-465d-8dcd-f0c20e6d7789 [] toolsbeta: toolsbeta-prometheus-1 d110734e-aa20-404a-9f1e-a58c266f9fc5 [] toolsbeta: toolsbeta-proxy-5 1870a647-224b-4b60-b16f-64714e15cf7b [] toolsbeta: toolsbeta-proxy-6 64e17166-af72-4aa6-a0b7-544a12569655 [] toolsbeta: toolsbeta-puppetdb-03 56691327-d836-4f78-a5fd-93bdef49bfa9 [] toolsbeta: toolsbeta-redis-4 44007ae1-c81b-4cc8-b733-536daf5c14e6 [] toolsbeta: toolsbeta-redis-5 f794f6fd-59fa-4fd1-9179-f348e2a0c9ab [] toolsbeta: toolsbeta-redis-6 019519e7-2169-4615-8985-424af8d0e0d7 [] toolsbeta: toolsbeta-static-2 13f3646e-000e-4cbb-a643-54f73eecae4e [] toolsbeta: toolsbeta-test-k8s-control-10 7d35bbd7-cc41-48b3-87a5-35604980c40d [] toolsbeta: toolsbeta-test-k8s-control-11 44513a1b-807b-47cb-9f47-8ae343fe5f82 [] toolsbeta: toolsbeta-test-k8s-control-12 652f13e1-1c7f-46cd-9ca1-d7bebdc5abba [] toolsbeta: toolsbeta-test-k8s-haproxy-5 b65d430f-8873-48b4-9a08-681ab0552739 [] toolsbeta: toolsbeta-test-k8s-haproxy-6 8fbe70cd-a471-4f8b-b3e8-0136c20e8a3e [] toolsbeta: toolsbeta-test-k8s-ingress-10 5d4584f9-559c-4d3e-9af5-6542200b5158 [] toolsbeta: toolsbeta-test-k8s-ingress-11 5130e84c-f8fc-4e02-8c93-9412dab3f630 [] toolsbeta: toolsbeta-test-k8s-ingress-9 72561491-3b1e-437c-878d-e1db110728f3 [] toolsbeta: toolsbeta-test-k8s-worker-12 e5f37100-ebe9-4238-98ae-4869237baf36 [] toolsbeta: toolsbeta-test-k8s-worker-nfs-10 8a0ca656-8179-4544-9154-7d11b8941899 [] toolsbeta: toolsbeta-test-k8s-worker-nfs-5 60e69fe7-f6dd-48d4-85f6-3c13a946c2da [] toolsbeta: toolsbeta-test-k8s-worker-nfs-7 cd12b691-19cb-4e28-a1f1-433e53378f8e [] toolsbeta: toolsbeta-test-k8s-worker-nfs-8 93848139-6952-4edc-a927-9f207fab6b35 [] tools: tools-acme-chief-3 452dc8d3-6ee0-412c-90ba-66f21f9d09c1 [] tools: tools-acme-chief-4 2cfcfc76-8443-4b04-b129-ba381e6fbf22 [] tools: tools-checker-5 f0936015-b1ef-4b8c-ac6e-74766ed882ce [] tools: tools-cumin-1 5c8b0a31-4074-4629-a996-b4c156cdaac6 [] tools: tools-db-4 230c30f1-37a1-42a7-a829-4369448b73f5 [] tools: tools-db-5 51a560c1-d6dd-42b9-bec9-bd2b2504f2d8 [] tools: tools-harbor-1 243f3a30-eb5f-4726-9862-bcb1a35e267b [] tools: tools-k8s-control-7 67dbc73c-a7f8-40e4-96ef-ee838766ecff [] tools: tools-k8s-control-8 1d0839d5-0a11-4ed2-83b8-b0c15c62a5ae [] tools: tools-k8s-control-9 c195fc23-61a1-4093-9c9c-02f72e793380 [] tools: tools-k8s-haproxy-5 1de296d9-e469-47ab-a1ce-eb08c38d05b5 [] tools: tools-k8s-haproxy-6 469f0822-42df-4268-9e75-32e6973bf86e [] tools: tools-k8s-ingress-7 a96f04aa-9200-4b1a-86a8-3bfb836843bc [] tools: tools-k8s-ingress-8 54a3d128-ddfc-4346-bf84-57df94409fd7 [] tools: tools-k8s-ingress-9 959a47f2-87b1-43f3-bce5-f302fa3d1a3a [] tools: tools-k8s-worker-102 1bac7184-6f29-43b2-a686-1eba60ffbfed [] tools: tools-k8s-worker-103 1f64b6ab-4318-48e5-b885-c80151d4ab8f [] tools: tools-k8s-worker-105 0a3ebade-abf5-496e-88d5-ca29ceb52965 [] tools: tools-k8s-worker-106 eec8ebe0-64bf-40c1-b470-8e017b8e20e9 [] tools: tools-k8s-worker-107 da629018-4983-441c-b83d-7ecf90b6dd48 [] tools: tools-k8s-worker-108 3019cf57-1301-46f0-97f7-234b006e8be6 [] tools: tools-k8s-worker-nfs-10 7a497017-e462-473b-bdf5-5798f08b3179 [] tools: tools-k8s-worker-nfs-11 91080d53-f5a6-4d96-b622-722405de8982 [] tools: tools-k8s-worker-nfs-12 81a33d19-55cf-407c-b888-5b06438a095f [] tools: tools-k8s-worker-nfs-13 8155a7ef-5889-421a-a1fe-d43c00d60a0e [] tools: tools-k8s-worker-nfs-14 524da4e8-c4f6-4ed1-b18a-5bbcc51900ac [] tools: tools-k8s-worker-nfs-17 00aaf7e9-b8bb-489f-965a-bb99ecc050e2 [] tools: tools-k8s-worker-nfs-1 7b355092-f5e4-4be0-af94-b1307983c8e9 [] tools: tools-k8s-worker-nfs-24 77140f83-1a12-43b7-9e47-0e779503a525 [] tools: tools-k8s-worker-nfs-32 cab36feb-ef28-4f5f-88db-83270d181138 [] tools: tools-k8s-worker-nfs-3 5312e2f8-035c-464d-903e-79670197ab8f [] tools: tools-k8s-worker-nfs-39 8fb68878-67fe-4118-bb08-2f88b6e5f095 [] tools: tools-k8s-worker-nfs-40 728ae870-9430-4cd9-9fe3-d8138b97169b [] tools: tools-k8s-worker-nfs-43 18dc79d9-df55-4bf6-8bcf-5f40fb22aecd [] tools: tools-k8s-worker-nfs-45 62a59163-64c4-4295-bac0-f7f7aaabadf7 [] tools: tools-k8s-worker-nfs-46 1842d2b7-ca46-4c16-857b-8d588d5df6c6 [] tools: tools-k8s-worker-nfs-47 1423479b-afaa-4e16-b970-f4fb81db76f0 [] tools: tools-k8s-worker-nfs-50 3c99f41d-8a93-4347-8bc1-3ad5c63f55b1 [] tools: tools-k8s-worker-nfs-54 761089d7-d849-4b5d-8f8e-7368738f70dd [] tools: tools-k8s-worker-nfs-66 f9015696-9010-4a1c-a7cf-9bc8e8098df4 [] tools: tools-k8s-worker-nfs-71 e06b79a7-990b-4142-844e-5e826f3d4569 [] tools: tools-k8s-worker-nfs-74 11266010-8581-4750-9375-03fd1efb9e98 [] tools: tools-k8s-worker-nfs-9 33bfa2d5-31c5-4fe0-9a77-aaefd42eef1d [] tools: tools-legacy-redirector-2 7db5fc0e-34d2-495e-8045-8602639fe2bf [] tools: tools-mail-4 1484e4c9-0e93-4476-8882-56a7da42ecaf [] tools: tools-package-builder-04 62045cf9-59ed-44b9-a268-1c9f171b5aae [] tools: tools-proxy-7 31457fd1-ba1b-4231-ac7b-5e1df1d7f458 [] tools: tools-proxy-8 24d99fcf-6a44-4321-a653-00f7e839c479 [] tools: tools-puppetdb-2 94166aa7-59e0-4f8c-9bec-e8b018fda855 [] tools: tools-redis-5 5d256217-56b6-4e10-b4a8-422738b6d244 [] tools: tools-redis-6 10457456-a444-49e9-9468-ffbdf5a73703 [] tools: tools-redis-7 5ee8dd81-2224-4a6a-bdbe-b0a498f101e5 [] traffic: traffic-acmechief01 b49ed32f-fac3-4c19-b031-abb3f3bc2e36 [] traffic: traffic-cache-bookworm 4bca356d-86cc-4e10-a7a6-db574cf706b3 [] traffic: traffic-cache-bullseye ab5877a3-5184-485f-9d5a-afec6fa6b4cf [] traffic: traffic-cache-text-bullseye e634e20e-a0c3-4332-a1dc-0a75812a03a8 [] traffic: traffic-cache-upload-bullseye 3c766c63-f92a-486c-900c-1f752619332b [] trove: maps-test-2 b615abe0-9cc2-4cee-a224-c1f0b90e7957 [] trove: quarry-db-02 4b932658-2b4d-4157-b62e-1e73faaed8aa [] trove: quarry-k8s d277044f-a5db-4939-b1e4-5a142116f9d0 [] twl: glitchtip-twl 3927f0a3-d8f8-488e-a00b-27482202e737 [] twl: prod0 56cd3c88-7fcf-4bfb-a192-a86ba04e44a0 [] twl: staging a04ce916-1310-49bf-ac50-e5aca7738a5e [] twl: twl-nfs-1 64e0fcfb-4f8b-4912-ab79-83984205d016 [] usdtest: primary-app 6356dbde-37f8-40fa-ae3e-26c5913702be [] utrs: utrs-alpha 5aed0a3d-c48d-4c36-b9d5-f216de0dbfc1 [] utrs: utrs-database 5203247c-7191-4bde-88d4-7dea51a31920 [] videocuttool: beta-videocuttool-bookworm 565fe5fd-1b53-46bc-a5ec-410ab5e65537 [] video: encoding02 9a9b2610-e1ac-4dda-8309-2dc816f78dab [] video: encoding04 8ab8234a-5e85-4de6-a42c-031c7f05850d [] video: video-dev-bookworm f17b4b49-c1f0-43aa-8cd4-16a07288f704 [] video: video-nfs-1 6aa18f0e-d275-4c05-ab9f-65da1b7a7c3b [] video: video-redis-bookworm 02961b09-1520-4b10-b6d6-6118133cba20 [] videowiki: db-instance2 79f8e042-3bbd-4777-9d9d-509363b436a1 [] videowiki: videowiki-app-1 577a10c0-dfce-4bf4-8452-db0b1c68b248 [] videowiki: videowiki-db-1 2acb2154-ec20-4649-b7bb-6d619676e4ad [] vuessr: prototype2 cdb5e683-ddc7-4714-9d3d-951cf01af9ef [] wcdo: wcdo02 d28c57b4-2c75-4e24-a900-52bcd9c2e402 [] webperformancetest: crux-metrics 241f3730-4e8d-48c9-90f7-8c67e271d273 [] webperformancetest: queue b54d0173-24e7-426a-97db-7d9ca9489009 [] webperf: wikidiff2-demo 8cd85dcf-5ef9-4f6e-8428-e0312ff0ccb0 [] wikiapiary: scripts d2e8dc6a-0d72-4e84-99f1-ba669b12cf09 [] wikibase-registry: redir1 fb6a81ce-0337-4133-8928-47c2b5bb158d [] wikicite: wikicite-refsdb-proc-1 a11237f6-9436-4ad8-a8f9-0af4e020663c [] wikidata-dev: special-new-lexeme-testing 4f62099c-b9bc-4094-8c4f-db435781fdf7 [] wikidata-dev: wikidata-icinga-2024 504cb696-f783-4806-9e95-b1b6c2d88096 [] wikidata-reconciliation: wikidata-reconciliation 692086ce-c855-4f66-8b92-58426516a880 [] wikidocumentaries: hupu2 c13fc11c-3856-4588-96b9-8473878d752d [] wikidumpparse: wikidumpparse-nfs-1 fea0bea6-8b82-434f-849d-b837f98ff9cc [] wikifunctions: piglet 6143e81e-b683-4b0c-ac84-1ecc7a9b0cf0 [] wikifunctions: porky 11c29cdd-fd9b-455a-90d3-9593f17f44b7 [] wikilink: glitchtip 258ed0d6-41a5-4b96-b057-6ca71037b3f7 [] wikimania-mautic: mautic-web d92e8df1-0e53-4607-ab9c-1e59de1fba97 [] wikipeoplestats: wikipeoplestats-prod01 c2f528c1-6668-4619-91f9-b6ef53d3d8a3 [] wikiqlever: puppetmaster 57d26b02-dcb3-447e-b2eb-ac667852932c [] wikiqlever: qlever1 b2c96f28-03bb-4ff4-ab83-f79cc943d31d [] wikisource: ia-upload-test2 f6e22480-0e2c-4cd3-8d67-08250ab32709 [] wikisource: kraken-ocr 62e585db-7abf-4a0a-b076-595cdc0521d1 [] wikisource: ocr-prod02 22aa1613-9328-43f0-a0a9-f9bb48dcc635 [] wikisource: ocr-test02 d9acbf41-02da-4604-ac9d-93e1fe769b66 [] wikisource: wsexport-test 1328e006-5595-417c-864c-f771f027c4a1 [] wikisp: diana 4a74b89d-185e-41ca-82b3-fd121b477c47 [] wikispeech: demo-wiki b010ef0b-475a-4c6d-aa87-3f33efac317d [] wikispeech: tts-dev 24adb48b-ee87-4399-8036-0d0daefcb99d [] wikispore: owmf-test 0b1e12a4-dfff-42ae-84ca-0754959ffbb3 [] wikispore: wiki-nyc-owmf f6074ec9-28eb-4653-ba84-e7d735e3122c [] wikispore: wikispore-prod-02 5a312470-f9be-45ed-848b-6406211b3e82 [] wikispore: wikispore-test-bullseye 30dd5c6d-0444-4ed0-8386-fe232e0588b5 [] wikistats: wikistats-bookworm cf582bc8-0c2f-4ec5-8064-2cad2110b2ad [] wikiwho: wikiwho01 1b0a01e4-e8cb-4a68-ad6c-306b9869db05 [] wlm-it-visual: wlmitvisual01 1931dcad-42dd-4cdd-8246-0d13bdb38ad1 [] wm-bot: wm-bot-pg2-bookworm 63b82d38-3026-408f-8dcc-0ecbd1a3c870 [] wmcs-uptime: test-01 b96b8d02-164b-4d08-9046-1fd2b467a564 [] wmde-techwishes-survey: beta 664e04b3-34b7-4b87-9296-bfb1cbb5f872 [] wmf-research-tools: cloud-modeling dce8372c-fa22-4f73-a0d5-a855e342810e [] wmf-research-tools: dp-pageview-visualization c5688475-f082-460b-9c3f-81ebf1ff78cb [] wmf-research-tools: geo-provenance 3351eecb-e2ca-4889-a5cf-dad5b40add0f [] wmf-research-tools: knowledge-gap-index-tool 948c8c55-6fa8-45f7-9e3e-f3d923f2172c [] wmf-research-tools: mwparserfromhtml 609f5c13-499a-4fad-8b1d-6bf70e6e3b36 [] wmf-research-tools: outlink-model-embeddings 00d1b8e5-1211-4ec1-b975-0a5955fdcee7 [] wmf-research-tools: person-topic-test 9b9ecd8f-1ff1-4dec-a48b-722c15cf0bde [] wmf-research-tools: quality-article 90239a84-0dd4-4414-a6ea-5f72d98846d5 [] wmf-research-tools: quality-bulk-prototype 128d77c8-8c86-42c9-a3d5-6c94c1b8b860 [] wmf-research-tools: reader-embedding 7e5361b4-f195-4316-ada1-a156da2c8b69 [] wmf-research-tools: research-api-template 3ded4501-74a9-4752-9fdb-7d8f03ee9742 [] wmf-research-tools: section-ranker 507f6cad-3313-49c8-b4aa-a7ff7f8bc3bb [] wmf-research-tools: topic-model-prototype c7354620-5323-4888-9fcf-e5d55d263d4d [] wmf-research-tools: vector-search 3684f788-7edd-4e5d-a7aa-240d54f9c4a2 [] wmf-research-tools: wikidata-quality 58ca6c1c-e6b0-4963-8687-f3d81776fab6 [] wmf-research-tools: wiki-region-test 1f42bdbd-6dae-4387-b81b-f79dda18c9a4 [] xtools: xtools-dev06 1edf4d4e-fced-4815-b945-ba94b20f6011
_______________________________________________ Cloud-announce mailing list -- cloud-announce@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.o...