March 2023 - Cloud-admin-feed - lists.wikimedia.org

[FIRING:1] PuppetStaleCertificates clouddb-services (clouddb-services-puppetmaster-2 warn)
by root＠wmflabs.org 28 Mar '23

28 Mar '23

1 0

** RECOVERY alert - clouddumps1001/Persistent high iowait is OK **
by nagios＠alert1001.wikimedia.org 28 Mar '23

28 Mar '23

Notification Type: RECOVERY Service: Persistent high iowait Host: clouddumps1001 Address: 208.80.154.142 State: OK Date/Time: Tue Mar 28 22:39:44 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Labstore https://grafana.wikimedia.org/d/000000568/labstore1004-1005 Acknowledged by : Additional Info: (C)10 ge (W)5 ge 2.656

1 0

** PROBLEM alert - clouddumps1001/Persistent high iowait is CRITICAL **
by nagios＠alert1001.wikimedia.org 28 Mar '23

28 Mar '23

Notification Type: PROBLEM Service: Persistent high iowait Host: clouddumps1001 Address: 208.80.154.142 State: CRITICAL Date/Time: Tue Mar 28 22:32:06 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Labstore https://grafana.wikimedia.org/d/000000568/labstore1004-1005 Acknowledged by : Additional Info: 56.91 ge 10

1 0

[FIRING:1] PuppetStaleCertificates clouddb-services (clouddb-services-puppetmaster-2 warn)
by root＠wmflabs.org 28 Mar '23

28 Mar '23

1 0

** RECOVERY alert - cloudvirt1020/Disk space is OK **
by nagios＠alert1001.wikimedia.org 28 Mar '23

28 Mar '23

Notification Type: RECOVERY Service: Disk space Host: cloudvirt1020 Address: 10.64.20.39 State: OK Date/Time: Tue Mar 28 20:26:14 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space https://grafana.wikimedia.org/d/000000377/host-overview?var-server=cloudvir… Acknowledged by : Additional Info: DISK OK

1 0

** PROBLEM alert - cloudvirt1020/Disk space is CRITICAL **
by nagios＠alert1001.wikimedia.org 28 Mar '23

28 Mar '23

Notification Type: PROBLEM Service: Disk space Host: cloudvirt1020 Address: 10.64.20.39 State: CRITICAL Date/Time: Tue Mar 28 20:05:18 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space https://grafana.wikimedia.org/d/000000377/host-overview?var-server=cloudvir… Acknowledged by : Additional Info: DISK CRITICAL - free space: /srv 239794 MB (3% inode=99%): /var/lib/nova/instances 239794 MB (3% inode=99%):

1 0

** RECOVERY alert - cloudbackup1004/Check unit status of backup_vms is OK **
by nagios＠alert1001.wikimedia.org 28 Mar '23

28 Mar '23

Notification Type: RECOVERY Service: Check unit status of backup_vms Host: cloudbackup1004 Address: 10.64.20.23 State: OK Date/Time: Tue Mar 28 17:10:24 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Check_u… Acknowledged by : Additional Info: OK: Status of the systemd unit backup_vms

1 0

[RESOLVED] ProbeDown tools (ip4 probes/custom http_admin_toolforge_org_ip4 warning prometheus wmcs)
by root＠wmflabs.org 28 Mar '23

28 Mar '23

1 0

[FIRING:2] ProbeDown tools (ip4 probes/custom http_admin_toolforge_org_ip4 warning prometheus wmcs)
by root＠wmflabs.org 28 Mar '23

28 Mar '23

1 0

[RESOLVED] ProbeDown tools (ip4 probes/custom http_admin_toolforge_org_ip4 warning prometheus wmcs)
by root＠wmflabs.org 28 Mar '23

28 Mar '23

1 0

2025

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed March 2023