March 2023 - Cloud-admin-feed - lists.wikimedia.org

** PROBLEM alert - labstore1004/Persistent high iowait is CRITICAL **
by nagios＠alert1001.wikimedia.org 17 Mar '23

17 Mar '23

Notification Type: PROBLEM Service: Persistent high iowait Host: labstore1004 Address: 10.64.37.19 State: CRITICAL Date/Time: Fri Mar 17 17:11:47 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Labstore https://grafana.wikimedia.org/d/000000568/labstore1004-1005 Acknowledged by : Additional Info: 10.27 ge 10

1 0

** RECOVERY alert - cloudbackup1004/Check unit status of backup_vms is OK **
by nagios＠alert1001.wikimedia.org 17 Mar '23

17 Mar '23

Notification Type: RECOVERY Service: Check unit status of backup_vms Host: cloudbackup1004 Address: 10.64.20.23 State: OK Date/Time: Fri Mar 17 17:08:45 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Check_u… Acknowledged by : Additional Info: OK: Status of the systemd unit backup_vms

1 0

** PROBLEM alert - cloudcephosd1025/IPMI Sensor Status is CRITICAL **
by nagios＠alert1001.wikimedia.org 17 Mar '23

17 Mar '23

Notification Type: PROBLEM Service: IPMI Sensor Status Host: cloudcephosd1025 Address: 10.64.148.2 State: CRITICAL Date/Time: Fri Mar 17 13:49:30 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_… Acknowledged by : Additional Info: Sensor Type(s) Temperature, Power_Supply Status: Critical [Status = Critical, PS Redundancy = Critical]

1 0

** RECOVERY alert - cloudbackup1003/Check unit status of backup_glance_images is OK **
by nagios＠alert1001.wikimedia.org 17 Mar '23

17 Mar '23

Notification Type: RECOVERY Service: Check unit status of backup_glance_images Host: cloudbackup1003 Address: 10.64.20.22 State: OK Date/Time: Fri Mar 17 13:04:32 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state Acknowledged by : Additional Info: OK: Status of the systemd unit backup_glance_images

1 0

[FIRING:1] InstanceDown paws (paws-123-dsvw4d5vbk5v-master-0 node warn)
by root＠wmflabs.org 17 Mar '23

17 Mar '23

1 0

** PROBLEM alert - cloudbackup1004/Check unit status of backup_vms is CRITICAL **
by nagios＠alert1001.wikimedia.org 17 Mar '23

17 Mar '23

Notification Type: PROBLEM Service: Check unit status of backup_vms Host: cloudbackup1004 Address: 10.64.20.23 State: CRITICAL Date/Time: Fri Mar 17 07:39:19 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Check_u… Acknowledged by : Additional Info: CRITICAL: Status of the systemd unit backup_vms

1 0

** PROBLEM alert - cloudbackup1003/Check unit status of backup_vms is CRITICAL **
by nagios＠alert1001.wikimedia.org 17 Mar '23

17 Mar '23

Notification Type: PROBLEM Service: Check unit status of backup_vms Host: cloudbackup1003 Address: 10.64.20.22 State: CRITICAL Date/Time: Fri Mar 17 05:15:25 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Check_u… Acknowledged by : Additional Info: CRITICAL: Status of the systemd unit backup_vms

1 0

** RECOVERY alert - cloudbackup1003/Check unit status of backup_vms is OK **
by nagios＠alert1001.wikimedia.org 17 Mar '23

17 Mar '23

Notification Type: RECOVERY Service: Check unit status of backup_vms Host: cloudbackup1003 Address: 10.64.20.22 State: OK Date/Time: Fri Mar 17 05:02:33 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Check_u… Acknowledged by : Additional Info: OK: Status of the systemd unit backup_vms

1 0

[RESOLVED] InstanceDown paws (node warn)
by root＠wmflabs.org 17 Mar '23

17 Mar '23

1 0

[FIRING:1] InstanceDown paws (node warn)
by root＠wmflabs.org 17 Mar '23

17 Mar '23

1 0

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed March 2023