October 2022 - Cloud-admin-feed - lists.wikimedia.org

** PROBLEM alert - cloudrabbit1001/Check systemd state is CRITICAL **
by nagios＠alert1001.wikimedia.org 21 Oct '22

21 Oct '22

Notification Type: PROBLEM Service: Check systemd state Host: cloudrabbit1001 Address: 208.80.154.147 State: CRITICAL Date/Time: Fri Oct 21 06:40:16 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state Acknowledged by : Additional Info: CRITICAL - degraded: The following units failed: drain_rabbitmq_notification_error.service

1 0

[FIRING:1] InstanceDown tools (tools-filesystemtest-1 node warn)
by root＠wmflabs.org 21 Oct '22

21 Oct '22

1 0

RECOVERY Host cloudcontrol1005.mgmt - PING OK - Packet loss = 0%, RTA = 4.05 ms
by nagios＠alert1001.wikimedia.org 20 Oct '22

20 Oct '22

Notification Type: RECOVERY Host: cloudcontrol1005.mgmt State: UP Address: 10.65.4.188 Info: PING OK - Packet loss = 0%, RTA = 4.05 ms Date/Time: Thu Oct 20 14:00:36 UTC 2022 Acknowledged by :

1 0

ACKNOWLEDGEMENT Host cloudcontrol1005.mgmt - PING CRITICAL - Packet loss = 100%
by nagios＠alert1001.wikimedia.org 20 Oct '22

20 Oct '22

Notification Type: ACKNOWLEDGEMENT Host: cloudcontrol1005.mgmt State: DOWN Address: 10.65.4.188 Info: PING CRITICAL - Packet loss = 100% Date/Time: Thu Oct 20 13:43:03 UTC 2022 Acknowledged by Cathal Mooney: Hosts down due to failure of rack C5 mgmt switch msw-c5-eqiad - The acknowledgement expires at: 2022-10-21 13:42:26.

1 0

RECOVERY Host cloudmetrics1004.mgmt - PING OK - Packet loss = 0%, RTA = 0.12 ms
by nagios＠alert1001.wikimedia.org 20 Oct '22

20 Oct '22

Notification Type: RECOVERY Host: cloudmetrics1004.mgmt State: UP Address: 0.0.0.0 Info: PING OK - Packet loss = 0%, RTA = 0.12 ms Date/Time: Thu Oct 20 13:30:25 UTC 2022 Acknowledged by :

1 0

PROBLEM Host cloudmetrics1004.mgmt - PING CRITICAL - Packet loss = 100%
by nagios＠alert1001.wikimedia.org 20 Oct '22

20 Oct '22

Notification Type: PROBLEM Host: cloudmetrics1004.mgmt State: DOWN Address: 10.65.1.63 Info: PING CRITICAL - Packet loss = 100% Date/Time: Thu Oct 20 13:12:33 UTC 2022 Acknowledged by :

1 0

PROBLEM Host cloudcontrol1005.mgmt - PING CRITICAL - Packet loss = 100%
by nagios＠alert1001.wikimedia.org 20 Oct '22

20 Oct '22

Notification Type: PROBLEM Host: cloudcontrol1005.mgmt State: DOWN Address: 10.65.4.188 Info: PING CRITICAL - Packet loss = 100% Date/Time: Thu Oct 20 13:09:13 UTC 2022 Acknowledged by :

1 0

** PROBLEM alert - cloudcontrol1005/Check unit status of backup_cinder_volumes is CRITICAL **
by nagios＠alert1001.wikimedia.org 20 Oct '22

20 Oct '22

Notification Type: PROBLEM Service: Check unit status of backup_cinder_volumes Host: cloudcontrol1005 Address: 208.80.154.85 State: CRITICAL Date/Time: Thu Oct 20 09:33:18 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Check_u… Acknowledged by : Additional Info: CRITICAL: Status of the systemd unit backup_cinder_volumes

1 0

** RECOVERY alert - cloudbackup1004/Check unit status of backup_vms is OK **
by nagios＠alert1001.wikimedia.org 20 Oct '22

20 Oct '22

Notification Type: RECOVERY Service: Check unit status of backup_vms Host: cloudbackup1004 Address: 10.64.20.23 State: OK Date/Time: Wed Oct 19 20:49:38 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Check_u… Acknowledged by : Additional Info: OK: Status of the systemd unit backup_vms

1 0

** RECOVERY alert - cloudrabbit1001/Check systemd state is OK **
by nagios＠alert1001.wikimedia.org 20 Oct '22

20 Oct '22

Notification Type: RECOVERY Service: Check systemd state Host: cloudrabbit1001 Address: 208.80.154.147 State: OK Date/Time: Wed Oct 19 18:36:22 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state Acknowledged by : Additional Info: OK - running: The system is fully operational

1 0

2025

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed October 2022