Notification Type: RECOVERY
Service: Galera haproxy failover
Host: cloudcontrol1003
Address: 208.80.154.23
State: OK
Date/Time: Thu Feb 24 17:40:59 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/HAProxy
Acknowledged by :
Additional Info:
OK check_failover servers up 31 down 0:
Notification Type: RECOVERY
Service: mysql -galera- process
Host: cloudcontrol1005
Address: 208.80.154.85
State: OK
Date/Time: Thu Feb 24 17:37:29 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Troubleshooting
Acknowledged by :
Additional Info:
PROCS OK: 1 process with command name mariadbd
Notification Type: PROBLEM
Service: mysql -galera- process
Host: cloudcontrol1005
Address: 208.80.154.85
State: CRITICAL
Date/Time: Thu Feb 24 17:31:09 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Troubleshooting
Acknowledged by :
Additional Info:
PROCS CRITICAL: 0 processes with command name mariadbd
Notification Type: RECOVERY
Service: mysql -galera- process
Host: cloudcontrol1004
Address: 208.80.154.132
State: OK
Date/Time: Thu Feb 24 16:58:51 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Troubleshooting
Acknowledged by :
Additional Info:
PROCS OK: 1 process with command name mariadbd
Notification Type: RECOVERY
Service: Check unit status of backup_vms
Host: cloudvirt1027
Address: 10.64.20.51
State: OK
Date/Time: Thu Feb 24 16:17:57 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
OK: Status of the systemd unit backup_vms
Notification Type: ACKNOWLEDGEMENT
Service: Galera haproxy failover
Host: cloudcontrol1005
Address: 208.80.154.85
State: CRITICAL
Date/Time: Thu Feb 24 00:55:23 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/HAProxy
Acknowledged by andrew bogott: Expected -- mariadb is turned off on cloudcontrol1004.
Additional Info:
CRITICAL check_failover servers up 31 down 1:
Notification Type: ACKNOWLEDGEMENT
Service: Galera haproxy failover
Host: cloudcontrol1003
Address: 208.80.154.23
State: CRITICAL
Date/Time: Thu Feb 24 00:55:23 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/HAProxy
Acknowledged by andrew bogott: Expected -- mariadb is turned off on cloudcontrol1004.
Additional Info:
CRITICAL check_failover servers up 31 down 1: