Notification Type: ACKNOWLEDGEMENT
Service: Check unit status of wmcs_monitoring_graphite_rsync
Host: cloudmetrics1004
Address: 10.64.37.6
State: CRITICAL
Date/Time: Wed Dec 15 02:37:20 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by andrew bogott: investigating
Additional Info:
CRITICAL: Status of the systemd unit wmcs_monitoring_graphite_rsync
Notification Type: ACKNOWLEDGEMENT
Service: Check systemd state
Host: cloudmetrics1003
Address: 10.64.4.6
State: CRITICAL
Date/Time: Wed Dec 15 02:37:20 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
Acknowledged by andrew bogott: investigating
Additional Info:
CRITICAL - degraded: The following units failed: wmcs_monitoring_graphite_rsync.service
Notification Type: PROBLEM
Service: Check unit status of wmcs_monitoring_graphite_rsync
Host: cloudmetrics1004
Address: 10.64.37.6
State: CRITICAL
Date/Time: Wed Dec 15 02:26:16 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
CRITICAL: Status of the systemd unit wmcs_monitoring_graphite_rsync
Notification Type: PROBLEM
Service: Check systemd state
Host: cloudmetrics1003
Address: 10.64.4.6
State: CRITICAL
Date/Time: Wed Dec 15 02:24:30 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
Acknowledged by :
Additional Info:
CRITICAL - degraded: The following units failed: wmcs_monitoring_graphite_rsync.service
Notification Type: RECOVERY
Service: Check unit status of wmcs_monitoring_graphite_rsync
Host: cloudmetrics1002
Address: 10.64.4.15
State: OK
Date/Time: Wed Dec 15 01:42:11 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
OK: Status of the systemd unit wmcs_monitoring_graphite_rsync
Notification Type: RECOVERY
Service: Check systemd state
Host: cloudmetrics1002
Address: 10.64.4.15
State: OK
Date/Time: Wed Dec 15 01:41:39 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
Acknowledged by :
Additional Info:
OK - running: The system is fully operational
Notification Type: ACKNOWLEDGEMENT
Service: Check unit status of wmcs_monitoring_graphite_rsync
Host: cloudmetrics1001
Address: 10.64.37.13
State: CRITICAL
Date/Time: Wed Dec 15 01:00:31 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by andrew bogott: work in progress, en route to being decomd
Additional Info:
CRITICAL: Status of the systemd unit wmcs_monitoring_graphite_rsync
Notification Type: RECOVERY
Service: Check unit status of wikitech_run_jobs
Host: cloudweb2001-dev
Address: 208.80.153.60
State: OK
Date/Time: Tue Dec 14 20:36:26 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
OK: Status of the systemd unit wikitech_run_jobs
Notification Type: ACKNOWLEDGEMENT
Service: Check unit status of wikitech_run_jobs
Host: cloudweb2001-dev
Address: 208.80.153.60
State: CRITICAL
Date/Time: Tue Dec 14 20:27:42 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by andrew bogott: I dont know what this is, but mediawiki behavior in codfw1dev barely matters -- its a rapidly deprecating test/dev site.
Additional Info:
CRITICAL: Status of the systemd unit wikitech_run_jobs
Notification Type: PROBLEM
Service: Check unit status of wikitech_run_jobs
Host: cloudweb2001-dev
Address: 208.80.153.60
State: CRITICAL
Date/Time: Tue Dec 14 20:26:00 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
CRITICAL: Status of the systemd unit wikitech_run_jobs
Notification Type: RECOVERY
Service: Check unit status of wmcs_monitoring_graphite_rsync
Host: cloudmetrics1004
Address: 10.64.37.6
State: OK
Date/Time: Tue Dec 14 16:00:45 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
OK: Status of the systemd unit wmcs_monitoring_graphite_rsync
Notification Type: ACKNOWLEDGEMENT
Service: Check unit status of wmcs_monitoring_graphite_rsync
Host: cloudmetrics1004
Address: 10.64.37.6
State: CRITICAL
Date/Time: Tue Dec 14 00:24:02 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by andrew bogott: work in progress
Additional Info:
CRITICAL: Status of the systemd unit wmcs_monitoring_graphite_rsync
Notification Type: ACKNOWLEDGEMENT
Service: Check systemd state
Host: cloudmetrics1004
Address: 10.64.37.6
State: CRITICAL
Date/Time: Tue Dec 14 00:24:02 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
Acknowledged by andrew bogott: work in progress
Additional Info:
CRITICAL - degraded: The following units failed: wmcs_monitoring_graphite_rsync.service
Notification Type: ACKNOWLEDGEMENT
Service: Check unit status of wmcs_monitoring_graphite_rsync
Host: cloudmetrics1002
Address: 10.64.4.15
State: CRITICAL
Date/Time: Tue Dec 14 00:24:02 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by andrew bogott: work in progress
Additional Info:
CRITICAL: Status of the systemd unit wmcs_monitoring_graphite_rsync
Notification Type: ACKNOWLEDGEMENT
Service: Check systemd state
Host: cloudmetrics1002
Address: 10.64.4.15
State: CRITICAL
Date/Time: Tue Dec 14 00:24:02 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
Acknowledged by andrew bogott: work in progress
Additional Info:
CRITICAL - degraded: The following units failed: wmcs_monitoring_graphite_rsync.service
Notification Type: PROBLEM
Service: Check unit status of wmcs_monitoring_graphite_rsync
Host: cloudmetrics1004
Address: 10.64.37.6
State: CRITICAL
Date/Time: Tue Dec 14 00:07:15 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
CRITICAL: Status of the systemd unit wmcs_monitoring_graphite_rsync
Notification Type: PROBLEM
Service: Check unit status of wmcs_monitoring_graphite_rsync
Host: cloudmetrics1002
Address: 10.64.4.15
State: CRITICAL
Date/Time: Tue Dec 14 00:04:29 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
CRITICAL: Status of the systemd unit wmcs_monitoring_graphite_rsync
Notification Type: PROBLEM
Service: Check systemd state
Host: cloudmetrics1004
Address: 10.64.37.6
State: CRITICAL
Date/Time: Tue Dec 14 00:04:11 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
Acknowledged by :
Additional Info:
CRITICAL - degraded: The following units failed: wmcs_monitoring_graphite_rsync.service
Notification Type: PROBLEM
Service: Check systemd state
Host: cloudmetrics1002
Address: 10.64.4.15
State: CRITICAL
Date/Time: Tue Dec 14 00:03:43 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
Acknowledged by :
Additional Info:
CRITICAL - degraded: The following units failed: wmcs_monitoring_graphite_rsync.service
Notification Type: RECOVERY
Service: Check unit status of backup_vms
Host: cloudvirt1024
Address: 10.64.20.43
State: OK
Date/Time: Mon Dec 13 15:38:06 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
OK: Status of the systemd unit backup_vms
Notification Type: PROBLEM
Service: Check unit status of backup_vms
Host: cloudvirt1024
Address: 10.64.20.43
State: CRITICAL
Date/Time: Mon Dec 13 07:05:35 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
CRITICAL: Status of the systemd unit backup_vms
Notification Type: RECOVERY
Service: Check unit status of backup_vms
Host: cloudvirt1024
Address: 10.64.20.43
State: OK
Date/Time: Mon Dec 13 02:05:54 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
OK: Status of the systemd unit backup_vms
Notification Type: PROBLEM
Service: Check unit status of backup_vms
Host: cloudvirt1024
Address: 10.64.20.43
State: CRITICAL
Date/Time: Sun Dec 12 04:16:30 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
CRITICAL: Status of the systemd unit backup_vms
Notification Type: RECOVERY
Service: Check unit status of backup_vms
Host: cloudvirt1024
Address: 10.64.20.43
State: OK
Date/Time: Sun Dec 12 02:00:20 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
OK: Status of the systemd unit backup_vms
Notification Type: PROBLEM
Service: Check unit status of backup_vms
Host: cloudvirt1024
Address: 10.64.20.43
State: CRITICAL
Date/Time: Sat Dec 11 07:09:53 UTC 2021
Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time…
Acknowledged by :
Additional Info:
CRITICAL: Status of the systemd unit backup_vms