October 2021 - Cloud-admin-feed - lists.wikimedia.org

** RECOVERY alert - labstore1004/NFS Share Volume Space /srv/tools is OK **
by nagios＠alert1001.wikimedia.org 30 Oct '21

30 Oct '21

Notification Type: RECOVERY Service: NFS Share Volume Space /srv/tools Host: labstore1004 Address: 10.64.37.19 State: OK Date/Time: Sat Oct 30 00:08:02 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Shared_stora… https://grafana.wikimedia.org/d/50z0i4XWz/tools-overall-nfs-storage-utiliza… Acknowledged by : Additional Info: DISK OK - free space: /srv/tools 2192511 MB (27% inode=76%):

1 0

** ACKNOWLEDGEMENT alert - labstore1004/NFS Share Volume Space /srv/tools is CRITICAL **
by nagios＠alert1001.wikimedia.org 29 Oct '21

29 Oct '21

Notification Type: ACKNOWLEDGEMENT Service: NFS Share Volume Space /srv/tools Host: labstore1004 Address: 10.64.37.19 State: CRITICAL Date/Time: Thu Oct 28 22:55:15 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Shared_stora… https://grafana.wikimedia.org/d/50z0i4XWz/tools-overall-nfs-storage-utiliza… Acknowledged by andrew bogott: I will run a find and delete some things Additional Info: DISK CRITICAL - free space: /srv/tools 1264262 MB (15% inode=76%):

1 0

** PROBLEM alert - labstore1004/NFS Share Volume Space /srv/tools is CRITICAL **
by nagios＠alert1001.wikimedia.org 29 Oct '21

29 Oct '21

Notification Type: PROBLEM Service: NFS Share Volume Space /srv/tools Host: labstore1004 Address: 10.64.37.19 State: CRITICAL Date/Time: Thu Oct 28 22:41:31 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Shared_stora… https://grafana.wikimedia.org/d/50z0i4XWz/tools-overall-nfs-storage-utiliza… Acknowledged by : Additional Info: DISK CRITICAL - free space: /srv/tools 1264262 MB (15% inode=76%):

1 0

[FIRING:1] InstanceDown tools (tools-sgeexec-0937 node warn)
by root＠wmflabs.org 29 Oct '21

29 Oct '21

1 0

** RECOVERY alert - cloudcontrol1005/Check unit status of backup_glance_images is OK **
by nagios＠alert1001.wikimedia.org 28 Oct '21

28 Oct '21

Notification Type: RECOVERY Service: Check unit status of backup_glance_images Host: cloudcontrol1005 Address: 208.80.154.85 State: OK Date/Time: Thu Oct 28 08:04:24 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: OK: Status of the systemd unit backup_glance_images

1 0

** PROBLEM alert - cloudcontrol1005/Check unit status of backup_glance_images is CRITICAL **
by nagios＠alert1001.wikimedia.org 28 Oct '21

28 Oct '21

Notification Type: PROBLEM Service: Check unit status of backup_glance_images Host: cloudcontrol1005 Address: 208.80.154.85 State: CRITICAL Date/Time: Wed Oct 27 17:21:06 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: CRITICAL: Status of the systemd unit backup_glance_images

1 0

** RECOVERY alert - cloudvirt1024/Check unit status of backup_vms is OK **
by nagios＠alert1001.wikimedia.org 27 Oct '21

27 Oct '21

Notification Type: RECOVERY Service: Check unit status of backup_vms Host: cloudvirt1024 Address: 10.64.20.43 State: OK Date/Time: Wed Oct 27 08:15:06 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: OK: Status of the systemd unit backup_vms

1 0

** PROBLEM alert - cloudvirt1024/Check unit status of backup_vms is CRITICAL **
by nagios＠alert1001.wikimedia.org 27 Oct '21

27 Oct '21

Notification Type: PROBLEM Service: Check unit status of backup_vms Host: cloudvirt1024 Address: 10.64.20.43 State: CRITICAL Date/Time: Wed Oct 27 06:58:20 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: CRITICAL: Status of the systemd unit backup_vms

1 0

[FIRING:1] InstanceDown tools (tools-sgeexec-0901 node warn)
by root＠wmflabs.org 26 Oct '21

26 Oct '21

1 0

[FIRING:1] InstanceDown tools (tools-sgeexec-0910 node warn)
by root＠wmflabs.org 25 Oct '21

25 Oct '21

1 0

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed October 2021