June 2022 - Cloud-admin-feed - lists.wikimedia.org

[FIRING:1] PuppetAgentFailure tools (tools-sgeweblight-10-27 node warn)
by root＠wmflabs.org 26 Jun '22

26 Jun '22

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: All k8s etcd nodes are healthy is CRITICAL **
by nagios＠alert1001.wikimedia.org 26 Jun '22

26 Jun '22

Notification Type: PROBLEM Service: toolschecker: All k8s etcd nodes are healthy Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Mon Jun 27 01:54:04 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/etcd/k8s - 443 bytes in 0.075 second response time

1 0

[FIRING:1] InstanceDown quarry (quarry-worker-04 node warn)
by root＠wmflabs.org 24 Jun '22

24 Jun '22

1 0

[FIRING:1] InstanceDown quarry (quarry-worker-04 node warn)
by root＠wmflabs.org 24 Jun '22

24 Jun '22

1 0

[FIRING:1] InstanceDown quarry (quarry-worker-04 node warn)
by root＠wmflabs.org 24 Jun '22

24 Jun '22

1 0

[FIRING:1] InstanceDown quarry (quarry-worker-04 node warn)
by root＠wmflabs.org 24 Jun '22

24 Jun '22

1 0

** RECOVERY alert - cloudcontrol1003/tools project instance distribution is OK **
by nagios＠alert1001.wikimedia.org 23 Jun '22

23 Jun '22

Notification Type: RECOVERY Service: tools project instance distribution Host: cloudcontrol1003 Address: 208.80.154.23 State: OK Date/Time: Thu Jun 23 22:39:21 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Troubleshooting Acknowledged by : Additional Info: OK: All critical instances are spread out enough

1 0

** RECOVERY alert - cloudcontrol1004/tools project instance distribution is OK **
by nagios＠alert1001.wikimedia.org 23 Jun '22

23 Jun '22

Notification Type: RECOVERY Service: tools project instance distribution Host: cloudcontrol1004 Address: 208.80.154.132 State: OK Date/Time: Thu Jun 23 22:29:27 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Troubleshooting Acknowledged by : Additional Info: OK: All critical instances are spread out enough

1 0

** RECOVERY alert - cloudcontrol1005/tools project instance distribution is OK **
by nagios＠alert1001.wikimedia.org 23 Jun '22

23 Jun '22

Notification Type: RECOVERY Service: tools project instance distribution Host: cloudcontrol1005 Address: 208.80.154.85 State: OK Date/Time: Thu Jun 23 22:25:09 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Troubleshooting Acknowledged by : Additional Info: OK: All critical instances are spread out enough

1 0

** PROBLEM alert - cloudcontrol1004/tools project instance distribution is CRITICAL **
by nagios＠alert1001.wikimedia.org 23 Jun '22

23 Jun '22

Notification Type: PROBLEM Service: tools project instance distribution Host: cloudcontrol1004 Address: 208.80.154.132 State: CRITICAL Date/Time: Thu Jun 23 18:19:56 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Troubleshooting Acknowledged by : Additional Info: CRITICAL: sgewebgrid-generic class instances not spread out enough

1 0

2025

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed June 2022