- Cloud-admin-feed - lists.wikimedia.org

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: start a job and verify on stretch is OK **
by nagios＠icinga1001.wikimedia.org 19 Feb '20

19 Feb '20

Notification Type: RECOVERY Service: toolschecker: start a job and verify on stretch Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Tue Feb 18 22:35:55 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/portal:toolforge/admin/toolschecker Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 0.279 second response time

1 0

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: check mtime mod from tools cron job is OK **
by nagios＠icinga1001.wikimedia.org 19 Feb '20

19 Feb '20

Notification Type: RECOVERY Service: toolschecker: check mtime mod from tools cron job Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Tue Feb 18 22:35:21 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 0.005 second response time

1 0

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: expect a long running job on stretch is OK **
by nagios＠icinga1001.wikimedia.org 19 Feb '20

19 Feb '20

Notification Type: RECOVERY Service: toolschecker: expect a long running job on stretch Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Tue Feb 18 22:34:09 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/portal:toolforge/admin/toolschecker Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 0.138 second response time

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: start a job and verify on stretch is CRITICAL **
by nagios＠icinga1001.wikimedia.org 19 Feb '20

19 Feb '20

Notification Type: PROBLEM Service: toolschecker: start a job and verify on stretch Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Tue Feb 18 22:24:49 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/portal:toolforge/admin/toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/grid/start/stretch - 177 bytes in 1.237 second response time

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: check mtime mod from tools cron job is CRITICAL **
by nagios＠icinga1001.wikimedia.org 19 Feb '20

19 Feb '20

Notification Type: PROBLEM Service: toolschecker: check mtime mod from tools cron job Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Tue Feb 18 22:23:25 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/cron - 177 bytes in 0.007 second response time

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: expect a long running job on stretch is CRITICAL **
by nagios＠icinga1001.wikimedia.org 19 Feb '20

19 Feb '20

Notification Type: PROBLEM Service: toolschecker: expect a long running job on stretch Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Tue Feb 18 22:23:07 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/portal:toolforge/admin/toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/grid/continuous/stretch - 177 bytes in 2.174 second response time

1 0

** RECOVERY alert - cloudcontrol1003/tools project instance distribution is OK **
by nagios＠icinga1001.wikimedia.org 18 Feb '20

18 Feb '20

Notification Type: RECOVERY Service: tools project instance distribution Host: cloudcontrol1003 Address: 208.80.154.23 State: OK Date/Time: Mon Feb 17 13:30:06 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Troubleshooting Acknowledged by : Additional Info: OK: All critical instances are spread out enough

1 0

** ACKNOWLEDGEMENT alert - cloudvirt1024/MegaRAID is CRITICAL **
by nagios＠icinga1001.wikimedia.org 15 Feb '20

15 Feb '20

Notification Type: ACKNOWLEDGEMENT Service: MegaRAID Host: cloudvirt1024 Address: 10.64.20.43 State: CRITICAL Date/Time: Sat Feb 15 00:32:12 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/MegaCli%23Monitoring Acknowledged by nagiosadmin: RAID handler auto-ack: https://phabricator.wikimedia.org/T245324 Additional Info: CRITICAL: 1 failed LD(s) (Degraded)

1 0

** ACKNOWLEDGEMENT alert - cloudvirt1024/MegaRAID is CRITICAL **
by nagios＠icinga1001.wikimedia.org 15 Feb '20

15 Feb '20

Notification Type: ACKNOWLEDGEMENT Service: MegaRAID Host: cloudvirt1024 Address: 10.64.20.43 State: CRITICAL Date/Time: Fri Feb 14 16:40:38 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/MegaCli%23Monitoring Acknowledged by nagiosadmin: RAID handler auto-ack: https://phabricator.wikimedia.org/T245282 Additional Info: CRITICAL: 1 failed LD(s) (Degraded)

1 0

** RECOVERY alert - labstore1004/High 1m load average is OK **
by nagios＠icinga1001.wikimedia.org 14 Feb '20

14 Feb '20

Notification Type: RECOVERY Service: High 1m load average Host: labstore1004 Address: 10.64.37.19 State: OK Date/Time: Fri Feb 14 12:15:12 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Labstore https://grafana.wikimedia.org/dashboard/db/labs-monitoring Acknowledged by : Additional Info: All metrics within thresholds.

1 0

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed