September 2019 - Cloud-admin-feed

** PROBLEM alert - labstore1004/High 1m load average is CRITICAL **
by nagios＠icinga1001.wikimedia.org 16 Sep '19

16 Sep '19

Notification Type: PROBLEM Service: High 1m load average Host: labstore1004 Address: 10.64.37.19 State: CRITICAL Date/Time: Mon Sept 16 13:16:08 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Labstore https://grafana.wikimedia.org/dashboard/db/labs-monitoring Acknowledged by : Additional Info: cluster=labsnfs instance=labstore1004:9100 job=node site=eqiad

1 0

** PROBLEM alert - cloudweb2001-dev/Check the last execution of wikitech_run_jobs is CRITICAL **
by nagios＠icinga1001.wikimedia.org 16 Sep '19

16 Sep '19

Notification Type: PROBLEM Service: Check the last execution of wikitech_run_jobs Host: cloudweb2001-dev Address: 208.80.153.60 State: CRITICAL Date/Time: Mon Sept 16 08:16:06 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: CRITICAL: Status of the systemd unit wikitech_run_jobs

1 0

** RECOVERY alert - cloudweb2001-dev/Check the last execution of wikitech_run_jobs is OK **
by nagios＠icinga1001.wikimedia.org 16 Sep '19

16 Sep '19

Notification Type: RECOVERY Service: Check the last execution of wikitech_run_jobs Host: cloudweb2001-dev Address: 208.80.153.60 State: OK Date/Time: Mon Sept 16 08:05:02 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: OK: Status of the systemd unit wikitech_run_jobs

1 0

** ACKNOWLEDGEMENT alert - cloudweb2001-dev/Check the last execution of wikitech_run_jobs is CRITICAL **
by nagios＠icinga1001.wikimedia.org 15 Sep '19

15 Sep '19

Notification Type: ACKNOWLEDGEMENT Service: Check the last execution of wikitech_run_jobs Host: cloudweb2001-dev Address: 208.80.153.60 State: CRITICAL Date/Time: Sun Sept 15 17:20:14 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by andrew bogott: These were silenced until a second ago. Additional Info: NRPE: Command check_check_wikitech_run_jobs_status not defined

1 0

** PROBLEM alert - cloudweb2001-dev/Check the last execution of wikitech_run_jobs is CRITICAL **
by nagios＠icinga1001.wikimedia.org 15 Sep '19

15 Sep '19

Notification Type: PROBLEM Service: Check the last execution of wikitech_run_jobs Host: cloudweb2001-dev Address: 208.80.153.60 State: CRITICAL Date/Time: Sun Sept 15 17:19:18 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: NRPE: Command check_check_wikitech_run_jobs_status not defined

1 0

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: toolsdb read/write is OK **
by nagios＠icinga1001.wikimedia.org 13 Sep '19

13 Sep '19

Notification Type: RECOVERY Service: toolschecker: toolsdb read/write Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Fri Sept 13 20:48:41 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 33.466 second response time

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: toolsdb read/write is CRITICAL **
by nagios＠icinga1001.wikimedia.org 13 Sep '19

13 Sep '19

Notification Type: PROBLEM Service: toolschecker: toolsdb read/write Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Fri Sept 13 20:41:05 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 504 Gateway Time-out - string OK not found on http://checker.tools.wmflabs.org:80/db/toolsdb - 340 bytes in 60.023 second response time

1 0

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: toolsdb read/write is OK **
by nagios＠icinga1001.wikimedia.org 13 Sep '19

13 Sep '19

Notification Type: RECOVERY Service: toolschecker: toolsdb read/write Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Fri Sept 13 16:30:07 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 0.051 second response time

1 0

** ACKNOWLEDGEMENT alert - checker.tools.wmflabs.org/toolschecker: toolsdb read/write is CRITICAL **
by nagios＠icinga1001.wikimedia.org 13 Sep '19

13 Sep '19

Notification Type: ACKNOWLEDGEMENT Service: toolschecker: toolsdb read/write Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Fri Sept 13 16:26:46 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by Bstorm: investigating Additional Info: HTTP CRITICAL: HTTP/1.1 504 Gateway Time-out - string OK not found on http://checker.tools.wmflabs.org:80/db/toolsdb - 340 bytes in 60.016 second response time

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: toolsdb read/write is CRITICAL **
by nagios＠icinga1001.wikimedia.org 13 Sep '19

13 Sep '19

Notification Type: PROBLEM Service: toolschecker: toolsdb read/write Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Fri Sept 13 16:22:59 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 504 Gateway Time-out - string OK not found on http://checker.tools.wmflabs.org:80/db/toolsdb - 340 bytes in 60.005 second response time

1 0

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed September 2019