September 2019 - Cloud-admin-feed

Host UP alert for labstore1007!
by nagios＠icinga1001.wikimedia.org 20 Sep '19

20 Sep '19

Notification Type: RECOVERY Host: labstore1007 State: UP Address: 208.80.155.106 Info: PING OK - Packet loss = 0%, RTA = 0.25 ms Date/Time: Sat Sept 21 01:19:37 UTC 2019 Acknowledged by :

1 0

Host UP alert for cloudstore1008!
by nagios＠icinga1001.wikimedia.org 20 Sep '19

20 Sep '19

Notification Type: RECOVERY Host: cloudstore1008 State: UP Address: 208.80.155.125 Info: PING WARNING - Packet loss = 54%, RTA = 100.03 ms Date/Time: Sat Sept 21 01:19:35 UTC 2019 Acknowledged by :

1 0

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: check mtime mod from tools cron job is OK **
by nagios＠icinga1001.wikimedia.org 20 Sep '19

20 Sep '19

Notification Type: RECOVERY Service: toolschecker: check mtime mod from tools cron job Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Sat Sept 21 01:17:11 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 0.008 second response time

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: check mtime mod from tools cron job is CRITICAL **
by nagios＠icinga1001.wikimedia.org 20 Sep '19

20 Sep '19

Notification Type: PROBLEM Service: toolschecker: check mtime mod from tools cron job Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Sat Sept 21 01:13:59 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/cron - 177 bytes in 0.006 second response time

1 0

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: Make sure enwiki dumps are not empty is OK **
by nagios＠icinga1001.wikimedia.org 20 Sep '19

20 Sep '19

Notification Type: RECOVERY Service: toolschecker: Make sure enwiki dumps are not empty Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Sat Sept 21 01:01:59 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 52.041 second response time

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: Make sure enwiki dumps are not empty is CRITICAL **
by nagios＠icinga1001.wikimedia.org 20 Sep '19

20 Sep '19

Notification Type: PROBLEM Service: toolschecker: Make sure enwiki dumps are not empty Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Sat Sept 21 00:01:34 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 504 Gateway Time-out - string OK not found on http://checker.tools.wmflabs.org:80/nfs/dumps - 340 bytes in 60.014 second response time

1 0

Host DOWN alert for labstore1007!
by nagios＠icinga1001.wikimedia.org 20 Sep '19

20 Sep '19

Notification Type: PROBLEM Host: labstore1007 State: DOWN Address: 208.80.155.106 Info: PING CRITICAL - Packet loss = 100% Date/Time: Fri Sept 20 23:59:54 UTC 2019 Acknowledged by :

1 0

Host DOWN alert for cloudstore1008!
by nagios＠icinga1001.wikimedia.org 20 Sep '19

20 Sep '19

Notification Type: PROBLEM Host: cloudstore1008 State: DOWN Address: 208.80.155.125 Info: PING CRITICAL - Packet loss = 100% Date/Time: Fri Sept 20 23:59:52 UTC 2019 Acknowledged by :

1 0

** ACKNOWLEDGEMENT alert - labsdb1009/IPMI Sensor Status is CRITICAL **
by nagios＠icinga1001.wikimedia.org 18 Sep '19

18 Sep '19

Notification Type: ACKNOWLEDGEMENT Service: IPMI Sensor Status Host: labsdb1009 Address: 10.64.4.14 State: CRITICAL Date/Time: Wed Sept 18 18:34:12 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_… Acknowledged by Ayounsi: https://phabricator.wikimedia.org/T233248 Additional Info: Sensor Type(s) Temperature, Power_Supply Status: Critical [Power Supply 1 = Critical, Power Supplies = Critical]

1 0

Host DOWN alert for cloudvirt1015!
by nagios＠icinga1001.wikimedia.org 17 Sep '19

17 Sep '19

Notification Type: ACKNOWLEDGEMENT Host: cloudvirt1015 State: DOWN Address: 10.64.20.31 Info: PING CRITICAL - Packet loss = 100% Date/Time: Tue Sept 17 19:27:03 UTC 2019 Acknowledged by andrew bogott: not in service -- hardware issues

1 0

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed September 2019