September 2023 - Cloud-admin-feed

** PROBLEM alert - 208.80.154.143/Recursive DNS is CRITICAL **
by nagios＠alert1001.wikimedia.org 14 Sep '23

14 Sep '23

Notification Type: PROBLEM Service: Recursive DNS Host: 208.80.154.143 Address: 208.80.154.143 State: CRITICAL Date/Time: Thu Sept 14 15:08:28 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/DNS Acknowledged by : Additional Info: DNS_QUERY CRITICALError response or zero answers:

1 0

PROBLEM Host cloudservices1005 - PING CRITICAL - Packet loss = 100%
by nagios＠alert1001.wikimedia.org 14 Sep '23

14 Sep '23

Notification Type: PROBLEM Host: cloudservices1005 State: DOWN Address: 208.80.154.148 Info: PING CRITICAL - Packet loss = 100% Date/Time: Thu Sept 14 14:49:35 UTC 2023 Acknowledged by :

1 0

[RESOLVED] PuppetSyncFailure project-proxy (project-proxy-puppetmaster-01 node /var/lib/git/operations/puppet warn wmcs)
by root＠wmflabs.org 14 Sep '23

14 Sep '23

1 0

[FIRING:1] PuppetSyncFailure project-proxy (project-proxy-puppetmaster-01 node /var/lib/git/operations/puppet warn wmcs)
by root＠wmflabs.org 14 Sep '23

14 Sep '23

1 0

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: Redis set/get is OK **
by nagios＠alert1001.wikimedia.org 13 Sep '23

13 Sep '23

Notification Type: RECOVERY Service: toolschecker: Redis set/get Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Wed Sept 13 13:52:55 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 0.016 second response time

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: Redis set/get is CRITICAL **
by nagios＠alert1001.wikimedia.org 13 Sep '23

13 Sep '23

Notification Type: PROBLEM Service: toolschecker: Redis set/get Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Wed Sept 13 13:51:45 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/redis - 270 bytes in 17.041 second response time

1 0

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: check mtime mod from tools cron job is OK **
by nagios＠alert1001.wikimedia.org 13 Sep '23

13 Sep '23

Notification Type: RECOVERY Service: toolschecker: check mtime mod from tools cron job Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Wed Sept 13 12:51:03 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 0.004 second response time

1 0

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: Redis set/get is OK **
by nagios＠alert1001.wikimedia.org 13 Sep '23

13 Sep '23

Notification Type: RECOVERY Service: toolschecker: Redis set/get Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Wed Sept 13 12:49:33 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 0.022 second response time

1 0

** RECOVERY alert - cloudservices1005/Memcached is OK **
by nagios＠alert1001.wikimedia.org 13 Sep '23

13 Sep '23

Notification Type: RECOVERY Service: Memcached Host: cloudservices1005 Address: 208.80.154.148 State: OK Date/Time: Wed Sept 13 12:48:23 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Memcached Acknowledged by : Additional Info: TCP OK - 0.000 second response time on 208.80.154.148 port 11000

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: check mtime mod from tools cron job is CRITICAL **
by nagios＠alert1001.wikimedia.org 13 Sep '23

13 Sep '23

Notification Type: PROBLEM Service: toolschecker: check mtime mod from tools cron job Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Wed Sept 13 12:48:09 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/cron - 177 bytes in 0.002 second response time

1 0

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed September 2023