April 2021 - Cloud-admin-feed - lists.wikimedia.org

[FIRING:1] WidespreadPuppetAgentFailures gratitude (crit)

by root＠wmflabs.org

3 years, 1 month

1
0
0 0

[FIRING:1] InstanceDown tools (tools-sgebastion-07 node warn)

by root＠wmflabs.org

3 years, 1 month

1
0
0 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: expect a long running job on stretch is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: toolschecker: expect a long running job on stretch Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Tue Apr 27 15:32:00 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/grid/continuous/stretch - 177 bytes in 0.075 second response time

3 years, 1 month

1
0
0 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: start a job and verify on stretch is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: toolschecker: start a job and verify on stretch Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Tue Apr 27 15:30:44 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/grid/start/stretch - 177 bytes in 0.039 second response time

3 years, 1 month

1
0
0 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: check mtime mod from tools cron job is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: toolschecker: check mtime mod from tools cron job Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Tue Apr 27 15:19:58 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/cron - 264 bytes in 0.004 second response time

3 years, 1 month

1
0
0 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: NFS read/writeable on labs instances is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: toolschecker: NFS read/writeable on labs instances Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Tue Apr 27 15:18:54 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/nfs/home - 298 bytes in 0.007 second response time

3 years, 1 month

1
0
0 0

** RECOVERY alert - labstore1007/Device not healthy -SMART- is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: Device not healthy -SMART- Host: labstore1007 Address: 208.80.155.106 State: OK Date/Time: Tue Apr 27 11:31:48 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/SMART%23Alerts https://grafana.wikimedia.org/dashboard/db/host-overview?var-server=labstor… Acknowledged by : Additional Info: All metrics within thresholds.

3 years, 1 month

1
0
0 0

** PROBLEM alert - labstore1007/Device not healthy -SMART- is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: Device not healthy -SMART- Host: labstore1007 Address: 208.80.155.106 State: CRITICAL Date/Time: Tue Apr 27 11:00:28 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/SMART%23Alerts https://grafana.wikimedia.org/dashboard/db/host-overview?var-server=labstor… Acknowledged by : Additional Info: cluster=wmcs device=1I:1:5 instance=labstore1007 job=node site=eqiad

3 years, 1 month

1
0
0 0

** RECOVERY alert - labstore1007/HTTP is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: HTTP Host: labstore1007 Address: 208.80.155.106 State: OK Date/Time: Tue Apr 27 09:29:35 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Dumps/XML-SQL_Dumps%23A_labstore_host_d… Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 301 Moved Permanently - 383 bytes in 0.001 second response time

3 years, 1 month

1
0
0 0

** PROBLEM alert - labstore1007/HTTP is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: HTTP Host: labstore1007 Address: 208.80.155.106 State: CRITICAL Date/Time: Tue Apr 27 09:21:55 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Dumps/XML-SQL_Dumps%23A_labstore_host_d… Acknowledged by : Additional Info: CRITICAL - Socket timeout after 10 seconds

3 years, 1 month

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed April 2021