May 2021 - Cloud-admin-feed - lists.wikimedia.org

** PROBLEM alert - labstore1007/HTTP is CRITICAL **
by nagios＠alert1001.wikimedia.org 24 May '21

24 May '21

Notification Type: PROBLEM Service: HTTP Host: labstore1007 Address: 208.80.155.106 State: CRITICAL Date/Time: Mon May 24 06:12:21 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Dumps/XML-SQL_Dumps%23A_labstore_host_d… Acknowledged by : Additional Info: CRITICAL - Socket timeout after 10 seconds

1 0

** RECOVERY alert - labstore1007/HTTP is OK **
by nagios＠alert1001.wikimedia.org 24 May '21

24 May '21

Notification Type: RECOVERY Service: HTTP Host: labstore1007 Address: 208.80.155.106 State: OK Date/Time: Mon May 24 05:46:07 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Dumps/XML-SQL_Dumps%23A_labstore_host_d… Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 301 Moved Permanently - 383 bytes in 0.001 second response time

1 0

** PROBLEM alert - labstore1007/HTTP is CRITICAL **
by nagios＠alert1001.wikimedia.org 24 May '21

24 May '21

Notification Type: PROBLEM Service: HTTP Host: labstore1007 Address: 208.80.155.106 State: CRITICAL Date/Time: Mon May 24 05:40:47 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Dumps/XML-SQL_Dumps%23A_labstore_host_d… Acknowledged by : Additional Info: CRITICAL - Socket timeout after 10 seconds

1 0

** RECOVERY alert - labstore1007/Device not healthy -SMART- is OK **
by nagios＠alert1001.wikimedia.org 24 May '21

24 May '21

Notification Type: RECOVERY Service: Device not healthy -SMART- Host: labstore1007 Address: 208.80.155.106 State: OK Date/Time: Mon May 24 04:03:27 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/SMART%23Alerts https://grafana.wikimedia.org/dashboard/db/host-overview?var-server=labstor… Acknowledged by : Additional Info: All metrics within thresholds.

1 0

** ACKNOWLEDGEMENT alert - labstore1007/Device not healthy -SMART- is CRITICAL **
by nagios＠alert1001.wikimedia.org 22 May '21

22 May '21

Notification Type: ACKNOWLEDGEMENT Service: Device not healthy -SMART- Host: labstore1007 Address: 208.80.155.106 State: CRITICAL Date/Time: Sat May 22 02:12:55 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/SMART%23Alerts https://grafana.wikimedia.org/dashboard/db/host-overview?var-server=labstor… Acknowledged by Bstorm: Flapping/failing disk T281045 Additional Info: cluster=wmcs device=1I:1:5 instance=labstore1007 job=node site=eqiad

1 0

** PROBLEM alert - labstore1007/Device not healthy -SMART- is CRITICAL **
by nagios＠alert1001.wikimedia.org 22 May '21

22 May '21

Notification Type: PROBLEM Service: Device not healthy -SMART- Host: labstore1007 Address: 208.80.155.106 State: CRITICAL Date/Time: Sat May 22 01:49:52 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/SMART%23Alerts https://grafana.wikimedia.org/dashboard/db/host-overview?var-server=labstor… Acknowledged by : Additional Info: cluster=wmcs device=1I:1:5 instance=labstore1007 job=node site=eqiad

1 0

** RECOVERY alert - clouddb1019/MariaDB memory is OK **
by nagios＠alert1001.wikimedia.org 22 May '21

22 May '21

Notification Type: RECOVERY Service: MariaDB memory Host: clouddb1019 Address: 10.64.48.9 State: OK Date/Time: Fri May 21 20:45:30 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting Acknowledged by : Additional Info: OK Memory 94% used

1 0

** PROBLEM alert - clouddb1019/MariaDB memory is CRITICAL **
by nagios＠alert1001.wikimedia.org 22 May '21

22 May '21

Notification Type: PROBLEM Service: MariaDB memory Host: clouddb1019 Address: 10.64.48.9 State: CRITICAL Date/Time: Fri May 21 20:11:22 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting Acknowledged by : Additional Info: CRIT Memory 95% used. Largest process: mysqld (1847) = 73.6%

1 0

** PROBLEM alert - clouddb1019/MariaDB memory is CRITICAL **
by nagios＠alert1001.wikimedia.org 22 May '21

22 May '21

Notification Type: PROBLEM Service: MariaDB memory Host: clouddb1019 Address: 10.64.48.9 State: CRITICAL Date/Time: Fri May 21 18:45:14 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting Acknowledged by : Additional Info: CRIT Memory 95% used. Largest process: mysqld (1847) = 73.6%

1 0

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: All k8s worker nodes are healthy is OK **
by nagios＠alert1001.wikimedia.org 21 May '21

21 May '21

Notification Type: RECOVERY Service: toolschecker: All k8s worker nodes are healthy Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Fri May 21 16:26:34 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 0.091 second response time

1 0

2025

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed May 2021