May 2024 - Discovery-alerts - lists.wikimedia.org

** RECOVERY alert - graphite1005/CirrusSearch more_like eqiad 95th percentile latency is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: CirrusSearch more_like eqiad 95th percentile latency Host: graphite1005 Address: 10.64.16.81 State: OK Date/Time: Tue May 28 23:24:48 UTC 2024 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/d/000000455/elasticsearch-percentiles?orgId=1… Acknowledged by : Additional Info: OK: Less than 20.00% above the threshold [1000.0]

5 days, 11 hours

1
0
0 0

** RECOVERY alert - graphite1005/CirrusSearch comp_suggest eqiad 95th percentile latency is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: CirrusSearch comp_suggest eqiad 95th percentile latency Host: graphite1005 Address: 10.64.16.81 State: OK Date/Time: Tue May 28 23:12:16 UTC 2024 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/d/000000455/elasticsearch-percentiles?orgId=1… Acknowledged by : Additional Info: OK: Less than 20.00% above the threshold [100.0]

5 days, 11 hours

1
0
0 0

** PROBLEM alert - graphite1005/CirrusSearch comp_suggest eqiad 95th percentile latency is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: CirrusSearch comp_suggest eqiad 95th percentile latency Host: graphite1005 Address: 10.64.16.81 State: CRITICAL Date/Time: Tue May 28 22:59:16 UTC 2024 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/d/000000455/elasticsearch-percentiles?orgId=1… Acknowledged by : Additional Info: CRITICAL: 40.00% of data above the critical threshold [250.0]

5 days, 11 hours

1
0
0 0

** PROBLEM alert - graphite1005/CirrusSearch more_like eqiad 95th percentile latency is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: CirrusSearch more_like eqiad 95th percentile latency Host: graphite1005 Address: 10.64.16.81 State: CRITICAL Date/Time: Tue May 28 22:55:48 UTC 2024 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/d/000000455/elasticsearch-percentiles?orgId=1… Acknowledged by : Additional Info: CRITICAL: 50.00% of data above the critical threshold [1500.0]

5 days, 11 hours

1
0
0 0

** PROBLEM alert - graphite1005/CirrusSearch full_text eqiad 95th percentile latency is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: CirrusSearch full_text eqiad 95th percentile latency Host: graphite1005 Address: 10.64.16.81 State: CRITICAL Date/Time: Tue May 28 22:54:12 UTC 2024 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/d/000000455/elasticsearch-percentiles?orgId=1… Acknowledged by : Additional Info: CRITICAL: 30.00% of data above the critical threshold [1000.0]

5 days, 11 hours

1
0
0 0

** PROBLEM alert - alert1001/MediaWiki CirrusSearch update rate - eqiad is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: MediaWiki CirrusSearch update rate - eqiad Host: alert1001 Address: 208.80.154.88 State: CRITICAL Date/Time: Tue May 28 21:23:45 UTC 2024 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Acknowledged by : Additional Info: CRITICAL: 20.00% of data under the critical threshold [50.0]

5 days, 13 hours

1
0
0 0

[FIRING:1] CirrusSearchNodeIndexingNotIncreasing elasticsearch search-platform (true true true false production-search-eqiad 10.64.16.132 elastic1056:9108 elasticsearch elastic1056-production-search-eqiad ops warning eqiad prometheus)

by sre-observability＠wikimedia.org

5 days, 16 hours

1
0
0 0

** RECOVERY alert - graphite1005/CirrusSearch full_text eqiad 95th percentile latency is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: CirrusSearch full_text eqiad 95th percentile latency Host: graphite1005 Address: 10.64.16.81 State: OK Date/Time: Tue May 28 17:30:53 UTC 2024 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/d/000000455/elasticsearch-percentiles?orgId=1… Acknowledged by : Additional Info: OK: Less than 20.00% above the threshold [500.0]

5 days, 16 hours

1
0
0 0

** PROBLEM alert - graphite1005/CirrusSearch full_text eqiad 95th percentile latency is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: CirrusSearch full_text eqiad 95th percentile latency Host: graphite1005 Address: 10.64.16.81 State: CRITICAL Date/Time: Tue May 28 17:15:51 UTC 2024 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/d/000000455/elasticsearch-percentiles?orgId=1… Acknowledged by : Additional Info: CRITICAL: 20.00% of data above the critical threshold [1000.0]

5 days, 17 hours

1
0
0 0

** RECOVERY alert - graphite1005/CirrusSearch full_text eqiad 95th percentile latency is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: CirrusSearch full_text eqiad 95th percentile latency Host: graphite1005 Address: 10.64.16.81 State: OK Date/Time: Tue May 28 17:02:51 UTC 2024 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/d/000000455/elasticsearch-percentiles?orgId=1… Acknowledged by : Additional Info: OK: Less than 20.00% above the threshold [500.0]

5 days, 17 hours

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts May 2024