May 2024 - Discovery-alerts - lists.wikimedia.org

[FIRING:1] RdfStreamingUpdaterFlinkJobUnstable search-platform (WDQS_Streaming_Updater rdf-streaming-updater k8s wikidata critical codfw prometheus)

by sre-observability＠wikimedia.org

12 hours, 21 minutes

1
0
0 0

[FIRING:1] CirrusStreamingUpdaterFlinkJobUnstable search-platform (cirrus_streaming_updater_producer_codfw cirrus-streaming-updater k8s producer critical codfw prometheus)

by sre-observability＠wikimedia.org

12 hours, 21 minutes

1
0
0 0

[FIRING:1] RdfStreamingUpdaterHighConsumerUpdateLag wdqs search-platform (wdqs1014:9101 jmx_query_service_streaming_updater ops thanos-rule warning eqiad thanos)

by sre-observability＠wikimedia.org

14 hours, 2 minutes

1
0
0 0

[FIRING:2] ProbeDown search-platform (wdqs1014:443 probes/custom ops info eqiad prometheus)

by sre-observability＠wikimedia.org

14 hours, 38 minutes

1
0
0 0

[FIRING:2] ProbeDown search-platform (wdqs1014:443 probes/custom ops info eqiad prometheus)

by sre-observability＠wikimedia.org

18 hours, 38 minutes

1
0
0 0

[FIRING:1] ProbeDown search-platform (2620:0:861:10a:10:64:131:25 ip6 wdqs1018:443 probes/custom http_wdqs_external_sparql_endpoint_search_ip6 ops info eqiad prometheus)

by sre-observability＠wikimedia.org

2 days, 15 hours

1
0
0 0

[FIRING:1] SystemdUnitFailed search-platform (elastic2090:9100 logrotate.service ops critical codfw prometheus)

by sre-observability＠wikimedia.org

3 days, 9 hours

1
0
0 0

[FIRING:2] ProbeDown search-platform (wdqs1012:443 probes/custom ops info eqiad prometheus)

by sre-observability＠wikimedia.org

3 days, 13 hours

1
0
0 0

** PROBLEM alert - alert1001/MediaWiki CirrusSearch update rate - codfw is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: MediaWiki CirrusSearch update rate - codfw Host: alert1001 Address: 208.80.154.88 State: CRITICAL Date/Time: Fri May 17 07:34:29 UTC 2024 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Acknowledged by : Additional Info: CRITICAL: 30.00% of data under the critical threshold [50.0]

4 days, 2 hours

1
0
0 0

** PROBLEM alert - alert1001/MediaWiki CirrusSearch update rate - codfw is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: MediaWiki CirrusSearch update rate - codfw Host: alert1001 Address: 208.80.154.88 State: CRITICAL Date/Time: Fri May 17 05:16:35 UTC 2024 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Acknowledged by : Additional Info: CRITICAL: 10.00% of data under the critical threshold [50.0]

4 days, 4 hours

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts May 2024