March 2022 - Discovery-alerts - lists.wikimedia.org

** RECOVERY alert - relforge1003/ElasticSearch health check for shards on 9400 is OK **
by nagios＠alert1001.wikimedia.org 15 Mar '22

15 Mar '22

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9400 Host: relforge1003 Address: 10.64.5.37 State: OK Date/Time: Tue Mar 15 22:08:46 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status relforge-eqiad-small-alpha: active_primary_shards: 37, unassigned_shards: 5, active_shards: 37, status: yellow, task_max_waiting_in_queue_millis: 0, timed_out: False, active_shards_percent_as_number: 88.09523809523809, number_of_nodes: 2, number_of_pending_tasks: 0, initializing_shards: 0, number_of_data_nodes: 2, relocating_shards: 0, number_of_in_flight_fetch: 0, cluster_name: relforge-eqiad-small-alpha, delayed_unassigned_shards: 0

1 0

** PROBLEM alert - relforge1003/ElasticSearch health check for shards on 9200 is CRITICAL **
by nagios＠alert1001.wikimedia.org 15 Mar '22

15 Mar '22

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9200 Host: relforge1003 Address: 10.64.5.37 State: CRITICAL Date/Time: Tue Mar 15 22:03:18 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - elasticsearch inactive shards 145 threshold =0.15 breach: number_of_nodes: 1, task_max_waiting_in_queue_millis: 0, number_of_pending_tasks: 0, active_shards: 146, unassigned_shards: 145, active_primary_shards: 146, status: red, number_of_data_nodes: 1, number_of_in_flight_fetch: 0, timed_out: False, delayed_unassigned_shards: 0, initializing_shards: 0, relocating_shards: 0, active_shards_percent_as_number: 50.171821305841924, cluster_name: relforge-eqiad

1 0

** PROBLEM alert - relforge1003/ElasticSearch health check for shards on 9400 is CRITICAL **
by nagios＠alert1001.wikimedia.org 15 Mar '22

15 Mar '22

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9400 Host: relforge1003 Address: 10.64.5.37 State: CRITICAL Date/Time: Tue Mar 15 22:03:04 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - elasticsearch inactive shards 21 threshold =0.15 breach: relocating_shards: 0, initializing_shards: 0, unassigned_shards: 21, number_of_in_flight_fetch: 0, number_of_nodes: 1, task_max_waiting_in_queue_millis: 0, active_shards: 21, cluster_name: relforge-eqiad-small-alpha, active_shards_percent_as_number: 50.0, timed_out: False, number_of_data_nodes: 1, active_primary_shards: 21, number_of_pending_tasks: 0, delayed_unassigned_shards: 0, status: red

1 0

Update on build jenkins-wikidata-query-rdf-maven-release-docker-90
by Release Engineering 15 Mar '22

15 Mar '22

The build 90 for wikidata/query/rdf has finished. See https://integration.wikimedia.org/ci/job/wikidata-query-rdf-maven-release-d… for details.

1 0

[FIRING:1] RdfStreamingUpdaterFlinkProcessingLatencyIsHigh (WCQS_Streaming_Updater rdf-streaming-updater k8s warning eqiad prometheus search-platform)
by sre-observability＠wikimedia.org 15 Mar '22

15 Mar '22

1 0

selenium-daily-beta-CirrusSearch - Build # 1312 - Failure!
by Release Engineering 15 Mar '22

15 Mar '22

1 0

selenium-daily-beta-RelatedArticles - Build # 1344 - Failure!
by Release Engineering 14 Mar '22

14 Mar '22

1 0

[FIRING:1] RdfStreamingUpdaterFlinkProcessingLatencyIsHigh (WCQS_Streaming_Updater rdf-streaming-updater k8s warning eqiad prometheus search-platform)
by sre-observability＠wikimedia.org 11 Mar '22

11 Mar '22

1 0

[FIRING:1] RdfStreamingUpdaterFlinkProcessingLatencyIsHigh (WDQS_Streaming_Updater rdf-streaming-updater k8s warning eqiad prometheus search-platform)
by sre-observability＠wikimedia.org 11 Mar '22

11 Mar '22

1 0

** RECOVERY alert - relforge1004/ElasticSearch health check for shards on 9200 is OK **
by nagios＠alert1001.wikimedia.org 10 Mar '22

10 Mar '22

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: relforge1004 Address: 10.64.21.126 State: OK Date/Time: Thu Mar 10 22:16:45 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status relforge-eqiad: number_of_data_nodes: 2, relocating_shards: 0, active_shards_percent_as_number: 100.0, active_primary_shards: 163, status: green, number_of_pending_tasks: 0, timed_out: False, task_max_waiting_in_queue_millis: 0, active_shards: 293, number_of_in_flight_fetch: 0, number_of_nodes: 2, initializing_shards: 0, cluster_name: relforge-eqiad, delayed_unassigned_shards: 0, unassigned_shards: 0

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts March 2022