- Discovery-alerts - lists.wikimedia.org

** RECOVERY alert - cloudelastic1004/ElasticSearch health check for shards on 9200 is OK **
by nagios＠icinga1001.wikimedia.org 05 Aug '19

05 Aug '19

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: cloudelastic1004 Address: 208.80.155.107 State: OK Date/Time: Mon Aug 5 21:22:25 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Additional Info: OK - elasticsearch status cloudelastic-chi-eqiad: cluster_name: cloudelastic-chi-eqiad, active_shards_percent_as_number: 85.38913362701909, number_of_pending_tasks: 0, active_shards: 1163, unassigned_shards: 199, status: red, initializing_shards: 0, active_primary_shards: 423, delayed_unassigned_shards: 0, number_of_in_flight_fetch: 0, number_of_data_nodes: 4, task_max_waiting_in_queue_millis: 0, number_of_nodes: 4, timed_out: False, relocating_shards: 0

1 0

** RECOVERY alert - cloudelastic1002/ElasticSearch health check for shards on 9200 is OK **
by nagios＠icinga1001.wikimedia.org 05 Aug '19

05 Aug '19

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: cloudelastic1002 Address: 208.80.154.133 State: OK Date/Time: Mon Aug 5 21:22:25 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Additional Info: OK - elasticsearch status cloudelastic-chi-eqiad: active_shards_percent_as_number: 85.38913362701909, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, cluster_name: cloudelastic-chi-eqiad, status: red, timed_out: False, number_of_in_flight_fetch: 0, unassigned_shards: 199, number_of_data_nodes: 4, initializing_shards: 0, active_shards: 1163, task_max_waiting_in_queue_millis: 0, active_primary_shards: 423, number_of_nodes: 4, relocating_shards: 0

1 0

** PROBLEM alert - cloudelastic1001/ElasticSearch health check for shards on 9200 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 05 Aug '19

05 Aug '19

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9200 Host: cloudelastic1001 Address: 208.80.154.8 State: CRITICAL Date/Time: Mon Aug 5 21:14:19 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Additional Info: CRITICAL - elasticsearch inactive shards 165 threshold =0.15 breach: delayed_unassigned_shards: 0, number_of_nodes: 4, unassigned_shards: 165, number_of_pending_tasks: 0, active_shards_percent_as_number: 73.5576923076923, timed_out: False, number_of_in_flight_fetch: 0, initializing_shards: 0, active_primary_shards: 183, status: red, task_max_waiting_in_queue_millis: 0, cluster_name: cloudelastic-chi-eqiad, number_of_data_nodes: 4, relocating_shards: 0, active_shards: 459

1 0

** PROBLEM alert - cloudelastic1003/ElasticSearch health check for shards on 9200 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 05 Aug '19

05 Aug '19

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9200 Host: cloudelastic1003 Address: 208.80.154.83 State: CRITICAL Date/Time: Mon Aug 5 21:14:19 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Additional Info: CRITICAL - elasticsearch inactive shards 165 threshold =0.15 breach: delayed_unassigned_shards: 0, task_max_waiting_in_queue_millis: 0, number_of_pending_tasks: 0, status: red, active_shards: 459, unassigned_shards: 165, number_of_data_nodes: 4, timed_out: False, cluster_name: cloudelastic-chi-eqiad, relocating_shards: 0, number_of_in_flight_fetch: 0, active_primary_shards: 183, number_of_nodes: 4, initializing_shards: 0, active_shards_percent_as_number: 73.5576923076923

1 0

** PROBLEM alert - cloudelastic1004/ElasticSearch health check for shards on 9200 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 05 Aug '19

05 Aug '19

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9200 Host: cloudelastic1004 Address: 208.80.155.107 State: CRITICAL Date/Time: Mon Aug 5 21:13:47 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Additional Info: CRITICAL - elasticsearch inactive shards 163 threshold =0.15 breach: timed_out: False, initializing_shards: 0, number_of_data_nodes: 4, active_primary_shards: 179, cluster_name: cloudelastic-chi-eqiad, number_of_pending_tasks: 0, status: red, delayed_unassigned_shards: 0, number_of_in_flight_fetch: 0, unassigned_shards: 163, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 73.36601307189542, relocating_shards: 0, number_of_nodes: 4, active_shards: 449

1 0

** PROBLEM alert - cloudelastic1002/ElasticSearch health check for shards on 9200 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 05 Aug '19

05 Aug '19

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9200 Host: cloudelastic1002 Address: 208.80.154.133 State: CRITICAL Date/Time: Mon Aug 5 21:13:47 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Additional Info: CRITICAL - elasticsearch inactive shards 163 threshold =0.15 breach: active_primary_shards: 179, delayed_unassigned_shards: 0, number_of_data_nodes: 4, timed_out: False, unassigned_shards: 163, number_of_pending_tasks: 0, relocating_shards: 0, task_max_waiting_in_queue_millis: 0, initializing_shards: 0, number_of_nodes: 4, cluster_name: cloudelastic-chi-eqiad, active_shards_percent_as_number: 73.36601307189542, status: red, active_shards: 449, number_of_in_flight_fetch: 0

1 0

** RECOVERY alert - icinga1001/Mediawiki Cirrussearch update rate - eqiad is OK **
by nagios＠icinga1001.wikimedia.org 05 Aug '19

05 Aug '19

Notification Type: RECOVERY Service: Mediawiki Cirrussearch update rate - eqiad Host: icinga1001 Address: 208.80.154.84 State: OK Date/Time: Mon Aug 5 14:26:58 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Additional Info: OK: Less than 1.00% under the threshold [80.0]

1 0

** RECOVERY alert - icinga1001/Mediawiki Cirrussearch update rate - codfw is OK **
by nagios＠icinga1001.wikimedia.org 05 Aug '19

05 Aug '19

Notification Type: RECOVERY Service: Mediawiki Cirrussearch update rate - codfw Host: icinga1001 Address: 208.80.154.84 State: OK Date/Time: Mon Aug 5 14:25:38 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Additional Info: OK: Less than 1.00% under the threshold [80.0]

1 0

** PROBLEM alert - icinga1001/Mediawiki Cirrussearch update rate - eqiad is CRITICAL **
by nagios＠icinga1001.wikimedia.org 05 Aug '19

05 Aug '19

Notification Type: PROBLEM Service: Mediawiki Cirrussearch update rate - eqiad Host: icinga1001 Address: 208.80.154.84 State: CRITICAL Date/Time: Mon Aug 5 14:03:02 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Additional Info: CRITICAL: 40.00% of data under the critical threshold [50.0]

1 0

** PROBLEM alert - icinga1001/Mediawiki Cirrussearch update rate - codfw is CRITICAL **
by nagios＠icinga1001.wikimedia.org 05 Aug '19

05 Aug '19

Notification Type: PROBLEM Service: Mediawiki Cirrussearch update rate - codfw Host: icinga1001 Address: 208.80.154.84 State: CRITICAL Date/Time: Mon Aug 5 14:01:42 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Additional Info: CRITICAL: 30.00% of data under the critical threshold [50.0]

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts