February 2020 - Discovery-alerts - lists.wikimedia.org

** RECOVERY alert - logstash1009/ElasticSearch health check for shards on 9200 is OK **

by nagios＠icinga1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: logstash1009 Address: 10.64.32.27 State: OK Date/Time: Thu Feb 27 21:49:14 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status production-logstash-eqiad: number_of_pending_tasks: 0, status: yellow, number_of_nodes: 5, number_of_data_nodes: 2, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, relocating_shards: 0, active_primary_shards: 484, unassigned_shards: 369, active_shards: 747, cluster_name: production-logstash-eqiad, delayed_unassigned_shards: 0, timed_out: False, initializing_shards: 8, active_shards_percent_as_number: 66.45907473309609

4 years, 1 month

1
0
0 0

** RECOVERY alert - logstash1007/ElasticSearch health check for shards on 9200 is OK **

by nagios＠icinga1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: logstash1007 Address: 10.64.0.37 State: OK Date/Time: Thu Feb 27 21:49:06 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status production-logstash-eqiad: relocating_shards: 0, status: yellow, initializing_shards: 8, number_of_data_nodes: 2, active_primary_shards: 484, task_max_waiting_in_queue_millis: 0, active_shards: 747, timed_out: False, number_of_nodes: 5, cluster_name: production-logstash-eqiad, unassigned_shards: 369, active_shards_percent_as_number: 66.45907473309609, number_of_in_flight_fetch: 0, number_of_pending_tasks: 0, delayed_unassigned_shards: 0

4 years, 1 month

1
0
0 0

** RECOVERY alert - logstash1008/ElasticSearch health check for shards on 9200 is OK **

by nagios＠icinga1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: logstash1008 Address: 10.64.0.90 State: OK Date/Time: Thu Feb 27 21:48:44 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status production-logstash-eqiad: number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards: 746, number_of_nodes: 5, relocating_shards: 0, active_primary_shards: 484, cluster_name: production-logstash-eqiad, number_of_data_nodes: 2, number_of_pending_tasks: 0, timed_out: False, delayed_unassigned_shards: 0, unassigned_shards: 370, active_shards_percent_as_number: 66.37010676156584, initializing_shards: 8, status: yellow

4 years, 1 month

1
0
0 0

** RECOVERY alert - logstash1011/ElasticSearch health check for shards on 9200 is OK **

by nagios＠icinga1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: logstash1011 Address: 10.64.16.30 State: OK Date/Time: Thu Feb 27 21:48:16 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status production-logstash-eqiad: status: yellow, number_of_nodes: 5, initializing_shards: 8, task_max_waiting_in_queue_millis: 0, number_of_data_nodes: 2, active_primary_shards: 484, relocating_shards: 0, cluster_name: production-logstash-eqiad, number_of_pending_tasks: 0, timed_out: False, active_shards: 744, active_shards_percent_as_number: 66.19217081850533, number_of_in_flight_fetch: 0, delayed_unassigned_shards: 0, unassigned_shards: 372

4 years, 1 month

1
0
0 0

** RECOVERY alert - logstash1012/ElasticSearch health check for shards on 9200 is OK **

by nagios＠icinga1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: logstash1012 Address: 10.64.48.177 State: OK Date/Time: Thu Feb 27 21:48:14 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status production-logstash-eqiad: number_of_nodes: 5, number_of_pending_tasks: 0, active_shards: 744, relocating_shards: 0, task_max_waiting_in_queue_millis: 0, timed_out: False, active_primary_shards: 484, number_of_data_nodes: 2, unassigned_shards: 372, cluster_name: production-logstash-eqiad, number_of_in_flight_fetch: 0, initializing_shards: 8, active_shards_percent_as_number: 66.19217081850533, delayed_unassigned_shards: 0, status: yellow

4 years, 1 month

1
0
0 0

** PROBLEM alert - logstash1009/ElasticSearch health check for shards on 9200 is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9200 Host: logstash1009 Address: 10.64.32.27 State: CRITICAL Date/Time: Thu Feb 27 21:42:38 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - elasticsearch inactive shards 947 threshold =0.34 breach: unassigned_shards: 939, timed_out: False, active_shards_percent_as_number: 15.747330960854091, task_max_waiting_in_queue_millis: 141613, relocating_shards: 0, delayed_unassigned_shards: 0, cluster_name: production-logstash-eqiad, status: red, number_of_pending_tasks: 111, number_of_nodes: 5, active_primary_shards: 164, number_of_in_flight_fetch: 0, active_shards: 177, initializing_shards: 8, number_of_data_nodes: 2

4 years, 1 month

1
0
0 0

** PROBLEM alert - logstash1007/ElasticSearch health check for shards on 9200 is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9200 Host: logstash1007 Address: 10.64.0.37 State: CRITICAL Date/Time: Thu Feb 27 21:42:32 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - elasticsearch inactive shards 958 threshold =0.34 breach: active_shards: 166, cluster_name: production-logstash-eqiad, number_of_in_flight_fetch: 0, initializing_shards: 8, relocating_shards: 0, unassigned_shards: 950, number_of_data_nodes: 2, number_of_nodes: 5, active_primary_shards: 153, timed_out: False, task_max_waiting_in_queue_millis: 135437, active_shards_percent_as_number: 14.768683274021353, status: red, delayed_unassigned_shards: 0, number_of_pending_tasks: 75

4 years, 1 month

1
0
0 0

** PROBLEM alert - logstash1008/ElasticSearch health check for shards on 9200 is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9200 Host: logstash1008 Address: 10.64.0.90 State: CRITICAL Date/Time: Thu Feb 27 21:42:10 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - elasticsearch inactive shards 994 threshold =0.34 breach: number_of_nodes: 5, status: red, number_of_pending_tasks: 78, initializing_shards: 8, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 112472, delayed_unassigned_shards: 0, active_shards: 130, unassigned_shards: 986, relocating_shards: 0, number_of_data_nodes: 2, active_shards_percent_as_number: 11.565836298932384, timed_out: False, cluster_name: production-logstash-eqiad, active_primary_shards: 117

4 years, 1 month

1
0
0 0

** PROBLEM alert - logstash1011/ElasticSearch health check for shards on 9200 is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9200 Host: logstash1011 Address: 10.64.16.30 State: CRITICAL Date/Time: Thu Feb 27 21:41:40 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - elasticsearch inactive shards 1051 threshold =0.34 breach: status: red, cluster_name: production-logstash-eqiad, unassigned_shards: 1043, relocating_shards: 0, task_max_waiting_in_queue_millis: 83481, active_shards: 73, number_of_data_nodes: 2, number_of_in_flight_fetch: 0, active_shards_percent_as_number: 6.494661921708185, number_of_nodes: 5, number_of_pending_tasks: 78, active_primary_shards: 65, initializing_shards: 8, delayed_unassigned_shards: 0, timed_out: False

4 years, 1 month

1
0
0 0

** PROBLEM alert - logstash1012/ElasticSearch health check for shards on 9200 is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9200 Host: logstash1012 Address: 10.64.48.177 State: CRITICAL Date/Time: Thu Feb 27 21:41:40 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - elasticsearch inactive shards 1051 threshold =0.34 breach: task_max_waiting_in_queue_millis: 81936, delayed_unassigned_shards: 0, number_of_nodes: 5, active_shards_percent_as_number: 6.494661921708185, timed_out: False, status: red, number_of_in_flight_fetch: 0, unassigned_shards: 1043, number_of_pending_tasks: 76, initializing_shards: 8, number_of_data_nodes: 2, active_primary_shards: 65, cluster_name: production-logstash-eqiad, relocating_shards: 0, active_shards: 73

4 years, 1 month

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts February 2020