March 2023 - Discovery-alerts - lists.wikimedia.org

** RECOVERY alert - cloudelastic1003/ElasticSearch health check for shards on 9400 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9400 Host: cloudelastic1003 Address: 208.80.154.83 State: OK Date/Time: Tue Mar 7 14:52:28 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status cloudelastic-omega-eqiad: cluster_name: cloudelastic-omega-eqiad, status: yellow, timed_out: False, number_of_nodes: 6, number_of_data_nodes: 6, active_primary_shards: 779, active_shards: 1369, relocating_shards: 0, initializing_shards: 4, unassigned_shards: 186, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 87.81270044900577

1 year, 2 months

1
0
0 0

** RECOVERY alert - cloudelastic1006/ElasticSearch health check for shards on 9400 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9400 Host: cloudelastic1006 Address: 208.80.154.138 State: OK Date/Time: Tue Mar 7 14:51:54 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status cloudelastic-omega-eqiad: cluster_name: cloudelastic-omega-eqiad, status: yellow, timed_out: False, number_of_nodes: 6, number_of_data_nodes: 6, active_primary_shards: 779, active_shards: 1323, relocating_shards: 0, initializing_shards: 4, unassigned_shards: 232, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 84.86209108402822

1 year, 2 months

1
0
0 0

** RECOVERY alert - cloudelastic1002/ElasticSearch health check for shards on 9400 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9400 Host: cloudelastic1002 Address: 208.80.154.133 State: OK Date/Time: Tue Mar 7 14:51:50 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status cloudelastic-omega-eqiad: cluster_name: cloudelastic-omega-eqiad, status: yellow, timed_out: False, number_of_nodes: 6, number_of_data_nodes: 6, active_primary_shards: 779, active_shards: 1317, relocating_shards: 0, initializing_shards: 4, unassigned_shards: 238, delayed_unassigned_shards: 0, number_of_pending_tasks: 1, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 84.47722899294419

1 year, 2 months

1
0
0 0

** RECOVERY alert - cloudelastic1005/ElasticSearch health check for shards on 9400 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9400 Host: cloudelastic1005 Address: 208.80.154.9 State: OK Date/Time: Tue Mar 7 14:51:38 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status cloudelastic-omega-eqiad: cluster_name: cloudelastic-omega-eqiad, status: yellow, timed_out: False, number_of_nodes: 6, number_of_data_nodes: 6, active_primary_shards: 779, active_shards: 1302, relocating_shards: 0, initializing_shards: 4, unassigned_shards: 253, delayed_unassigned_shards: 0, number_of_pending_tasks: 1, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 83.51507376523412

1 year, 2 months

1
0
0 0

** RECOVERY alert - cloudelastic1004/ElasticSearch health check for shards on 9400 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9400 Host: cloudelastic1004 Address: 208.80.155.107 State: OK Date/Time: Tue Mar 7 14:51:38 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status cloudelastic-omega-eqiad: cluster_name: cloudelastic-omega-eqiad, status: yellow, timed_out: False, number_of_nodes: 6, number_of_data_nodes: 6, active_primary_shards: 779, active_shards: 1300, relocating_shards: 0, initializing_shards: 3, unassigned_shards: 256, delayed_unassigned_shards: 0, number_of_pending_tasks: 4, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 2072, active_shards_percent_as_number: 83.38678640153945

1 year, 2 months

1
0
0 0

** RECOVERY alert - cloudelastic1001/ElasticSearch health check for shards on 9400 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9400 Host: cloudelastic1001 Address: 208.80.154.8 State: OK Date/Time: Tue Mar 7 14:51:34 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status cloudelastic-omega-eqiad: cluster_name: cloudelastic-omega-eqiad, status: yellow, timed_out: False, number_of_nodes: 6, number_of_data_nodes: 6, active_primary_shards: 779, active_shards: 1293, relocating_shards: 0, initializing_shards: 4, unassigned_shards: 262, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 82.93778062860808

1 year, 2 months

1
0
0 0

** PROBLEM alert - cloudelastic1005/ElasticSearch health check for shards on 9400 is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9400 Host: cloudelastic1005 Address: 208.80.154.9 State: CRITICAL Date/Time: Tue Mar 7 14:44:08 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - elasticsearch inactive shards 389 threshold =0.2 breach: cluster_name: cloudelastic-omega-eqiad, status: yellow, timed_out: False, number_of_nodes: 6, number_of_data_nodes: 6, active_primary_shards: 779, active_shards: 1170, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 389, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 75.04810776138551

1 year, 2 months

1
0
0 0

** PROBLEM alert - cloudelastic1001/ElasticSearch health check for shards on 9400 is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9400 Host: cloudelastic1001 Address: 208.80.154.8 State: CRITICAL Date/Time: Tue Mar 7 14:44:02 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - elasticsearch inactive shards 389 threshold =0.2 breach: cluster_name: cloudelastic-omega-eqiad, status: yellow, timed_out: False, number_of_nodes: 6, number_of_data_nodes: 6, active_primary_shards: 779, active_shards: 1170, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 389, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 75.04810776138551

1 year, 2 months

1
0
0 0

** RECOVERY alert - relforge1004/ElasticSearch health check for shards on 9200 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: relforge1004 Address: 10.64.21.126 State: OK Date/Time: Tue Mar 7 14:36:30 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status relforge-eqiad: cluster_name: relforge-eqiad, status: green, timed_out: False, number_of_nodes: 2, number_of_data_nodes: 2, active_primary_shards: 176, active_shards: 352, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 0, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 100.0

1 year, 2 months

1
0
0 0

** RECOVERY alert - relforge1004/ElasticSearch health check for shards on 9400 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9400 Host: relforge1004 Address: 10.64.21.126 State: OK Date/Time: Tue Mar 7 14:34:48 UTC 2023 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status relforge-eqiad-small-alpha: cluster_name: relforge-eqiad-small-alpha, status: green, timed_out: False, number_of_nodes: 2, number_of_data_nodes: 2, active_primary_shards: 5, active_shards: 10, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 0, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 100.0

1 year, 2 months

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts March 2023