December 2022 - Discovery-alerts - lists.wikimedia.org

** RECOVERY alert - relforge1004/ElasticSearch unassigned shard check - 9400 is OK **
by nagios＠alert1001.wikimedia.org 13 Dec '22

13 Dec '22

Notification Type: RECOVERY Service: ElasticSearch unassigned shard check - 9400 Host: relforge1004 Address: 10.64.21.126 State: OK Date/Time: Tue Dec 13 00:38:13 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - All good

1 0

** RECOVERY alert - relforge1003/ElasticSearch unassigned shard check - 9400 is OK **
by nagios＠alert1001.wikimedia.org 13 Dec '22

13 Dec '22

Notification Type: RECOVERY Service: ElasticSearch unassigned shard check - 9400 Host: relforge1003 Address: 10.64.5.37 State: OK Date/Time: Tue Dec 13 00:38:13 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - All good

1 0

** RECOVERY alert - relforge1004/ElasticSearch unassigned shard check - 9200 is OK **
by nagios＠alert1001.wikimedia.org 13 Dec '22

13 Dec '22

Notification Type: RECOVERY Service: ElasticSearch unassigned shard check - 9200 Host: relforge1004 Address: 10.64.21.126 State: OK Date/Time: Tue Dec 13 00:33:49 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - All good

1 0

** RECOVERY alert - relforge1003/ElasticSearch unassigned shard check - 9200 is OK **
by nagios＠alert1001.wikimedia.org 13 Dec '22

13 Dec '22

Notification Type: RECOVERY Service: ElasticSearch unassigned shard check - 9200 Host: relforge1003 Address: 10.64.5.37 State: OK Date/Time: Tue Dec 13 00:33:21 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - All good

1 0

** RECOVERY alert - relforge1003/ElasticSearch health check for shards on 9400 is OK **
by nagios＠alert1001.wikimedia.org 12 Dec '22

12 Dec '22

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9400 Host: relforge1003 Address: 10.64.5.37 State: OK Date/Time: Mon Dec 12 16:40:47 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status relforge-eqiad-small-alpha: cluster_name: relforge-eqiad-small-alpha, status: green, timed_out: False, number_of_nodes: 2, number_of_data_nodes: 2, active_primary_shards: 5, active_shards: 10, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 0, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 100.0

1 0

** RECOVERY alert - relforge1004/ElasticSearch health check for shards on 9400 is OK **
by nagios＠alert1001.wikimedia.org 12 Dec '22

12 Dec '22

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9400 Host: relforge1004 Address: 10.64.21.126 State: OK Date/Time: Mon Dec 12 16:40:31 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status relforge-eqiad-small-alpha: cluster_name: relforge-eqiad-small-alpha, status: green, timed_out: False, number_of_nodes: 2, number_of_data_nodes: 2, active_primary_shards: 5, active_shards: 10, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 0, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 100.0

1 0

** RECOVERY alert - relforge1003/ElasticSearch health check for shards on 9200 is OK **
by nagios＠alert1001.wikimedia.org 12 Dec '22

12 Dec '22

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: relforge1003 Address: 10.64.5.37 State: OK Date/Time: Mon Dec 12 14:08:17 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status relforge-eqiad: cluster_name: relforge-eqiad, status: yellow, timed_out: False, number_of_nodes: 2, number_of_data_nodes: 2, active_primary_shards: 159, active_shards: 315, relocating_shards: 0, initializing_shards: 2, unassigned_shards: 1, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 99.05660377358491

1 0

** RECOVERY alert - relforge1004/ElasticSearch health check for shards on 9200 is OK **
by nagios＠alert1001.wikimedia.org 12 Dec '22

12 Dec '22

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: relforge1004 Address: 10.64.21.126 State: OK Date/Time: Mon Dec 12 14:07:37 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status relforge-eqiad: cluster_name: relforge-eqiad, status: yellow, timed_out: False, number_of_nodes: 2, number_of_data_nodes: 2, active_primary_shards: 159, active_shards: 315, relocating_shards: 0, initializing_shards: 2, unassigned_shards: 1, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 99.05660377358491

1 0

** ACKNOWLEDGEMENT alert - relforge1004/ElasticSearch unassigned shard check - 9400 is CRITICAL **
by nagios＠alert1001.wikimedia.org 12 Dec '22

12 Dec '22

Notification Type: ACKNOWLEDGEMENT Service: ElasticSearch unassigned shard check - 9400 Host: relforge1004 Address: 10.64.21.126 State: CRITICAL Date/Time: Mon Dec 12 07:59:54 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by Guillaume Lederrey: https://phabricator.wikimedia.org/T324939 Additional Info: CRITICAL - mw_cirrus_metastore[2](2022-12-08T19:59:50.731Z), mw_cirrus_metastore[4](2022-12-08T19:59:50.729Z), mw_cirrus_metastore[1](2022-12-08T19:59:50.731Z), mw_cirrus_metastore[3](2022-12-08T19:59:50.731Z), mw_cirrus_metastore[0](2022-12-08T19:59:50.731Z)

1 0

** ACKNOWLEDGEMENT alert - relforge1004/ElasticSearch health check for shards on 9400 is CRITICAL **
by nagios＠alert1001.wikimedia.org 12 Dec '22

12 Dec '22

Notification Type: ACKNOWLEDGEMENT Service: ElasticSearch health check for shards on 9400 Host: relforge1004 Address: 10.64.21.126 State: CRITICAL Date/Time: Mon Dec 12 07:59:54 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by Guillaume Lederrey: https://phabricator.wikimedia.org/T324939 Additional Info: CRITICAL - elasticsearch inactive shards 5 threshold =0.15 breach: cluster_name: relforge-eqiad-small-alpha, status: yellow, timed_out: False, number_of_nodes: 2, number_of_data_nodes: 2, active_primary_shards: 5, active_shards: 5, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 5, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 50.0

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts December 2022