Notification Type: PROBLEM

Service: ElasticSearch health check for shards on 9443 Host: search.svc.codfw.wmnet Address: 10.2.1.30 State: CRITICAL

Date/Time: Wed Mar 27 20:09:56 UTC 2019

Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration

Additional Info:

CRITICAL - elasticsearch inactive shards 556 threshold =0.15 breach: relocating_shards: 0, number_of_nodes: 15, active_primary_shards: 1108, number_of_in_flight_fetch: 0, status: yellow, number_of_pending_tasks: 0, unassigned_shards: 546, number_of_data_nodes: 15, initializing_shards: 10, active_shards_percent_as_number: 83.26813120674089, task_max_waiting_in_queue_millis: 0, timed_out: False, cluster_name: production-search-omega-codfw, active_shards: 2767, delayed_unassigned_shards: 0