Notification Type: PROBLEM
Service: ElasticSearch health check for shards on 9243 Host: search.svc.codfw.wmnet Address: 10.2.1.30 State: CRITICAL
Date/Time: Thu May 16 18:21:17 UTC 2019
Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration
Additional Info:
CRITICAL - elasticsearch inactive shards 988 threshold =0.2 breach: number_of_data_nodes: 29, delayed_unassigned_shards: 0, active_shards_percent_as_number: 74.4438696326953, relocating_shards: 0, timed_out: False, active_primary_shards: 1246, task_max_waiting_in_queue_millis: 0, number_of_nodes: 29, number_of_in_flight_fetch: 0, active_shards: 2878, unassigned_shards: 988, initializing_shards: 0, status: red, cluster_name: production-search-codfw, number_of_pending_tasks: 0