Notification Type: ACKNOWLEDGEMENT
Service: CirrusSearch eqiad 95th percentile latency Host: graphite1004 Address: 10.64.16.149 State: CRITICAL
Date/Time: Tue Jul 28 21:06:04 UTC 2020
Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelId...
Acknowledged by Ryan Kemper: suspected to be related to a config value that determines the number of results that are run through the ML models per-shard on a search request. patch will be opened up
Additional Info:
CRITICAL: 90.00% of data above the critical threshold [1000.0]