Notification Type: ACKNOWLEDGEMENT
Service: CirrusSearch eqiad 95th percentile latency
Host: graphite1004
Address: 10.64.16.149
State: CRITICAL
Date/Time: Tue Jul 28 21:06:04 UTC 2020
Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoringhttps://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI…
Acknowledged by Ryan Kemper: suspected to be related to a config value that determines the number of results that are run through the ML models per-shard on a search request. patch will be opened up
Additional Info:
CRITICAL: 90.00% of data above the critical threshold [1000.0]