March 2018 - Discovery-alerts - lists.wikimedia.org

** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL **
by icinga＠einsteinium.wikimedia.org 28 Mar '18

28 Mar '18

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: CRITICAL Date/Time: Wed Mar 28 09:33:23 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: CRITICAL: 30.00% of data above the critical threshold [1000.0]

1 0

** RECOVERY alert - graphite1001/CirrusSearch eqiad 95th percentile latency is OK **
by icinga＠einsteinium.wikimedia.org 27 Mar '18

27 Mar '18

Notification Type: RECOVERY Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: OK Date/Time: Tue Mar 27 15:41:56 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: OK: Less than 20.00% above the threshold [500.0]

1 0

** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL **
by icinga＠einsteinium.wikimedia.org 27 Mar '18

27 Mar '18

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: CRITICAL Date/Time: Tue Mar 27 15:13:46 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: CRITICAL: 30.00% of data above the critical threshold [1000.0]

1 0

** RECOVERY alert - graphite1001/CirrusSearch eqiad 95th percentile latency is OK **
by icinga＠einsteinium.wikimedia.org 26 Mar '18

26 Mar '18

Notification Type: RECOVERY Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: OK Date/Time: Mon Mar 26 18:39:01 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: OK: Less than 20.00% above the threshold [500.0]

1 0

** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL **
by icinga＠einsteinium.wikimedia.org 26 Mar '18

26 Mar '18

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: CRITICAL Date/Time: Mon Mar 26 17:23:35 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: CRITICAL: 30.00% of data above the critical threshold [1000.0]

1 0

** RECOVERY alert - graphite1001/CirrusSearch eqiad 95th percentile latency is OK **
by icinga＠einsteinium.wikimedia.org 26 Mar '18

26 Mar '18

Notification Type: RECOVERY Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: OK Date/Time: Mon Mar 26 16:34:26 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: OK: Less than 20.00% above the threshold [500.0]

1 0

** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL **
by icinga＠einsteinium.wikimedia.org 26 Mar '18

26 Mar '18

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: CRITICAL Date/Time: Mon Mar 26 16:00:16 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: CRITICAL: 20.00% of data above the critical threshold [1000.0]

1 0

** RECOVERY alert - graphite1001/CirrusSearch eqiad 95th percentile latency is OK **
by icinga＠einsteinium.wikimedia.org 26 Mar '18

26 Mar '18

Notification Type: RECOVERY Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: OK Date/Time: Mon Mar 26 15:36:15 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: OK: Less than 20.00% above the threshold [500.0]

1 0

** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL **
by icinga＠einsteinium.wikimedia.org 26 Mar '18

26 Mar '18

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: CRITICAL Date/Time: Mon Mar 26 15:12:05 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: CRITICAL: 30.00% of data above the critical threshold [1000.0]

1 0

** RECOVERY alert - search.svc.eqiad.wmnet/ElasticSearch health check for shards is OK **
by icinga＠einsteinium.wikimedia.org 01 Mar '18

01 Mar '18

Notification Type: RECOVERY Service: ElasticSearch health check for shards Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: OK Date/Time: Thu Mar 1 16:04:22 UTC 2018 Notes URLs: Additional Info: OK - elasticsearch status production-search-eqiad: status: yellow, number_of_nodes: 35, unassigned_shards: 780, number_of_pending_tasks: 60, number_of_in_flight_fetch: 0, timed_out: False, active_primary_shards: 3127, task_max_waiting_in_queue_millis: 261965, cluster_name: production-search-eqiad, relocating_shards: 23, active_shards_percent_as_number: 91.5699840341, active_shards: 8603, initializing_shards: 12, number_of_data_nodes: 35, delayed_unassigned_shards: 0

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts March 2018