- Discovery-alerts - lists.wikimedia.org

** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL **

by icinga＠einsteinium.wikimedia.org

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: CRITICAL Date/Time: Tue Jun 19 02:43:12 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: CRITICAL: 20.00% of data above the critical threshold [1000.0]

5 years, 11 months

1
0
0 0

** RECOVERY alert - search.svc.eqiad.wmnet/ElasticSearch health check for shards is OK **

by icinga＠einsteinium.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: OK Date/Time: Mon Jun 4 15:24:35 UTC 2018 Notes URLs: Additional Info: OK - elasticsearch status production-search-eqiad: status: yellow, number_of_nodes: 35, unassigned_shards: 788, number_of_pending_tasks: 253, number_of_in_flight_fetch: 0, timed_out: False, active_primary_shards: 3114, task_max_waiting_in_queue_millis: 45525, cluster_name: production-search-eqiad, relocating_shards: 4, active_shards_percent_as_number: 91.3940560188, active_shards: 8549, initializing_shards: 17, number_of_data_nodes: 35, delayed_unassigned_shards: 0

6 years

1
0
0 0

** PROBLEM alert - search.svc.eqiad.wmnet/ElasticSearch health check for shards is CRITICAL **

by icinga＠einsteinium.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch health check for shards Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: CRITICAL Date/Time: Mon Jun 4 15:22:15 UTC 2018 Notes URLs: Additional Info: CRITICAL - elasticsearch inactive shards 1001 threshold =0.1 breach: status: yellow, number_of_nodes: 32, unassigned_shards: 996, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, timed_out: False, active_primary_shards: 3114, task_max_waiting_in_queue_millis: 0, cluster_name: production-search-eqiad, relocating_shards: 5, active_shards_percent_as_number: 89.3521488133, active_shards: 8358, initializing_shards: 0, number_of_data_nodes: 32, delayed_unassigned_shards: 0

6 years

1
0
0 0

Fatal Error - Oozie Job discovery-transfer_to_es-discovery.popularity_score-2018, 5, 28->https://elastic2010.codfw.wmnet:9243-wf

by oozie＠analytics1003.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: transfer - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

6 years

1
0
0 0

Fatal Error - Oozie Job discovery-transfer_to_es-discovery.popularity_score-2018, 5, 28->https://elastic1017.eqiad.wmnet:9243-wf

by oozie＠analytics1003.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: transfer - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

6 years

1
0
0 0

Fatal Error - Oozie Job discovery-transfer_to_es-discovery.popularity_score-2018, 5, 14->https://elastic1017.eqiad.wmnet:9243-wf

by oozie＠analytics1003.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: transfer - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

6 years

1
0
0 0

** RECOVERY alert - graphite1001/CirrusSearch eqiad 95th percentile latency is OK **

by icinga＠einsteinium.wikimedia.org

Notification Type: RECOVERY Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: OK Date/Time: Mon Apr 23 21:01:00 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: OK: Less than 20.00% above the threshold [500.0]

6 years, 1 month

1
0
0 0

** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL **

by icinga＠einsteinium.wikimedia.org

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: CRITICAL Date/Time: Mon Apr 23 20:44:00 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: CRITICAL: 20.00% of data above the critical threshold [1000.0]

6 years, 1 month

1
0
0 0

** RECOVERY alert - graphite1001/CirrusSearch eqiad 95th percentile latency is OK **

by icinga＠einsteinium.wikimedia.org

Notification Type: RECOVERY Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: OK Date/Time: Mon Apr 23 18:51:15 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: OK: Less than 20.00% above the threshold [500.0]

6 years, 1 month

1
0
0 0

** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL **

by icinga＠einsteinium.wikimedia.org

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1001 Address: 10.64.32.155 State: CRITICAL Date/Time: Mon Apr 23 18:40:15 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Additional Info: CRITICAL: 20.00% of data above the critical threshold [1000.0]

6 years, 1 month

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts