- Discovery-alerts - lists.wikimedia.org

** ACKNOWLEDGEMENT alert - search.svc.eqiad.wmnet/ElasticSearch shard size check - 9643 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 15 Aug '19

15 Aug '19

Notification Type: ACKNOWLEDGEMENT Service: ElasticSearch shard size check - 9643 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: CRITICAL Date/Time: Thu Aug 15 02:40:11 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Additional Info: usage: check_elasticsearch_shard_size.py [-h] [--url URL] [--timeout SECONDS]

1 0

** PROBLEM alert - search.svc.codfw.wmnet/ElasticSearch shard size check - 9643 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 15 Aug '19

15 Aug '19

Notification Type: PROBLEM Service: ElasticSearch shard size check - 9643 Host: search.svc.codfw.wmnet Address: 10.2.1.30 State: CRITICAL Date/Time: Thu Aug 15 00:35:07 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Additional Info: usage: check_elasticsearch_shard_size.py [-h] [--url URL] [--timeout SECONDS]

1 0

** PROBLEM alert - search.svc.codfw.wmnet/ElasticSearch shard size check - 9243 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 14 Aug '19

14 Aug '19

Notification Type: PROBLEM Service: ElasticSearch shard size check - 9243 Host: search.svc.codfw.wmnet Address: 10.2.1.30 State: CRITICAL Date/Time: Wed Aug 14 22:23:20 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Additional Info: usage: check_elasticsearch_shard_size.py [-h] [--url URL] [--timeout SECONDS]

1 0

** RECOVERY alert - search.svc.eqiad.wmnet/ElasticSearch health check for shards on 9443 is OK **
by nagios＠icinga1001.wikimedia.org 14 Aug '19

14 Aug '19

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9443 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: OK Date/Time: Wed Aug 14 08:56:17 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Additional Info: OK - elasticsearch status production-search-omega-eqiad: unassigned_shards: 589, number_of_in_flight_fetch: 0, number_of_data_nodes: 18, active_shards: 3811, number_of_pending_tasks: 6, status: yellow, active_shards_percent_as_number: 86.49568769859283, delayed_unassigned_shards: 0, number_of_nodes: 18, cluster_name: production-search-omega-eqiad, initializing_shards: 6, relocating_shards: 0, active_primary_shards: 1469, task_max_waiting_in_queue_millis: 67554, timed_out: False

1 0

** PROBLEM alert - search.svc.eqiad.wmnet/ElasticSearch health check for shards on 9443 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 14 Aug '19

14 Aug '19

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9443 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: CRITICAL Date/Time: Wed Aug 14 08:53:03 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Additional Info: CRITICAL - elasticsearch inactive shards 1098 threshold =0.2 breach: active_shards_percent_as_number: 75.07943713118475, timed_out: False, unassigned_shards: 1098, delayed_unassigned_shards: 0, relocating_shards: 0, number_of_pending_tasks: 0, cluster_name: production-search-omega-eqiad, number_of_in_flight_fetch: 0, number_of_nodes: 15, task_max_waiting_in_queue_millis: 0, active_shards: 3308, status: yellow, active_primary_shards: 1469, number_of_data_nodes: 15, initializing_shards: 0

1 0

** RECOVERY alert - icinga1001/Mediawiki Cirrussearch update rate - eqiad is OK **
by nagios＠icinga1001.wikimedia.org 14 Aug '19

14 Aug '19

Notification Type: RECOVERY Service: Mediawiki Cirrussearch update rate - eqiad Host: icinga1001 Address: 208.80.154.84 State: OK Date/Time: Wed Aug 14 02:07:03 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Additional Info: OK: Less than 1.00% under the threshold [80.0]

1 0

** RECOVERY alert - icinga1001/Mediawiki Cirrussearch update rate - codfw is OK **
by nagios＠icinga1001.wikimedia.org 14 Aug '19

14 Aug '19

Notification Type: RECOVERY Service: Mediawiki Cirrussearch update rate - codfw Host: icinga1001 Address: 208.80.154.84 State: OK Date/Time: Wed Aug 14 02:05:01 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Additional Info: OK: Less than 1.00% under the threshold [80.0]

1 0

** PROBLEM alert - icinga1001/Mediawiki Cirrussearch update rate - codfw is CRITICAL **
by nagios＠icinga1001.wikimedia.org 13 Aug '19

13 Aug '19

Notification Type: PROBLEM Service: Mediawiki Cirrussearch update rate - codfw Host: icinga1001 Address: 208.80.154.84 State: CRITICAL Date/Time: Tue Aug 13 23:43:59 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Additional Info: CRITICAL: 10.00% of data under the critical threshold [50.0]

1 0

** PROBLEM alert - icinga1001/Mediawiki Cirrussearch update rate - eqiad is CRITICAL **
by nagios＠icinga1001.wikimedia.org 13 Aug '19

13 Aug '19

Notification Type: PROBLEM Service: Mediawiki Cirrussearch update rate - eqiad Host: icinga1001 Address: 208.80.154.84 State: CRITICAL Date/Time: Tue Aug 13 23:42:47 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Additional Info: CRITICAL: 10.00% of data under the critical threshold [50.0]

1 0

** PROBLEM alert - icinga1001/Mediawiki Cirrussearch update rate - codfw is CRITICAL **
by nagios＠icinga1001.wikimedia.org 13 Aug '19

13 Aug '19

Notification Type: PROBLEM Service: Mediawiki Cirrussearch update rate - codfw Host: icinga1001 Address: 208.80.154.84 State: CRITICAL Date/Time: Tue Aug 13 22:19:39 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Additional Info: CRITICAL: 20.00% of data under the critical threshold [50.0]

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts