October 2019 - Discovery-alerts - lists.wikimedia.org

** RECOVERY alert - search.svc.eqiad.wmnet/ElasticSearch unassigned shard check - 9243 is OK **
by nagios＠icinga1001.wikimedia.org 30 Oct '19

30 Oct '19

Notification Type: RECOVERY Service: ElasticSearch unassigned shard check - 9243 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: OK Date/Time: Wed Oct 30 18:42:18 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - All good!

1 0

** RECOVERY alert - search.svc.eqiad.wmnet/ElasticSearch numbers of masters eligible - 9243 is OK **
by nagios＠icinga1001.wikimedia.org 30 Oct '19

30 Oct '19

Notification Type: RECOVERY Service: ElasticSearch numbers of masters eligible - 9243 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: OK Date/Time: Wed Oct 30 14:49:56 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Expected_eligible_masters_chec… Acknowledged by : Additional Info: OK - All good

1 0

** PROBLEM alert - search.svc.eqiad.wmnet/ElasticSearch numbers of masters eligible - 9243 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 30 Oct '19

30 Oct '19

Notification Type: PROBLEM Service: ElasticSearch numbers of masters eligible - 9243 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: CRITICAL Date/Time: Wed Oct 30 14:31:44 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Expected_eligible_masters_chec… Acknowledged by : Additional Info: CRITICAL - Found 2 eligible masters.

1 0

** RECOVERY alert - search.svc.codfw.wmnet/ElasticSearch shard size check - 9243 is OK **
by nagios＠icinga1001.wikimedia.org 24 Oct '19

24 Oct '19

Notification Type: RECOVERY Service: ElasticSearch shard size check - 9243 Host: search.svc.codfw.wmnet Address: 10.2.1.30 State: OK Date/Time: Thu Oct 24 18:26:22 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Acknowledged by : Additional Info: OK - All good!

1 0

** PROBLEM alert - search.svc.codfw.wmnet/ElasticSearch shard size check - 9243 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 23 Oct '19

23 Oct '19

Notification Type: PROBLEM Service: ElasticSearch shard size check - 9243 Host: search.svc.codfw.wmnet Address: 10.2.1.30 State: CRITICAL Date/Time: Wed Oct 23 18:26:12 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Acknowledged by : Additional Info: CRITICAL - commonswiki_content_1556151793(67gb)

1 0

** RECOVERY alert - graphite1004/CirrusSearch eqiad 95th percentile latency is OK **
by nagios＠icinga1001.wikimedia.org 21 Oct '19

21 Oct '19

Notification Type: RECOVERY Service: CirrusSearch eqiad 95th percentile latency Host: graphite1004 Address: 10.64.16.149 State: OK Date/Time: Mon Oct 21 16:39:07 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Acknowledged by : Additional Info: OK: Less than 20.00% above the threshold [500.0]

1 0

** PROBLEM alert - graphite1004/CirrusSearch eqiad 95th percentile latency is CRITICAL **
by nagios＠icinga1001.wikimedia.org 21 Oct '19

21 Oct '19

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1004 Address: 10.64.16.149 State: CRITICAL Date/Time: Mon Oct 21 16:26:19 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Acknowledged by : Additional Info: CRITICAL: 40.00% of data above the critical threshold [1000.0]

1 0

** PROBLEM alert - search.svc.eqiad.wmnet/ElasticSearch unassigned shard check - 9243 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 20 Oct '19

20 Oct '19

Notification Type: PROBLEM Service: ElasticSearch unassigned shard check - 9243 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: CRITICAL Date/Time: Sun Oct 20 21:06:29 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - dewiki_content_1566659363[5](2019-10-17T11:16:36.023Z), enwiki_content_1546970425[1](2019-10-17T11:16:49.266Z), enwiki_content_1546970425[6](2019-10-17T11:16:49.267Z)

1 0

** RECOVERY alert - graphite1004/CirrusSearch eqiad 95th percentile latency is OK **
by nagios＠icinga1001.wikimedia.org 20 Oct '19

20 Oct '19

Notification Type: RECOVERY Service: CirrusSearch eqiad 95th percentile latency Host: graphite1004 Address: 10.64.16.149 State: OK Date/Time: Sun Oct 20 19:47:53 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Acknowledged by : Additional Info: OK: Less than 20.00% above the threshold [500.0]

1 0

** PROBLEM alert - graphite1004/CirrusSearch eqiad 95th percentile latency is CRITICAL **
by nagios＠icinga1001.wikimedia.org 20 Oct '19

20 Oct '19

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1004 Address: 10.64.16.149 State: CRITICAL Date/Time: Sun Oct 20 19:38:15 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Acknowledged by : Additional Info: CRITICAL: 40.00% of data above the critical threshold [1000.0]

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts October 2019