August 2019 - Discovery-alerts - lists.wikimedia.org

selenium-daily-beta-CirrusSearch - Build # 367 - Still Failing!
by jenkins-bot 06 Sep '19

06 Sep '19

1 18

** RECOVERY alert - search.svc.codfw.wmnet/ElasticSearch shard size check - 9243 is OK **
by nagios＠icinga1001.wikimedia.org 29 Aug '19

29 Aug '19

Notification Type: RECOVERY Service: ElasticSearch shard size check - 9243 Host: search.svc.codfw.wmnet Address: 10.2.1.30 State: OK Date/Time: Thu Aug 29 02:52:05 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Acknowledged by : Additional Info: OK - All good!

1 0

** RECOVERY alert - search.svc.eqiad.wmnet/ElasticSearch shard size check - 9243 is OK **
by nagios＠icinga1001.wikimedia.org 28 Aug '19

28 Aug '19

Notification Type: RECOVERY Service: ElasticSearch shard size check - 9243 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: OK Date/Time: Wed Aug 28 19:34:43 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Acknowledged by : Additional Info: OK - All good!

1 0

** PROBLEM alert - search.svc.codfw.wmnet/ElasticSearch shard size check - 9243 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 28 Aug '19

28 Aug '19

Notification Type: PROBLEM Service: ElasticSearch shard size check - 9243 Host: search.svc.codfw.wmnet Address: 10.2.1.30 State: CRITICAL Date/Time: Wed Aug 28 02:49:39 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Acknowledged by : Additional Info: CRITICAL - commonswiki_content_1556151793(82gb)

1 0

** PROBLEM alert - search.svc.eqiad.wmnet/ElasticSearch shard size check - 9243 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 27 Aug '19

27 Aug '19

Notification Type: PROBLEM Service: ElasticSearch shard size check - 9243 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: CRITICAL Date/Time: Tue Aug 27 19:34:10 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Acknowledged by : Additional Info: CRITICAL - commonswiki_content_1556235298(60.333333333333336gb)

1 0

** RECOVERY alert - graphite1004/CirrusSearch eqiad 95th percentile latency is OK **
by nagios＠icinga1001.wikimedia.org 26 Aug '19

26 Aug '19

Notification Type: RECOVERY Service: CirrusSearch eqiad 95th percentile latency Host: graphite1004 Address: 10.64.16.149 State: OK Date/Time: Mon Aug 26 16:33:03 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Acknowledged by : Additional Info: OK: Less than 20.00% above the threshold [500.0]

1 0

** PROBLEM alert - graphite1004/CirrusSearch eqiad 95th percentile latency is CRITICAL **
by nagios＠icinga1001.wikimedia.org 26 Aug '19

26 Aug '19

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1004 Address: 10.64.16.149 State: CRITICAL Date/Time: Mon Aug 26 16:17:29 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Acknowledged by : Additional Info: CRITICAL: 40.00% of data above the critical threshold [1000.0]

1 0

** RECOVERY alert - graphite1004/Number of backend failures per minute from CirrusSearch is OK **
by nagios＠icinga1001.wikimedia.org 22 Aug '19

22 Aug '19

Notification Type: RECOVERY Service: Number of backend failures per minute from CirrusSearch Host: graphite1004 Address: 10.64.16.149 State: OK Date/Time: Thu Aug 22 08:26:51 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?orgId=… Acknowledged by : Additional Info: OK: Less than 20.00% above the threshold [300.0]

1 0

** PROBLEM alert - graphite1004/Number of backend failures per minute from CirrusSearch is CRITICAL **
by nagios＠icinga1001.wikimedia.org 22 Aug '19

22 Aug '19

Notification Type: PROBLEM Service: Number of backend failures per minute from CirrusSearch Host: graphite1004 Address: 10.64.16.149 State: CRITICAL Date/Time: Thu Aug 22 08:24:33 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?orgId=… Acknowledged by : Additional Info: CRITICAL: 20.00% of data above the critical threshold [600.0]

1 0

** RECOVERY alert - stat1007/Check the last execution of search-drop-query-clicks is OK **
by nagios＠icinga1001.wikimedia.org 20 Aug '19

20 Aug '19

Notification Type: RECOVERY Service: Check the last execution of search-drop-query-clicks Host: stat1007 Address: 10.64.21.118 State: OK Date/Time: Tue Aug 20 00:56:19 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: OK: Status of the systemd unit search-drop-query-clicks

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts August 2019