March 2020 - Discovery-alerts - lists.wikimedia.org

** ACKNOWLEDGEMENT alert - search.svc.eqiad.wmnet/ElasticSearch shard size check - 9243 is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: ACKNOWLEDGEMENT Service: ElasticSearch shard size check - 9243 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: CRITICAL Date/Time: Wed Mar 4 10:49:50 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Acknowledged by Elukey: T246882 Additional Info: CRITICAL - commonswiki_content_1582399079(65gb)

3 years, 9 months

1
0
0 0

** PROBLEM alert - search.svc.eqiad.wmnet/ElasticSearch shard size check - 9243 is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch shard size check - 9243 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: CRITICAL Date/Time: Tue Mar 3 23:13:47 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Acknowledged by : Additional Info: CRITICAL - commonswiki_content_1582399079(65gb)

3 years, 9 months

1
0
0 0

** RECOVERY alert - icinga1001/Mediawiki Cirrussearch update rate - codfw is OK **

by nagios＠icinga1001.wikimedia.org

Notification Type: RECOVERY Service: Mediawiki Cirrussearch update rate - codfw Host: icinga1001 Address: 208.80.154.84 State: OK Date/Time: Tue Mar 3 15:39:27 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Acknowledged by : Additional Info: OK: Less than 1.00% under the threshold [80.0]

3 years, 9 months

1
0
0 0

** PROBLEM alert - icinga1001/Mediawiki Cirrussearch update rate - codfw is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: Mediawiki Cirrussearch update rate - codfw Host: icinga1001 Address: 208.80.154.84 State: CRITICAL Date/Time: Tue Mar 3 14:53:04 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23No_updates https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?panelId=44… Acknowledged by : Additional Info: CRITICAL: 10.00% of data under the critical threshold [50.0]

3 years, 9 months

1
0
0 0

** RECOVERY alert - cloudelastic1001/Old JVM GC check - cloudelastic1001-cloudelastic-chi-eqiad is OK **

by nagios＠icinga1001.wikimedia.org

Notification Type: RECOVERY Service: Old JVM GC check - cloudelastic1001-cloudelastic-chi-eqiad Host: cloudelastic1001 Address: 208.80.154.8 State: OK Date/Time: Tue Mar 3 11:05:22 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Using_jstack_or_jmap_or_other_… https://grafana.wikimedia.org/d/000000462/elasticsearch-memory?orgId=1 Acknowledged by : Additional Info: (C)100 gt (W)80 gt 75.25

3 years, 9 months

1
0
0 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-3-3-6-wf

by oozie＠an-coord1001.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

3 years, 9 months

1
0
0 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-3-3-5-wf

by oozie＠an-coord1001.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

3 years, 9 months

1
0
0 0

** PROBLEM alert - cloudelastic1001/Old JVM GC check - cloudelastic1001-cloudelastic-chi-eqiad is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: Old JVM GC check - cloudelastic1001-cloudelastic-chi-eqiad Host: cloudelastic1001 Address: 208.80.154.8 State: CRITICAL Date/Time: Tue Mar 3 09:16:49 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Using_jstack_or_jmap_or_other_… https://grafana.wikimedia.org/d/000000462/elasticsearch-memory?orgId=1 Acknowledged by : Additional Info: 136 gt 100

3 years, 9 months

1
0
0 0

** RECOVERY alert - cloudelastic1001/Old JVM GC check - cloudelastic1001-cloudelastic-chi-eqiad is OK **

by nagios＠icinga1001.wikimedia.org

Notification Type: RECOVERY Service: Old JVM GC check - cloudelastic1001-cloudelastic-chi-eqiad Host: cloudelastic1001 Address: 208.80.154.8 State: OK Date/Time: Tue Mar 3 04:22:39 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Using_jstack_or_jmap_or_other_… https://grafana.wikimedia.org/d/000000462/elasticsearch-memory?orgId=1 Acknowledged by : Additional Info: (C)100 gt (W)80 gt 71.19

3 years, 9 months

1
0
0 0

** PROBLEM alert - cloudelastic1001/Old JVM GC check - cloudelastic1001-cloudelastic-chi-eqiad is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: Old JVM GC check - cloudelastic1001-cloudelastic-chi-eqiad Host: cloudelastic1001 Address: 208.80.154.8 State: CRITICAL Date/Time: Tue Mar 3 02:20:35 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Using_jstack_or_jmap_or_other_… https://grafana.wikimedia.org/d/000000462/elasticsearch-memory?orgId=1 Acknowledged by : Additional Info: 132.2 gt 100

3 years, 9 months

1
0
0 0

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts March 2020