October 2020 - Discovery-alerts - lists.wikimedia.org

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-10-21-16-wf

by oozie＠an-coord1001.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

3 years, 5 months

1
0
0 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-10-21-4-wf

by oozie＠an-coord1001.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

3 years, 6 months

1
0
0 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-10-21-2-wf

by oozie＠an-coord1001.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

3 years, 6 months

1
0
0 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-10-21-1-wf

by oozie＠an-coord1001.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

3 years, 6 months

1
0
0 0

** RECOVERY alert - graphite1004/CirrusSearch eqiad 95th percentile latency is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: CirrusSearch eqiad 95th percentile latency Host: graphite1004 Address: 10.64.16.149 State: OK Date/Time: Wed Oct 21 04:11:31 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Acknowledged by : Additional Info: OK: Less than 20.00% above the threshold [500.0]

3 years, 6 months

1
0
0 0

** PROBLEM alert - graphite1004/CirrusSearch eqiad 95th percentile latency is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1004 Address: 10.64.16.149 State: CRITICAL Date/Time: Wed Oct 21 04:04:57 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Acknowledged by : Additional Info: CRITICAL: 30.00% of data above the critical threshold [1000.0]

3 years, 6 months

1
0
0 0

** ACKNOWLEDGEMENT alert - search.svc.eqiad.wmnet/ElasticSearch shard size check - 9243 is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: ACKNOWLEDGEMENT Service: ElasticSearch shard size check - 9243 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: CRITICAL Date/Time: Tue Oct 20 17:25:38 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Acknowledged by Ryan Kemper: this was supposed to have been fixed looking into if a value was missed in the puppet repo somewhere Additional Info: CRITICAL - commonswiki_content_1587080795(60.833333333333336gb), commonswiki_file_1595354515(54.510416666666664gb)

3 years, 6 months

1
0
0 0

** PROBLEM alert - search.svc.eqiad.wmnet/ElasticSearch shard size check - 9243 is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch shard size check - 9243 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: CRITICAL Date/Time: Tue Oct 20 16:10:43 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23If_it_has_been_indexed Acknowledged by : Additional Info: CRITICAL - commonswiki_content_1587080795(60.833333333333336gb), commonswiki_file_1595354515(54.510416666666664gb)

3 years, 6 months

1
0
0 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-10-20-9-wf

by oozie＠an-coord1001.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

3 years, 6 months

1
0
0 0

** RECOVERY alert - graphite1004/CirrusSearch eqiad 95th percentile latency is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: CirrusSearch eqiad 95th percentile latency Host: graphite1004 Address: 10.64.16.149 State: OK Date/Time: Tue Oct 20 06:32:51 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Acknowledged by : Additional Info: OK: Less than 20.00% above the threshold [500.0]

3 years, 6 months

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts October 2020