July 2020 - Discovery-alerts - lists.wikimedia.org

** PROBLEM alert - graphite1004/CirrusSearch eqiad 95th percentile latency is CRITICAL **
by nagios＠icinga1001.wikimedia.org 18 Jul '20

18 Jul '20

Notification Type: PROBLEM Service: CirrusSearch eqiad 95th percentile latency Host: graphite1004 Address: 10.64.16.149 State: CRITICAL Date/Time: Sat Jul 18 19:43:39 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Acknowledged by : Additional Info: CRITICAL: 20.00% of data above the critical threshold [1000.0]

1 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-7-18-15-wf
by oozie＠an-coord1001.eqiad.wmnet 18 Jul '20

18 Jul '20

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

1 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-7-18-14-wf
by oozie＠an-coord1001.eqiad.wmnet 18 Jul '20

18 Jul '20

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

1 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-7-18-10-wf
by oozie＠an-coord1001.eqiad.wmnet 18 Jul '20

18 Jul '20

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

1 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-7-18-4-wf
by oozie＠an-coord1001.eqiad.wmnet 18 Jul '20

18 Jul '20

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

1 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-7-18-0-wf
by oozie＠an-coord1001.eqiad.wmnet 17 Jul '20

17 Jul '20

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

1 0

** RECOVERY alert - search.svc.eqiad.wmnet/ElasticSearch unassigned shard check - 9243 is OK **
by nagios＠icinga1001.wikimedia.org 17 Jul '20

17 Jul '20

Notification Type: RECOVERY Service: ElasticSearch unassigned shard check - 9243 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: OK Date/Time: Sat Jul 18 01:52:33 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - All good

1 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-7-17-19-wf
by oozie＠an-coord1001.eqiad.wmnet 17 Jul '20

17 Jul '20

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

1 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-7-17-17-wf
by oozie＠an-coord1001.eqiad.wmnet 17 Jul '20

17 Jul '20

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

1 0

** ACKNOWLEDGEMENT alert - search.svc.eqiad.wmnet/ElasticSearch unassigned shard check - 9243 is CRITICAL **
by nagios＠icinga1001.wikimedia.org 17 Jul '20

17 Jul '20

Notification Type: ACKNOWLEDGEMENT Service: ElasticSearch unassigned shard check - 9243 Host: search.svc.eqiad.wmnet Address: 10.2.2.30 State: CRITICAL Date/Time: Fri Jul 17 18:19:48 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by Ryan Kemper: Failure to assign shard due to unforeseen implications of row awareness. Working on a patch for this upcoming monday Additional Info: CRITICAL - dewiki_content_1594699515[0](2020-07-14T04:05:15.243Z)

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts July 2020