March 2020 - Discovery-alerts - lists.wikimedia.org

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-3-2-4-wf

by oozie＠an-coord1001.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

4 years, 2 months

1
0
0 0

** RECOVERY alert - stat1007/Check the last execution of search-drop-query-clicks is OK **

by nagios＠icinga1001.wikimedia.org

Notification Type: RECOVERY Service: Check the last execution of search-drop-query-clicks Host: stat1007 Address: 10.64.21.118 State: OK Date/Time: Mon Mar 2 02:22:41 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: OK: Status of the systemd unit search-drop-query-clicks

4 years, 2 months

1
0
0 0

** PROBLEM alert - stat1007/Check the last execution of search-drop-query-clicks is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: Check the last execution of search-drop-query-clicks Host: stat1007 Address: 10.64.21.118 State: CRITICAL Date/Time: Mon Mar 2 02:11:31 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: connect to address 10.64.21.118 port 5666: Connection refused

4 years, 2 months

1
0
0 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2020-3-1-22-wf

by oozie＠an-coord1001.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

4 years, 2 months

1
0
0 0

** PROBLEM alert - cloudelastic1004/ElasticSearch unassigned shard check - 9200 is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch unassigned shard check - 9200 Host: cloudelastic1004 Address: 208.80.155.107 State: CRITICAL Date/Time: Sun Mar 1 23:46:38 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - ebernhardson_deleteme[6](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[6](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[4](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[4](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[2](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[5](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[5](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[3](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[1](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[7](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[7](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[0](2020-02-27T17:24:54.480Z)

4 years, 2 months

1
0
0 0

** PROBLEM alert - cloudelastic1003/ElasticSearch unassigned shard check - 9200 is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch unassigned shard check - 9200 Host: cloudelastic1003 Address: 208.80.154.83 State: CRITICAL Date/Time: Sun Mar 1 23:26:34 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - ebernhardson_deleteme[1](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[4](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[4](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[3](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[7](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[7](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[5](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[5](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[6](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[6](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[2](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[0](2020-02-27T17:24:54.480Z)

4 years, 2 months

1
0
0 0

** PROBLEM alert - cloudelastic1001/ElasticSearch unassigned shard check - 9200 is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch unassigned shard check - 9200 Host: cloudelastic1001 Address: 208.80.154.8 State: CRITICAL Date/Time: Sun Mar 1 22:51:04 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - ebernhardson_deleteme[6](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[6](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[2](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[3](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[1](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[7](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[7](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[4](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[4](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[5](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[5](2020-02-27T17:24:54.480Z), ebernhardson_deleteme[0](2020-02-27T17:24:54.480Z)

4 years, 2 months

1
0
0 0

** RECOVERY alert - stat1007/Check the last execution of search-drop-query-clicks is OK **

by nagios＠icinga1001.wikimedia.org

Notification Type: RECOVERY Service: Check the last execution of search-drop-query-clicks Host: stat1007 Address: 10.64.21.118 State: OK Date/Time: Sun Mar 1 22:47:56 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: OK: Status of the systemd unit search-drop-query-clicks

4 years, 2 months

1
0
0 0

** PROBLEM alert - stat1007/Check the last execution of search-drop-query-clicks is CRITICAL **

by nagios＠icinga1001.wikimedia.org

Notification Type: PROBLEM Service: Check the last execution of search-drop-query-clicks Host: stat1007 Address: 10.64.21.118 State: CRITICAL Date/Time: Sun Mar 1 22:25:36 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: connect to address 10.64.21.118 port 5666: Connection refused

4 years, 2 months

1
0
0 0

** RECOVERY alert - stat1007/Check the last execution of search-drop-query-clicks is OK **

by nagios＠icinga1001.wikimedia.org

Notification Type: RECOVERY Service: Check the last execution of search-drop-query-clicks Host: stat1007 Address: 10.64.21.118 State: OK Date/Time: Sun Mar 1 22:12:14 UTC 2020 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: OK: Status of the systemd unit search-drop-query-clicks

4 years, 2 months

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts March 2020