January 2021 - Discovery-alerts - lists.wikimedia.org

** PROBLEM alert - logstash1010/Rate of JVM GC Old generation-s runs - logstash1010-production-logstash-eqiad is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: Rate of JVM GC Old generation-s runs - logstash1010-production-logstash-eqiad Host: logstash1010 Address: 10.64.0.181 State: CRITICAL Date/Time: Wed Jan 13 14:57:25 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Using_jstack_or_jmap_or_other_… https://grafana.wikimedia.org/d/000000462/elasticsearch-memory?orgId=1&var-… Acknowledged by : Additional Info: 101.7 gt 100

3 years, 4 months

1
0
0 0

** RECOVERY alert - logstash1011/Rate of JVM GC Old generation-s runs - logstash1011-production-logstash-eqiad is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: Rate of JVM GC Old generation-s runs - logstash1011-production-logstash-eqiad Host: logstash1011 Address: 10.64.16.30 State: OK Date/Time: Wed Jan 13 10:40:26 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Using_jstack_or_jmap_or_other_… https://grafana.wikimedia.org/d/000000462/elasticsearch-memory?orgId=1&var-… Acknowledged by : Additional Info: (C)100 gt (W)80 gt 77.29

3 years, 4 months

1
0
0 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2021-1-13-6-wf

by oozie＠an-coord1001.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

3 years, 4 months

1
0
0 0

** PROBLEM alert - logstash1011/Rate of JVM GC Old generation-s runs - logstash1011-production-logstash-eqiad is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: Rate of JVM GC Old generation-s runs - logstash1011-production-logstash-eqiad Host: logstash1011 Address: 10.64.16.30 State: CRITICAL Date/Time: Wed Jan 13 07:13:11 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Using_jstack_or_jmap_or_other_… https://grafana.wikimedia.org/d/000000462/elasticsearch-memory?orgId=1&var-… Acknowledged by : Additional Info: 100.7 gt 100

3 years, 4 months

1
0
0 0

Fatal Error - Oozie Job discovery-query-clicks-hourly-discovery.query_clicks_hourly-2021-1-13-1-wf

by oozie＠an-coord1001.eqiad.wmnet

Error details: - Job id: UNSET - Failing action: aggregate - Error code: UNSET - Error message: UNSET - Hue link: https://hue.wikimedia.org/oozie/list_oozie_workflows Please have a look and take necessary action ! Thanks :) -- Oozie

3 years, 4 months

1
0
0 0

** RECOVERY alert - logstash1011/Rate of JVM GC Old generation-s runs - logstash1011-production-logstash-eqiad is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: Rate of JVM GC Old generation-s runs - logstash1011-production-logstash-eqiad Host: logstash1011 Address: 10.64.16.30 State: OK Date/Time: Wed Jan 13 01:31:20 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Using_jstack_or_jmap_or_other_… https://grafana.wikimedia.org/d/000000462/elasticsearch-memory?orgId=1&var-… Acknowledged by : Additional Info: (C)100 gt (W)80 gt 77.29

3 years, 4 months

1
0
0 0

** RECOVERY alert - logstash1008/ElasticSearch health check for shards on 9200 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: logstash1008 Address: 10.64.0.90 State: OK Date/Time: Wed Jan 13 00:50:44 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status production-logstash-eqiad: unassigned_shards: 0, active_shards: 916, number_of_nodes: 6, relocating_shards: 4, active_shards_percent_as_number: 100.0, number_of_in_flight_fetch: 0, initializing_shards: 0, delayed_unassigned_shards: 0, active_primary_shards: 483, cluster_name: production-logstash-eqiad, status: green, task_max_waiting_in_queue_millis: 0, number_of_data_nodes: 3, timed_out: False, number_of_pending_tasks: 0

3 years, 4 months

1
0
0 0

** PROBLEM alert - logstash1008/ElasticSearch health check for shards on 9200 is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9200 Host: logstash1008 Address: 10.64.0.90 State: CRITICAL Date/Time: Wed Jan 13 00:49:10 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - elasticsearch http://localhost:9200/_cluster/health error while fetching: HTTPConnectionPool(host=localhost, port=9200): Max retries exceeded with url: /_cluster/health (Caused by NewConnectionError(requests.packages.urllib3.connection.HTTPConnection object at 0x7f2a92cbe4e0: Failed to establish a new connection: [Errno 111] Connection refused,))

3 years, 4 months

1
0
0 0

** PROBLEM alert - logstash1011/Rate of JVM GC Old generation-s runs - logstash1011-production-logstash-eqiad is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: Rate of JVM GC Old generation-s runs - logstash1011-production-logstash-eqiad Host: logstash1011 Address: 10.64.16.30 State: CRITICAL Date/Time: Wed Jan 13 00:42:48 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Using_jstack_or_jmap_or_other_… https://grafana.wikimedia.org/d/000000462/elasticsearch-memory?orgId=1&var-… Acknowledged by : Additional Info: 100.7 gt 100

3 years, 4 months

1
0
0 0

Update on build jenkins-wikidata-query-rdf-maven-release-docker-39

by jenkins-bot

The build 39 for wikidata/query/rdf has finished. See https://integration.wikimedia.org/ci/job/wikidata-query-rdf-maven-release-d… for details.

3 years, 4 months

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts January 2021