Discovery-alerts April 2021

discovery-alerts@lists.wikimedia.org

4 participants
472 discussions

Start a nNew thread

[airflow] SLA miss on DAG=export_queries_to_relforge
by noreply＠wikimedia.org 18 Apr '21

18 Apr '21

1 0

[airflow] SLA miss on DAG=export_queries_to_relforge
by noreply＠wikimedia.org 18 Apr '21

18 Apr '21

1 0

** RECOVERY alert - cloudelastic.wikimedia.org/WMF Cloud -Chi Cluster- - Public Internet Port - HTTPS is OK **
by nagios＠alert1001.wikimedia.org 17 Apr '21

17 Apr '21

Notification Type: RECOVERY Service: WMF Cloud -Chi Cluster- - Public Internet Port - HTTPS Host: cloudelastic.wikimedia.org Address: 208.80.154.241 State: OK Date/Time: Sat Apr 17 16:58:00 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 673 bytes in 0.006 second response time

1 0

** PROBLEM alert - cloudelastic.wikimedia.org/WMF Cloud -Chi Cluster- - Public Internet Port - HTTPS is CRITICAL **
by nagios＠alert1001.wikimedia.org 17 Apr '21

17 Apr '21

Notification Type: PROBLEM Service: WMF Cloud -Chi Cluster- - Public Internet Port - HTTPS Host: cloudelastic.wikimedia.org Address: 208.80.154.241 State: CRITICAL Date/Time: Sat Apr 17 16:55:46 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - Socket timeout after 10 seconds

1 0

[airflow] SLA miss on DAG=export_queries_to_relforge
by noreply＠wikimedia.org 17 Apr '21

17 Apr '21

1 0

** RECOVERY alert - elastic1060/Rate of JVM GC Old generation-s runs - elastic1060-production-search-eqiad is OK **
by nagios＠alert1001.wikimedia.org 17 Apr '21

17 Apr '21

Notification Type: RECOVERY Service: Rate of JVM GC Old generation-s runs - elastic1060-production-search-eqiad Host: elastic1060 Address: 10.64.48.130 State: OK Date/Time: Sat Apr 17 16:25:16 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Using_jstack_or_jmap_or_other_… https://grafana.wikimedia.org/d/000000462/elasticsearch-memory?orgId=1&var-… Acknowledged by : Additional Info: (C)100 gt (W)80 gt 61.02

1 0

** RECOVERY alert - cloudelastic1005/ElasticSearch health check for shards on 9200 is OK **
by nagios＠alert1001.wikimedia.org 17 Apr '21

17 Apr '21

Notification Type: RECOVERY Service: ElasticSearch health check for shards on 9200 Host: cloudelastic1005 Address: 208.80.154.9 State: OK Date/Time: Sat Apr 17 16:06:08 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status cloudelastic-chi-eqiad: timed_out: False, number_of_pending_tasks: 0, cluster_name: cloudelastic-chi-eqiad, relocating_shards: 0, active_shards_percent_as_number: 100.0, number_of_data_nodes: 6, number_of_in_flight_fetch: 0, initializing_shards: 0, number_of_nodes: 6, status: green, task_max_waiting_in_queue_millis: 0, active_shards: 1793, delayed_unassigned_shards: 0, unassigned_shards: 0, active_primary_shards: 895

1 0

** PROBLEM alert - cloudelastic1005/ElasticSearch health check for shards on 9200 is CRITICAL **
by nagios＠alert1001.wikimedia.org 17 Apr '21

17 Apr '21

Notification Type: PROBLEM Service: ElasticSearch health check for shards on 9200 Host: cloudelastic1005 Address: 208.80.154.9 State: CRITICAL Date/Time: Sat Apr 17 16:03:50 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: CRITICAL - elasticsearch http://localhost:9200/_cluster/health error while fetching: HTTPConnectionPool(host=localhost, port=9200): Read timed out. (read timeout=4)

1 0

** RECOVERY alert - alert1001/Mediawiki CirrusSearch pool counter rejections rate is OK **
by nagios＠alert1001.wikimedia.org 17 Apr '21

17 Apr '21

Notification Type: RECOVERY Service: Mediawiki CirrusSearch pool counter rejections rate Host: alert1001 Address: 208.80.154.88 State: OK Date/Time: Sat Apr 17 15:41:46 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Pool_Counter_rejections_%28sea… https://grafana.wikimedia.org/d/qrOStmdGk/elasticsearch-pool-counters?viewP… Acknowledged by : Additional Info: OK: Less than 1.00% above the threshold [500.0]

1 0

** RECOVERY alert - graphite1004/CirrusSearch eqiad 95th percentile latency is OK **
by nagios＠alert1001.wikimedia.org 17 Apr '21

17 Apr '21

Notification Type: RECOVERY Service: CirrusSearch eqiad 95th percentile latency Host: graphite1004 Address: 10.64.16.149 State: OK Date/Time: Sat Apr 17 15:38:40 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/dashboard/db/elasticsearch-percentiles?panelI… Acknowledged by : Additional Info: OK: Less than 20.00% above the threshold [500.0]

1 0

← Newer
1
...
18
19
20
21
22
23
24
...
48
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts April 2021