September 2022 - Discovery-alerts

** PROBLEM alert - graphite1004/CirrusSearch codfw 95th percentile latency - more_like is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: CirrusSearch codfw 95th percentile latency - more_like Host: graphite1004 Address: 10.64.16.149 State: CRITICAL Date/Time: Mon Sept 12 20:40:47 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Health/Activity_Monitoring https://grafana.wikimedia.org/d/000000455/elasticsearch-percentiles?orgId=1… Acknowledged by : Additional Info: CRITICAL: 80.00% of data above the critical threshold [2000.0]

1 year, 8 months

1
0
0 0

** RECOVERY alert - logstash2027/OpenSearch health check for shards on 9200 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: OpenSearch health check for shards on 9200 Host: logstash2027 Address: 10.192.16.150 State: OK Date/Time: Mon Sept 12 17:07:41 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - elasticsearch status production-elk7-codfw: cluster_name: production-elk7-codfw, status: green, timed_out: False, number_of_nodes: 16, number_of_data_nodes: 10, discovered_master: True, active_primary_shards: 562, active_shards: 1281, relocating_shards: 2, initializing_shards: 0, unassigned_shards: 0, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 100.0

1 year, 8 months

1
0
0 0

Update on build jenkins-wikidata-query-rdf-maven-release-docker-101

by Release Engineering

The build 101 for wikidata/query/rdf has finished. See https://integration.wikimedia.org/ci/job/wikidata-query-rdf-maven-release-d… for details.

1 year, 8 months

1
0
0 0

selenium-daily-beta-CirrusSearch - Build # 1493 - Failure!

by Release Engineering

1 year, 8 months

1
0
0 0

[FIRING:1] RdfStreamingUpdaterFlinkJobUnstable (WCQS_Streaming_Updater rdf-streaming-updater k8s critical eqiad prometheus search-platform)

by sre-observability＠wikimedia.org

1 year, 8 months

1
0
0 0

[FIRING:1] RdfStreamingUpdaterNotEnoughTaskSlots (flink-session-cluster jobmanager 10_64_67_128 10.64.67.128:9102 k8s-pods rdf-streaming-updater flink-session-cluster-main-7f78557b85-7zfq2 7f78557b85 k8s main main warning eqiad prometheus search-platform)

by sre-observability＠wikimedia.org

1 year, 8 months

1
0
0 0

** RECOVERY alert - elastic2047/ElasticSearch setting check - 9400 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch setting check - 9400 Host: elastic2047 Address: 10.192.32.177 State: OK Date/Time: Fri Sept 9 21:29:22 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - All good!

1 year, 8 months

1
0
0 0

** RECOVERY alert - elastic2042/ElasticSearch setting check - 9400 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch setting check - 9400 Host: elastic2042 Address: 10.192.16.89 State: OK Date/Time: Fri Sept 9 21:29:22 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - All good!

1 year, 8 months

1
0
0 0

** RECOVERY alert - elastic2025/ElasticSearch setting check - 9200 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch setting check - 9200 Host: elastic2025 Address: 10.192.0.77 State: OK Date/Time: Fri Sept 9 21:29:22 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - All good!

1 year, 8 months

1
0
0 0

** RECOVERY alert - elastic2042/ElasticSearch setting check - 9200 is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: ElasticSearch setting check - 9200 Host: elastic2042 Address: 10.192.16.89 State: OK Date/Time: Fri Sept 9 21:29:22 UTC 2022 Notes URLs: https://wikitech.wikimedia.org/wiki/Search%23Administration Acknowledged by : Additional Info: OK - All good!

1 year, 8 months

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

Discovery-alerts September 2022