Hello,
We are currently investigating widespread intermittent DNS resolution issues within the Toolforge Kubernetes cluster that began on Sunday. These issues are causing some jobs and deployments to fail, particularly on NFS worker nodes.
Impact:
- Some tools may experience failed deployments or crashes - Job execution may be inconsistent - Image pulls may fail intermittently
Our team is actively investigating and working to resolve the issue. We will send an update once we have more information or when the incident is resolved.
Thank you for your patience,
Slavina, on behalf of the WMCS Team -- Slavina Stefanova (she/her) Software Engineer | Cloud Services
Wikimedia Foundation