Hi,
I woke up this morning to "Puppet failure on...." alerts for every cloud VPS server I have access to. I basically just ignored it because I assumed that many failures was something systemic, and I would wait for an announcement from the cloud services team.
Before I try to debug something this gnarly, *is* there something going on? Or is it just coincidence that 8 machines across two projects got these failures this morning?
Thanks, -Travis
On Tue, Oct 22, 2024 at 3:33 PM Travis Briggs audiodude@gmail.com wrote:
Hi,
I woke up this morning to "Puppet failure on...." alerts for every cloud VPS server I have access to. I basically just ignored it because I assumed that many failures was something systemic, and I would wait for an announcement from the cloud services team.
Before I try to debug something this gnarly, *is* there something going on? Or is it just coincidence that 8 machines across two projects got these failures this morning?
You can likely ignore any Puppet failure messages you got on 2024-10-22. There was an unexpected failure of the shared Puppet server that caused all Puppet runs to fail until it was corrected. See https://phabricator.wikimedia.org/T377803 for more details.
Bryan
Thanks Bryan!
-Travis
On Tue, Oct 22, 2024 at 5:01 PM Bryan Davis bd808@wikimedia.org wrote:
On Tue, Oct 22, 2024 at 3:33 PM Travis Briggs audiodude@gmail.com wrote:
Hi,
I woke up this morning to "Puppet failure on...." alerts for every cloud
VPS server I have access to. I basically just ignored it because I assumed that many failures was something systemic, and I would wait for an announcement from the cloud services team.
Before I try to debug something this gnarly, *is* there something going
on? Or is it just coincidence that 8 machines across two projects got these failures this morning?
You can likely ignore any Puppet failure messages you got on 2024-10-22. There was an unexpected failure of the shared Puppet server that caused all Puppet runs to fail until it was corrected. See https://phabricator.wikimedia.org/T377803 for more details.
Bryan
Bryan Davis Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808 _______________________________________________ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/