Hi! This was caused by a broken change that was merged to the Puppet Git repository. I just merged a patch to fix that. Sorry for the unnecessary alert!
Taavi
On Wed, May 22, 2024 at 12:47 PM Sebastian Berlin sebastian.berlin@wikimedia.se wrote:
How do I fix this? I followed the steps for debugging on wt:Portal:Cloud_VPS/Admin/Runbooks/Cloud VPS alert Puppet failure on and got the same error. I couldn't find any instructions for it on that page.
Full log: sebastian-berlin-wmse@producer:~$ sudo run-puppet-agent 2024-05-22 09:28:12.274041 WARN puppetlabs.facter - locale environment variables were bad; continuing with LANG=C LC_ALL=C 2024-05-22 09:28:13.088451 WARN puppetlabs.facter - locale environment variables were bad; continuing with LANG=C LC_ALL=C Info: Using environment 'production' Info: Retrieving pluginfacts Info: Retrieving plugin Info: Loading facts Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Class[Memcached]: parameter 'memcached_user' expects a String value, got Undef (file: /srv/puppet_code/environments/production/modules/profile/manifests/simplelamp2.pp, line: 32, column: 5) on node producer.wikispeech.eqiad1.wikimedia.cloud Warning: Not using cache on failed catalog Error: Could not retrieve catalog; skipping run
Sebastian Berlin Utvecklare/Developer Wikimedia Sverige (WMSE)
E-post/E-Mail: sebastian.berlin@wikimedia.se Telefon/Phone: (+46) 0707 - 92 03 84
---------- Forwarded message --------- From: root root@producer.wikispeech.eqiad1.wikimedia.cloud Date: Wed, 22 May 2024 at 10:15 Subject: [Cloud VPS alert][wikispeech] Puppet failure on producer.wikispeech.eqiad1.wikimedia.cloud (172.16.0.200) To: sebastian.berlin@wikimedia.se
Puppet is having issues on the "producer.wikispeech.eqiad1.wikimedia.cloud (172.16.0.200)" instance in project wikispeech in Wikimedia Cloud VPS.
Puppet is running with failures.
Working Puppet runs are needed to maintain instance security and logins. As long as Puppet continues to fail, this system is in danger of becoming unreachable.
You are receiving this email because you are listed as member for the project that contains this instance. Please take steps to repair this instance or contact a Cloud VPS admin for assistance.
If your host is expected to fail puppet runs and you want to disable this alert, you can create a file under /.no-puppet-checks, that will skip the checks.
You might find some help here: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Cloud_VP...
For further support, visit #wikimedia-cloud on libera.chat or https://wikitech.wikimedia.org
Some extra info follows: ---- Last run summary: application: converged_environment: production initial_environment: production run_mode: agent changes: total: 0 events: failure: 0 success: 0 total: 0 resources: changed: 0 corrective_change: 0 failed: 0 failed_to_restart: 0 out_of_sync: 0 restarted: 0 scheduled: 0 skipped: 0 total: 0 time: fact_generation: 0.5950437020510435 last_run: 1716364257 plugin_sync: 0.8315697535872459 startup_time: 0.862627795 total: 4.96828438 version: config: null puppet: 7.23.0
---- Failed resources if any:
No failed resources.
--- Last run log:
ERR: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Class[Memcached]: parameter 'memcached_user' expects a String value, got Undef (file: /srv/puppet_code/environments/production/modules/profile/manifests/simplelamp2.pp, line: 32, column: 5) on node producer.wikispeech.eqiad1.wikimedia.cloud WARNING: Not using cache on failed catalog ERR: Could not retrieve catalog; skipping run
---- Exceptions that happened when running the script if any: No exceptions happened.
Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/