Hi! This was caused by a broken change that was merged to the Puppet
Git repository. I just merged a patch to fix that. Sorry for the
unnecessary alert!

Taavi

On Wed, May 22, 2024 at 12:47 PM Sebastian Berlin
<sebastian.berlin@wikimedia.se> wrote:
>
> How do I fix this? I followed the steps for debugging on wt:Portal:Cloud_VPS/Admin/Runbooks/Cloud VPS alert Puppet failure on and got the same error. I couldn't find any instructions for it on that page.
>
> Full log:
> sebastian-berlin-wmse@producer:~$ sudo run-puppet-agent
> 2024-05-22 09:28:12.274041 WARN  puppetlabs.facter - locale environment variables were bad; continuing with LANG=C LC_ALL=C
> 2024-05-22 09:28:13.088451 WARN  puppetlabs.facter - locale environment variables were bad; continuing with LANG=C LC_ALL=C
> Info: Using environment 'production'
> Info: Retrieving pluginfacts
> Info: Retrieving plugin
> Info: Loading facts
> Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Class[Memcached]: parameter 'memcached_user' expects a String value, got Undef (file: /srv/puppet_code/environments/production/modules/profile/manifests/simplelamp2.pp, line: 32, column: 5) on node producer.wikispeech.eqiad1.wikimedia.cloud
> Warning: Not using cache on failed catalog
> Error: Could not retrieve catalog; skipping run
>
> Sebastian Berlin
> Utvecklare/Developer
> Wikimedia Sverige (WMSE)
>
> E-post/E-Mail: sebastian.berlin@wikimedia.se
> Telefon/Phone: (+46) 0707 - 92 03 84
>
>
> ---------- Forwarded message ---------
> From: root <root@producer.wikispeech.eqiad1.wikimedia.cloud>
> Date: Wed, 22 May 2024 at 10:15
> Subject: [Cloud VPS alert][wikispeech] Puppet failure on producer.wikispeech.eqiad1.wikimedia.cloud (172.16.0.200)
> To: <sebastian.berlin@wikimedia.se>
>
>
>
> Puppet is having issues on the "producer.wikispeech.eqiad1.wikimedia.cloud (172.16.0.200)" instance in project
> wikispeech in Wikimedia Cloud VPS.
>
> Puppet is running with failures.
>
> Working Puppet runs are needed to maintain instance security and logins.
> As long as Puppet continues to fail, this system is in danger of becoming
> unreachable.
>
> You are receiving this email because you are listed as member for the
> project that contains this instance.  Please take steps to repair
> this instance or contact a Cloud VPS admin for assistance.
>
> If your host is expected to fail puppet runs and you want to disable this
> alert, you can create a file under /.no-puppet-checks, that will skip the checks.
>
> You might find some help here:
>     https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Cloud_VPS_alert_Puppet_failure_on
>
> For further support, visit #wikimedia-cloud on libera.chat or
> <https://wikitech.wikimedia.org>
>
> Some extra info follows:
> ---- Last run summary:
> application:
>   converged_environment: production
>   initial_environment: production
>   run_mode: agent
> changes:
>   total: 0
> events:
>   failure: 0
>   success: 0
>   total: 0
> resources:
>   changed: 0
>   corrective_change: 0
>   failed: 0
>   failed_to_restart: 0
>   out_of_sync: 0
>   restarted: 0
>   scheduled: 0
>   skipped: 0
>   total: 0
> time:
>   fact_generation: 0.5950437020510435
>   last_run: 1716364257
>   plugin_sync: 0.8315697535872459
>   startup_time: 0.862627795
>   total: 4.96828438
> version:
>   config: null
>   puppet: 7.23.0
>
>
> ---- Failed resources if any:
>
>   No failed resources.
>
> --- Last run log:
>
> ERR: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Class[Memcached]: parameter 'memcached_user' expects a String value, got Undef (file: /srv/puppet_code/environments/production/modules/profile/manifests/simplelamp2.pp, line: 32, column: 5) on node producer.wikispeech.eqiad1.wikimedia.cloud
> WARNING: Not using cache on failed catalog
> ERR: Could not retrieve catalog; skipping run
>
> ---- Exceptions that happened when running the script if any:
>   No exceptions happened.
>
> _______________________________________________
> Cloud mailing list -- cloud@lists.wikimedia.org
> List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/



--
Taavi Väänänen (he/him)
Site Reliability Engineer, Cloud Services
Wikimedia Foundation
_______________________________________________
Cloud mailing list -- cloud@lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/