[Labs-l] Service restored

[Nuria Ruiz]

We are also having problems running puppet (we had no issues yesterday
morning and changes had not been made):

These are the errors that we are getting on our self hosted puppet master:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER:
Could not find data item ganglia_class in any Hiera data file and no
default supplied at /etc/puppet/manifests/ganglia.pp:22 on node
wikimetrics1.eqiad.wmflabs
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

On Tue, Mar 31, 2015 at 6:40 AM, Mjbmr <mjbmri at gmail.com> wrote:

> Crontab is not working.
>
>
> On 3/31/2015 11:07 AM, Yuvi Panda wrote:
>
>> Services are all back up again. I've restarted the webservices pointed
>> at me as being down.  Load on all machines is back to normal as well -
>> abnormal load was caused by NFS messing up again. Should be ok atm.
>>
>> Detailed postmortem to follow tomorrow.
>>
>> _______________________________________________
>> Labs-l mailing list
>> Labs-l at lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/labs-l
>>
>
>
> _______________________________________________
> Labs-l mailing list
> Labs-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/labs-l
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-l/attachments/20150331/ac861833/attachment.html>