[Labs-admin] ** PROBLEM alert - tools-exec-1433/Puppet run is CRITICAL **

[Chase Pettet]

Well I found this on tools-exec-1433 to coincide with the Puppet run alert:

Notice:
/Stage[main]/Role::Labs::Nfsclient/Labstore::Nfs_mount[tools-project-on-labstore-secondary]/Exec[ensure-nfs-tools-project-on-labstore-secondary]/returns:
mount.nfs: mount(2): Device or resource busy
Notice:
/Stage[main]/Role::Labs::Nfsclient/Labstore::Nfs_mount[tools-project-on-labstore-secondary]/Exec[ensure-nfs-tools-project-on-labstore-secondary]/returns:
mount.nfs: /mnt/nfs/labstore-secondary-tools-project is busy or already
mounted
Notice:
/Stage[main]/Role::Labs::Nfsclient/Labstore::Nfs_mount[tools-project-on-labstore-secondary]/Exec[ensure-nfs-tools-project-on-labstore-secondary]/returns:
mount.nfs: trying text-based options
'vers=4,bg,intr,sec=sys,proto=tcp,port=0,lookupcache=all,nofsc,hard,addr=10.64.37.18,clientaddr=10.68.21.151'
Error: /usr/local/sbin/nfs-mount-manager mount
/mnt/nfs/labstore-secondary-tools-project returned 32 instead of one of [0]
Error:
/Stage[main]/Role::Labs::Nfsclient/Labstore::Nfs_mount[tools-project-on-labstore-secondary]/Exec[ensure-nfs-tools-project-on-labstore-secondary]/returns:
change from notrun to 0 failed: /usr/local/sbin/nfs-mount-manager mount
/mnt/nfs/labstore-secondary-tools-project returned 32 instead of one of [0]
Notice: /Stage[main]/Role::Labs::Nfsclient/File[/data/project]: Dependency
Exec[ensure-nfs-tools-project-on-labstore-secondary] has failures: true
Warning: /Stage[main]/Role::Labs::Nfsclient/File[/data/project]: Skipping
because of failed dependencies


So I guess we can stop laying the blame on wsexport (or all of it anyway).
I don't get this transient failure but something is up here still it seems.

On Wed, Apr 19, 2017 at 6:54 PM, shinken <
shinken at shinken-01.shinken.eqiad.wmflabs> wrote:

> Notification Type: PROBLEM
>
> Service: Puppet run
> Host: tools-exec-1433
> Address: 10.68.21.151
> State: CRITICAL
>
> Date/Time: Wed 19 Apr 23:54:16 UTC 2017
>
> Additional Info:
>
> CRITICAL: 60.00% of data above the critical threshold [0.0]




-- 
Chase Pettet
Engineering Manager -- Labs
chasemp on phabricator <https://phabricator.wikimedia.org/p/chasemp/> and
IRC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-admin/attachments/20170419/50d84a89/attachment.html>