We had issues with NFS mounts in Toolforge as an unintended side effect of OpenStack upgrades earlier today. While NFS mounts were broken, many (all?) cronjobs failed. The emails telling maintainers about these failures did not go out in real time because the mail system also relies on NFS. When we fixed the mail server thousands of queued messages started going out. There are still a few emails sending as I write this, but the queue is almost empty. As far as we can tell, cronjobs are starting as expected now and the emails are just left over noise. Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808 _______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce