It turned out BIOS settings on phab1001 were such that disks were running under "legacy IDE" mode and write cache was turned off. This was really bad for performance. Changing that in BIOS unfortunately also means complete data loss. So we had to go back to phab1003 yesterday, fix phab1001, reimage it, rsync all the data again and finally (today) switch to phab1001 for real. The good part is that the process to switch is now simplified and there was basically no downtime for users besides the rarely used git-ssh service, which is also fixed now. This is done now. On Wed, Dec 4, 2019 at 12:14 AM Daniel Zahn <dzahn(a)wikimedia.org> wrote: -- Daniel Zahn <dzahn(a)wikimedia.org> Operations Engineer