Hello, While trying to troubleshoot performance issues on our NFS server, it became apparent that we have a peak of requests durings hours 0-3 every day. This is currently hard to properly manage and results in slower performance for everybody. Here's a breakdown of the number of cronjobs scheduled to start in the new SGE cluster, per hour: 473 * # hourly 213 0 106 1 82 2 78 3 54 9 50 4 48 7 45 12 42 5 You can see that, apart from hourly jobs, we have most cronjobs starting in hours 1-3. My advice is that, if your workload permits, *schedule your cronjobs outside of that time window so you are not competing for resources with everybody else*. The more spread throughout the day your cronjobs are, the better. Also beware of step values like "*/2" which will make your cronjob run on hours 0 and 2 too. If you have any questions, please find us on #wikimedia-cloud Regards, -- Giovanni Tirloni Operations Engineer Wikimedia Foundation _______________________________________________ Wikimedia Cloud Services mailing list Cloud(a)lists.wikimedia.org (formerly labs-l(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud

Hello,  While trying to troubleshoot performance issues on our NFS server, it became apparent that we have a peak of requests durings hours 0-3 every day. This is currently hard to properly manage and results in slower performance for everybody.  Here's a breakdown of the number of cronjobs scheduled to start in the new SGE cluster, per hour:     473 *  # hourly     213 0     106 1      82 2      78 3      54 9      50 4      48 7      45 12      42 5  You can see that, apart from hourly jobs, we have most cronjobs starting in hours 1-3.  My advice is that, if your workload permits, *schedule your cronjobs outside of that time window so you are not competing for resources with everybody else*. The more spread throughout the day your cronjobs are, the better.  Also beware of step values like "*/2" which will make your cronjob run on hours 0 and 2 too.  If you have any questions, please find us on #wikimedia-cloud Regards,