Also posted in the Virtualmin forum:
Some of our users are starting to complain about delays and timeouts on our server, especially on the hour and right after midnight. Let's face it, there are a huge number of tasks running out of cron, and the list has been growing as new features and new clients are added. Some of these jobs are scheduled at random times, which helps a lot, but the real resource-intensive ones are causing an impact on the system at set times every day or every hour.
I think I have a solution: Let's be nice! Instead of running these jobs at the normal priority and making the users compete with them, let's schedule them at a lower priority. In particular, the following jobs seem ripe for this treatment:
0,5,10,15,20,25,30,35,40,45,50,55 * * * * nice /etc/webmin/status/monitor.pl
0,10,20,30,40,50 * * * * nice /etc/webmin/quota/email.pl
0 * * * * nice /etc/webmin/virtual-server/bw.pl
@hourly nice /etc/webmin/virtual-server/quotas.pl
@daily nice /etc/webmin/virtual-server/backup.pl
For now, I have simply added the word nice in front of the command name in cron, but there may be a better way to handle this. Also, I want to make sure that the various Webmin/Virtualmin modules recognize this when automatically scheduling jobs so they don't get overwritten, or worse, duplicated with a whole second job.