About two updates ago, my backups started freaking out. I've got regular rsync backups running on large files, so I use the Virtualmin backup feature to grab databases and configs. Now, when a backup runs it doesn't exclude items in the "Files to exclude from each domain" field. The effect was a completely full hard drive and consequently a denial of service for my clients. Clients of which have a contractual service level agreements. How do I get my backups working right again?
Status:
Closed (fixed)
Comments
Submitted by JamieCameron on Mon, 05/21/2012 - 18:08 Comment #1
What paths are you entering in the exclude list - full paths, or just relative to the domain's home directory?
Also, did the excludes stop working when you upgraded to Virtualmin 3.92?
The paths in this field are:
cgi-bindomains
etc
fcgi-bin
logs
tmp
users
www
Correct. Updating to Virtualmin 3.92 is when I encountered the problem. I could have sworn it was in the update prior, but nonetheless it happened at the same time the New Feature Log mentioned "Encrypted backups."
Submitted by JamieCameron on Mon, 05/21/2012 - 19:27 Comment #3
Ok, that should work.
Which of those directories contained the files that you wanted to exclude?
All of them. None of them are currently working. There were before the recent update.
Please note, I'm using the virtualmin backup routines for database dumps and configs. Everything else gets rsync-ed regularly.
Submitted by JamieCameron on Mon, 05/21/2012 - 21:08 Comment #5
Are you using the default tar.gz backup format, or have you switched to something like ZIP or bzip2?
My default is 'pbzip2 -p3', but I also tried the normal bzip2 binary.
Submitted by JamieCameron on Tue, 05/22/2012 - 00:09 Comment #7
That should work fine..
Is the backup including
public_htmlin top-level virtual servers, in sub-servers, or both?The 'public_html' folder has been renamed 'www' on my server. As you can see above, 'www' has been excluded.
I'm not sure why or how, but I seem to have gotten my backups working again. It stopped working with the last update, but after flipping switches trying to diagnose this issue it's working like it was previously. There is definitely a bug, I just don't know what it is. Wish I could be of more help.
Feel free to close this issue as you see fit. If I can provide further details, I'm available to assist.
Submitted by JamieCameron on Tue, 05/22/2012 - 13:03 Comment #10
Ok, I'm glad it is working again now. Unfortunately I couldn't re-produce this, or see how any code changes could cause it.
By the way, a better way to make backups when you are planning to use rsync to copy home directories is as follows :
The resulting backup will include Apache configs, MySQL, DNS and so on, but nothing under each domains' home directory.
Thanks for the suggestion on excluding home and web directories. I'll definitely have to check that out.
It is possible that the 'Excluded Directories' for each virtual server got wiped? Some of my home directories have data in them that is specifically excluded above and beyond the general exclusion list. This is data like photo gallery files not in the web directory. For such instances, I've added those specific directories in [virtual server] -> Administration Options -> Excluded Directories. In my trying to figure this bug out, I noticed that nothing was in those fields. I believe once I corrected that, everything started to work like it had in the past. Plausible?
Submitted by JamieCameron on Thu, 05/24/2012 - 11:03 Comment #12
That page being empty would certainly explain the problem.
Do you happen to know roughly when the exclude lists got emptied - perhaps when you upgraded?
My backups started randomly failing after the upgrade. Then eventually my disk ran out of space as the backups consumed the hard drive. It looks like the exclude lists for each virtual server got emptied when I upgraded to the latest version.
Submitted by JamieCameron on Thu, 05/24/2012 - 13:16 Comment #14
I did a test upgrade on a system with some excluded directories set, but I wasn't able to re-produce the problem of the excludes disappearing..
If a tree falls in a forest and no one is around to hear it, does it make a sound? I guess the tech version is "If a bug fails to reproduce, does it exist?
On the test system, where you using general exclude directories or specific virtual server excludes?
Submitted by JamieCameron on Thu, 05/24/2012 - 15:13 Comment #16
Specific virtual server excludes. Which ones were you setting?
Pull up one of the virtual servers (e.g. example.org, example.com, etc). Flip down Administration Options. Then go to Excluded Directories. Add something in the field labeled "Directories under home to skip." Then try an upgrade. After the upgrade is complete, is there still data in that field?
Submitted by JamieCameron on Thu, 05/24/2012 - 15:30 Comment #18
Yes, that's exactly what I did.
Bug can't be reproduced. Closing ticket.