These forums are locked and archived, but all topics have been migrated to the new forum. You can search for this topic on the new forum: Search for Best practice backup strategy to redundant hardware and USB external drives? on the new forum.
Web Hosting and Cloud Computing Control Panels
Here's what we do:
Webmin->System->Filesystem Backup : We use this to make a full system backup once per week, and an incremental backup of only changed files every day. This requires two scheduled backups (one "full" and one "incremental").
Virtualmin->Backup and Restore->Scheduled Backups : We use this to make a daily backup of all of our virtual servers on the system. Again, we have two schedules: one for a full backup once per week, and one for a daily incremental backup. We keep four weeks worth (one week is probably sufficient; as long as you always have one full backup in existence, since incrementals are worthless without a full backup to go along with them), since we have tons of disk space.
The Virtualmin backups are the easiest to work with, and they're what we'd use if we had a catastrophic failure and needed to bring the website back up quickly. Virtualmin backups are designed to be restorable on any Virtualmin system. The full system backup would be something I'd restore locally (on a spare disk attached to my PC) and selectively restore the bits I needed (extra stuff in /etc, etc.).
On our old box, we also used Webmin's Backup Configuration Files module, which is actually quite handy. I'll probably set it up again soon on the new box. This one is mainly defense against administrator stupidity (and I've got plenty of that to go around!)...if I break something, but I'm not sure what I did, I can go back to the previous days config file backups and use "diff -u" to compare. Basically this module just formalizes and simplifies a sysadmin practice as old as UNIX itself: Backing up /etc periodically. As a system administrator in the past, I would always make a backup of /etc (in addition to full backups) whenever doing anything that might be disruptive...like system upgrades, migrations, installation of large new software, etc. Tarring all of /etc/ up only takes a minute or two, likewise untarring it. So, it's a really cheap way to have quick access to the most likely to break part of your system.
Once all this stuff is setup, you can just forget about it. You should be getting daily reports via email from your system, assuming you've setup the root alias correctly to point to your email address, so you'll hear about it if there are any problems. One less thing to think about.
--
Check out the forum guidelines!
I have setup two jobs to backup to S3:
1) One full backup every sunday at 1am, retention of 28 days. File under bucket = full-%Y-%m-%d
2) One incremental backup every day but sunday at 1am, retention of 28 days. File under bucket = inc-%Y-%m-%d
Both use: Bucket name = kvm1. Backup format = One file per server, Do strftime-style time substitutions on file or directory name = ON.
But they are separate backup jobs. How does the incremental one know when the last full backup was done?
Howdy,
But they are separate backup jobs. How does the incremental one know when the last full backup was done?
Virtualmin has a log of all the backup jobs, so as an incremental runs, it just queries Virtualmin and asks when the last full backup was. If Virtualmin tells it "3 days ago", than the incremental backup is run so that all files changed or added since then are included in the backup.
-Eric
Thanks Joe. I'd love to replicate this. Where do you make these backups? To a spare identical system? or to offsite/usb storage?
I guess I'm asking if you have a cold/warm failover system ready to be turned on and enabled if the primary system goes down. I'm not working in a VM environment, so need to work at the server/disk level.
What I'd love a script on, or help creating one myself, is how to create a "warm" failover system that is essentially a mirror of the primary system, which can be enable fairly quickly if the main system goes down. Is this possible to set up with webmin/virtualmin? Incremental backups to the warm system could occur on an hourly or daily basis, but the key is that the backup system have everything it needs to become primary given a significant issue (minus up to 24 hours of recent changes).
Am I making sense?
We don't have a failover system. It's merely stored on a NAS at our hosting provider, which we pay a little extra for each month. I occasionally plug in a USB drive to my desktop machine and use rsync to copy the backups to a local disk). Pretty low-tech, but it works.
I'm considering S3 for our off-site Virtualmin backups, as it'd provide a little extra safety in the event of catastrophe at our hosting provider.
For us, the cost of a couple hours of downtime, in the unlikely event of a major failure that we weren't prepared for (SMART can warn us of oncoming disk problems, and Webmin has SMART monitors in the System and Server Status module), is lower than the cost of having a fully redundant setup. It'd cost an extra ~$200 per month to have a second identical server, or $2400 per year. We will probably worry more about that now that our customer base is growing pretty rapidly. We're also about to introduce some rather more expensive versions of our products for enterprise use (that cost an order of magnitude more, while including more serious support and service level agreements that have nasty teeth if we fail to deliver), so being "always on" will probably be more important. We'll have to see how all that shakes out.
--
Check out the forum guidelines!
Thanks. These servers are low end, on-site systems, already in hand. I'll start poking around to see if there are some good mirror approaches or rsync scripts to do this.
Dumb question: if I use virtualmin to backup the sites, what do I need to do to backup virtualmin's settings?
So If I understand this correctly, if I suffer a major failure, the recovery would be
1) rebuild server 2) install virtualmin 3) recover virtualmin settings from backups 4) recover sites from backups
Is this correct?
You might also use the Webmin backup/restore feature for its settings which are important; and optionally include the configuration files for the various servers (mail, apache, etc.)
Having back is safe as always. Wont take risk in any new method out there, When I bought my first unit at Spectra, i did get some detailed instruction but manage to create some back up just in case something gets crazy.