Copying domains

Outside of the normal back and restore features for domains within Virtualmin is there a way to copy domains from one server to another? I ask because we are in the process of migrating a lot of domains from one server to another, upwards of 500 including sub-servers and aliases.

My concern is that going through the integrated restore procedures might be extremely time consuming on such a large number.

If you have any documentation that you might be able to refer me to that would be helpful.

Thanks, Chris

Status: 
Closed (fixed)

Comments

Howdy -- my suggestion there would be to use the command line tools to perform that restore. That would allow you to type one command on the command line, walk away for a few hours, then come back when it's finished.

First, you'd want to generate a set of backups -- you can do that from within Virtualmin, using Backup and Restore.

Then, copy those backups to your new server.

Lastly, you can perform a restore with the command line tools -- you can run "virtualmin restore-domain" to see a list of options.

There is a document here on how to migrate domains to a new server. But, rather than using the GUI as it shows, you can use the command line tools as mentioned above:

https://virtualmin.com/documentation/system/migrate

So there is no clear solution to copy /home, OS users and groups as well as any pertinent webmin/virtualmin files for users and domains?

Per our engineer "We can't do the VM restore because too much data will change by the time the data is copied to the cloud. "

So, I am really in need of the other alternative.

Well, you'll need to use the Virtualmin backups in order to transfer the account information.

However, to handle data that changed since the backup was performed -- you could always use rsync to pull in newer files.

So if your backups are a day old, once you restore those onto the new server, you can then use rsync to pull in files that have changed over the course of that day.

One thing you could do is a full backup and restore, then shut down the webserver on your old system (to prevent updates), then a final incremental backup and restore. Because the incremental backup is smaller, your downtime window should be reduced.

The only down-side is that the incremental feature doesn't include detecting diffs in database contents, so they will still be fully backed up.

What about the possibility of performing a complete VM backup MINUS the /hone directory. Then import the full backup over the internet and rsync the /home directory to update it. Then attach VM on the new server to the recently updated /home directory. This would give is around only 4-8hours of downtime...theoretically.

Or, is there is a way to perform a full backup and import of a single to a few domains at a time while running a parallel server in the cloud until everything is moved over. We could then migrate a few domains at a time until every domain has been migrated.

Thanks!

What about the possibility of performing a complete VM backup MINUS the /hone directory.

You can do that -- when you generate the backup in Virtualmin's Backup and Restore, go into Features and Settings, and disable the "Server's home directory and web pages" option.

is there is a way to perform a full backup and import of a single to a few domains at a time while running a parallel server in the cloud until everything is moved over. We could then migrate a few domains at a time until every domain has been migrated.

You can migrate as many domains at a time as you like -- so if you just want to move a handful at any given point, that's certainly an option.

That all sounds great. For the first item the procedures are pretty clear. Would you mind giving a few more specifics or point me to information on the second part?

We'd be happy to help.

Can you clarify what you'd like additional information on?

To do a backup minus the home directory using the command-line API, you can use a command like :

virtualmin backup-domain --domain whatever.com --separate --all-features --except-feature dir --dest /tmp/backupdir

The important flag is --separate , which selects a backup format that uses one file per domain, and can be used without including the home directory contents.

Thanks for all your help Eric and Jamie,

So, doing the full backup minus the /home directory (Jamie, I believe that's the '--except-feature dir' option, correct) this will give me everything we'll need to restore to a parallel server.

Eric, in follow up to your question, following the first procedure, what would be necessary to migrate the remaining data for the individual domains (either singularly or en-mass)? Would a separate VM backup be required or could we use something like rsync?

Jamie, if we used the same API command with out the --separate flag, would there be any downside? It'd be nice to let the system run through everything without having to restore each domain individually. Without the home directory the backup shouldn't be that large, right? Obviously that's relative, but my /home directory is over 800 GB.

So, doing the full backup minus the /home directory (Jamie, I believe that's the '--except-feature dir' option, correct) this will give me everything we'll need to restore to a parallel server.

That would fully backup your Virtual Servers, minus the home directory data.

following the first procedure, what would be necessary to migrate the remaining data for the individual domains (either singularly or en-mass)? Would a separate VM backup be required or could we use something like rsync?

You wouldn't need a Virtualmin backup -- you could use a tool such as rsync to copy the files from /home.

The Virtualmin backup without the home directory data will re-create all the account information, metadata, databases, and the like -- and then a tool such as rsync can be used to copy your home directory data.

That sounds like what I thought but I really wanted to be sure.

So, understanding that the Virtualmin backup without /home will create all the account information, I assume all the accounts will be visible within Virtualmin, all the apache, DNS, conf files will be put in place and they would operate with the expectation that /home existed though until it is rsync'd it does not. Am I stating this correctly?

Would this be the correct command if I want to backup all domains minus /home is a single file

virtualmin backup-domain --all-domains --all-features --except-feature dir --dest /tmp/backupdir

So, understanding that the Virtualmin backup without /home will create all the account information, I assume all the accounts will be visible within Virtualmin, all the apache, DNS, conf files will be put in place and they would operate with the expectation that /home existed though until it is rsync'd it does not. Am I stating this correctly?

The accounts will be visible, and Apache, DNS, and config files will be put into place.

However, nothing in /home will exist until you copy it over. Daemons such as Apache will require you to copy that over before they work properly... you may see errors about the missing home directories until that data is copied.

Cross posted... Eric, that's perfect!

What about the above API command?

To backup all domains to a single file, you can use a command like :

virtualmin backup-domain --all-domains --all-features --except-feature dir --dest /tmp/backupfile.tar.gz