Failover

We are currently working on the virtualization of a web+ftp+mail server using KVM and Cloudmin Pro.

When this is over and stable, we will set up another remote server for geographical proximity and SEO purpose.

Both servers (one on each side of the Atlantic) will then be behind load-balancers that will take care of proximity load-balancing and failover).

Both servers will then have to share the same exact data and virtual domains.

From Cloudmin's documentation, it's not clear for the moment how Cloudmin could facilitate or complicate the setup. The only thing we know is that we won't be able to use Cloudmin's "Host Failovers Groups", as these require shared storage.

Will we need a second Cloudmin Pro license ? Will we have to use Cloudmin master replication ?

Thanks for your answer as we have to make technological choices now.

Status: 
Closed (fixed)

Comments

Cloudmin's failover feature may not help much here, as you don't have shared storage between the host systems. It was designed for the case where you have multiple KVM hosts on the same LAN, sharing storage using NFS, iSCSI or cluster LVM. If one host goes down, its virtual systems can be brought up on a new host automatically.

Cloudmin's master replication feature won't really help either. It sounds like what you actually want is regular syncing of your web content to a remote host ... is that correct?

Yep. and also the virtual domains.

Is it possible to "sync" two Cloudmin Pro installs ?

You can do a one-way sync of cloudmin pro, so that if the master system fails you can have a spare ready to do.

As for the website, are you using Virtualmin to host it? Because cloudmin has support for regular one-way replication of Virtualmin domains..

Yes, we use Virtualmin GPL to manage the virtual domains.

So we can use Virtualmin's replication feature, from each VM ?

Do we need two Cloudmin Pro licenses ?

Yes, you can use Cloudmin's "Virtual Server Replication" feature. Only one Cloudmin licence is needed.

This just works by regularly backing up domains and restoring them on target system(s). So it is not real-time replication ..

OK. But as we don't create new domains and new users too often, I guess it'll make it.

Thanks a lot !

Automatically closed -- issue fixed for 2 weeks with no activity.

Sorry to "up" this post again... I'm actually working on this replication scenario...

Would it be possible to replicate the Virtual Machines, and not the Virtual Domains, on the secont host ?

There's the "Move System" option, but as stated, it "moves" it, and thus deletes the VM from the original host...

Would it be possible to doit using full backup of a VM (firts backup, transfer the backup archive, and then restore the backup) ?

And could we imagine doing so using a live backup of a VM (without shutting down the VM), providing there's no MySQL or other DB service running ? What backup technique does Cloudmin use when backuping a VM system without shutting it down ?

What you could do is replica a VM daily (or even more often) to a new host using a cron job that calls Cloudmin API functions. Something like :

cloudmin create-image --host sourcesystem --id replication-image --overwrite --desc "Image for replication"
cloudmin delete-system --host destinationsystem
cloudmin create-system --host destinationsystem --type xen --image replication-image --ssh-pass foo --desc "Replicated system"

The only down-side is that this will shut down the source system during the image creation.

In fact, the problem is just for the initial duplication of our VMs. After that, data sync will be done through rsync scripts and mysql master-master replication.

So we are obliged to shutdown a VM to replicate it on another server, aren't we ?

Are the Cloudmin API functions "create-image" and "create-system" those used by the GUI "Move system" function, and by the System Backup and Restore function ?

Such a script could do the trick. The only problem is down time. Can the function "create-image" use LVM snapshot to diminish down time ? Does it re-start the VM once the image is created ?

I would strongly recommend a shutdown to do replication, or else you can't guarantee that the filesystem is in a consistent state. Even LVM snapshots don't avoid that ..

The create-image function will restart a system if it was shut down.

Also, another option would be to clone the source system and then move the clone to the new Xen host. I would recommend that actually ..

Ok, I agree with the last solution. I think that's what we will do : stop the VM to clone it, if possible via LVM snapshot, restart the original one, move the clone on the new host, and then configure data synchronisation. When everything's up and running, after the necessary tests, we can configure load-balancing and failover.

Thanks again.

Great! Let us know how it goes ..

Thanks.

Regarding data sync and Cloudmin/Virtualmin, I still have a concern.

Here's the setup : - On each host, we have a "web" VM and a "SQL" VM. - Virtualmin on the "web" VM is configured to use "SQL" VM as database server. This works fine. - When the two hosts will be up. both "SQL" VM will be replicating in a "Master/Master" scheme. That means all transcations on each sql VM will be replicated on the other (auto-increment offset). - Each "web" VM will act as "client" and will provide failover via two load-balancers in ther front-end. - It means that all of our Virtual Domains will be duplicated on each "web" VM. - Static content on the "web" VMs will be replicated via rsync scripts.

Imagine we create a new Virtual Domain in one of our "web" VM : users will be created not only in the VD, but also in the database, and thus the dabases users will be replicated in the "SQL2" VM, and will be available in the "web2" VM.

If we replicate de Virtual Domain in the "web2" VM, either manually or via Virtualmin's VD replication feature, will duplicate databases and databases users be created (and replicated in the first "SQL" VM) ?? And if yes, is there a way to avoid that ?

I know it's a bit confusing... but as we don't know exactly how Cloudmin/Virtualmin functions "under the hood", it's a point we have to clarify.

Thanks in advance for you patience...

This situation won't be handled too well unfortunately - the replication feature assumes that the target system is completely separate, so if the database already exists on the target the backup made during replication may fail to restore. You may be able to avoid this by de-selecting the MySQL feature when replicating, but that hasn't been heavily tested..

Do you have a shared home directory (via NFS) between the frontends?

No, there's no shared home dir, as the hosts won't be in the same subnet (they'll be distant).

I guess de-selecting MySQL feature on the second host will do the trick. That will have to be done in Virtualmin's "Systems Settings" / "Features and Plugins", right ?

Before going to production, we will test the procedure with a second local server.

Yes, that should work too. Let me know how it goes ..

Automatically closed -- issue fixed for 2 weeks with no activity.