Replication and Shared Storage

Introduction

Sometimes a single system is not enough to host a website - either because of the traffic load it receives, or because you are concerned that the system could go down. The traditional solution to this is to have the site replicated across multiple systems, and use either a front-end load balancer or multiple DNS entries to distribute traffic between those systems.

Virtualmin versions 4.17 and above support this configuration via new features designed to allow domain replication using Virtualmin's existing backup and restore process. Cloudmin can also be used to enhance these functions by providing load balancing using either DNS or an Apache proxy, and automatic scheduled replication.

All replication in Virtualmin is one-way - a master system is the source of truth as to what domains and users exist, and that state is periodically copied to one or more replica systems. This includes mailboxes, passwords, cron jobs, spam filtering settings and DNS records.

Requirements

Domain replication works best when all the Virtualmin systems share domain home directories from a central server, typically using NFS to share the entire /home directory. Also, MySQL databases should be stored on a remote server, and all Virtualmin systems configured to use it. For dynamic websites like blogs, shopping carts or wikis, this is absolutely required to ensure that all application state remains in sync between systems. However, for sites that have only static web content it is possible to avoid setting up file sharing.

Setting up NFS exports and mounts is beyond the scope of this document, as it is covered elsewhere :

When databases and home directories are shared, only metadata about the domains needs to be copied, such as users, mail aliases, DNS records and spam filtering settings.

Replicating Your First Domain

One all the Virtualmin systems have been setup and configured to use shared storage, you can replicate a domain as follows. These steps assume that the domain is named example.com , the master system is named master.yourdomain.com and the replica is named replica.yourdomain.com :

  1. Create the domain on the master system first, setup the website and validate that it works.
  2. SSH into the master system as root, and run the command virtualmin transfer-domain --domain example.com --host replica.yourdomain.com --pass XYZ --replication --output --overwrite (replacing XYZ with the root password for the replica system)
  3. Assuming this command succeeds, update any DNS records or load balancer configuration for the example.com domain to use the IPs of both master.yourdomain.com and replica.yourdomain.com .
  4. Test that the site is accessible and functions correctly when traffic is being sent to both systems.

Scheduled Replication

One replication has been confirmed to work manually, it is recommended that you setup regular scheduled replication to keep settings like mailbox users in sync between the master and other systems. If you don't have Cloudmin installed, this can be done by creating a cron job (using the Scheduled Cron Jobs module on the master system) that runs a command like :

virtualmin transfer-domain --domain example.com --host replica.yourdomain.com --pass XYZ --replication --overwrite

If home directory and MySQL database contents are shared, this could be setup to run once every 10 minutes, as the amount of data to transfer will be small.

Deleting Domains

When deleting a domain from a single replica, care must be take not to remove shared home directory or database contents. The delete-domain API command has a --preserve-remote flag for this purpose, and there is a checkbox in the Virtualmin UI that is displayed when deleting domains with shared storage that has the same effect.

When you want to completely remove a domain, we recommend deleting it from all replicas first as described above, and the performing a regular delete from the master system. This will remove all databases and the home directory contents, even if they are stored on remote servers.

Cloudmin Domain Synchronization

Cloudmin's built-in domain synchronization feature will automatically use replication mode when the destination system is running Virtualmin 4.17 or later. For more information, see https://www.virtualmin.com/documentation/cloudmin/vm/vsync