Virtualmin restore to new server does not restore DNS Zones

The backup on the original server completed without errors and contains entries as below, showing the DNS data for relevant Virtual Server is copied.

Copying virtual server configuration .. .. done

Backing up Cron jobs .. .. none defined.

Copying records in DNS domain .. .. done

Saving mail aliases .. .. done

Saving mail and FTP users .. .. done

Backing up mail and FTP user Cron jobs .. .. none to backup

Backing up Dovecot control files .. .. done

Copying Apache virtual host configuration .. .. done

Copying Apache log files .. .. done

Copying Webalizer configuration files .. .. done

Copying Logrotate configuration .. .. done

Dumping MySQL database bizventures .. .. done

Backing up Webmin ACL files .. .. done

Backing up AWstats configuration file .. .. done

Creating TAR file of home directory .. .. done

However when restored only 4 of the 20+ Virtual Servers with "DNS Domain enabled" checked had their dns data restored. I used the commands exactly (replacing the correct path)

https://www.virtualmin.com/documentation/system/migrate

I have tried now since yesterday with multiple backups and still no dice so I am worried that other things may not have been restored correctly. How can I troubleshoot this issue.

My configuration is as follows

Original Server : CentOS 6 - Virtualmin GPL New Server : CentOS 7 - Virtualmin Pro * No errors thrown during backup or restore procedures.

I also want any previous restore to be completely overwritten by the latest restore which I notice also was not done as a previous Virtual Server (now deleted form the original server) was left on the new server after complete restore. I am sure this is just a command option I am missing.

Many thanks for any assistance.

Status: 
Closed (fixed)

Comments

Restoring a domain should definitely restore the DNS settings, and we can certainly troubleshoot why that's not working in your case.

If you look in System Settings -> Features and Plugins on the new server, is DNS enabled there?

Also, can you share the output you receive when restoring a domain that has this problem?

  1. Yes DNS is enabled in Virtualmin > System Settings > Features and Plugins
  2. Below is output for a specific Virtual Server Backup which will be followed by the output from the restore made on the new server

Copying virtual server configuration .. .. done

Backing up Cron jobs .. .. none defined.

Copying records in DNS domain .. .. done

Saving mail aliases .. .. done

Saving mail and FTP users .. .. done

Backing up mail and FTP user Cron jobs .. .. none to backup

Backing up Dovecot control files .. .. none found

Copying Apache virtual host configuration .. .. done

Copying Apache log files .. .. done

Copying Webalizer configuration files .. .. done

Copying Logrotate configuration .. .. done

Dumping MySQL database kidsafemaryland .. .. done

Copying Procmail and SpamAssassin configuration files .. .. done

Backing up Webmin ACL files .. .. done

Backing up AWstats configuration file .. .. done

Creating incremental TAR file of home directory .. .. done

  1. Restore log for the exact same domain (notice DNS log entry is missing)

Restoring virtual server password, quota and other details .. .. done

Updating administration password and quotas .. .. done

Restoring Cron jobs .. .. done

Extracting TAR file of home directory .. .. done

Setting ownership of home directory .. .. done

Restoring Apache virtual host configuration .. .. done

Checking restored PHP execution mode .. .. mode FCGId OK for this system

Updating home directory in PHP configuration .. .. done

Restoring Apache log files .. .. done

Restoring Webalizer configuration files and Cron job .. .. done

Restoring Logrotate configuration .. .. done

Restoring allowed MySQL hosts .. .. done

Deleting old MySQL databases .. .. done

Re-loading MySQL database ..

Creating MySQL database ..
.. done

.. done

Restoring SpamAssassin and Procmail configurations .. .. done

Restoring Webmin ACL files .. .. done

Restoring AWstats configuration file .. .. done

Re-creating mail and FTP users .. .. done

Re-creating mail aliases .. .. done

Restoring mail and FTP user Cron jobs .. .. done

Updating Webmin user .. .. done

  1. Finally I will note when looking at the Virtual Server which has been restored to the new server that has the DNZ Zones missing it has the "DNS domain enabled" features disabled which would explain why the DNZ was not restored but when you look at the same Virtual Server on the original server which was backed up it has the DNS feature enabled.

As a follow up to the above I just did a new backup on one of this same virtual server above from the original server. I then copied to the new server and this is the output from the restore operation showing what will be restored. You can see it notes DNS data

Virtual server password, description and other details
Administration user's password, quota and Cron jobs
Records in DNS domain, except SOA
Mail/FTP users and mail aliases
Apache webserver configuration
Webalizer configuration and schedule
Logrotate configuration for log file
Contents of server's MySQL databases
Spam filtering
Webmin ACL files
Server's home directory and web pages 
AWstats configuration file 

Then after completing the restore action below is the output (which you can see as before is missing the DNS restore procedure)

 Restoring virtual server password, quota and other details ..
.. done

Updating administration password and quotas ..
.. done

Restoring Cron jobs ..
.. done

Extracting TAR file of home directory ..
.. done

Setting ownership of home directory ..
.. done

Restoring Apache virtual host configuration ..
.. done

Checking restored PHP execution mode ..
.. mode FCGId OK for this system

Updating home directory in PHP configuration ..
.. done

Restoring Apache log files ..
.. done

Restoring Webalizer configuration files and Cron job ..
.. done

Restoring Logrotate configuration ..
.. done

Restoring allowed MySQL hosts ..
.. done

Deleting old MySQL databases ..
.. done

Re-loading MySQL database ..
    Creating MySQL database ..
    .. done

.. done

Restoring SpamAssassin and Procmail configurations ..
.. done

Restoring Webmin ACL files ..
.. done

Restoring AWstats configuration file ..
.. done

Re-creating mail and FTP users ..
.. done

Re-creating mail aliases ..
.. done

Restoring mail and FTP user Cron jobs ..
.. done

Updating Webmin user ..
.. done

Applying web server configuration .. .. done

Re-loading Webmin .. .. done

.. restore complete.

Jamie, do you know what might be going on here?

It looks like it's intending to restore DNS data when the restore starts, but it doesn't actually seem to be doing so.

Only other thing I can add is that all of these domains originally had DNS disabled and I was managing it manually in BIND from within Webmin. I recently deleted all these zones from BIND in Webmin and enabled the feature for each of the Virtual Servers which then re-added all the zones.

In short I did not have this feature enabled when the virtual server for each was first created, although that should not matter I thought I'd mention it.

OK, I found the cause of the problem and a workaround to fix it. My scenario with workaround is documented below

  1. (Original) Server has all the Virtualmin settings and Virtual Servers, none of the servers have DNS feature enabled as all DNS zones were managed directly/manually in BIND.
  2. (Original) Server full backup performed and copied to (Destination) Server.
  3. (Destination) Server full restore performed <- At this point restoring all virtualmin servers with the DNS Feature disabled.
  4. (Original) Server - realized the error in my ways and deleted most DNS zones from BIND and went through each applicable Virtual Server and "enabled" the DNS feature so all deleted zones were now re-created by Virtualmin as required.
  5. (Original) Server - performed a new full backup and copied to (Destination) Server.
  6. (Destination) Server full restore performed with feature checked "Delete and Re-create Virtual Servers" <-- THIS IS WHERE I BELIEVE THERE TO BE THE BUG
  7. (Destination) Server all virtual servers and features were restored except for the DNS zones (which were enabled and backed up in #4 above.
  8. (Destination) Server - surmised the issue maybe due to #1 that the restore was ignoring the new enabled feature from the (Original) Server so I tested this by first deleting ALL Virtual Servers (so Virtualmin is now empty) and performing the same full backup using the same file as I did previously in #6. This time all features including DNS zone data were restored correctly.

As noted I believe this to be a bug as a new restore should "enable" any new features and "disable" any features deactivated as present in the backup being restored.

OK, So I spoke to soon. MY approach above fixed the DNS issue but now I have an Apache issue were almost all the virtual hosts are missing. In looking in /etc/httpd/conf/httpd.conf which I have not touched, it is 100% managed by Virtualmin on this new server there was an error where a "virtualhost" tag was not closed causing all virtual hosts below it to not be recognized and Apache to not be able to restart. As the restore restarts Apache after every Virtual Server successful restore this leaves only the hosts created before the erroneous one offline.

I had found this issue once a few years back but thought I must have screwed something up so never reported it.

In light of being a bit concerned by this situation before making this new server live I am going do more testing by deleting all virtual servers again and looking at the httpd.conf file to see if it is clean and if not manually clean it and perform the restore again. I definitely understand there is a lot of moving parts here but just wanted to report in hopes of maybe helping someone else struggling with this down the road on a production server.

  • Update 1 - Deleting all Virtual Server indeed cleaned the http.conf perfectly of all virtual hosts created by virtualmin
  • Update 2 - After the latest clean delete and restore, all DNS zones and Apache virtual hosts were restored perfectly however this time I used the command line and not the virtualmin interface fyi.

Yeah we'll have to get Jamie's input to know for sure regarding what the issues there may have been.

I see you have a few updates there -- are you saying that things are working properly for you now after your latest batch of restores?

Yes I am good now, essentially the restores worked well when I did a complete delete of everything first and if you look above Step #6 I believe contains the source of the bug you should look at.

I think the cause is that the DNS feature in Virtualmin wasn't enabled when the backup was made, hence the records didn't get included in the backup or restored. Even though you created the domain in the BIND module manually, this isn't known to Virtualmin.

I am not sure you read my report in full, I did later create the domains in Virtualmin after enabling the DNS feature so there is actually a bug in the backup sequence as it relates to the DNS when the DNS feature was not enabled from the beginning but later enabled.

My apologies - I see now. The "delete and re-create" option should be re-creating with the features from the backup, rather than what exists on the destination system.

For step 5 in comment #6, did you copy all the backup files (including those with .info and .dom extensions) to the new system?

Actually, I did some more testing and found the cause of this! A fix will be included in the next Virtualmin release.

Status: Active ยป Fixed

Yes, I did a full backup and copied the entire archive as I used the same backup to eventually solve my issue. The issue appears that the restore is not enabling new features added from a new backup (in may only be DNS as that is only one that affected me) unless you do a full delete of the virtual server first on the target server (not really ideal for production servers). Restoring to an existing virtual server appears to only restore those features that are "Currently" enabled on the target virtual server (regardless of what is in the backup) and thus ignoring any new features and the related data for that feature. Further even restoring (with new enabled features) after a full delete only worked 100% using the command line (not using Virtualmin web client)