Restore SSL Site Failure on SSL Certificates

When trying to backup & restore a site running SSL, I am receiving this error:

Restoring SSL Apache virtual host configuration and certificate ..
Restore failed : Missing file to open at WebminCore::../web-lib-funcs.pl line 2695

I was able to reproduce the issue too many times to count. I did fresh backups and ensured everything was being included.

While trying to troubleshoot the issue, I observed that Apache was trying to restart but missing the SSL CA (${HOME}/ssl.ca) file that was uploaded to the primary server.

I peeked inside the backup domain.tar.gz file and saw the following files (related to SSL):
domain.com_ssl
domain.com_ssl_cert
domain.com_ssl_key

Just creating the domain.com_ssl_ca file and recreating the domain.com.tar.gz archive didn't resolve the issue.

I've tried many different things, but I've had zero success using any of the built-in restore tools. What I can (kinda) do is restore all wanted items except the SSL files (and MySQL, but that's a different story), then enable SSL on the domain, and then reupload my certs into the Virtualmin interface.

Unfortunately, it's kludgy and won't be efficient in a pinch. I would love to provide some log data (or see it for myself), but I don't know if/where webmin/Virtualmin log any information to.

Any and all help will be greatly appreciated. Since this is currently a non-production server I can give Virtualmin staff access to the server to recreate the issue.

My Virtualmin backup settings:

  • All Servers, subservers, and all plans
  • Features
    • Virtual server password, description and other details
    • Administration user's password, quota and Cron jobs
    • Records in DNS domain, except SOA
    • Apache webserver configuration
    • Webalizer configuration and schedule
    • Apache SSL webserver configuration and certificate
    • Logrotate configuration for log file
    • Webmin ACL files
  • Virtualmin settings to also backup
    • Module configuration
    • Server templates
    • Custom fields, links and shells
  • Backup Format: One file per server (old format) <== I can't use the new format since I'm not backing up the Home directory apparently
  • Action on error: Continue
  • Backup level: Full
Status: 
Closed (fixed)

Comments

That's definately a bug ..

So I can track down the cause, could you add the line error_stack=1 to /etc/webmin/config , re-try the restore, and post the full error message here?

Thanks for the quick reply. Here is the full output from the restore.

Starting restore of 1 domains from local file /virtual-servers/09-11-2009-11-07/deepoutdoors.com.tar.gz ..

Extracting backup archive file .. .. done

Re-creating virtual server deepoutdoors.com ..

  Creating administration group deepoutdoors ..
  .. done

  Creating administration user deepoutdoors ..
  .. done

  Creating aliases for administration user ..
  .. done

  Adding administration user to groups ..
  .. done

  Creating home directory ..
  .. done

  Creating mailbox for administration user ..
  .. done

  Adding new DNS zone ..
  .. done

  Adding new virtual website ..
  .. done

  Adding Apache user www-data to server's group ..
  .. done

  Performing other Apache configuration ..
  .. done

  Creating SSL certificate and private key ..
  .. done

  Adding new SSL virtual website ..
  .. done

  Setting up log file rotation ..
  .. done

  Creating Webmin user ..
  .. done

  Re-starting DNS server ..
  .. done

  Stopping and re-starting web server ..
  .. not running!

  Re-loading Webmin ..
  .. done

  Saving server details ..
  .. done

Restoring backup for virtual server deepoutdoors.com ..

  Restoring virtual server password, quota and other details ..
  .. done

  Updating administration password and quotas ..
  .. done

  Restoring Cron jobs ..
  .. done

  Re-creating records in DNS domain ..
  .. done

  Restoring Apache virtual host configuration ..
  .. done

  Restoring SSL Apache virtual host configuration and certificate ..
  Restore failed : Missing file to open at WebminCore::../web-lib-funcs.pl line 2695
  Call Stack Trace
  File  Line    Function
  ../web-lib-funcs.pl   7391    WebminCore::error
  ../web-lib-funcs.pl   2695    WebminCore::open_tempfile
  /usr/share/webmin/virtual-server/feature-ssl.pl   646     WebminCore::flush_file_lines
  /usr/share/webmin/virtual-server/backups-lib.pl   1306    virtual_server::restore_ssl
  /usr/share/webmin/virtual-server/restore.cgi  227     virtual_server::restore_domains
  (eval 25)     6   (eval)
  /usr/share/webmin/miniserv.pl     2145    (eval)
  /usr/share/webmin/miniserv.pl     753     miniserv::handle_request
  <-  Return to previous page

Ok, thanks .. it looks like the problem is that Virtualmin exects to find a <Virtualhost> block in the Apache config for port 443, but isn't.

Could you check if this in your Apache config, or attach the /etc/apache2/sites-enabled/deepoutdoors.com.conf file to this bug report.

This is in my /etc/httpd/conf/httpd.conf file, which is where all my Apache configs are currently stored:

SuexecUserGroup deepoutdoors deepoutdoors
ServerName deepoutdoors.com
ServerAlias www.deepoutdoors.com deepoutdoors.com
DocumentRoot /home/deepoutdoors/public_html
ErrorLog "/home/deepoutdoors/logs/error_log"
CustomLog "/home/deepoutdoors/logs/access_log" combined
ScriptAlias /cgi-bin/ /home/deepoutdoors/cgi-bin/

Options Indexes IncludesNOEXEC FollowSymLinks
allow from all
AllowOverride All

allow from all

CBandScoreboard /etc/httpd/scoreboards/deepoutdoors.com

SuexecUserGroup deepoutdoors deepoutdoors
ServerName deepoutdoors.com
ServerAlias www.deepoutdoors.com deepoutdoors.com
DocumentRoot /home/deepoutdoors/public_html
ErrorLog "/home/deepoutdoors/logs/error_log"
CustomLog "/home/deepoutdoors/logs/access_log" combined
ScriptAlias /cgi-bin/ /home/deepoutdoors/cgi-bin/

Options Indexes IncludesNOEXEC FollowSymLinks
allow from all
AllowOverride All

allow from all

SSLEngine On
SSLCertificateFile /home/deepoutdoors/ssl.cert
SSLCertificateKeyFile /home/deepoutdoors/ssl.key

CBandScoreboard /etc/httpd/scoreboards/s.deepoutdoors.com
SSLCACertificateFile /home/deepoutdoors/ssl.ca

In addition, this is in deepoutdoors.com_ssl (in the backup tar.gz) file:

SuexecUserGroup deepoutdoors deepoutdoors
ServerName deepoutdoors.com
ServerAlias www.deepoutdoors.com deepoutdoors.com
DocumentRoot /home/deepoutdoors/public_html
ErrorLog "/home/deepoutdoors/logs/error_log"
CustomLog "/home/deepoutdoors/logs/access_log" combined
ScriptAlias /cgi-bin/ /home/deepoutdoors/cgi-bin/

Options Indexes IncludesNOEXEC FollowSymLinks
allow from all
AllowOverride All

allow from all

SSLEngine On
SSLCertificateFile /home/deepoutdoors/ssl.cert
SSLCertificateKeyFile /home/deepoutdoors/ssl.key

CBandScoreboard /etc/httpd/scoreboards/s.deepoutdoors.com
SSLCACertificateFile /home/deepoutdoors/ssl.ca

Is this what you were looking for? Or is there something that is managed within Virtualmin that I missed?

That looks OK to me ...

Could you try running the following command, and letting me know what it outputs :

virtualmin validate-domains --domain deepoutdoors.com --all-features

@linode1:~# virtualmin validate-domains --domain deepoutdoors.com --all-features
deepoutdoors.com
    Administration user : Encrypted password for Unix user deepoutdoors does not match virtual server

I get this for a few domains when I use the Virtualmin Validate Servers tool, but I'm not sure how to fix it.... also, it doesn't seem to impact the import of non-SSL servers.

Ok, that looks harmless .. it can happen if you changed a password outside of Virtualmin.

So if you re-try the restore for this domain from the same source, does it fail again?

Yes, it fails on each retry.

BTW, how did you get to the point where you figured out it was looking for the VirtualHost directive? I would like to be able to troubleshoot this further, but can't see any more log files.

I was only able to tell that the error was possible due to a missing VirtualHost directive because I looked at the line numbers in the error message..

I you try a restore of just the "Apache SSL webserver configuration and certificate" feature, does it still fail?

I hadn't tried it... I just deleted the domain from Virtualmin and tried importing just the one feature:

Starting restore of 1 domains from local file /virtual-servers/09-11-2009-11-07/deepoutdoors.com.tar.gz ..

Extracting backup archive file ..
.. done

Re-creating virtual server deepoutdoors.com ..

      Creating SSL certificate and private key ..
      .. SSL website failed! : Failed to open /home/deepoutdoors/ssl.cert.webmintmp.7366 : No such file or directory at ../web-lib-funcs.pl line 1323, line 1.

      Saving server details ..
      .. done

Restoring backup for virtual server deepoutdoors.com ..

.. restore complete.

In Virtualmin I see deepoutdoors.com created, but no SSL certs (and of course no website or DNS, etc.).

Actually, I was thinking of doing a full restore (which would presumably fail), then doing a restore of just the SSL feature. There's no need to delete the domain after the full restore.

OK, I tried that, too. I did a full restore (failed) and then an SSL-only restore, and here's the output:

Starting restore of 1 domains from local file /virtual-servers/09-11-2009-11-07/deepoutdoors.com.tar.gz ..

Extracting backup archive file ..
.. done

Restoring backup for virtual server deepoutdoors.com ..

      Restoring SSL Apache virtual host configuration and certificate ..
      Restore failed : Missing file to open at WebminCore::../web-lib-funcs.pl line 2695
      Call Stack Trace
      File Line Function
      ../web-lib-funcs.pl 7391 WebminCore::error
      ../web-lib-funcs.pl 2695 WebminCore::open_tempfile
      /usr/share/webmin/virtual-server/feature-ssl.pl 646 WebminCore::flush_file_lines
      /usr/share/webmin/virtual-server/backups-lib.pl 1306 virtual_server::restore_ssl
      /usr/share/webmin/virtual-server/restore.cgi 227 virtual_server::restore_domains
      (eval 29) 6 (eval)
      /usr/share/webmin/miniserv.pl 2145 (eval)
      /usr/share/webmin/miniserv.pl 753 miniserv::handle_request

I thought I should also mention that this is not isolated to this one domain. I have the same problem with another domain that I'm trying to restore from my primary server.

Is there anything else I should be looking at in my configuration? I have had these domains running for a while and have done many hand-tweaks of configurations (see the above apache conf).

Thanks for the continued help...

Would it be possible for you to send me one of the backup files so I can do a test restore and see if anything is confusing Virtualmin? My email is jcameron@virtualmin.com

Thanks .. however, it seems that the restore worked fine on all my test systems.

If you like, I could login to your system and try a restore there myself to see what is going wrong?

Server info sent, thanks for the help.

I installed Webmin from the 1.480 .deb and used the upgrade in the UI to upgrade to 1.490. I installed Virtualmin from the .tar.gz file on the Webmin site.

I hope all this helps... It sounds like this isn't a bug as much as it is my own issue...

So I took a look at the deepoutdoors.com domain, and found that it was missing the block for SSL completely .. this was revealed by running validation on that domain.

The fix was to disable and then re-enable the SSL feature with the commands :

virtualmin disable-feature --domain deepoutdoors.com --ssl
virtualmin enable-feature --domain deepoutdoors.com --ssl

The restore then worked fine..

Hmm, odd. On the primary server, SSL is enabled in Virtualmin (i.e. the SSL Enabled box is checked). Is there something in the config on the primary server I can verify so the backup and restore doesn't require that manual step?

I did verify that the /etc/webmin/virtual-servers/ for deepoutdoors has ssl=1.

I did a little more investigation. In the backup tar.gz, deepoutdoors.com_webmin has feature-ssl=0. That might be why it's disabled... I would change it to 1, but the _webmin file is a binary file, so I can't edit it directly safely.

Do you mind if I delete and then re-create from the backup that deepoutdoors.com domain on your system?

On the server which I emailed you? Sure, that is my backup system - it's a playground for now :)

Ok, I found and fixed the problem .. there was a Virtualmin bug triggered by having an extra / at the end of the directory to add Apache config files to! Once I changed that on your system, all was good.

I will include code in the next Virtualmin release to better handle this case.

Can you point me to the fix? I want to do a fresh install of the system and test my build and recovery procedures.

You're awesome!

It was in the /etc/webmin/apache/config file , in the line that contains /etc/httpd/conf.d .. there was an extra / at the end.

Thanks!! I changed that setting on my primary server before doing the backup & restore, and everything worked perfectly.

Automatically closed -- issue fixed for 2 weeks with no activity.