Unable to renew Let's Encrypt certificate via DNS validation

Hi,

I am receiving the following error message in Virtualmin when I attempt to renew a Let's Encrypt certificate for one of my domains. Usually the renewal happens automatically but it failed this morning and the certificate expires early tomorrow morning so I'm trying to get this solved as quickly as possible as the site is currently live and cannot be taken offline.

Here is the error:

Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org Obtaining a new certificate Performing the following challenges: dns-01 challenge for katcallinginc.com dns-01 challenge for www.katcallinginc.com Hook command "/etc/webmin/webmin/letsencrypt-dns.pl" returned error code 25 Error output from letsencrypt-dns.pl: Undefined subroutine &main::get_bind_zone_for_domain called at /usr/libexec/webmin/webmin/letsencrypt-dns.pl line 21.

Hook command "/etc/webmin/webmin/letsencrypt-dns.pl" returned error code 25 Error output from letsencrypt-dns.pl: Undefined subroutine &main::get_bind_zone_for_domain called at /usr/libexec/webmin/webmin/letsencrypt-dns.pl line 21.

Waiting for verification... An unexpected error occurred: The request message was malformed :: Unable to update challenge :: authorization is not pending

What does this mean? As far as I know I am not having any DNS issues or BIND server problems, and setting "Check connectivity" to "Yes" in the Virtualmin GUI for Let's Encrypt returns "no problems found." Thank you in advance for any support provided.

Status: 
Active

Comments

Unfortunately that link has nothing to do with my issue. The error messages in my original post do not mention anything about an agreement whatsoever; rather, they seem to indicate a problem with an undefined subroutine in the Webmin code.

It sounds like you could be seeing a bug -- Jamie will get with you shortly about that.

Can you check which Webmin version you're running there? The command rpm -q webmin will tell you.

I'm running the latest, Webmin 1.860 for CentOS 7 (latest version of my OS as well, yum update does not report anything there).

Just to make sure you aren't seeing an issue we already fixed, can you upgrade to the latest development version of Webmin? You can do this with a command like :

rpm -U https://download.webmin.com/devel/rpm/webmin-1.861-1.noarch.rpm

I've upgraded to the latest development version of Webmin as indicated in the last post, but am still seeing the error in my original post. Alternatively, I sometimes see a "no zone named www.domain.com" message from the same Webmin script that produces the original error message, but when this happens versus when I see the normal error I've already mentioned is not easily determinable.

I've been experiencing the same problem exactly. It looks like it may be a bug as I have seen reports in letsencrypt forums as well.

The way I resolved it was manually editing the DNS Records for the domain to delete the "www" A and AAAA records, then adding a CNAME for "www" to the "domain.com" address. After doing this the certificate renewal was successful.

Hope that helps.

Good morning,

I have updated to the latest Webmin development release (1.862) as Jamie detailed in his post, but still get the same error that started this whole issue:

DNS-based validation failed : Saving debug log to /var/log/letsencrypt/letsencrypt.log Plugins selected: Authenticator manual, Installer None Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org Obtaining a new certificate Performing the following challenges: dns-01 challenge for deveauxminimart.com dns-01 challenge for www.deveauxminimart.com Hook command "/etc/webmin/webmin/letsencrypt-dns.pl" returned error code 25 Error output from letsencrypt-dns.pl: Undefined subroutine &main::get_bind_zone_for_domain called at /usr/libexec/webmin/webmin/letsencrypt-dns.pl line 20.

Hook command "/etc/webmin/webmin/letsencrypt-dns.pl" returned error code 25 Error output from letsencrypt-dns.pl: Undefined subroutine &main::get_bind_zone_for_domain called at /usr/libexec/webmin/webmin/letsencrypt-dns.pl line 20.

Waiting for verification... Cleaning up challenges Hook command "/etc/webmin/webmin/letsencrypt-cleanup.pl" returned error code 25 Error output from letsencrypt-cleanup.pl: Undefined subroutine &main::get_bind_zone_for_domain called at /usr/libexec/webmin/webmin/letsencrypt-cleanup.pl line 18.

Hook command "/etc/webmin/webmin/letsencrypt-cleanup.pl" returned error code 25 Error output from letsencrypt-cleanup.pl: Undefined subroutine &main::get_bind_zone_for_domain called at /usr/libexec/webmin/webmin/letsencrypt-cleanup.pl line 18.

Failed authorization procedure. deveauxminimart.com (dns-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.deveauxminimart.com IMPORTANT NOTES: - The following errors were reported by the server:

Domain: deveauxminimart.com Type: connection Detail: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.deveauxminimart.com

That is very very odd - can you post the output from the following command on your system :

perl -c /usr/libexec/webmin/webmin/letsencrypt-lib.pl

Here is the output of the command:

/usr/libexec/webmin/webmin/letsencrypt-lib.pl syntax OK

Can you attach that /usr/libexec/webmin/webmin/letsencrypt-lib.pl file from your system to this bug report?

Here's a copy of the script as requested. Also, I've noticed over the past few days that sometimes the script will work just fine, and other times I will get the error that started this entire thread; so far it seems to be completely random as to when it works and when it doesn't, as every domain I've used with Let's Encrypt (every virtual server, I should say) over the past few days has valid DNS records in BIND on my system.

It may be working most of the time because DNS-based validation isn't being used - instead, the web-based validation is completing successfully instead.

However, that doesn't explain why DNS validation isn't working on your system. I have one idea though, which will be implemented in the next Webmin release.

That's certainly possible; maybe when Web-based validation fails for some reason (unrelated to Webmin) it falls back to DNS-based validation, which fails because of the error we're trying to diagnose. What's weird though is that, sometimes Webmin will report the undefined subroutine error in my original post, and other times the same script will just report, "No zone named www.domain.com found" when there clearly is a zone with that name in BIND and a virtual server with that name too (because it's the one for which I'm trying to get an SSL certificate).

Hi,

Any update on this issue? I am still getting DNS-based validation failures with either the undefined subroutine error or the "no zone named www.domain.com" error even when a DNS zone exists in BIND. Furthermore, Web-based validation only works if I first remove all SSL redirects that Virtualmin put in place and then try to get a certificate, but I thought Virtualmin was supposed to do this automatically and then put them back in place after the certificate was issued?

If you wait a couple of days, we'll have a new version of Webmin out that should fix this.

That sounds good. Do you have a more specific ETA for when this new version may be available? I've already got a few certificates expiring as early as tomorrow, December 5, unfortunately. Really looking forward to finally being rid of these Let's Encrypt issues though :)

I still see this issue on Version: 1.870 of webmin. Os is Ubuntu 14.04

The error message is: Hook command "/etc/webmin/webmin/letsencrypt-dns.pl" returned error code 25 Error output from letsencrypt-dns.pl: Undefined subroutine &main::get_bind_zone_for_domain called at /usr/share/webmin/webmin/letsencrypt-dns.pl line 24.

and on cleanup Hook command "/etc/webmin/webmin/letsencrypt-cleanup.pl" returned error code 25 Error output from letsencrypt-cleanup.pl: Undefined subroutine &main::get_bind_zone_for_domain called at /usr/share/webmin/webmin/letsencrypt-cleanup.pl line 22.

file seems ok

perl -c /etc/webmin/webmin/letsencrypt-dns.pl /etc/webmin/webmin/letsencrypt-dns.pl syntax OK perl -c /usr/share/webmin/webmin/letsencrypt-cleanup.pl /usr/share/webmin/webmin/letsencrypt-cleanup.pl syntax OK

perl is 5.18.2

i added

require './letsencrypt-lib.pl';

to letsencrypt-dns.pl and letsencrypt-cleanup.pl

above error is gone, but i get new errors, will investigate more

ok what i found out so far is that adding: require './letsencrypt-lib.pl'; helps.

i can request certs for domains where i request certs for "Domains associated with this server" which fallback to dns verification. But for domains where i have "Domain names listed here" selected the verification fails with: Error: urn:acme:error:malformed :: The request message was malformed :: Unable to update challenge :: authorization is not pending 2017-12-20 20:04:16,711:ERROR:certbot.log:An unexpected error occurred: 2017-12-20 20:04:16,711:ERROR:certbot.log:The request message was malformed :: Unable to update challenge :: authorization is not pending

What domains did you enter in the "Domain names listed here" box?

UPDATE: Success

As per #22 above, I modified two files to contain a

require './letsencrypt-lib.pl';

letsencrypt-dns.pl AND letsencrypt-cleanup.pl in directory /usr/share/webmin/webmin/

ORIGINAL ISSUE:

Requesting letsencrypt certificate I get a bunch of these:

...
2017-12-31 02:33:25,984:ERROR:certbot.hooks:Hook command "/etc/webmin/webmin/letsencrypt-dns.pl" returned error code 25
2017-12-31 02:33:25,984:ERROR:certbot.hooks:Error output from letsencrypt-dns.pl:
Undefined subroutine &main::get_bind_zone_for_domain called at /usr/share/webmin/webmin/letsencrypt-dns.pl line 24.
...

and

# perl -c /usr/share/webmin/webmin/letsencrypt-lib.pl
/usr/share/webmin/webmin/letsencrypt-lib.pl syntax OK
# ls -al `locate letsencrypt-dns.pl`
-rwxr-xr-x 1 root root  487 Dec 31 02:33 /etc/webmin/webmin/letsencrypt-dns.pl
-rwxr-xr-x 1 root root 1480 Dec  6 18:21 /usr/share/webmin/webmin/letsencrypt-dns.pl

and

Operating system Debian Linux 8
Webmin version 1.870  Usermin version 1.730
Virtualmin version 6.02  Theme version Authentic Theme 19.04
Time on system Saturday, December 30, 2017 8:56 PM Kernel and CPU Linux 4.9.50-x86_64-linode86 on x86_64
Processor information Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz, 2 cores System uptime 43 days, 1 hours, 48 minutes
Running processes 181 CPU load averages 0.06 (1 min) 0.08 (5 mins) 0.08 (15 mins)
Real memory 3.77 GB total / 1.26 GB used Virtual memory 250 MB total / 55.68 MB used
Local disk space 19.43 GB total / 11.91 GB free / 7.53 GB used Package updates All installed packages are up to date

"Domain names listed here", all of which, remotely executed dig indicates resolve, via A record, to the server's IP address.

domain1.com
www.domain1.com
ftp.domain1.com
mail.domain1.com
smtp.domain1.com
pop3.domain1.com
pop.domain1.com
domain2.com
www.domain2.com
ftp.domain2.com
mail.domain2.com
smtp.domain2.com
pop3.domain2.com
pop.domain2.com
domain3.com
www.domain3.com
ftp.domain3.com
mail.domain3.com
smtp.domain3.com
pop3.domain3.com
pop.domain3.com
domain4.com
www.domain4.com
ftp.domain4.com
mail.domain4.com
smtp.domain4.com
pop3.domain4.com
pop.domain4.com

This is getting a bit worrisome, since expiration hits on Jan 7.

It's very unusual that adding this require helps, as in webmin-lib.pl which is also require'd there is already a call to 'letsencrypt-lib.pl`

Can you post the first 15 lines of /usr/share/webmin/webmin/letsencrypt-dns.pl on your system?

i attached the files from my system as "letsencrypt-files-fox.zip"

That actually looks fine.

If you SSH in as root and run :

CERTBOT_DOMAIN=example.com CERTBOT_VALIDATION=test /etc/webmin/webmin/letsencrypt-dns.pl

what does it output?

# head -n 15 /usr/share/webmin/webmin/letsencrypt-dns.pl
#!/usr/bin/perl
# Called by the let's encrypt client to add a DNS record for validation

$no_acl_check++;
$ENV{'WEBMIN_CONFIG'} ||= "/etc/webmin";
$ENV{'WEBMIN_VAR'} ||= "/var/webmin";
if ($0 =~ /^(.*\/)[^\/]+$/) {
        chdir($1);
        }
else {
        chop($pwd = `pwd`);
        }
$0 = "$pwd/letsencrypt-dns.pl";
require './webmin-lib.pl';
require './letsencrypt-lib.pl'; # LOCAL MODIFICATION AS PER https://www.virtualmin.com/comment/789139#comment-789139
# CERTBOT_DOMAIN=example.com CERTBOT_VALIDATION=test ;/etc/webmin/webmin/letsencrypt-dns.pl
Error: Script was not run with full path (failed to find /letsencrypt-dns.pl under /usr/share/webmin)

-----
Script was not run with full path (failed to find /letsencrypt-dns.pl under /usr/share/webmin)
-----
# pwd
/root

This is why:

# ls /usr/share/webmin/letsencrypt-dns.pl
ls: cannot access /usr/share/webmin/letsencrypt-dns.pl: No such file or directory
# ls /usr/share/webmin/webmin/letsencrypt-dns.pl
/usr/share/webmin/webmin/letsencrypt-dns.pl

Ok, I see the cause of this bug - it will be fixed in Webmin 1.872, which is being rolled out to Virtualmin customers.

ok i am on the latest version and i am seeing error ..

DNS-based validation failed : Failed to request certificate : Undefined subroutine &main::get_bind_zone_for_domain called at /usr/share/webmin/webmin/letsencrypt-dns.pl line 24. inter-stellar.tech challenge did not pass: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.inter-stellar.tech

Sorry to post of the original users post.

masterg0g0 - what happens if you try running the commands from comment #30 ?

Subscribing to this topic because I have basically the exact same problem.

(edit: What are those attachments? I didn't add those! lol)

So I'm not very familiar with perl, but my source file "letsencrypt-dns.pl" over here only seems to have an include for "webmin-lib.pl" and a foreign_require("bind8")

But the thing is, that function get_bind_zone_for_domain() doesn't seem to exist in the webmin-lib.pl file, and I'm actually running bind9, so ... maybe the include and function name needs to be updated or something.

(just glancing; could be wrong)

Undefined subroutine &main::get_bind_zone_for_domain called at /usr/libexec/webmin/webmin/letsencrypt-dns.pl line 24 Wen trying letsencrypt on maindomain.

Also here with wen using mail.maindomain.com subserver where This virtual server shares its SSL certificate with maindomain.com, so it cannot be edited on this page. Use its Manage SSL Certificate page to change SSL settings.

Operating system CentOS Linux 7.4.1708 Webmin version 1.872 Usermin version 1.732 Virtualmin version 6.02 Theme version Authentic Theme 19.07

Error Is with subserver mail.... wen i manual listed the domain and remove the mail.maindomain.com out of that list it is working!

So i think something with subservers, subdomains and co

Workarround that seems to work here ( on domains created 2017), is removing virtual server shares its SSL certificate with maindomain.com, then letsencrypt for that subserver (mail) itself, then again doing letsencrypt on maindomain, with manual list where mail. subserver is in then no error ... but this is not the way it should. ;)

# Functions for cert creation with Let's Encrypt

if ($config{'letsencrypt_cmd'}) {
$letsencrypt_cmd = &has_command($config{'letsencrypt_cmd'});
}
else {
$letsencrypt_cmd = &has_command("letsencrypt-auto") ||
   &has_command("letsencrypt") ||
   &has_command("certbot-auto") ||
   &has_command("certbot");
}

$account_key = "$module_config_directory/letsencrypt.pem";

$letsencrypt_chain_urls = [
"https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem",
];

sub get_letsencrypt_python_cmd
{
return &has_command("python2.7") || &has_command("python27") ||
       &has_command("python2.6") || &has_command("python26") ||
       &has_command("python");
}

# check_letsencrypt()
# Returns undef if all dependencies are installed, or an error message
sub check_letsencrypt
{
if (&has_command($letsencrypt_cmd)) {
# Use built-in client
return undef;
}
my $python = &get_letsencrypt_python_cmd();
if (!$python || !&has_command("openssl")) {
return $text{'letsencrypt_ecmds'};
}
my $out = &backquote_command("$python -c 'import argparse' 2>&1");
if ($?) {
return &text('letsencrypt_epythonmod', 'argparse');
}
my $ver = &backquote_command("$python --version 2>&1");
if ($ver !~ /Python\s+([0-9\.]+)/) {
return &text('letsencrypt_epythonver',
     "<tt>".&html_escape($out)."</tt>");
}
$ver = $1;
if ($ver < 2.5) {
return &text('letsencrypt_epythonver2', '2.5', $ver);
}
return undef;
}

# request_letsencrypt_cert(domain|&domains, webroot, [email], [keysize],
#    [request-mode], [use-staging])
# Attempt to request a cert using a generated key with the Let's Encrypt client
# command, and write it to the given path. Returns a status flag, and either
# an error message or the paths to cert, key and chain files.
sub request_letsencrypt_cert
{
my ($dom, $webroot, $email, $size, $mode, $staging) = @_;
my @doms = ref($dom) ? @$dom : ($dom);
$email ||= "root\@$doms[0]";
$mode ||= "web";

my $challenge;
if ($mode eq "web") {
# Create a challenges directory under the web root
$challenge = "$webroot/.well-known/acme-challenge";
my @st = stat($webroot);
my $user = getpwuid($st[4]);
if (!-d $challenge) {
my $cmd = "mkdir -p -m 755 ".quotemeta($challenge);
if ($user && $user ne "root") {
$cmd = &command_as_user($user, 0, $cmd);
}
my $out = &backquote_logged("$cmd 2>&1");
if ($?) {
return (0, "mkdir failed : $out");
}
}

# Create a .htaccess file to ensure the directory is accessible
if (&foreign_installed("apache")) {
&foreign_require("apache");
my $htaccess = "$challenge/.htaccess";
if (!-r $htaccess && $apache::httpd_modules{'core'} >= 2.2) {
&open_tempfile(HT, ">$htaccess");
&print_tempfile(HT, "AuthType None\n");
&print_tempfile(HT, "Require all granted\n");
&print_tempfile(HT, "Satisfy any\n");
&close_tempfile(HT);
&set_ownership_permissions(
$user, undef, 0755, $htaccess);
}
}
}
elsif ($mode eq "dns") {
# Make sure all the DNS zones exist
&foreign_require("bind8");
foreach my $d (@doms) {
my $z = &get_bind_zone_for_domain($d);
$z || return (0, "Neither DNS zone $d or any of its ".
"sub-domains exist on this system");
}
}

# Create DNS hook wrapper scripts
my $dns_hook = "$module_config_directory/letsencrypt-dns.pl";
my $cleanup_hook = "$module_config_directory/letsencrypt-cleanup.pl";
if ($mode eq "dns") {
&foreign_require("cron");
&cron::create_wrapper($dns_hook, $module_name,
      "letsencrypt-dns.pl");
&cron::create_wrapper($cleanup_hook, $module_name,
      "letsencrypt-cleanup.pl");
}

if ($letsencrypt_cmd && -d "/etc/letsencrypt/accounts") {
# Use the native Let's Encrypt client if possible
my $temp = &transname();
&open_tempfile(TEMP, ">$temp");
&print_tempfile(TEMP, "email = $email\n");
&print_tempfile(TEMP, "text = True\n");
&close_tempfile(TEMP);
my $dir = $letsencrypt_cmd;
$dir =~ s/\/[^\/]+$//;
$size ||= 2048;
my $out;
if ($mode eq "web") {
# Webserver based validation
&clean_environment();
$out = &backquote_command(
"cd $dir && (echo A | $letsencrypt_cmd certonly".
" -a webroot ".
join("", map { " -d ".quotemeta($_) } @doms).
" --webroot-path ".quotemeta($webroot).
" --duplicate".
" --manual-public-ip-logging-ok".
" --config $temp".
" --rsa-key-size $size".
($staging ? " --test-cert" : "").
" 2>&1)");
&reset_environment();
}
elsif ($mode eq "dns") {
# DNS based validation, via hook script
&clean_environment();
$out = &backquote_command(
"cd $dir && (echo A | $letsencrypt_cmd certonly".
" --manual".
join("", map { " -d ".quotemeta($_) } @doms).
" --preferred-challenges=dns".
" --manual-auth-hook $dns_hook".
" --manual-cleanup-hook $cleanup_hook".
" --duplicate".
" --manual-public-ip-logging-ok".
" --config $temp".
" --rsa-key-size $size".
($staging ? " --test-cert" : "").
" 2>&1)");
&reset_environment();
}
else {
return (0, "Bad mode $mode");
}
if ($?) {
return (0, "<pre>".&html_escape($out || "No output from $letsencrypt_cmd")."</pre>");
}
my ($full, $cert, $key, $chain);
if ($out =~ /(\/etc\/letsencrypt\/(?:live|archive)\/[a-zA-Z0-9\.\_\-\/\r\n ]*\.pem)/) {
# Output contained the full path
$full = $1;
$full =~ s/\s//g;
}
else {
# Try searching common paths
my @fulls = glob("/etc/letsencrypt/live/$doms[0]-*/cert.pem");
if (@fulls) {
my %stats = map { $_, [ stat($_) ] } @fulls;
@fulls = sort { $stats{$a}->[9] <=> $stats{$b}->[9] }
      @fulls;
$full = pop(@fulls);
}
else {
&error("Output did not contain a PEM path!");
}
}
-r $full && -s $full || return (0, &text('letsencrypt_efull', $full));
$full =~ s/\/[^\/]+$//;
$cert = $full."/cert.pem";
-r $cert && -s $cert || return (0, &text('letsencrypt_ecert', $cert));
$key = $full."/privkey.pem";
-r $key && -s $key || return (0, &text('letsencrypt_ekey', $key));
$chain = $full."/chain.pem";
$chain = undef if (!-r $chain);
&set_ownership_permissions(undef, undef, 0600, $cert);
&set_ownership_permissions(undef, undef, 0600, $key);
&set_ownership_permissions(undef, undef, 0600, $chain);
return (1, $cert, $key, $chain);
}
else {
# Fall back to local Python client
$size ||= 4096;

# But first check if the native Let's Encrypt client was used previously
# for this system - if so, it must be used in future due to the account
# key.
-d "/etc/letsencrypt/accounts" &&
return (0, &text('letsencrypt_enative', '/etc/letsencrypt'));

# Generate the account key if missing
if (!-r $account_key) {
my $out = &backquote_logged(
"openssl genrsa 4096 2>&1 >$account_key");
if ($?) {
return (0, &text('letsencrypt_eaccountkey',
&html_escape($out)));
}
}

# Generate a key for the domain
my $key = &transname();
my $out = &backquote_logged("openssl genrsa $size 2>&1 >$key");
if ($?) {
return (0, &text('letsencrypt_ekeygen', &html_escape($out)));
}

# Generate a CSR
my $csr = &transname();
my ($ok, $csr) = &generate_ssl_csr($key, undef, undef, undef,
   undef, undef, \@doms, undef);
if (!$ok) {
return &text('letsencrypt_ecsr', $csr);
}
&copy_source_dest($csr, "/tmp/lets.csr", 1);

# Find a reasonable python version
my $python = &get_letsencrypt_python_cmd();

# Request the cert and key
my $cert = &transname();
my $out = &backquote_logged(
"$python $module_root_directory/acme_tiny.py ".
"--account-key ".quotemeta($account_key)." ".
"--csr ".quotemeta($csr)." ".
($mode eq "web" ? "--acme-dir ".quotemeta($challenge)." "
: "--dns-hook $dns_hook ".
  "--cleanup-hook $cleanup_hook ").
($staging ? "--ca https://acme-staging.api.letsencrypt.org "
  : "").
"--quiet ".
"2>&1 >".quotemeta($cert));
if ($?) {
my @lines = split(/\r?\n/, $out);
my $trace;
for(my $i=1; $i<@lines; $i++) {
if ($lines[$i] =~ /^Traceback\s+/) {
$trace = $i;
last;
}
}
if ($trace) {
@lines = @lines[0 .. $trace-1];
$out = join("\n", @lines);
}
return (0, &text('letsencrypt_etiny',
"<pre>".&html_escape($out))."</pre>");
}
-r $cert && -s $cert || return (0, &text('letsencrypt_ecert', $cert));

# Download the latest chained cert files
my $chain = &transname();
foreach my $url (@$letsencrypt_chain_urls) {
my $cout;
my ($host, $port, $page, $ssl) = &parse_http_url($url);
my $err;
&http_download($host, $port, $page, \$cout, \$err, undef, $ssl);
if ($err) {
return (0, &text('letsencrypt_echain', $err));
}
my $fh = "CHAIN";
&open_tempfile($fh, ">>$chain");
&print_tempfile($fh, $cout);
&close_tempfile($fh);
}

# Copy the per-domain files
my $certfinal = "$module_config_directory/$doms[0].cert";
my $keyfinal = "$module_config_directory/$doms[0].key";
my $chainfinal = "$module_config_directory/$doms[0].chain";
&copy_source_dest($cert, $certfinal, 1);
&copy_source_dest($key, $keyfinal, 1);
&copy_source_dest($chain, $chainfinal, 1);
&set_ownership_permissions(undef, undef, 0600, $certfinal);
&set_ownership_permissions(undef, undef, 0600, $keyfinal);
&set_ownership_permissions(undef, undef, 0600, $chainfinal);
&unlink_file($cert);
&unlink_file($key);
&unlink_file($chain);

return (1, $certfinal, $keyfinal, $chainfinal);
}
}

# get_bind_zone_for_domain(domain)
# Given a hostname like www.foo.com, return the local BIND zone that contains
# it like foo.com
sub get_bind_zone_for_domain
{
my ($d) = @_;
&foreign_require("bind8");
my $bd = $d;
while ($bd =~ /\./) {
my $z = &bind8::get_zone_name($bd, "any");
if ($z) {
return ($z, $bd);
}
$bd =~ s/^[^\.]+\.//;
}
return ( );
}

1;

If anyone who is having this problem could send me login details to their system (via email at jcameron@virtualmin.com), I'm sure we can get it resolved quickly. This is a hard issue to debug, as even though a few users are reporting it, it doesn't happen on any of my test systems!

Before we had to add/create new actual virtual subserver for mail.maindomain.com to have letsencrypt and so on get working, install was virtualmin server in august 2017 running with some install bugs, hopefully solved after that ;)

Is this now changed somehow so not needed to have to create a hard real subserver mail.?

We have no PRO! so i'm only posting here while maybe same kind of problem some others have, hope it helps somehow sometime someone. ;)

Could also be paths, htaccess rewrite and so on, also we have other paths var/.. while other apache repo coditguru... and remiphp, with older version virtualmin 6 ... i had to create the "welknow acme directory's" in subservers domain/..path by hand before LE to work for that.

So i think problem is mainly with subserver/allias and so on, and also mayby on boxes that where installed updated before with one of the early VM 6.x install scripts versions.

Offtopic but important! Logins and so on out of germany hmm , for all your information: sending, giving LOGIN credentials for EU for external service support are becomming very strict under some LAW..... in a short time, Germany it is already.

Maybe someone can help you with this future problem (giving login credentials EU for external support and new privacy law rules) to have a kind of contract text, that is waterproof for these situation and a howto i'm no good with English and lawtxts blabl.. ;) This should be a good idea i hope, to have a howto and default a extra separate (root) support user with that standard contract text to aply before login, and have a button to renew password, and switch on/of in panel for that?

I think we can get around the issue of sending out login details by instead using our Remote Support feature in Virtualmin, which grants access via an SSH key, and thus doesn't require you to send us your root password or anything.

I have just set up the Virtualmin remote logins feature on my system to enable you to log in as the root user via an SSH key and diagnose this problem for me. The login expires on January 10, 2018, and Virtualmin informed me that it sent a notification of the login access to remote@virtualmin.com

Thanks! Also, can you tell me which specific domain you are having trouble requesting a cert for, and if you're doing it using the web UI, the command line, or if this is an automatic renewal?

Jamie, do you need another system to test this on? If so, how do I get the remote login module going?

Yes, if you have a system and domain that reliably fails.

I'm having the same issue. Jamie you need access to my server also ?

I did, but now it's showing different after running the commands from comment 30. I'll step back until I can confirm.

Yep with subdomains. Have some extra info Partly depending when extern DNS. So when created also a mail.maindomain subserver , and have this automaticly shared with maindomain her goes wrong

Also autodiscover.maindomain ( new created virtualserver for that domain domain is years old extern dns with also autodiscover.maindomain resolving) And worse autoconfig.maindomain same error. ( you can't put this in external dns, not allowed)

Gives error Letsencrypt

autodiscover.themaindomain challenge did not pass: Invalid response from http://autodiscover.themaindomain/.well-known/acme-challenge/thecertcodehere: "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p" ,

So its is i believe going wrong here different ways depending extern dns with subdomains / subservers and no real web/disk path/space. Also some dns panels or settings external are maybe not 100% or guys as me are doing things wrong there! ;)

  • FYI: the problem to give login is not only the security with the new EU LAWS, it has to do with signed contracts when some company in EU let other external Support on their Server, to comply with the new dataprivacy LAWS EU. ( IN that contract their should be a kind of rule howto keep secret and data not used/leaking and so on.. blabla sorry i'm no Lawyer.) ( so for companys out of EU giving you support acces to their Servers where Customer info's are on.... )

But try yourself some different constillations with external dns, (so not on the virtualmin server) and then subdomains, subservers and so on. ( also the mail, autodiscover, while autodiscover was is wrong right now here but thinking it the dns i hope. Also try some different contillations with ipv6 in partly set for domains, where the virtualmin box has ipv6 also but domain only ipv4, dns external yes and no with ipv6, i think some could go wrong to where not all is ipv6 set. ,/b> For the FAQ / howto / Docu Letsencrypt virtualmin sake

External dns have no *.maindomain option, in this case

Jamie, my apologies for the delay in getting back to you about this. While I don't have any domains expiring right now or that have already expired, my next expiring domain is southtownswine.com and it expires January 20th. Before I implemented the fix of adding the require line to letsencrypt-dns.pl, I was unable to renew certificates via DNS validation. But of course this fix was from a member of the user community, not an official fix, and I don't know if it works all the time, so I was hoping you could log into my system and apply a more official patch or at least better observe the problem. The other issue that has still not been fixed is that Web-based validations fail even when they used to succeed perfectly and would never have to fall back to DNS-based validations.

Please keep in mind though that my system is part of a production environment as this issue currently doesn't exist on my development system. Its certificates can somehow renew just fine via either Web-based or DNS-based validation (I'm not sure which).

Sorry i am in your topic could be something complete different the posts i mad.

For your domain in the error message a check on that domain gives these errors and warnings ( could also be a timeout, but then you should have a log message somewhere i presume)

smtp  mail2  Failed To Connect  
smtp  mail2  IP Resolution Error  
smtp  mail2.jemediacorp.com  Failed To Connect  
smtp  mail2.jemediacorp.com  IP Resolution Error
dns  deveauxminimart.com  Name Servers are on the Same Subnet  
dns  deveauxminimart.com  SOA Serial Number Format is Invalid  
dns  deveauxminimart.com  SOA Expire Value out of recommended range 
smtp  mail.deveauxminimart.com  6.800 seconds - Warning on Connection time  
smtp  mail.deveauxminimart.com  8.869 seconds - Not good! on Transaction Time

Don't know if this could be blocking something. mxtoolbox healthcheck https://mxtoolbox.com/domain/deveauxminimart.com/

Is there a fix for this yet ?

getting something similar with version webmin 1.872, virtualmin 6.02.

request failed : Web-based validation failed : Failed to request certificate :
Traceback (most recent call last):
  File "/usr/share/webmin/webmin/acme_tiny.py", line 250, in <module>
    main(sys.argv[1:])
  File "/usr/share/webmin/webmin/acme_tiny.py", line 246, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, args.dns_hook, args.cleanup_hook, log=LOGGER, CA=args.ca)
  File "/usr/share/webmin/webmin/acme_tiny.py", line 196, in get_crt
    log.error("{0} challenge did not pass: {1}".format(domain, challenge_status['error']['detail']))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa9' in position 152: ordinal not in range(128)
DNS-based validation failed : Failed to request certificate :
Undefined subroutine &main::get_bind_zone_for_domain called at /usr/share/webmin/webmin/letsencrypt-dns.pl line 24.
mydomain.com challenge did not pass: No TXT record found at _acme-challenge.mydomain.com

Another "me too". I get exactly the same problem as clemsontiger in #50:

DNS-based validation failed : Failed to request certificate :
Undefined subroutine &main::get_bind_zone_for_domain called at /usr/share/webmin/webmin/letsencrypt-dns.pl line 24.
mydomain.com challenge did not pass: No TXT record found at _acme-challenge.mydomain.com

Ok this has been a problem for well over a month and if this can't be fixed tell us cause it's killing websites left and right and now I got clients up my ass because now they are having to purchase ssl certs.

For anyone else who's seeing this - if we could get remote access to your system to figure out what's going wrong, that would be very useful.

Hi Jamie,

I'd be delighted to give you access, but the only problem is I don't have a webmin-virtualmin-support package available and can't find the optional Virtualmin packages link as described on the Support Requests and Remote Login Access page.

How could we proceed?

Thanks, Steve

strange i tried again and it went through

Previously I was using Certbot client to update my LE certs, but due to a security issue I decided to give the Virtualmin native LE a go.

Much like everyone else, I was getting the error

DNS-based validation failed : Failed to request certificate :
Undefined subroutine &main::get_bind_zone_for_domain called at /usr/share/webmin/webmin/letsencrypt-dns.pl line 24.
mydomain.com challenge did not pass: No TXT record found at _acme-challenge.mydomain.com

Then I remembered I was actually using AWS for my dns, so the DNS was failing because the Virtualmin DNS module didn't have control. Switched away from AWS, ran the LE and success.

I highly doubt this will ever be fixed since no one had attempted to in months.

Frankly let's encrypt should be removed from Virtualmin and Webmin completely because it's junk code. Bloatware for sure.

I had to stop using using as clients were really pissed off.

Yes, the DNS validate mode won't work unless the DNS zone is hosted locally.

That said, you shouldn't be getting an error about get_bind_zone_for_domain . Annoyingly, this only happens on some systems (and never on mine!) so it's hard to debug.

Uh i don't uderstand not using DNS local bit only extern

This was proofable working till 18-01 the last renew sofar.

Only problem we have is with mail. and some "subdomains" if external dns not allowed autoconfig.domain... and so on

I don't know how it is working now we didn't have renews after that time., but i hope it is still minimal working as before while there i was able to do it manual wen failed the auto...

If now nothing external dns then sh...

So is external dns used working with the Virtualmin 6.x LE cert solution if dns done right ( but not the dns validate mode)? ( so maybe i am understanding dns here wrong..? ) so you are meaning only the dns way of letsencrypt (the DNS validate mode) ? sorry then for misunderstanding

I don't know if this will work for everyone but the following solved the issue for me:

I noticed the protocol in the error message is "http:". However for my site, I force https via .htaccess file for anyone trying to access my site via http. I commented out the section that forces https in the .htaccess file to allow http which the error message is showing for the protocol it's using. Then I tried letscencrypt again and it worked. Set https back in the .htaccess file and accessed the site normally checking to make sure the certificate is the newly created one and it is.

Hopefully this helps some who are experiencing this issue! Good luck!

DonX - Thanks for trying to help but this didn't seem to be my issue. For others to read: I had an .htaccess rule that forced the www prefix. Took that out but it didn't help.

Also I don't know if this helps, but my web request always fails with "challenge did not pass: Invalid response from [domain]" and then a 404 error.

But then I take the exact URL that it failed to find, plug it into my browser, and it's there.

So ... maybe there's a race condition? Failure to specify the hostname or some other http protocol thing?

I wanted to throw in a "me too" as well.

Undefined subroutine &main::get_bind_zone_for_domain called at /usr/share/webmin/webmin/letsencrypt-dns.pl line 24.
XXX.net challenge did not pass: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.XXX.net

I also get 404 errors on web-based validation.

If I comment out RedirectMatch ^/(?!.well-known)(.)$ https://XXX.net/$1 in the Apache conf file for the *:80 site, web-based *usually works. But that's not always an option, there's a few sites I need the DNS-based to work for.

It might also be worth mentioning that I poked around in the letsencrypt-dns.pl file and it makes a lot of references to BIND8 commands, I have BIND9 installed, which is what got installed by the install.sh script for Virtualmin. I don't know if that's the cause of the issues or not.

I'd be happy to provide access to my server for support staff, but I also can't locate the webmin-virtualmin-support package or the referenced section in the control panel that it's in. I'd gladly make someone an admin login though if that would work.

Yeah, this only happens on some systems, and only when falling back to DNS verification.

mindleak - I can login to your system and take a look, if you email me details at jcameron@virtualmin.com (including the domain you're having trouble renewing the cert for)

JamieCameron, I just emailed you login info for it.

I'm afraid I have a sort of "me too". Sort of in that DNS is not enabled for my problem site, but I am getting the error "

pfcc.co.nz challenge did not pass: Invalid response from http://mydomain/.well-known/acme-challenge/npmwKZMLc1NhoS3THDu5QzTQiVyRk... " <html xmlns="htt"

Other domains on the same Virtualmin server seem to have been working OK, and this one did until the recent renewal.

I am running Centos 7 on a Xen hosted VPS. Webmin v 1.872, Virtualmin 6.02 (GPL). All installed packages are up to date.

The http line that is purported to be failing works perfectly when pasted into a browser.

I am happy for Virtualmin support staff to log in and look around - although this is a production server.

My problem solved. The external DNS had residual IPV6 records in it pointing to some random website that was in use before they transferred the thing to my server - which is IPV4 only.

Sorry to have intruded.

Adding another "me too."

.. request failed : Web-based validation failed : Failed to request certificate :

Traceback (most recent call last):
  File "/usr/share/webmin/webmin/acme_tiny.py", line 250, in <module>
    main(sys.argv[1:])
  File "/usr/share/webmin/webmin/acme_tiny.py", line 246, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, args.dns_hook, args.cleanup_hook, log=LOGGER, CA=args.ca)
  File "/usr/share/webmin/webmin/acme_tiny.py", line 155, in get_crt
    resp_data = resp.read().decode('utf8').strip()
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xc2 in position 84345: invalid continuation byte

DNS-based validation failed : Failed to request certificate :

Undefined subroutine &main::get_bind_zone_for_domain called at /usr/share/webmin/webmin/letsencrypt-dns.pl line 24.
website.name challenge did not pass: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.website.name

Edit: Debian Linux 9, Webmin 1.872, Virtualmin 6.02-3

For anyone who's seeing this, please try upgrading to the 1.880 Webmin package from http://www.webmin.com/devel.html , and let us know if it helps.

Just want to make sure I get this right. On Debian, if I do:

wget http://download.webmin.com/devel/deb/webmin_1.880_all.deb
dpkg –install webmin_1.880_all.deb

...will this upgrade the current install or does upgrading require different commands? Thanks.

Did upgrade (for those copying/pasting, apologies for the typo: it should be dpkg --install webmin_1.880_all.deb). The upgrade to 1.880 didn't solve the problem - still getting the following errors:

Requesting a certificate for website.name, www.website.name from Let's Encrypt ..
.. request failed : Web-based validation failed : Failed to request certificate :

Traceback (most recent call last):
  File "/usr/share/webmin/webmin/acme_tiny.py", line 250, in <module>
    main(sys.argv[1:])
  File "/usr/share/webmin/webmin/acme_tiny.py", line 246, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, args.dns_hook, args.cleanup_hook, log=LOGGER, CA=args.ca)
  File "/usr/share/webmin/webmin/acme_tiny.py", line 155, in get_crt
    resp_data = resp.read().decode('utf8').strip()
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xc2 in position 84350: invalid continuation byte

DNS-based validation failed : Failed to request certificate :

website.name challenge did not pass: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.website.name

Edit: Wanted to point out that this appears to only be for one domain - the other domain renewals appear to be working properly. This might be a clue as to what's wrong.

Not sure if this applies but wanted to add what I found when a domain wouldn’t update its cert. I had moved the domain to a new dedicated IP. All DNS had propagated fine (or so I thought), all websites worked. But upon doing a dig@ 8.8.8.8, it returned blank info, even after waiting a week. All other dns servers I dug returned new IP fine. But letsencrypt was unable to find the website. So it was obviously resolution issue somewhere.

Turned out the issue was I had setup DNSSEC keys with the registrar on the old IP. Once I asked the registrar to remove the DNSSEC keys and waited for propagation, googles DNS starting showing the new IP correctly and letsencrpyt completed with no problem. So letsencrypt seems to only be using googles dns and respecting its DNSSEC settings.

Just one more thing to check. Although Highly unlikely issue because you would have to change IP after having set up DNSSEC not only in bind but also with your registrar.

@c_prompt - also, check if that domain is properly setup at your DNS registrar.

For anyone reading: I gave up and started using one of the scripts directly from letsencrypt and it ended up being incredibly easy. After figuring out the very basic command line parameters, you just make a script that can call it and copy the output SSL certs to wherever your virtualmin install expects the cert files (ie the account's folder above public_html in the two special files ssl.cert and ssl.key or somesuch).

After a few tests, I added my two new scripts to weekly cron and now my troubles are completely over. The LetsEncrypt script even has a mode where it'll listen on port 80, allowing you to verify other servers that don't actually host a website (ie: mail server, etc).

To top it off (THANK YOU VIRTUALMIN for this!), the virtualmin module will detect that you've gone and verified manually and self-disable, so you no longer have to worry about it again.

The only negative seems to be that the virtualmin module seems to refuse to ever execute again once you've done the LetsEncrypt way, so keep that in mind (ie you can never go back to the vmin module, at least the way it's programmed right now).

Hope this helps anyone.

@JamieCameron: thanks for the suggestion. Yes, it is properly set up (as a slave just as all other domains are) and the DNS records all look good. So the problem must be elsewhere.

A thousand apologies - I just spent a full day discovering this is likely because of my user's error (which explains why SSL renewal worked fine for other domains).

Great, we're glad to hear it's working now! Feel free to let us know if you have any additional questions.

I had the DNS validation error as well. What I've done is added

require './letsencrypt-lib.pl';

to both following files:

/usr/share/webmin/webmin/letsencrypt-cleanup.pl

/usr/share/webmin/webmin/letsencrypt-dns.pl

Just add it after the first "require" in each file. This sorted the issue for me.

Webmin 1.872

@c_prompt so... what was your user's error? what did you do to resolve it?

@clemsontiger, I'm embarrassed to say my user forgot to renew the user account on my custom-built CMS platform which was, then, redirecting her non-working custom domain to my main site. In other words, the issue wasn't related to Let's Encrypt or Webmin at all and it was my fault for not checking the account first before posting here. Once the account was renewed, the Let's Encrypt renewal worked properly.

(removed the prior post) After reading the earlier posts and make the recommended changes per the link in the acme_tiny.py file. After doing so, the script took several minutes to run but it worked.

Was that supposed to pop-up to be read/agreed? If so, no. I have read that previously but did not see a link or pop-up notice.

I had a DNS problem similar to at least one case mentioned above when I first tried to install LetsEncrypt. I solved my problem with a simple configuration change.

When I created the virtual domain, Webmin/Virtualmin only enabled IPv4 address, which is why I could access my domain with a regular http. By default, IPv6 is not configured in the virtual domains created with Virtualmin.

However, LetsEncrypt demands the IPv6 is configured correctly, so you can either recreate the domain, carefully enabling this option, or seach the virtual domain configuration and enabling it later.

Both solutions worked and afterwards LetsEncrypt was installed very easily. Sorry I cannot access the interface now to show you the precise location of these options. If you cannot find them, please send a message and I explain it later.

I hope this helps someone! Good luck!

Hello.

I'm facing the same issue, I can't generate the certificate to my mail dns record, mail.domain.com

I use external DNS servers, I've disabled the DNS funcionality for the virtual server, I've removed IPV6 completly from the server and still can't generate the certificate.

Operating system Ubuntu Linux 16.04.4 Webmin version 1.890 Usermin version 1.741 Virtualmin version 6.03

This causes failures in sslchecker for the mail entry, since the certificate doesn't contain the mail.domain.com in it.

How can I go over this?

Thanks

I had the same problems and eventually solved the issue by going to Server Configuration -> Change IP Address and for New IPv6 address I checked Shared address with the IPV6 already set in the dropdown.