Let's Encrypt Error: Gave up waiting for valiation

Submitted by Kintaro on Fri, 06/09/2017 - 05:55

I'm having problem during the renewal of the certificates.

Parsing account key...
Parsing CSR...
Registering account...
Already registered!
Verifying www.mydomain.com...
Traceback (most recent call last):
  File "/usr/share/webmin/webmin/acme_tiny.py", line 235, in <module>
    main(sys.argv[1:])
  File "/usr/share/webmin/webmin/acme_tiny.py", line 231, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, args.dns_hook, args.cleanup_hook, log=LOGGER, CA=args.ca)
  File "/usr/share/webmin/webmin/acme_tiny.py", line 171, in get_crt
    raise ValueError("Gave up waiting for valiation")
ValueError: Gave up waiting for valiation

I saw that some other user got the same issue here: https://www.virtualmin.com/node/52455

Related issues: 
#52235: Let's Encrypt stopped working and can't renew
Status: 
Active

Comments

Submitted by JamieCameron on Fri, 06/09/2017 - 15:45

Has Let's Encrypt worked previously on this server? If so, this could be a transient outage of the Let's Encrypt service.

Submitted by Kintaro on Tue, 06/13/2017 - 05:36

yes Jamie, Let's Encrypt worked previously on this server, the issue is still here. :(

Submitted by JamieCameron on Tue, 06/13/2017 - 17:03

Make sure that there's no outgoing firewall blocking access from your system to Let's Encrypt, and that there's no firewall that could block incoming port 80 traffic from the rest of the internet.

Submitted by Kintaro on Sat, 06/17/2017 - 08:08

The only thing changed in the server is that installed gitlab... but I don't know if the issue was already there or not.

If I try to connect to an http website I have in that server it works... what can I do to check if let's encrypt is reachable? If try to "only update renewal" it works without any problem, but I don't know if it check against let's encrypt servers.

Submitted by Kintaro on Sat, 06/17/2017 - 09:18

in my case I'm already with all the modules updated

I installed certbot and tryed this:

sudo certbot certonly --nginx -d test.com -n --agree-tos --email test@test.com --test-cert

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
tls-sni-01 challenge for test.com
Cleaning up challenges
Cannot find a VirtualHost matching domain test.com.

IMPORTANT NOTES:
- If you lose your account credentials, you can recover through
   e-mails sent to test@test.com.
- Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.

and:

$ nc -zv acme-v01.api.letsencrypt.org 443
DNS fwd/rev mismatch: e981.dscb.akamaiedge.net != a23-206-21-80.deploy.static.akamaitechnologies.com
e981.dscb.akamaiedge.net [23.206.21.80] 443 (https) open
$ nc -zv acme-v01.api.letsencrypt.org 80
DNS fwd/rev mismatch: e981.dscb.akamaiedge.net != a23-206-21-80.deploy.static.akamaitechnologies.com
e981.dscb.akamaiedge.net [23.206.21.80] 80 (http) open

I think the letsencrypt is reachable.

Submitted by JamieCameron on Sat, 06/17/2017 - 13:51

Can the website you're trying to renew be reached from the rest of the internet?

Submitted by Kintaro on Tue, 06/20/2017 - 03:48

I'm seeing that static website can renew and dynamic ones are failing.

My location directives from one of the "not working" virtual host was:

location ~ (^|/)\. {
        return 403;
    }
    location / {
        try_files $uri $uri/ /index.php?q=$uri&$args;
    }
   
location ~ \.php$ {
try_files $uri =404;
fastcgi_pass localhost:8009;
}

I found out the this location directive was the cause of the issue :

location ~ (^|/)\. {
        return 403;
    }

Moving it to the bottom like this solved my problem:

location / {
        try_files $uri $uri/ /index.php?q=$uri&$args;
    }
   
location ~ \.php$ {
try_files $uri =404;
fastcgi_pass localhost:8009;
}

    location ~ (^|/)\. {
        return 403;
    }

Is that added by default to nginx virtualhost configuration? In case yes maybe it can be useful to add a comment saying that it have stay at the bottom? Plus I found this: https://github.com/h5bp/server-configs/issues/126

Submitted by h4ns3n on Tue, 06/20/2017 - 05:38 Pro Licensee

I have the same issue. Let's Encrypt was working perfectly before the recently Webmin and Virtualmin updates.

Parsing account key...
Parsing CSR...
Registering account...
Already registered!
Verifying www.virtualbuzz.net...
Traceback (most recent call last):
  File "/usr/libexec/webmin/webmin/acme_tiny.py", line 235, in <module>
    main(sys.argv[1:])
  File "/usr/libexec/webmin/webmin/acme_tiny.py", line 231, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, args.dns_hook, args.cleanup_hook, log=LOGGER, CA=args.ca)
  File "/usr/libexec/webmin/webmin/acme_tiny.py", line 171, in get_crt
    raise ValueError("Gave up waiting for valiation")
ValueError: Gave up waiting for valiation

Submitted by skelgaard on Sun, 06/25/2017 - 11:32 Pro Licensee

this problem also happens on apache

Traceback (most recent call last):
  File "/usr/share/webmin/webmin/acme_tiny.py", line 235, in <module>
    main(sys.argv[1:])
  File "/usr/share/webmin/webmin/acme_tiny.py", line 231, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, args.dns_hook, args.cleanup_hook, log=LOGGER, CA=args.ca)
  File "/usr/share/webmin/webmin/acme_tiny.py", line 171, in get_crt
    raise ValueError("Gave up waiting for valiation")
ValueError: Gave up waiting for valiation

Submitted by skelgaard on Fri, 06/30/2017 - 03:52 Pro Licensee

can we please get this fixed or get the intergration changed to work directly with certbot instead, as that works fine... i have to manually start renewing domains, as they are beginning to expire cause of this error.

Submitted by Kintaro on Fri, 06/30/2017 - 10:16

@skelgaard have you double checked you apache virtualhosts configurations? in my case was the "403" location directive to cause the problem.

Submitted by skelgaard on Sat, 07/01/2017 - 02:47 Pro Licensee

@Kintato you where partly correct... i was looking for errors in the vhost file, and didn't find any... but i though lets try something, cause it looked none standart. so i ran disable-feature --web --ssl --logrotate --domain delete the vhost file from /etc/apache2/site-avaliable and then ran enable-feature --web --ssl --logrotate --domain and then went to the interface and pressed renew.... that worked... i gotta test it on the other domains that also gave me the error.

Submitted by mike0810 on Sat, 07/01/2017 - 03:03

I have the same problem! Worked before, after upgrade still fails.

Submitted by mike0810 on Thu, 07/06/2017 - 07:02

I get still the error "ValueError: Gave up waiting for valiation" on only one of my vhosts despite curl is working - curl -i http://mydomain.com/.well-known/acme-challenge/test and also browser access to http://mydomain.com/.well-known/acme-challenge/test and to the challenge file is also working. This is weird.

Apache Accesslog points to the wrong challenge file which does not exist: 10.200.1.1 - - [06/Jul/2017:13:57:29 +0200] "GET /.well-known/acme-challenge/NbL8p17oJo1mT6UJ6aQYb-4W5-dFdq8UzytMXXezUOI HTTP/1.1" 200 87 "-" "Python-urllib/2.7" 10.200.1.1 - - [06/Jul/2017:13:57:30 +0200] "GET /.well-known/acme-challenge/6LYmmK9jiQOTmJ2aasd1FTO1uzDLfb5ixZfuabU7bkI HTTP/1.1" 200 87 "-" "Python-urllib/2.7"

On every request wrong files get created and will not delete afterwards. They pile up.

Thanks for help!

Submitted by JamieCameron on Thu, 07/06/2017 - 20:43

Do you see new entries in the Apache log for this domain each time you attempt a Let's Encrypt cert request?

If not, double-check your DNS and registrar settings - the issue may be that your domain can't be resolved.

Submitted by mike0810 on Fri, 07/07/2017 - 04:46

Yes I get the entries in the access log. These are new ones from today from the auto renewal: 10.200.1.1 - - [07/Jul/2017:11:37:04 +0200] "GET /.well-known/acme-challenge/NbL8p17oJo1mT6UJ6aQYb-4W5-dFdq8UzytMXXezUOI HTTP/1.1" 200 87 "-" "Python-urllib/2.7" 10.200.1.1 - - [07/Jul/2017:11:37:06 +0200] "GET /.well-known/acme-challenge/6LYmmK9jiQOTmJ2aasd1FTO1uzDLfb5ixZfuabU7bkI HTTP/1.1" 200 87 "-" "Python-urllib/2.7"

However, the challenge file does not exist but there are 108 files now in the acme-challenge folder piling up since my last post. As I said, i can access them with curl and with the browser over http. Also the site is working (still with the old certs, but they are going to be invalid in some time)

curl -i http://mydomain.at/.well-known/acme-challenge/pFwTPanpIQR40NAzcckAe6Ug5l... HTTP/1.1 200 OK Date: Fri, 07 Jul 2017 09:44:40 GMT Server: Apache/2.4.6 Last-Modified: Fri, 07 Jul 2017 09:32:11 GMT ETag: "57-553b6e85e5aa6" Accept-Ranges: bytes Content-Length: 87

Submitted by JamieCameron on Sat, 07/08/2017 - 12:37

Normally those challenge files are created only for a very short time and then deleted once the Let's Encrypt service has fetched them. The old ones are useless, and can be safely removed.

You might want to check if the file referenced in the logs ever comes into existence (even for a short time) during the cert request process, by continually running ls on that directory.

Submitted by Gerritjan on Wed, 07/12/2017 - 06:38

Also can't issue new certificates after the 5.99 update, while it always used to work

Error:
Parsing account key...
Parsing CSR...
Registering account...
Already registered!
Verifying www.mydomain.nl...
Traceback (most recent call last):
  File "/usr/share/webmin/webmin/acme_tiny.py", line 235, in <module>
    main(sys.argv[1:])
  File "/usr/share/webmin/webmin/acme_tiny.py", line 231, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, args.dns_hook, args.cleanup_hook, log=LOGGER, CA=args.ca)
  File "/usr/share/webmin/webmin/acme_tiny.py", line 171, in get_crt
    raise ValueError("Gave up waiting for valiation")
ValueError: Gave up waiting for valiation

And the apache access log:

185.55.555.555 - - [12/Jul/2017:13:49:33 +0300] "GET /.well-known/acme-challenge/IggMbvRb2LpyEJkZGf4Btzq2jPtIXDe8WiclzfDeiPI HTTP/1.1" 200 439 "-" "Python-urllib/2.7"
185.55.555.555 - - [12/Jul/2017:13:56:35 +0300] "GET /.well-known/acme-challenge/lrKLl60uZmhQWgiOrSJH6wzkVmCoHKWke-zyJoh8v7I HTTP/1.1" 404 527 "-" "Python-urllib/2.7"
66.133.109.36 - - [12/Jul/2017:13:56:36 +0300] "GET /.well-known/acme-challenge/lrKLl60uZmhQWgiOrSJH6wzkVmCoHKWke-zyJoh8v7I HTTP/1.1" 404 527 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
185.55.555.555 - - [12/Jul/2017:13:58:42 +0300] "GET /.well-known/acme-challenge/IggMbvRb2LpyEJkZGf4Btzq2jPtIXDe8WiclzfDeiPI HTTP/1.1" 200 439 "-" "Python-urllib/2.7"
185.55.555.555 - - [12/Jul/2017:14:29:12 +0300] "GET /.well-known/acme-challenge/IggMbvRb2LpyEJkZGf4Btzq2jPtIXDe8WiclzfDeiPI HTTP/1.1" 200 439 "-" "Python-urllib/2.7"

Submitted by Gerritjan on Wed, 07/12/2017 - 11:38

Wanted to add that I CAN issue renewals for existing domains. I just can't seem to issue Certificates for new domains!

Submitted by Gerritjan on Wed, 07/12/2017 - 14:34

Jamie, Thanks for replying.

I had just found the same solution for the problem and was coming back here to report on this, you beat me to it. The same problem was reported in the acme_tiny github: https://github.com/diafygi/acme-tiny/issues/167

I have edited acme_tiny.py and everything is working smoothly again. Basically the webmin update wasn't the problem, me updating openssl to version 1.1 caused an incompatibility with the acme_tiny script.

I hope this solution fixes it for everyone!

Submitted by andreychek on Thu, 07/13/2017 - 11:43

You can use the "find" command to find files. In this case, you can use "find /usr -name acme_tiny.py" -- and that should produce something like this:

# find /usr -name acme_tiny.py
/usr/share/webmin/webmin/acme_tiny.py

The next Webmin version should correct this issue.

Submitted by JamieCameron on Fri, 07/14/2017 - 00:26

We are likely going to do a new Webmin release shortly to address this.

Submitted by KAtwitch on Mon, 08/07/2017 - 16:40

I've applied the acme_tiny.py code change and I'm still seeing the timeouts. Any suggestions?

Submitted by JamieCameron on Mon, 08/07/2017 - 17:47

KAtwitch - What's the specific domain (and hostnames) that you are trying to request a cert for?

Submitted by KAtwitch on Mon, 08/07/2017 - 18:05

domain is keith.photography hostname is zoe.thealangroup.net

Submitted by JamieCameron on Mon, 08/07/2017 - 23:49

That's an interesting domain name - it's possible that .photography isn't one of the TLDs that Let's Encrypt recogizes, since it's relatively new?

Submitted by KAtwitch on Wed, 08/09/2017 - 14:21

After reading JamieCameron's suggestion about the .photography domain possibly being a problem, I decided to try using letsencrypt with one of the other virtual servers. Unfortunately, that leads to a different (DNS) error. I've read that disabling https redirects works for most people. I did that days ago, but it doesn't seem to matter. The new error is: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.www.computertoystore.com

Submitted by JamieCameron on Thu, 08/10/2017 - 14:16

Is computertoystore.com a domain hosted on your Virtualmin system? When I check the NS records, it looks like the DNS hosting is by Google.

fakemoth's picture
Submitted by fakemoth on Fri, 08/11/2017 - 04:17

A month later I still have the exact same problem, opened a private ticket with the team. They have access to my live systems to investigate. Will post back if something useful emerges.

Cheers!

Submitted by KAtwitch on Fri, 08/11/2017 - 11:30

JamieCameron, I'm using Google to manage DNS (their DNS servers). The computertoystore.com domain is a virtual domain hosted on my virtualmin server. I prefer to use Google as the DNS servers. If that's not possible or I'm doing something wrong, I'd be willing to look at a change. I'm by no means a DNS expert, but I believe I have the records configured to correctly point to the virtualmin box.

Submitted by JamieCameron on Sun, 08/13/2017 - 11:46

In that case, I'd recommend turning off the DNS feature for this domain in Virtualmin - otherwise, Virtualmin will incorrectly think that the domain is local.

Submitted by KAtwitch on Mon, 08/14/2017 - 15:47

I have disabled DNS for the virtual server as recommended by unchecking "DNS Enabled" in the manage virtual server area. The lets encrypt is still failing. It's throwing an 'invalid response' error.

Following is the most recent error: ValueError: www.computertoystore.com challenge did not pass: {u'status': u'invalid', u'validationRecord': [{u'addressesResolved': [u'173.10.220.211'], u'url': u'http://www.computertoystore.com/.well-known/acme-challenge/0ogzWT1xDulOH...', u'hostname': u'www.computertoystore.com', u'addressesTried': [], u'addressUsed': u'173.10.220.211', u'port': u'80'}], u'keyAuthorization': u'0ogzWT1xDulOHcIKUIS74H6PNyxzB8LJM55GIpnvypc.usH11WrNVmfwD91d4EIdUoWBoYtxZ_10Pmb9f3sicwQ', u'uri': u'https://acme-v01.api.letsencrypt.org/acme/challenge/xlsYkE966sPP_5mBxLas...', u'token': u'0ogzWT1xDulOHcIKUIS74H6PNyxzB8LJM55GIpnvypc', u'error': {u'status': 403, u'type': u'urn:acme:error:unauthorized', u'detail': u'Invalid response from http://www.computertoystore.com/.well-known/acme-challenge/0ogzWT1xDulOH... "\n<ht"'}, u'type': u'http-01'}

Submitted by KAtwitch on Wed, 08/16/2017 - 13:06

I don't have any redirects or edits to an .htaccess file outside of the virtualmin GUI. What's weird is that I was able to get the letsencrypt to work by disabling the site in virtualmin and then re-enabling it. What's even more weird is that when I enable DNS on the site (something we covered above) the letsencrypt validation fails UNLESS I remove the autoconfig.computertoystore.com entry. If I remove the autoconfig line, it works. Correct me if I'm wrong here, but I seem to remember reading that if you are running a mail server on the virtual domain, Virtualmin says you must enable DNS.