Let's Encrypt renew retry frequency and ratelimit

Submitted by george.asenov on Wed, 07/13/2016 - 03:56

Hello,

We have hit "Too many currently pending authorizations" rate limit on Let's Encrypt so we start to investigate deep.

And we found: 1. Virtualmin try to renew certificates for suspended accounts/domains - while they obviously will fail 2. The renew task is run too frequently and forever. First issue here is that every time it try to renew and fail it send email to the client. Also hits let's encrypt servers rate limit. I think it try every 5 minutes. For five minutes what will change? I think it should be less frequently like every hour for example. And to try for example 5 times and if all of them fail to stop trying till manually restarted.

Status: 
Closed (fixed)

Comments

Submitted by JamieCameron on Wed, 07/13/2016 - 04:09

Good suggestion - the next release of Virtualmin will not attempt to issue certs for disabled domains.

Also, I have already implemented a rate limit that will be included in the next release.

Submitted by JamieCameron on Wed, 07/13/2016 - 04:11

Status: Active ยป Fixed

Submitted by george.asenov on Wed, 07/13/2016 - 04:31

One more thing. Many clients are using strange and buggy rewrite rules. It will be very good virtualmin to create and .htaccess file with "RewriteEngine off" in webroot/.well-known directory. This will prevent the case when users rewrite rules rewrite the request to the verification file.

Or add a global location directive which will override all domains location /.well-known to one global place where virtualmin will place the authorization file. This will prevent users form braking the renewal process.

Submitted by JamieCameron on Wed, 07/13/2016 - 04:59

Yeah, the fact that there can be redirects and rewrites that interfere with access to /.well-known is a problem.

I don't yet know of a good way to disable all of these ... however, your suggestion is a good start.

Submitted by george.asenov on Thu, 07/14/2016 - 01:22

another bug we found. It happen on couple of domains already. The auto-renew task start and do it's job and successfully renew the certificate but "forget" to change the last renew date in domains configuration file. This lead to constant retry to renew and then hit the rate limit and start send emails to the customer that it can't renew. But the cert is renewed successfully from the first time just the date isn't changed.

Submitted by JamieCameron on Thu, 07/14/2016 - 02:52

I'm pretty sure that is fixed, as I have several domains with automatic renewal of Let's Encrypt certs and haven't seen this error.

Submitted by george.asenov on Thu, 07/14/2016 - 06:16

We also have maybe more than 1000 domains using Let's Encrypt on different servers and this happen only on 3-4 domains already. We are unable to find the reason but it happen.

Submitted by JamieCameron on Thu, 07/14/2016 - 09:06

Are you sure that the domains for which the renewal is repeating aren't failing to renew?

Submitted by george.asenov on Fri, 07/15/2016 - 01:02

Yes I'm sure we have check trough virtualmin SSL certificates menu and with online SSL checker website and with the browser so we ware sure before manually edit the last renew date in domain configuration file.