=========================================================================================================
Scenario 1: A Virtualmin sub-server created to run a NextCloud server operation. The site functions properly and the SSL is still valid.
Virtualmin => Cloud Sub-Server => Manage SSL Certificate => Let's Encrypt =>
Last successful renewal: 02/23/2018 4:22 PM
Last failed renewal: 05/23/2018 10:17 PM
Renewal failed due to: Web-based validation failed : Failed to request certificate :
cloud.MySiteDN.com challenge did not pass: Invalid response from http://cloud.MySiteDN.com/.well-known/acme-challenge/5enH7qzzMxApz37pkOSwH14oub895o_mYTgKWW0AtLg: "<!DOCTYPE html>
Then when I go ahead and click the [ Request Certificate ] button, it produces:
Request Certificate
In domain cloud.MySiteDN.com
Requesting a certificate for cloud.MySiteDN.com from Let's Encrypt ..
.. request failed : Web-based validation failed : Failed to request certificate :
cloud.MySiteDN.com challenge did not pass: Invalid response from http://cloud.MySiteDN.com/.well-known/acme-challenge/eZhkBlESbJQCTs-gzqoQWx86i_X6hoom0hVpy1DUGBs: "<!DOCTYPE html>
<html class="ng-csp" data-placeholder-focus="false" lang="en" >
<head data-requesttoken="YQ/jmudHXlRWD7lX00V+3z"
DNS-based validation failed : Failed to request certificate :
cloud.MySiteDN.com challenge did not pass: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.cloud.MySiteDN.com
Opening caveat here; since I'm a lousy sys-admin I guessed that after already failing to renew and failure to request cert (earlier today to my try limit) that somehow my folders for /.well-known and so on were tainted so I deleted them and tried to request new, which didn't work, so I made the folders again including the .htaccess file.
Biggest idiosyncrasy is NextCloud complains about security issues with the .well-known folder in its web space. They suggest moving it elsewhere. Mine is now in a /var/www/SubFolder. My Apache .conf files have this:
Alias /.well-known/acme-challenge/ /var/www/SubFolder/.well-known/acme-challenge/.
Both SubFolder and its subfolders are owned by the named user for the virtual sub-server and the group is www-admin. The site is functioning fine right now, but its worth noting that a power outage 2 weeks ago made my DHCP IP number change. It's my second time at a DHCP change so I got through that.
=========================================================================================================
Scenario 2: A Virtualmin sub-server created to host an ordinary website. The site won't load with details of why below. The site works great when the Virtualmin, Services, Preview Website is selected.
Virtualmin > Cloud Sub-Server > Manage SSL Certificate > Let's Encrypt >
Last successful renewal: 02/20/2018 12:58 AM
Last failed renewal: 05/23/2018 10:37 PM
Renewal failed due to: Web-based validation failed : Failed to request certificate :
TheSiteDN.com challenge did not pass: No valid IP addresses found for TheSiteDN.com
This one is more of a cautionary than it is a request for help. Google fucked this one up. I have 3 blogs on Blogger.com and I also have my domains managed at domains.google.com as of now (hint: accepting suggestions to switch my domains to). I used the Blogger tools to connect them to a subdomain of TheSiteDN.com a long time ago, but Google changed something and they had some error messages. I clicked their link to correct the problem. A message said it fixed things, but the link to fix things remained. Eventually, I disconnected them and used the regular Synthetic records tools to make a subdomain redirect to each blog. What I didn't realize happened before this was Google caused one of the blogs to be the redirect for my www.TheSiteDN.com and TheSiteDN.com. So when Let's Encrypt tries to resolve, Google says, "maybe you should try this blog over here?" Its been 3 days now and I'm still getting redirected wrong and they haven't answered my request for service yet. I assume time will resolve this Scenario.
If this URL was important, I'd be sooooooooo screwed!
=========================================================================================================
Quick update on Scenario 1
I see Webmin has been emailing me the failure notifications, here's content from the message closest to the time of writing this:
An error occurred requesting a new certificate for cloud.MySiteDN.com from Let'sEncrypt : Web-based validation failed : Failed to request certificate : <pre>cloud.MySiteDN.comchallenge did not pass: Invalid response from http://cloud.MySiteDN.com/.well-known/acme-challenge/-gW94FaTKRiFMNSD4lOOtZ089ssmatNC9o3mPIgFirk:"<!DOCTYPE html><html class="ng-csp" data-placeholder-focus="false" lang="en" ><head data-requesttoken="SnFtaQQMHD1NUDpBddhuvE"</pre>DNS-based validation failed : Failed to request certificate : <pre>cloud.MySiteDN.comchallenge did not pass: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.cloud.MySiteDN.com</pre>As I flip through the messages (about 1:05 apart), I see the acme challenge string and the request token changing normally but the renewal operation fails each time. The first failure happened
19/05/2018 16:27(long after the DHCP IP change was resolved) and the message was:An error occurred requesting a new certificate for cloud.MySiteDN.com from Let'sEncrypt : Web-based validation failed : Failed to request certificate : <pre>cloud.MySiteDN.comchallenge did not pass: Invalid response from http://cloud.MySiteDN.com/.well-known/acme-challenge/PNtgRoBwXEENhyT-CU_Lyxkp5taTXcl8DRIFORfhZP8:"<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"><html><head><title>404 Not Found</title></head><body><h1>Not Found</h1><p"</pre>DNS-based validation failed : Failed to request certificate : <pre>cloud.MySiteDN.comchallenge did not pass: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.cloud.MySiteDN.com</pre>Furthermore, my
MySiteDN.comandwww.MySiteDN.comrenewed automatically on the 15th of this month so it's definitely a quirk with the NextCloud sub-server I made and not Virtualmin at all. I think it was on the 10th or the 11th of this month that I had all the DHCP problems mitigated.-- I'm remarkably average
Another Quick update on Scenario 1
I've passed the deadline for my certificate. @noisemarine or @Joe have any ideas? Should I delete the ssl.* files in that user directory and try to get a new one?
-- I'm remarkably average
Does this name exist in your DNS? Can you access the .well-known directory via a web browser? That is what the error is saying - that the DNS lookup for that domain is failing. Do you even need that subdomain? Can you remove it from your certificate request if not? Sorry, I don't use Cloudmin, so have little knowledge of its inner workings.
DNS problem: NXDOMAIN looking up TXT for _acme-challenge.cloud.MySiteDN.com@noisemarine I used the Virtualmin interface to create a sub-server of
MySiteDN.comand I named that sub-servercloud. The BIND DNS Server has the appropriate entries forcloud.MySiteDN.comand the address record points to the current IP of my router. Also,domains.google.comhas it listed in the Registered Hosts with the right IP address. Also,MySiteDN.comis functioning properly including cert renewal.I can't negotiate an attempt to access the
/.well-knowndirectory or its subdirectories because I can't find out where all the force https rewrite/redirect's are coming from and the cert is expired so browsers won't let me open the site at all. I just tried commenting out myAlias /.well-known/acme-challenge/ /var/www/SubFolder/.well-known/acme-challenge/statement in both Apache conf files, then recreating the natural folders for
/.well-known/acme-challenge/with the right permissions and ownership and then requesting a new cert from Let's Encrypt but that failed:Request Certificate
In domain cloud.MySiteDN.com
Requesting a certificate for cloud.MySiteDN.com from Let's Encrypt ..
.. request failed : Web-based validation failed : Failed to request certificate :
cloud.MySiteDN.com challenge did not pass: Invalid response from http://cloud.MySiteDN.com/.well-known/acme-challenge/B5zZZywzjybbaQ-aj5B8ih3c06PIcwn8O4eGxGY21IY: "<!DOCTYPE html><html class="ng-csp" data-placeholder-focus="false" lang="en" ><head data-requesttoken="l9rOej3jbdJM9Vg07zy1Cc"DNS-based validation failed : Failed to request certificate :
cloud.MySiteDN.com challenge did not pass: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.cloud.MySiteDN.com======================================================================================
To clear up this miscommunication, "Sorry, I don't use Cloudmin, so have little knowledge of its inner workings.", I'm not using Cloudmin either. I manually installed a NextCloud server program in the space prepared by Virtualmin by the create sub-server operation.
Just wondering, assume I have a new blank virtual server prepared and I have a https redirect in the port 80's Apache conf file, will that make Let's Encrypt fail? Like, does Let's Encrypt need a path to the http version to work? Also, why would a new request fail? I can understand a renewal failing, but not a new request, although I don't know what script runs after clicking the Virtualmin buttons for Let's Encrypt in Server Configuration - Manage SSL Certificate.
======================================================================================
Also, is there a way for me to use the Virtualmin interface for Let's Encrypt in Server Configuration - Manage SSL Certificate under the main virtual-server of
MySiteDN.comand get that cert set to cover the sub-server(s)?-- I'm remarkably average
I just reread your op. Don't use DNS-01 challenge. I've not seen anything to say that Virtualmin supports it. That's where the _acme-challenge name is coming from, and why LE can't find it.
Do you have http->https redirection active? LE needs to connect to the http:// version of your site. There are some LE compatible redirect methods that have been posted before.
I'm sad to report @noisemarine that I'm not knowledgeable enough to understand what you are suggesting in your first paragraph. I don't know what a DNS-01 challenge is or how I asked for it; maybe my NextCloud code did that? I googled around a bit and found these two posts on a thread that may solve my problem, but they are also cryptic enough to me that I can't fully decide how to proceed:
I think I understand that I would need to make a fake subdomain of an easy to talk to FQDN on the same server where some sort of a proxy challenge would suffice to Let's Encrypt. It seems like I would have to manually modify the BIND DNS Server on my machine as well as the new CNAME record at
domains.google.com. I feel very confused by this.I saw that the Let's Encrypt system will start allowing wildcard certificates. Can I get a wildcard cert for my FQDN using the Virtualmin interface?
About redirects, yes. most of the virtual servers and sub-servers I made force https in one way or another. My personal goto is this line in the Apache conf file for the port 80 handler:
Redirect permanent / https://[subdomain-name.]DOMAIN-NAME.com/and that was what I thought was redirecting my NextCloud system to https, but their's something else doing that too.Different systems I've installed on my Virtualmin managed server handle their own redirects. I can't tell you I know what they all are. My Moodle server has no Apache redirection but forces https for all connections, yet had successfully renewed automatically just this past Saturday via the original Virtualmin creation of it's cert. Moodle seems to only have the config.php using
$CFG->wwwroot = 'https://school.wnymathguy.com';to redirect.I'm looking all around my NextCloud files for possible redirections but I don't know the languages enough to really be useful to you without guidance.
In the .htaccess file I see:
<IfModule mod_rewrite.c>RewriteEngine on⋮RewriteCond %{REQUEST_URI} !^/.well-known/(acme-challenge|pki-validation)/.*⋮</IfModule>and the config/config.php file has
'overwrite.cli.url' => 'https://cloud.ruppssites.com',I commented out that last line in the config.php and the one in my Apache conf file then in a incognito window am able to go to the http version now. Also, I see since last night when I created the new
/.well-known/acme-challenge/folders, the attempts at autorenew have been populating theacme-challengefolder with the normal root owned 777 files with hashey filenames and contents about every 1 hour and 5 minutes.-- I'm remarkably average
Just checked,
Renewal failed due to Web-based validation failed : Failed to request certificate :
cloud.ruppssites.com challenge did not pass: Invalid response from http://cloud.ruppssites.com/.well-known/acme-challenge/RIbm8rGKcTi2tnw0YnrnThwhiSqXXzPSdX4kB7lj3iM: "<!DOCTYPE html>BUT the
/.well-known/acme-challenge/folder has a file namedRIbm8rGKcTi2tnw0YnrnThwhiSqXXzPSdX4kB7lj3iMowned by root and 777.-- I'm remarkably average
I have good news about my Scenario 1. The auto-renew of Virtualmin's system finally went through. Here's the state I was in at the time.
The NextCloud's config/config.php file has
'overwrite.cli.url' => 'https://cloud.MySiteDN.com',active, but it's not redirecting to https.In my Apache .conf files for port 80 and 443 have these commented out:.
#Redirect permanent / https://cloud.MySiteDN.com/#RewriteCond %{REQUEST_URI} /\.well\-known/acme\-challenge/#RewriteRule (.*) /.well-known/acme-challenge/$1 [L,QSA]In my Apache .conf files for 443 have these commented out:.
#<IfModule mod_headers.c>#Header always set Strict-Transport-Security "max-age=15552000; preload; includeSubDomains"#</IfModule>And I'm using the natural
.well-knownfolder that Virtualmin makes instead of my auxiliary folder invar/www.It appears that unbeknownst to me they fixed the NextCloud security warning problem of having the .well-known folder inside the same space as the cloud server. I'll be glad to come back here in 3 moonths if something new goes wrong.
My Scenario 2 is still in limbo cause Google sucks at their job.
-- I'm remarkably average
One new little groaning lulz in my Scenario 1. A related virtual sub-server named
officeis there to allow the LibreOffice file editor of Collabora to edit the documents in the cloud with a browser interface. Its SSL Certificate is about to expire and isn't auto-updating. I just did some preliminary changes to see if I could get it to fly right this time.-- I'm remarkably average
So the
officesub-server had a bunch of it's Apache .conf file directives commented out. I don't remember why I did that. When I uncommented them all, the SSL Cert renewed. However, the god-damn files don't load anymore. Time to start an issue on the Collabora GitHub pages.-- I'm remarkably average
In case I ever come back here to fix the same problems, because due to DHCP changing the IP I lose the docker connectivity. I have to run the following commands as the owner of the domain listed in Virtualmin:
cd domains/cloud.MySiteDN.com/public_html/nextcloud/apps/docker ps -adocker stop CONTAINER_IDfor collabora/codedocker rm CONTAINER_IDfor collabora/codedocker pull collabora/codedocker run -t -d -p 127.0.0.1:9980:9980 -e "domain=cloud\\.MySiteDN\\.com" --restart always --cap-add MKNOD collabora/codeand if the OCR software is installed
cd domains/cloud.MySiteDN.com/public_html/nextcloud/apps/ocr/redis/docker ps -adocker stop CONTAINER_IDfor DOMAIN_OWNER/ocrdocker rm CONTAINER_IDfor DOMAIN_OWNER/ocrdocker stop CONTAINER_IDfor DOMAIN_OWNER/redisdocker rm CONTAINER_IDfor DOMAIN_OWNER/redisdocker run --name redis-ocr --network=isolated_ocr -p 6380:6379 --restart always -d support/redisdocker run --name ocr --network=isolated_ocr --restart always -e "NODE_ENV=production" -e "REDIS_HOST=redis" -e "REDIS_PORT=6380" -e "REDIS_DB=0" -v /domains/cloud.MySiteDN.com/public_html/nextcloud/data:/home/node/data:ro -v /domains/cloud.MySiteDN.com/tmp:/home/node/output -d support/ocrdocker ps -aThen give the old beast a reboot if things don't work right away.
-- I'm remarkably average