SSL on one virtual server stopping incoming mail on another

Submitted by crewmark on Mon, 03/21/2016 - 16:12 Pro Licensee

I've search endlessly for a solution to this and have had no feed back on the forum. I have a number of virtual servers, some running SSL, some not. Some with their own IP addresses, some not. All is well. However, I have a virtual server (domain A) that, when I activate the SSL the incoming mail to another (not all) virtual server (domain B) stops being received. There is nothing is the logs showing errors and the server does not appear to receive the emails at all from the outside for domain B. Mail sent from within the server works. If I turn off SSL on Domain A then mail starts again for Domain B. Both domain A and domain B have their own independent IP addresses. Any help would be most appreciated.

Status: 
Active

Comments

Submitted by JamieCameron on Mon, 03/21/2016 - 23:11

If you send email from gmail to one of the domains that stopped receiving, do you get back any bounce message that explains why the message could not be delivered?

Or if this is happening right now, let me know the domain name and I will try some diagnostics to see what's going wrong.

Or you could try the Check Connectivity feature in Virtualmin.

Submitted by crewmark on Tue, 03/22/2016 - 02:44 Pro Licensee

If I send an email in from gmail etc it just disappears into the ether until I turn SSL off on the other server then it wanders in. I'm not getting any bouncebacks.

Checked the connectivity and I get the following error:

Failed to check connectivity HTTP/1.1 504 Gateway Timeout Verify that your DNS server is running, that software.virtualmin.com can be resolved, and that no firewall is blocking outgoing HTTP requests.

But I get this error with all the sites as I use external DNS records. Domain not receiving email is crewmark.com.au I'll leave as is for a few hours but will have to switch it back so messages can be received.

Any help would be great as it's got me.

Submitted by andreychek on Tue, 03/22/2016 - 08:49

Hmm, what is the output of these two commands:

cat /etc/resolv.conf
netstat -an | grep :25

Submitted by crewmark on Tue, 03/22/2016 - 17:41 Pro Licensee

Hmm indeed :)
cat /etc/resolv.conf

nameserver 127.0.0.1
nameserver 70.38.0.4
nameserver 70.38.0.3
search privatedns.com crewmark.net
# Generated by NetworkManager
-------------

netstat -an | grep :25
------------
tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN
tcp 0 0 184.107.201.180:25 212.21.56.255:60712 TIME_WAIT
tcp 1 16 184.107.201.180:25 49.0.204.57:2303 CLOSING
tcp 0 16 184.107.201.180:25 125.123.230.37:1128 FIN_WAIT1
tcp 0 0 184.107.201.180:25 187.94.241.167:36360 TIME_WAIT

Submitted by andreychek on Tue, 03/22/2016 - 23:23

According to the above netstat output, it looks like Postfix is configured to listen to all connections.

When you ran that command, were things working properly? Or was the issue you described occurring at that point?

Also, what domain name is it that isn't able to receive email?

Lastly, what is the output of this command:

host google.com

Submitted by crewmark on Wed, 03/23/2016 - 15:28 Pro Licensee

OK, I was sitting in a cafe when I sent that last info but hadn't checked if the SSL was on or off.
I've just done another test.
I've run these commands on 2 sites:
1. crewmark.com.au - this is the one that's mail stops when the other sites SSL is turned on
2. paperdog.com.au - this is the one that causes the mail issue when it's SSL is turned on.

First cmds were run while paperdog had it's SSL on:
------------------------------------
crewmark.com.au (SSL on PD)

cat /etc/resolv.conf
nameserver 127.0.0.1
nameserver 70.38.0.4
nameserver 70.38.0.3
search privatedns.com crewmark.net
# Generated by NetworkManager

netstat -an | grep :25
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
tcp 0 0 108.163.169.138:25 0.0.0.0:* LISTEN
tcp 0 0 184.107.201.178:25 0.0.0.0:* LISTEN
tcp 0 0 184.107.201.178:25 203.84.134.32:24057 ESTABLISHED
tcp 0 0 108.163.169.138:25 120.27.94.124:4795 ESTABLISHED

host google.com
google.com has address 172.217.2.142
google.com has IPv6 address 2607:f8b0:400b:80b::200e
google.com mail is handled by 40 alt3.aspmx.l.google.com.
google.com mail is handled by 20 alt1.aspmx.l.google.com.
google.com mail is handled by 50 alt4.aspmx.l.google.com.
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 30 alt2.aspmx.l.google.com.

—————————————

paperdog.com.au (SSL on)

cat /etc/resolv.conf
nameserver 127.0.0.1
nameserver 70.38.0.4
nameserver 70.38.0.3
search privatedns.com crewmark.net
# Generated by NetworkManager

netstat -an | grep :25
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
tcp 0 0 108.163.169.138:25 0.0.0.0:* LISTEN
tcp 0 0 184.107.201.178:25 0.0.0.0:* LISTEN
tcp 0 0 108.163.169.138:25 120.27.94.124:2817 ESTABLISHED
tcp 0 0 108.163.169.138:25 120.27.94.124:1798 ESTABLISHED
tcp 1 16 184.107.201.178:25 203.45.8.247:58384 CLOSING
tcp 1 16 184.107.201.178:25 203.45.8.247:58387 CLOSING
tcp 1 16 108.163.169.138:25 203.45.8.247:58383 CLOSING
tcp 1 16 108.163.169.138:25 203.45.8.247:58385 CLOSING

host google.com
google.com has address 172.217.2.142
google.com has IPv6 address 2607:f8b0:400b:80b::200e
google.com mail is handled by 20 alt1.aspmx.l.google.com.
google.com mail is handled by 50 alt4.aspmx.l.google.com.
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 30 alt2.aspmx.l.google.com.
google.com mail is handled by 40 alt3.aspmx.l.google.com.
----------------------------------------------------

The next results are from when the SSL on paperdog was turned off:
---------------------
crewmark.com.au (SSL off PD)

cat /etc/resolv.conf
nameserver 127.0.0.1
nameserver 70.38.0.4
nameserver 70.38.0.3
search privatedns.com crewmark.net
# Generated by NetworkManager

netstat -an | grep :25
tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN
tcp 0 0 184.107.201.178:25 206.222.179.198:54163 TIME_WAIT
tcp 0 1 184.107.201.180:25 219.238.58.248:1540 LAST_ACK
tcp 0 0 184.107.201.180:25 117.240.94.138:54314 TIME_WAIT

host google.com
google.com has address 172.217.2.142
google.com has IPv6 address 2607:f8b0:400b:80b::200e
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 30 alt2.aspmx.l.google.com.
google.com mail is handled by 40 alt3.aspmx.l.google.com.
google.com mail is handled by 20 alt1.aspmx.l.google.com.
google.com mail is handled by 50 alt4.aspmx.l.google.com.

———————————————

paperdog.com.au (SSL off)

cat /etc/resolv.conf
nameserver 127.0.0.1
nameserver 70.38.0.4
nameserver 70.38.0.3
search privatedns.com crewmark.net
# Generated by NetworkManager

netstat -an | grep :25
tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN
tcp 0 0 184.107.201.180:25 188.86.55.8:4618 SYN_RECV
tcp 0 1 108.163.169.138:25 120.27.94.124:1798 FIN_WAIT1
tcp 0 1 184.107.201.180:25 219.238.58.248:1540 LAST_ACK
tcp 0 0 184.107.201.180:25 67.197.92.214:2391 TIME_WAIT
tcp 0 37 184.107.201.180:25 189.90.247.173:53006 ESTABLISHED
tcp 0 0 184.107.201.180:25 177.10.56.104:3171 TIME_WAIT

host google.com
google.com has address 172.217.2.142
google.com has IPv6 address 2607:f8b0:400b:80b::200e
google.com mail is handled by 50 alt4.aspmx.l.google.com.
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 30 alt2.aspmx.l.google.com.
google.com mail is handled by 40 alt3.aspmx.l.google.com.
google.com mail is handled by 20 alt1.aspmx.l.google.com.
------------------------------------------

Hope you can make more from this than I can... Appreciating your help :)

Submitted by JamieCameron on Wed, 03/23/2016 - 22:32

What are the IPs of those two domains?

Submitted by crewmark on Wed, 03/23/2016 - 23:55 Pro Licensee

crewmark 184.107.201.179 paperdog 108.163.169.138

By the way, SSL is off at the moment on paperdog so I can receive emails to crewmark during the work day.

Submitted by JamieCameron on Thu, 03/24/2016 - 22:23

Any chance we could try turning it back on while logged in your system, to see what breaks?

Submitted by crewmark on Thu, 03/24/2016 - 22:25 Pro Licensee

OK, SSL is back on.

Submitted by crewmark on Fri, 03/25/2016 - 20:46 Pro Licensee

Got a bit further with this. When the SSL is on and I do a netstat on port 25 on the domain who's mail isn't working, all looks good. However, if I run a port scan on that domain from my location, port 25 isn't open. If I turn off SSL port 25 opens again. I've got no idea on why this would be happening on just one domain with none of the others being effected.

Submitted by JamieCameron on Sat, 03/26/2016 - 00:45

Does domain B (the one that does't work) have it's own private IP address? If so, when you enable SSL for domain A, does any change get made to the /etc/postfix/master.cf file on your system? You could compare them before and after the change to check.

Submitted by crewmark on Sat, 03/26/2016 - 01:31 Pro Licensee

Yes, both domains have their own private IP address.
I've checked the master.cf file in both scenarios.
Mostly the same except:
Row 1 changes when SSL is on to have an IP address before smtp
---------
184.107.201.178:smtp inet n - n - - smtpd -o smtpd_sasl_auth_enable=yes
--------
When SSL was off there was no 184.107.201.178. This IP address is the base server IP.

At the end of the master.cf file there were a few extra lines where it appears to be referencing domain A's IP
--------
184.107.201.178:submission inet n - n - - smtpd -o smtpd_sasl_auth_enable=yes
108.163.169.138:smtp inet n - n - - smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_cert_file=/home/admin17179
/ssl.cert -o smtpd_tls_key_file=/home/admin17179/ssl.key -o smtpd_tls_CAfile=/home/admin17179/ssl.ca
127.0.0.1:smtp inet n - n - - smtpd -o smtpd_sasl_auth_enable=yes
108.163.169.138:submission inet n - n - - smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_cert_file=/home/ad
min17179/ssl.cert -o smtpd_tls_key_file=/home/admin17179/ssl.key -o smtpd_tls_CAfile=/home/admin17179/ssl.ca
127.0.0.1:submission inet n - n - - smtpd -o smtpd_sasl_auth_enable=yes
--------

Submitted by JamieCameron on Sun, 03/27/2016 - 00:32

Does domain B have SSL enabled on it's website? If not, you should try turning that on as a work-around.

Alternately, if you don't need per-IP SMTP certificates, go to System Settings -> Virtualmin Configuration -> SSL settings, and change "Copy per-IP SSL certificates to Postfix?" to "No".

Submitted by crewmark on Sun, 03/27/2016 - 03:27 Pro Licensee

I've tried SSL on and off on domain B and it still doesn't receive mail. I've just turned "Copy per-IP SSL certificates to Postfix" to No and the mail is now working! Many thanks :)

However, sorry to be dumb, but what does "per-IP SMTP certificates" mean exactly? If I have other SSL certs on other domains with their own IP will they be able to use SSL mail through their own certificates?

Submitted by JamieCameron on Sun, 03/27/2016 - 12:23

What that option is supposed to do is configure Postfix to present the cert for each domain with SSL enabled and a private IP to SMTP connections to that IP - however, this requires that Virtualmin change the Postfix master.cf file to listen on each IP separately. Clearly this isn't working as expected in your case - we will look into it further.

Submitted by crewmark on Sun, 03/27/2016 - 16:40 Pro Licensee

Ah... ok, that makes sense. Thanks for all your help with this. I paid for a premium support ticket not realising I didn't need one. Please absorb that into your account if you can :)

Submitted by fuzzbawl on Wed, 03/15/2017 - 00:04 Pro Licensee

My apologies for highjacking the thread here, but we are seeing this exact same problem. We have most of our domains on dedicated IPs to prevent customers from causing blacklist issues with one another if their wordpress sites get hacked (seems to be a frequent problem). The problem seems to be exactly as described above. If we enable SSL for a domain, the certificate is added to the IP in Postfix, the shared IP on the server is added to the smtp, smtps and submission entries in master.cf. The problem is that the other dedicated IPs on the server that are handling mail for their domains, which don't have SSL certs, do not get added anywhere in master.cf. This causes Postfix to stop responding for mail requests on those IPs.

Any status update on this? We could urgently use a fix. The work-around of preventing SSL certs from being added to Postfix and Dovecot won't work in our environment, we have customers that want SSL certificates for their domains so their mail clients don't complain on mismatch hostname.

Submitted by crewmark on Wed, 03/15/2017 - 00:12 Pro Licensee

Thank god it's not just me :) I'm in the same boat... have customers that need certificates relating to their own domains so their mail can get through.

Submitted by JamieCameron on Wed, 03/15/2017 - 23:12

For anyone who is seeing this, I'd be very interested in logging into one of your systems if possible as the behavior of other dedicated IPs not being able to accept email isn't supposed to happen (and doesn't in tests). Please contact me directly at jcameron@virtualmin.com if remote access is possible.

Submitted by crewmark on Wed, 03/15/2017 - 23:48 Pro Licensee

On the 1st Feb you logged into our server and "patched a bug" but it didn't work. I've changed the login details back if you want to check again.

Submitted by JamieCameron on Thu, 03/16/2017 - 23:08

Can you re-send me the login details and reference this ticket - we get a lot of them, so I don't recall which one was yours.

Submitted by JamieCameron on Sat, 03/18/2017 - 00:24

Ok, I looked into this and realized that there's actually a bug in Virtualmin - it doesn't handle the case where a domain has it's own IP but doesn't have SSL or a website enabled (and thus doesn't have its own cert). In this case, the problem you saw of Postfix not listening on the IP will happen :-(

We can fix this, but it will take some work - stay tuned.

Submitted by JamieCameron on Sun, 03/19/2017 - 15:47

FYI, the work-around is to enable an SSL website on these domains that have an IP but no website. Even though it won't be used for anything, it will get a cert that will be copied to Postfix.

Submitted by crewmark on Sun, 03/19/2017 - 15:59 Pro Licensee

Bit confused by that? An SSL website that has it's own IP but no website? What's the point in that? I want to be able to have it so that if a domain has it's own IP address and I set it up on SSL and get a certificate (whether it be a Let's Encrypt one or any other) then that website will be able to use it's own certificate to verify it's own email etc rather than using another domains certificate. At the moment, all domains want to use the one SSL cert for mail. The domain that all other domains are trying to use for their certificate has it's SSL copied to Postfix etc but I can't "not" copy that off (if that makes sense). I will be moving the site that everything seems to default to this week and putting it on a new server so we'll see if removing it fixes other problems.

Submitted by JamieCameron on Sun, 03/19/2017 - 16:04

I think there may be two different issues being discussed here - what fuzzbawl was describing was domains that have a private IP, but no website or SSL cert not being able to accept email because Virtualmin is configuring Postfix to not listen on that IP.

Submitted by JamieCameron on Sun, 05/07/2017 - 13:18

The issue of a domain with an IP but no SSL cert not being properly setup in Postfix will be fixed in the next Virtualmin release.

Submitted by crewmark on Sun, 05/07/2017 - 16:51 Pro Licensee

One of the domains I setup with SSL set the servers certificate to a single entity, no matter which domain or IP address you used. ie. all domains using ssl are reading only one domains certificate. I ended up moving that particular domain to another server and off this server completely... somehow it's SSL certificate is still being read as the certificate for all other domains using SSL (even when they've got their own certificate. Hopefully this next update will fix the issue.

Submitted by andreychek on Mon, 05/08/2017 - 08:59

Let us know if the next update resolves that. If not, we can help take a closer look into what's going on. For example, one of us can log in if necessary.