So looks like my private email server hasn't been receiving emails since Jul 11th.
Around this time i moved the server from one location to another and updated a bunch of IPs. May be related but i'm not sure. DNS config looks good, MX records show up properly from outside. SSL certificates are in place. Reverse address is good. Not sure what else to check.
Here's what i can do, and what i can't at the moment:
Note: Server is setup to automatically forward any message received to my gmail account.
Let's assume i have me@domain1.com as my email with alias a@domain1.com to make it shorter. I have also a 2nd domain, let's assume admin@domain2.com is the email there. I can't receive emails here either by the way, so this is common between all domains on the servers.
Anyone got any goddamn idea what the hell is going on? I'm so confused i don't know where to start. Checked mail logs, nothing unusual.
Here's the log doing step #2 above after a restart (note - changed real info to the supposed data shown above for privacy reasons)
Oct 19 03:26:36 server1 postfix/postfix-script[19478]: stopping the Postfix mail system
Oct 19 03:26:36 server1 postfix/master[9620]: terminating on signal 15
Oct 19 03:26:36 server1 postfix/postfix-script[19488]: fatal: the Postfix mail system is not running
Oct 19 03:26:36 server1 postfix/postqueue[19561]: warning: Mail system is down -- accessing queue directly
Oct 19 03:26:54 server1 postfix/postfix-script[19878]: starting the Postfix mail system
Oct 19 03:26:54 server1 postfix/master[19880]: daemon started -- version 2.10.1, configuration /etc/postfix
Oct 19 03:27:11 server1 dovecot: pop3-login: Login: user=<me@domain1.com>, method=PLAIN, rip=my.ip.address, lip=my.server.ip, mpid=20206, session=<3u/4YD6VIoNJLEun>
Oct 19 03:27:11 server1 dovecot: pop3(me@domain1.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/7368, size=654099943
Oct 19 03:33:01 server1 dovecot: pop3-login: Login: user=<me@domain1.com>, method=PLAIN, rip=my.ip.address, lip=my.server.ip, mpid=22145, session=<iaXadT6VeINJLEun>
Oct 19 03:33:01 server1 dovecot: pop3(me@domain1.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/7368, size=654099943
Oct 19 03:36:06 server1 dovecot: pop3-login: Login: user=<me@domain1.com>, method=PLAIN, rip=my.ip.address, lip=my.server.ip, mpid=24308, session=<T1PcgD6VBIRJLEun>
Oct 19 03:36:06 server1 dovecot: pop3(me@domain1.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/7368, size=654099943
Oct 19 03:36:08 server1 dovecot: pop3-login: Login: user=<me@domain1.com>, method=PLAIN, rip=my.ip.address, lip=my.server.ip, mpid=24355, session=<euMCgT6VCIRJLEun>
Oct 19 03:36:08 server1 dovecot: pop3(me@domain1.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/7368, size=654099943
Oct 19 03:37:28 server1 postfix/cleanup[25316]: warning: database /etc/postfix/generic.db is older than source file /etc/postfix/generic
Oct 19 03:37:28 server1 postfix/pickup[19881]: 5A6CE811290E: uid=0 from=<me@domain1.com>
Oct 19 03:37:28 server1 postfix/cleanup[25316]: 5A6CE811290E: message-id=<1571470648.25308@domain1.com>
Oct 19 03:37:28 server1 postfix/qmgr[19882]: 5A6CE811290E: from=<me@domain1.com>, size=559, nrcpt=1 (queue active)
Oct 19 03:37:28 server1 postfix/trivial-rewrite[25317]: warning: do not list domain server1.mymainserver.com in BOTH mydestination and virtual_alias_domains
Oct 19 03:37:28 server1 postfix/trivial-rewrite[25317]: warning: do not list domain server1.mymainserver.com in BOTH mydestination and virtual_alias_domains
Oct 19 03:37:31 server1 postfix/local[25318]: 5A6CE811290E: to=<me-domain1.com@server1.mymainserver.com>, orig_to=<a@domain1.com>, relay=local, delay=3.2, delays=0.05/0.01/0/3.1, dsn=2.0.0, status=sent (delivered to command: /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME)
Oct 19 03:37:31 server1 postfix/cleanup[25316]: 60B898124D19: message-id=<1571470648.25308@domain1.com>
Oct 19 03:37:31 server1 postfix/local[25318]: 5A6CE811290E: to=<me-domain1.com@server1.mymainserver.com>, orig_to=<a@domain1.com>, relay=local, delay=3.2, delays=0.05/0.01/0/3.1, dsn=2.0.0, status=sent (forwarded as 60B898124D19)
Oct 19 03:37:31 server1 postfix/qmgr[19882]: 60B898124D19: from=<me@domain1.com>, size=706, nrcpt=1 (queue active)
Oct 19 03:37:31 server1 postfix/qmgr[19882]: 5A6CE811290E: removed
Oct 19 03:37:31 server1 postfix/smtp[25395]: 60B898124D19: to=<mygmailaddress>, orig_to=<a@domain1.com>, relay=gmail-smtp-in.l.google.com[74.125.132.27]:25, delay=3.5, delays=3.1/0.01/0.19/0.21, dsn=2.0.0, status=sent (250 2.0.0 OK 1571470651 c4si2849970ilj.32 - gsmtp)
Oct 19 03:37:31 server1 postfix/qmgr[19882]: 60B898124D19: removed
Oct 19 03:37:55 server1 dovecot: pop3-login: Login: user=<me@domain1.com>, method=PLAIN, rip=my.ip.address, lip=my.server.ip, mpid=25640, session=<95Vbhz6VWIRJLEun>
Oct 19 03:37:55 server1 dovecot: pop3(me@domain1.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/7368, size=654099943
Oct 19 03:40:48 server1 dovecot: pop3-login: Login: user=<me@domain1.com>, method=PLAIN, rip=my.ip.address, lip=my.server.ip, mpid=27730, session=<XB20kT6VGoVJLEun>
Oct 19 03:40:48 server1 dovecot: pop3(me@domain1.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/7368, size=654099943
Oct 19 03:47:18 server1 postfix/cleanup[29306]: warning: database /etc/postfix/generic.db is older than source file /etc/postfix/generic
Oct 19 03:47:18 server1 postfix/pickup[19881]: 618108124D1A: uid=0 from=<admin@domain2.com>
Oct 19 03:47:18 server1 postfix/cleanup[29306]: 618108124D1A: message-id=<1571471238.29302@domain2.com>
Oct 19 03:47:18 server1 postfix/qmgr[19882]: 618108124D1A: from=<admin@domain2.com>, size=595, nrcpt=1 (queue active)
Oct 19 03:47:18 server1 postfix/trivial-rewrite[29307]: warning: do not list domain server1.mymainserver.com in BOTH mydestination and virtual_alias_domains
Oct 19 03:47:18 server1 postfix/trivial-rewrite[29307]: warning: do not list domain server1.mymainserver.com in BOTH mydestination and virtual_alias_domains
Oct 19 03:47:20 server1 postfix/local[29308]: 618108124D1A: to=<me-domain1.com@server1.mymainserver.com>, orig_to=<a@domain1.com>, relay=local, delay=1.9, delays=0.04/0.01/0/1.9, dsn=2.0.0, status=sent (delivered to command: /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME)
Oct 19 03:47:20 server1 postfix/cleanup[29306]: 69915815BD20: message-id=<1571471238.29302@domain2.com>
Oct 19 03:47:20 server1 postfix/qmgr[19882]: 69915815BD20: from=<admin@domain2.com>, size=742, nrcpt=1 (queue active)
Oct 19 03:47:20 server1 postfix/local[29308]: 618108124D1A: to=<me-domain1.com@server1.mymainserver.com>, orig_to=<a@domain1.com>, relay=local, delay=2, delays=0.04/0.01/0/2, dsn=2.0.0, status=sent (forwarded as 69915815BD20)
Oct 19 03:47:20 server1 postfix/qmgr[19882]: 618108124D1A: removed
Oct 19 03:47:20 server1 postfix/smtp[29353]: 69915815BD20: to=<mygmailaddress>, orig_to=<a@domain1.com>, relay=gmail-smtp-in.l.google.com[74.125.132.26]:25, delay=2.5, delays=2/0.01/0.14/0.36, dsn=2.0.0, status=sent (250 2.0.0 OK 1571471240 a1si12452917ilp.82 - gsmtp)
Oct 19 03:47:20 server1 postfix/qmgr[19882]: 69915815BD20: removed
Oct 19 03:47:39 server1 postfix/pickup[19881]: 842E1815BD2C: uid=0 from=<admin@domain2.com>
Oct 19 03:47:39 server1 postfix/cleanup[29306]: 842E1815BD2C: message-id=<1571471259.29593@domain2.com>
Oct 19 03:47:39 server1 postfix/qmgr[19882]: 842E1815BD2C: from=<admin@domain2.com>, size=598, nrcpt=1 (queue active)
Oct 19 03:47:39 server1 postfix/trivial-rewrite[29307]: warning: do not list domain server1.mymainserver.com in BOTH mydestination and virtual_alias_domains
Oct 19 03:47:39 server1 postfix/trivial-rewrite[29307]: warning: do not list domain server1.mymainserver.com in BOTH mydestination and virtual_alias_domains
Oct 19 03:47:42 server1 postfix/local[29308]: 842E1815BD2C: to=<me-domain1.com@server1.mymainserver.com>, orig_to=<me@domain1.com>, relay=local, delay=2.7, delays=0.03/0/0/2.7, dsn=2.0.0, status=sent (delivered to command: /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME)
Oct 19 03:47:42 server1 postfix/cleanup[29306]: 8865A815BD2D: message-id=<1571471259.29593@domain2.com>
Oct 19 03:47:42 server1 postfix/local[29308]: 842E1815BD2C: to=<me-domain1.com@server1.mymainserver.com>, orig_to=<me@domain1.com>, relay=local, delay=2.7, delays=0.03/0/0/2.7, dsn=2.0.0, status=sent (forwarded as 8865A815BD2D)
Oct 19 03:47:42 server1 postfix/qmgr[19882]: 8865A815BD2D: from=<admin@domain2.com>, size=745, nrcpt=1 (queue active)
Oct 19 03:47:42 server1 postfix/qmgr[19882]: 842E1815BD2C: removed
Oct 19 03:47:42 server1 postfix/smtp[29353]: 8865A815BD2D: to=<mygmailaddress>, orig_to=<me@domain1.com>, relay=gmail-smtp-in.l.google.com[74.125.132.26]:25, delay=3.1, delays=2.7/0/0.09/0.29, dsn=2.0.0, status=sent (250 2.0.0 OK 1571471262 u11si10789116iom.109 - gsmtp)
Oct 19 03:47:42 server1 postfix/qmgr[19882]: 8865A815BD2D: removed
Oct 19 03:48:01 server1 dovecot: pop3-login: Login: user=<me@domain1.com>, method=PLAIN, rip=my.ip.address, lip=my.server.ip, mpid=29763, session=<+7KAqz6VwoVJLEun>
Oct 19 03:48:01 server1 dovecot: pop3(me@domain1.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/7368, size=654099943
A few things spring to mind:
Oct 19 03:26:36 server1 postfix/postfix-script[19488]: fatal: the Postfix mail system is not runningis a big flag. From Webmin - Server - Postfiox: check the status, start or stop\start if need be and check the system & mail logs at the time to see for errors.
From a command line type
postconf
to see your configuration and type
postconf -n
to see how it differs from a default setup. The following lines are not good
Oct 19 03:37:28 server1 postfix/cleanup[25316]: warning: database /etc/postfix/generic.db is older than source file /etc/postfix/genericOct 19 03:37:28 server1 postfix/trivial-rewrite[25317]: warning: do not list domain server1.mymainserver.com in BOTH mydestination and virtual_alias_domains
Those warnings should be dealt with. Given they are warnings - they shouldn't stop Postfix from starting\running - however you could end up with weird\unpredictable behaviour, so I would recommend dealing with them
1st one - looks like you need to regen the associated hash file. 2nd one - it's self explanatory. Once those issues are dealt with - Postfix should start\run better - if not - check the logs for additional errors and deal with them.
Then moving onto the mail:
HIH
Dibs
Thanks for the info!
I tried a few things and now it's even more confusing. May be a very odd mismatch of IP addresses, caused after moving server to another datacenter. Let me explain:
From outside, if i ping the server's hostname (server1.myhostname.com) i get the NEW ip address (all good) From the server itself, if i ping my hostname (same) i get the OLD ip address (wtf!?)
What gives? I am looking at network interfaces and i don't see that old IP anywhere, and i have no idea how to update the hostname of the server to point to the actual IP of the server itself. I think this may be the issue. Pointers would be greatly appreciated.
Thanks!
Anything odd in Host Addresses?
Looks good, this is all that's in there.
IP Address Enabled? Hostnames 127.0.0.1 Yes localhost , localhost.localdomain , localhost4 , localhost4.localdomain4 ::1 Yes localhost , localhost.localdomain , localhost6 , localhost6.localdomain6 NEWIP Yes server1.myhost.com
This was a dumb question - hosts file, duh. Let me see if that worked.
It didn't but doing a netstat -plnt | grep :25 still shows the old IP being listened on, not the new IP. Any idea where this can be changed!? I thought it was the hostname wrong but updating that (and i verified, it's correct now) didn't do the trick.
In /etc/postfix/main.cf file what does inet_interfaces say?
When you moved data centers and IP - you did update your DNS records? For ALL hosts\domains?
Yep, sure did, i can access everything very well. This is the only issue.
It isn't really about whether you can access everything - you don't have a DNS entry hanging about with the old IP? It's happened to the best of us. ;)
I will re-check. In the meantime i added to postfix config: smtp_bind_address = newip Restarted postfix, no change
Checked every config file i can find and no luck, that IP doesn't exist anywhere. I'm running a grep -rlw "old ip address" / to see if it digs up anything. Though this may take a while as i probably should not have searched recursively from root lol
So i got mad that i couldn't find anything and literally re-installed postfix, re-did all config files, etc. Whatever it was, it's now fixed.
Thanks for the help, much appreciated, i learned a few things!
Glad you got it sorted. Be very careful with forwarding emails to Gmail. Forwarding isn't ideal but a fairly good way to end up on an RBL or backscatterer.org unless you harden Postfix as best you can.