Mail not getting through Procmail with Spamassassin

9 posts / 0 new
Topic locked
Last post
#1 Wed, 04/07/2010 - 07:43
simp

Mail not getting through Procmail with Spamassassin

Hi:-

Strange problem began last week. Emails to some accounts were not getting through at all. I discovered that Spamassassin had stopped. When I restarted it, the problem persisted. Only by turning off Spamassassin in the config, would these accounts work normally. Meanwhile, some other accounts were working fine. And all accounts had previously been working fine.

I have done some testing, and have noted the following:-

SCENARIO 1: -> Spamassassin running, Spamassassin is active on foobar.com, active on info@foobar.com -> Send an email to info@foobar.com -> Email received by Postfix and handed off to Procmail -> Email disappears... -> In the Edit Mail Users panel for info@, there is notification of new mail at correct time -> But email doesn't show up in Maildir, and is not delivered to either IMAP or POP service

SCENARIO 2: -> Spamassassin running, Spamassassin is NOT active on foobar.com, NOT active on info@foobar.com -> Send an email to info@foobar.com -> Email received by Postfix and handed off to Procmail -> Email received...

So, somewhere between Spamassassin and Procmail the messages are being lost... Unfortunately, this is slightly beyond my server maintenance knowledge. I hope that someone out there can point me in the right direction!

Thanks,

Simp.

Wed, 04/07/2010 - 09:58
andreychek

Howdy,

It sounds like you've done a lot of detective work already! :-)

Since emails are getting to your server, that means you can use the logfiles in order to track down where the messages are going.

That's my recommendation of where to go next -- send yourself an email to an account you've had trouble with, and then look at the email logs (either /var/log/messages or /var/log/mail.log).

In there, you should see the email being received, as well as what all happens to it before it gets handed off to procmail. If there's errors with Postfix or SpamAssassin, they should show up there.

And then, if you don't notice anything at that point, do you see it being handed off to procmail? If so, the next step is to look in /var/log/procmail.log, and see what's happening in there.

You should be able to see what folder procmail puts it in. If there's an error of some sort, feel free to note that here and we can try to figure that out.

-Eric

Wed, 04/07/2010 - 10:08 (Reply to #2)
simp

Hi Eric:-

Thanks for your comments... I checked the Mail log and there are no errors for the missing emails - here's an example:-

--EXAMPLE OF WORKING EMAIL ACCOUNT

Mar 31 22:39:56 vps postfix/smtpd[25689]: EAD6219842566: client=unknown[109.78.102.85], sasl_method=PLAIN, sasl_username=user.mydomain

Mar 31 22:39:57 vps postfix/cleanup[13586]: EAD6219842566: message-id=520BAB31-A95C-4D59-9AFD-CD4C5B437917@mydomain.com

Mar 31 22:40:00 vps postfix/qmgr[26248]: EAD6219842566: from=user@mydomain.com, size=22551, nrcpt=1 (queue active)

Mar 31 22:40:20 vps postfix/local[13613]: EAD6219842566: to=mydomain@vps.myserver.com, orig_to=webmaster@mydomain.com, relay=local, delay=24, delays=4/0.01/0/20, dsn=2.0.0, status=sent (delivered to command: /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME)

Mar 31 22:40:20 vps postfix/qmgr[26248]: EAD6219842566: removed

--EXAMPLE OF BROKEN EMAIL ACCOUNT

Apr 1 10:41:39 vps postfix/smtpd[13821]: 715B719842F04: client=mail-wy0-f180.google.com[74.125.82.180]

Apr 1 10:41:39 vps postfix/cleanup[22384]: 715B719842F04: message-id=r2xa3cdfa541004010241m438ad692o84c4af35bd7781ee@mail.gmail.com

Apr 1 10:41:39 vps postfix/qmgr[13360]: 715B719842F04: from=account@gmail.com, size=1852, nrcpt=1 (queue active)

Apr 1 10:42:38 vps postfix/local[32558]: 715B719842F04: to=user.mydomain@vps.myserver.com, orig_to=user@mydomain.com, relay=local, delay=59, delays=0.09/0.08/0/59, dsn=2.0.0, status=sent (delivered to command: /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME)

Apr 1 10:42:38 vps postfix/qmgr[13360]: 715B719842F04: removed

So it must be something to do with Procmail - I will investigate the logs....

Thanks for your help, as ever!

Cheers,

Simp.

Wed, 04/07/2010 - 10:09
andreychek

Looks like you're correct, I don't see anything wrong above either; it's indeed just delivering those messages to procmail, so let's see what procmail has in store for us :-)

-Eric

Wed, 04/07/2010 - 10:29
simp

Okay - figured out what the problem is - bit of an RTFM, really.

The missing email was being processed correctly by procmail, but then ending up in the /Maildir/.spam folder

Doh!

Now - in my defense - I had not expected it to go there because in the Procmail Delivery Settings of Spamassassin the "Action for messages classified as spam" is set to "Deliver Normally" - I was expecting Spamassassin to just mark the headers and pass it on - not to quarantine it...

Any idea why it would be doing this?

Furthermore - it looks like one of the domains has ended up on a spam blacklist - so it was treated as a false positive...

So I will go back to the spamassassin settings and adjust... And try to get myself removed from the blacklist!

Thanks again for all your help.

Wed, 04/07/2010 - 10:41
andreychek

Just for fun, would you be willing to triple check that the SpamAssassin settings you're describing above are indeed set for this particular domain (rather than for some other domain on the system)?

It's possible to set those for one domain, and not for another.

You can view those by first choosing the domain from the drop-down on the top-left, then going into Server Configuration -> Spam and Virus Delivery.

If it is set the way you want it, with spam going into the Inbox -- try changing it to something else, then back again.

-Eric

Wed, 04/07/2010 - 10:54
simp

Thanks Eric. Looks like the local settings were indeed over-riding the global settings... Double doh! But to be honest, the quarantine approach suits my users best, so I think I'll stick with it.

Upon further investigation, the main problem here appears to have been a Spamassassin bug, whereby it marks an email dated from 2010 with "FH_DATE_PAST_20XX The date is grossly in the future." - this appears to be a known issue with older Spamassassin distros, e.g. https://secure.grepular.com/SpamAssassin_2010_Bug And this has been causing a lot of false positives since Jan 1st, 2010, hence my discovery of the problem.

There is a solution proposed here: http://wiki.apache.org/spamassassin/Rules/FH_DATE_PAST_20XX

If you need to disable this test manually, place the following in your local.cf file:

score FH_DATE_PAST_20XX 0.0

So I think I might just try this, and assume that the distro will be updated before 2020 ;-)

Thanks for all your help in solving this.

Regards,

Simp.

Wed, 04/07/2010 - 10:58
andreychek

More great detective work :-)

Yeah, that problem does crop up on some SpamAssassin installs. In addition to what you discovered above, running "sa-update" is supposedly able to correct it, as that will pull in more recent rule files that correct the problem.

However, to be safe, you might consider doing both for the time being :-)

-Eric

Wed, 04/07/2010 - 11:08
simp

Great - thanks - I have updated spamassassin and added the following daily cron:-

sa-update && service spamassassin restart

And I'll monitor the spam folders for this problem in future!

Topic locked