Extra Spam filtering options

9 posts / 0 new
Last post
#1 Mon, 06/29/2009 - 01:40
nickorr

Extra Spam filtering options

I'm using virtualmin on ubuntu and so I've got postfix for mail with postgrey enabled. I'm looking to increase the spam filtering, although spam assassin and postgrey do catch a fair bit. I was looking to increase some of the restrictions on helo and receiving mail. Specifically to change smtpd_helo_restrictions to :

smtpd_helo_restrictions = permit_mynetworks, reject_non_fqdn_hostname, reject_invalid_hostname, permit

and change smtpd_recipient_restrictions to :

permit_sasl_authenticated, reject_unauth_destination, reject_invalid_hostname, reject_non_fqdn_hostname, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unknown_sender_domain, reject_unknown_recipient_domain, permit_mynetworks, check_policy_service inet:127.0.0.1:60000, reject_rbl_client zen.spamhaus.org, permit

Is there anything wrong with this, and will it make any sort of a difference? Are there any other rbl lists that I should or shouldn't use?

Thanks, Nick

Mon, 06/29/2009 - 02:01
Joe
Joe's picture

I would recommend not using RBLs in your Postfix configuration using the reject_rbl_client directive. Unless you really trust that RBL not to return false positives, you're going to miss some legitimate mail.

Instead, crank up the value of RBL checks in your SpamAssassin configuration. You could make RBLs worth, say, 3.5 points...which would mean that just a wee bit more spamminess would push it over the top, but very non-spammy messages would come through unimpeded.

While blocking as early as possible definitely has value (which is why postgrey is a big win, particularly on hard-working mail servers), and Postfix is the first thing that can make any decisions about a message, it's risky to use some of its spam prevention directives, because they are true or false. There is no room for mistakes when making use of those options...and RBLs very regularly make mistakes. They do good things, and we all love that things like spamhaus.org exists, but they are also operated by and contributed to by humans and sometimes something gets added by mistake (or intentionally, but it is still a server that sends you legitimate mail).

This lack of nuance in Postfix options is partly why the policy server interface was invented. A policy server can take all of that information, look it over and compare many pieces of information, and finally make a decision which it returns to Postfix. And then Postfix can block the message...and all of this still happens quite early in the mail delivery process, so it is generally cheap (SpamAssassin and ClamAV are not at all cheap).

--

Check out the forum guidelines!

Mon, 06/29/2009 - 04:35
nickorr

Ok, thanks for that. Is there anything you would recommend to increase the spam blocking abilities?

Thanks, Nick

Mon, 06/29/2009 - 09:07 (Reply to #3)
andreychek

Hi Nick,

Some of this will depend on what sort of spam you're getting on your server.

If you're having spam get past the filters, what I might do is look at the headers, and see what's being marked in the X-Spam-Status header. This header tells you everything SpamAssassin finds "spammy" about that message.

If you see a common theme between a lot of spam getting through, you could try increasing the value of that rule (or rules).

If you haven't already, I might also recommend enabling Greylisting.

Finally, you might consider poking around on the SpamAssassin wiki, they have some neat tips and tricks throughout:

http://wiki.apache.org/spamassassin/

One of the things I do that they describe on there is using Pyzor, which I've found largely to be pretty helpful (but of course, your milage may vary ;-)

Some info on Pyzor and integrating it into SpamAssassin is here:

http://wiki.apache.org/spamassassin/UsingPyzor

Mon, 06/29/2009 - 18:05 (Reply to #4)
nickorr

Most of this spam is recent ones with long sentences and then a single line with a url inside it, but the url is broken up by spaces to make it not trigger any spam rules. The spam headers are just :

X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on mail.goya.com.au X-Spam-Level: * X-Spam-Status: No, score=1.0 required=5.0 tests=BAYES_60 autolearn=ham version=3.2.4

What does that mean? Also the email has in the receive headers :

Received: from zcjxtp.speedy.com.ar (unknown [190.174.103.17]) by test.goya.com.au (Postfix) with SMTP id 3DD201A61E

The reason I wanted to look at other blocking options is that it appears to me at least that that is an obvious dialup/temp IP or hostname, and should be easy to block on.

Cheers, Nick

Mon, 06/29/2009 - 18:15 (Reply to #5)
Joe
Joe's picture

It means that the Bayesian filter got a whiff of spam from the message...but not with a lot of confidence.

You could train the Bayes database on this message (and others like it) to make it more accurate. Usermin has a "Report as spam" button, which adds the message to the spam corpus. There's also optionally an alias you can email spam and ham to in order to train you system to more accurately detect spam using the Bayesian filter.

The autolearn=ham means that because this wasn't detected as spam, it is now in the ham corpus...which means it's actually training itself to be more likely to let this kind of spam through in the future. (Autolearn is a mixed blessing.)

Lack of reverse resolution here, "(unknown [190.174.103.17])", is definitely a spammy indicator, and I'm surprised SpamAssassin isn't treating it as such in your system. I don't actually know what rules are enabled by default in SpamAssassin, but reverse resolution is a good one. I suppose it is disabled by default because it can slow mail delivery quite a bit, particularly for mail coming from unreliable hosts.

Anyway, the test name is RDNS_NONE. It only has 0.1 point in SpamAssassin 3.2, by default, it looks like. It's probably safe to crank it to 1.0.

Anyway, frankly, it looks like this spam is perfectly designed to avoid being tagged as spam by SpamAssassin. This means the spammer has tested (possibly a lot) against spam filtering systems to make sure their message gets through.

--

Check out the forum guidelines!

Tue, 06/30/2009 - 00:51 (Reply to #6)
nickorr

Are any of the settings for spamassasin test values (like the RDNS_NONE one) in virtualmin, or do I have to modify config files?

Also one of the options on the wiki you pointed out to me (thanks) was to turn on the network tests, by removing the -L or --local setting. I can't find that anywhere (the module config in virtualmin, the bootup actions, the /etc/init.d or /etc/defaults files - would this be a setting that a stock virtualmin install wouldn't use, or am I just looking in the wrong place?

Thanks, Nick

Tue, 06/30/2009 - 08:39 (Reply to #7)
andreychek

To modify the test values, you'd want to look in Webmin -> Servers -> SpamAssassin -> Header and Body Tests, click "Switch to advanced mode", and you can add your tests to "SpamAssassin test scores".

For example, put "RDNS_NONE" as the test name, and something like "1.0" as the score.

As for the network tests -- how to enable SpamAssassin switches depends on your distro. Virtualmin really isn't involved in that, it just installs the standard SpamAssassin your distro provided.

That said, most setups I've seen already have network tests enabled. But for me on my Ubuntu system, I'd edit /etc/defaults/spamassassin, and modify the "OPTIONS" parameter they have in there.

-Eric

Tue, 06/30/2009 - 20:06 (Reply to #8)
nickorr

Thanks Eric. The defaults file doesn't have the -l or local option, so I'll assume on my box (ubuntu as well) that it's already in use.

As for the extra tests, I'll keep feeding these into the spam reporting and see if anything consistent turns up.

Thanks, Nick

Topic locked