Submitted by bbuhlman on Fri, 10/05/2012 - 11:18
I'm using Mailman and having trouble with spam. I have Spam Assassin running but is it configured to look at Mailman list mail? I've added many domain names into denied addresses but they still get through like SA never looked at the rulebase.
Thanks, Bill
Status:
Active
Comments
Submitted by andreychek on Fri, 10/05/2012 - 11:22 Comment #1
Howdy -- SpamAssassin should look at any email that's being delivered locally, if that recipient's Virtual Server has the "spam filtering" feature enabled.
SpamAssassin doesn't check outgoing mail though, only incoming mail.
If a user on your server is receiving spam, you may want to look at the mail headers for that mail... the "X-Spam-Status" header would show which rules it's triggering in SpamAssassin.
Submitted by bbuhlman on Fri, 10/05/2012 - 11:37 Comment #2
The virtual server does have spam filtering enabled. This virtual server is only for mailman lists and I am the only real user configured. Here is the config:
SpamAssassin client program spamc (Client for SpamAssassin filter server spamd)
Server host for spamc localhost
Maximum message size to process Default (500 kB) At most Allow mailbox users to create mail filters? No
Virus scanning program Server scanner (clamdscan) Remote server scanner (clamd-stream-client)
Server host for clamd-stream-client localhost
Should I be using stand alone spamassassin or the standalone scanner instead of what I am using?
Submitted by andreychek on Fri, 10/05/2012 - 14:42 Comment #3
All of those settings will work find in regard to scanning email for spam. If the spam feature is enabled, incoming email should be scanned.
You may need to look at the email headers for the X-Spam-Status header to get a better idea of what's going on.
Submitted by bbuhlman on Fri, 10/05/2012 - 15:21 Comment #4
how do I do that?
Submitted by bbuhlman on Fri, 10/05/2012 - 15:50 Comment #5
I don't think SA is integrated with Mailman. There seems to be quite a few threads about how to do this on the Internet. SA seems to need a "user" to be able to filter the mail and with my current setup Mailman lists are not users. I did find where SA was working with my own username though. Does this help:
Integrating SpamAssassin with Mailman
It is assumed that you already have a copy of Mailman installed on your system. The current version of my patches are designed to work with Mailman >= 2.1, so you should upgrade to 2.1.x if you haven't already.
If you are using 2.1, you may want to upgrade to 2.1.1, as it fixes a number of cross site scripting vulnerabilities. New releases are available from the Mailman website or on Sourceforge:
Consult the documentation included in the source tarball for instructions on building and installing Mailman. Installing SpamAssassin
SpamAssassin is available from the SpamAssassin website. Follow the installation instructions included in the tarball, or just install the RPMs (or rebuild the SRPM first).
You should install SpamAssassin >= 2.50, but more recent versions are preferable. Configuring spamd
The Mailman patches make use of the spamd daemon included in SpamAssassin, so it will be necessary to configure it to run at startup.
First create a new user account to run spamd as. The home directory for the user should be set to something like /var/lib/spamassassin. On most Linux distros, this can be done with the following command:
mkdir /var/lib/spamassassin useradd -r -d /var/lib/spamassassin -M -s /sbin/nologin \ -c 'SpamAssassin' spamassassin chown spamassassin.spamassassin /var/lib/spamassassin
The exact details will depend on your OS. We need to pass a number of arguments to spamd to get it to run in a locked down mode. The arguments I use are:
spamd -d -u spamassassin -x -a -P --virtual-config-dir=/var/lib/spamassassin/%u.prefs
These meanings of these arguments are:
-d fork spamd on startup -u spamassassin run as the spamassassin user account. -x don't create user_prefs files for individual users -a create automatic whitelists, to smooth out the scores that individuals receive. -P paranoid mode --virtual-config-dir=/var/lib/spamassassin/%u.prefs create per-user configuration directories under the spamassassin user's home directory. This way we can maintain separate automatic whitelists and Bayes databases for each mailing list.
If you are using the RPMs, you can put these options in a /etc/sysconfig/spamd file so that they will be passed to spamd when it is started:
Options to spamdOPTIONS="-d -u spamassassin -x -a -P --virtual-config-dir=/var/lib/spamassassin/%u.prefs"
don't bother with UTF-8 modeexport LANG=en_AU
Next, you will want to set up your init scripts to start spamd when your system starts up. If you are using the RPMs, this is trivial:
chkconfig --level 345 spamassassin on service spamassassin start
Adding the SpamAssassin Filter to Mailman
First you will need to download the filter, which is comprised of the following two files:
Both files should be installed into the Mailman/Handlers/ subdirectory under the Mailman install directory. You will then need to edit the Mailman/mm_cfg.py file to enable the filter:
GLOBAL_PIPELINE.insert(1, 'SpamAssassin')
After making the changes to Mailman/mm_cfg.py, you will need to restart the qrunner process. This can be achieved with the mailmanctl program:
mailmanctl restart
At this point, the SpamAssassin filter should operational. Configuration
With the mailman filter in place, every incoming message will be passed off to SpamAssassin's spamd daemon for scoring. The mailing list name will be sent as the user name. This allows us to maintain separate SpamAssassin data files for each list.
After scoring, a message can be discarded if the score is over a certain threshold (defaulting to 10), or held for moderation if it the score is over another threshold (defaulting to 5). Additionally, a "bonus" can be subtracted from the scores of messages sent by list subscribers (defaulting to 2) to reduce the chance that subscriber posts are held or discarded.
These settings can be tuned by editing the Mailman/mm_cfg.py file and adding the following variables:
SPAMASSASSIN_HOST The host spamd is running on. A string in hostname:port format. SPAMASSASSIN_DISCARD_SCORE If a message receives a score above this limit, the message will be discarded without moderation. The default value for this variable is 10. SPAMASSASSIN_HOLD_SCORE If a message receives a score above this limit, the message will be held for moderation. The default value for this variable is 5. SPAMASSASSIN_MEMBER_BONUS If the message was sent by a member of the list, an adjustment can be performed on the score. This makes it less likely that a message claiming to come from a list member will be held for moderation. The default value for this variable is 2.
As before, you will need to restart qrunner with mailmanctl after modifying the config file.
For the PyGTK mailing list, I use a discard score of 7.5 and a hold score of 5. Feeding the Bayes Database
By itself, SpamAssassin will filter the majority of spam directed at the lists. For better results, you should look at seeding the Bayes database for your list. This will customise the filter based on traffic to your list, which makes it more difficult for spammers to produce messages that get through. In turn, the amount of administration required for the list will be reduced.
To seed the Bayes database, you will need to feed it a corpus of spam and a corpus of "ham" (non-spam). This is used to help it differentiate spam from normal list postings. It is a good idea to use recent messages if possible, as they will better reflect the typical list traffic.
If you run a closed list, your list archives should make a pretty good "ham" corpus. Simply trim a few months off the bottom of the archive mbox (found in archives/private/listname.mbox), and pass them to sa-learn:
HOME=/var/lib/spamassassin/listname.prefs \ sa-learn --showdots --ham --mbox filename.mbox
If you don't have a collection of recent spam messages, an other option is to train the database on messages that get held in the moderation queue. To help with this, I wrote a small script called mmlearn. After removing all legitimate messages from the moderation queue, you can run it on the remaining spam:
mmlearn listname
Training the filter based on false negatives is a fairly effective way of improving the filter.
Submitted by andreychek on Fri, 10/05/2012 - 16:18 Comment #6
SpamAssassin is configured to scan email being delivered to users on your server.
If Mailman delivers email to a user on your server, it would be scanned by SpamAssassin.
I'm not familiar with the setup you're describing above, to integrate SpamAssassin into Mailman for all messages (and not just those being delivered locally). That may do what you want though... you could certainly give that a try and let us know how it works. I haven't tried that setup before though :-)
Submitted by bbuhlman on Fri, 10/05/2012 - 16:34 Comment #7
I'm sorry I can't get my point across, there are no configured local users on my server. The box was built as a Mailman mailing list server only. All mail recipients are on mailing lists, they don't exist as users on the server. Can SA be integrated to scan incoming Mailman mailing list mail?
What is happening is that I am getting a lot of spam addressed to the various mailing lists and I have to discard it one by one. I am hoping SA can be configured to help out here.
Thank you, Bill
Submitted by andreychek on Fri, 10/05/2012 - 16:54 Comment #8
I think understand what you're saying... what I'm offering is that Mailman isn't configured to do what I believe you're after :-)
SpamAssassin is only setup to scan email destined for a local user, not a mailing list.
If you don't have users on your system, if it's just a mailing list server -- that's not a setup where SpamAssassin will scan those emails, unless you perform some additional manual setup.
The instructions you pasted in above might work, but that's unfortunately not a setup I'm familiar with.
Submitted by bbuhlman on Wed, 10/10/2012 - 12:44 Comment #9
Is there some filter mechanism (or even global manual domain block list) that you guys approve of that can be used that postfix will look at before delivering list mail?
Submitted by andreychek on Thu, 10/11/2012 - 10:42 Comment #10
The suggestion they made that you pasted in above seems like a good one, we just hadn't tried that before.
It sounds like what you'd need to do is download and install some files, and then edit the mm_cfg.py file.
The "Mailman/Handlers" directory they mentioned is typically located in "/usr/lib/mailman/Mailman/Handlers" (it should already exist). It sounds like they want you to place the SpamAssassin filters in that directory.
The mm_cfg.py file they mentioned is typically in either /etc/mailman/mm_cfg.py or /usr/lib/mailman/Mailman/mm_cfg.py.
You can restart Mailman with this command:
/etc/init.d/mailman restart
You can hopefully use the above to get SpamAssassin to scan emails prior to them reaching your Mailman list members.
One other thing you could do, if you're looking for a way to filter specific domains before getting to Mailman, would be to configure Postfix with domains to block.
I'm a little rusty with the config syntax for this, but I believe it would work a bit like this --
smtpd_sender_restrictions = hash:/etc/postfix/access
/etc/postfix/access
, and in it, add domains to block:baddomain.com REJECT
anotherbaddomain.com REJECT
Run this command to generate a Postfix database: postmap hash:/etc/mail/access
Then restart Postfix: /etc/init.d/postfix restart
That should tell Postfix to refuse emails with senders from the above domains.