Spam & Virus

From SEDSWiki

(Redirected from Spam/Virus)
Jump to: navigation, search

Contents

Overview

SEDS employs a number of current technologies to try and reduce the onslaught of spam that is a fact of internet life today. SEDS.org experiences statistics similar to what's commonly mentioned in the press and in literature, or about 80% of all incoming mail is spam or attempts at it.

There a number of things built into Mac OS X Server to help with this, but most of them have been upgraded and supplemented.

The general way that things work is as follows:

  1. Email is checked against the DNS Realtime Blacklists
  2. Email is checked against the greylist (postgrey)
  3. Email is sent to Amavis where several things happen:
    • It is first virus-checked by ClamAV
    • It is then spam checked by Spam Assassin which in turn uses several other tools (described below)
    • In parallel, a bayesian-learning spam checker, DSPAM is used to inspect spam
  4. Headers are added to indicate status of the spam
    • If spam, it goes off to the quarantine (extremely high scoring spam is just dropped)
    • If ham, it makes it to your mail box (finally!).
  5. For the "SEDS Organizational" email addresses and aliases there is one more step, in that they are all forwarded to support@seds.org which gets further processed and managed by the ESupport system.

This is a constant process, and one of Chris's hobbies (ie, SpamTrapping, like the frontier days, except without the rodents).

DNS Realtime Blacklists

SEDS makes use of the following DNSRBL's for the first line of defense against spam. These services maintain lists of known-spammer addresses, and SEDS will refuse email to anyone trying to send email identified by these services.

This blocking is controlled by postfix directly and it can be modified in /etc/postfix/main.cf

Amavis

Amavisd-new coordinates spam and virus filtering on SEDS.org.

The current version installed is amavisd-new-2.5.2

Amavis runs as user clamav and must be started from that user. When making modifications to the config file, SIGHUP the master process, or kill it off entirely and let launchd restart a new one. To SIGHUP it, the following works well (used by rules_du_jour): killall -HUP -m 'amavisd (master)' The config file is in /etc/amavisd.conf

Spam Assassin

The current version of SpamAssassin installed is 3.2.3.

Compilation

The easiest way to download/compile SpamAssassin is through the use of CPAN. This will keep the configuration consistent with Apple's configuration, test things for you, and install them in the right place with the appropriate permissions. 

seds: root# perl -MCPAN -e shell

cpan> install Mail::SpamAssassin

Set the administrator to "root@seds.org"

Configuration

The configuration area for SpamAssassin is in /etc/mail/spamassassin

The main configuration file is "local.cf"

Note there are also some configuration options specific to accommodating DSPAM within SpamAssassin.

The default options for SEDS are as follows: 

# Bayesian Auto Learn
bayes_auto_learn 0

# enable the statistical filter
use_bayes               0

# Safe Reporting  (don't rewrite the messages, just add headers)
report_safe 0

# Required hits to be marked as spam
required_hits 5

# postfix Already does RBL checks
skip_rbl_checks 1

ClamAV

ClamAV performs virus scanning on incoming and outgoing mail, and is called by Amavisd-new.

The current version of ClamAV installed is 0.91.2 . The installation of it has been moved from /usr to /usr/local (the original Apple files have been removed), so this should be less sensitive to Apple updates.

Compilation 

./configure --include /usr/share/clamav/include --prefix /usr/local --sysconfdir /etc/spam/clamav/

Configuration

Tack the following onto the config files for the SEDS installation:

/etc/spam/clamav/clamd.conf: 

## Collection conf options different than defaults:
PidFile /var/clamav/clamd.pid
Foreground yes
UpdateLogFile /var/log/clamd.log
LogTime yes
LocalSocket /tmp/clamd.socket
User clamav
LogFileMaxSize 0

/etc/spam/clamav/freshclam.conf: 

## Collection conf options different than defaults:
PidFile /var/clamav/freshclam.pid
Foreground yes
UpdateLogFile /var/log/freshclam.log
DatabaseMirror db.us.clamav.net
LogFileMaxSize 0

DSPAM

The current version of DSPAM installed is 3.8.0.

The configuration file for dspam is at: /var/dspam/etc/dspam.conf

Compilation

Configuration parameters (Tiger):

./configure --prefix=/var/dspam --with-storage-driver=mysql_drv --with-mysql-libraries=/usr/lib/mysql --with-mysql-includes=/usr/include/mysql --enable-preferences-extension --with-dspam-home-owner=clamav --with-dspam-home-group=clamav --with-dspam-owner=clamav --with-dspam-group=clamav --without-local-delivery-agent --without-quarantine-agent --enable-debug --enable-shared=NO

If you get an error about /usr/lib/mysql/libmysqlclient.a, follow the instructions in doc/osx.txt

In order to make it work with the training aliases run those as user clamav, or the following change must be made: 

chmod 775 /var/dspam/bin/dspam

DCC

[www.rhyolite.com/anti-spam/dcc/ Distributing Checksum Clearing] house. Version 1.3.30 is installed on SEDS.

Compilation

Configuration parameters (10.4.5): 

./configure --with-uid=clamav --disable-server

DCC is used automatically by spamassassin (from within amavisd) to check for spam signatures in email.

Mail aliases & training

The aliases currently in use are: 

spam-quarantine:        "|/usr/bin/sed 1d |/usr/bin/cyrus/bin/deliver -a cyrusimap -m quarantine"
spam:   "|/usr/bin/sed 1d |/usr/bin/cyrus/bin/deliver -a cyrusimap -m spam"
ham:    "|/usr/bin/sed 1d |/usr/bin/cyrus/bin/deliver -a cyrusimap -m ham"

The "quarantine" folder is where all trapped spam temporarily gets quarantined. There isn't any public access to this folder, but if you're curious, you can get access to it via IMAP by asking Chris.

Chris periodically (about daily) runs scripts against the ham and spam folder to retrain the spam processors and alert the various authorities to caught spam.

Note that the "spam" and "ham" folders are shared to all mail users of SEDS as shared IMAP folders. If you subscribe to these folders, you can move spam or false-positive spam to "ham". This works for webmail users as well.

If you are a POP user, you can forward your spam to "spam@seds.org" for processing. If things get tagged as spam that shouldn't be, forward them to "ham@seds.org".


Crontab

Note that the SEDS configuration of DSPAM uses the Mysql database to store the spam message keys. This database is called "dspam" and there is a user called "dspam" to access it. See the configuration file for the password. Related to this, there is a cron-script which is called nightly to purge the database of stale data. 

# Purge out old DSPAM tokens nightly @ 3AM
0 3 * * * /usr/bin/mysql -udspam -pXXXXXXXX dspam < /var/dspam/etc/purge-4.1.sql

DSPAM & SpamAssassin

Finally, in order for Amavis/SpamAssassin to make use of the DSPAM result, the following needs to be in the SpamAssassin configuration file (/etc/mail/spamassassin/local.cf): 

### Place more weight on DSPAM's opinion
header DSPAM_SPAM X-DSPAM-Result =~ /^Spam$/
describe DSPAM_SPAM DSPAM claims it is spam
score DSPAM_SPAM 4.0

header DSPAM_HAM X-DSPAM-Result =~ /^Innocent$/
describe DSPAM_HAM DSPAM claims it is ham
score DSPAM_HAM -0.25

Apple, Launchd, amavisd-new, freshclam, clamd etc...

The Apple launchd configurations for keeping the "freshclam" and "clamd" processes running aren't terribly effective with the updated clamav. I've moved the following files from /System/Library/LaunchDaemons to /Library/LaunchDaemons and edited them a bit.

  • org.amavis.amavisd.plist
  • org.clamav.clamd.plist
  • org.clamav.freshclam.plist

Updated contents of org.amavis.amavisd.plist: (Keeps it in the forground so that launchd can start and monitor it properly) 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Label</key>
        <string>org.amavis.amavisd</string>
        <key>OnDemand</key>
        <false/>
        <key>Program</key>
        <string>/usr/local/bin/amavisd</string>
        <key>ProgramArguments</key>
        <array>
                <string>amavisd</string>
                <string>foreground</string>
        </array>
        <key>ServiceIPC</key>
        <false/>
        <key>UserName</key>
        <string>clamav</string>
</dict>
</plist>

Updated contents of org.clamav.clamd.plist (reads an updated config file which lets freshclam talk to clamd directly) 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Iterations</key>
        <integer>5</integer>
        <key>Label</key>
        <string>org.clamav.clamd</string>
        <key>OnDemand</key>
        <false/>
        <key>Program</key>
        <string>/usr/local/sbin/clamd</string>
        <key>ProgramArguments</key>
        <array>
                <string>clamd</string>
                <string>--config-file=/etc/spam/clamav/clamd.conf</string>
        </array>
        <key>ServiceIPC</key>
        <false/>
        <key>UserName</key>
        <string>root</string>
</dict>
</plist>

Updated contents of org.clamav.freshclam.plist (Runs it in daemon mode instead of periodically, an update to the config file keeps it in the foreground for launchd) 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Iterations</key>
        <integer>5</integer>
        <key>Label</key>
        <string>org.clamav.freshclam</string>
        <key>OnDemand</key>
        <false/>
        <key>Program</key>
        <string>/usr/local/bin/freshclam</string>
        <key>ProgramArguments</key>
        <array>
                <string>freshclam</string>
                <string>-d</string>
        </array>
        <key>ServiceIPC</key>
        <false/>
        <key>UserName</key>
        <string>clamav</string>
</dict>
</plist>

rulesdujour

The "rulesdujour" script available from here is used to periodically update SpamAssassin rulesets available from http://www.rulesemporium.com/. For SEDS, I've picked ones (listed below) that are conservative (rarely or never test positive on non-spam), but they still add a lot of value to the spam checker.

The script is installed in "/usr/local/sbin/rules_du_jour" and is current run manually.

The /etc/rulesdujour/config file for SEDS 

SA_DIR="/etc/mail/spamassassin"
MAIL_ADDRESS="root@seds.org"
SA_RESTART="killall -HUP -m 'amavisd \(master\)'"
TRUSTED_RULESETS="TRIPWIRE ANTIDRUG BOGUSVIRUS SARE_ADULT SARE_SPOOF SARE_WHITELIST SARE_REDIRECT_POST300 SARE_EVILNUMBERS0 SARE_HEADER0 SARE_SPECIFIC
 SARE_BML SARE_FRAUD SARE_RANDOM SARE_OEM SARE_GENLSUBJ0 SARE_URI0 SARE_UNSUB SARE_WHITELIST_SPF SARE_WHITELIST_RCVD SARE_OBFU0 SARE_STOCKS"

postgrey

SEDS uses greylisting to reduce the amount of spam. postgrey 1.31 is installed on SEDS. If you find this to be a problem for receiving or sending email on seds.org, please contact the administrators.

Retrieved from "http://wiki.seds.org/index.php/Spam_%26_Virus"
Personal tools