Dspam vpopmail

From Gentoo Linux Wiki

[edit] Intro

This HOWTO is for those interested in getting DSPAM working with vpopmail This method uses DSPAM in a non daemon mode and uses pipes and vpopmail to handle spam. If this is not what you are looking for please do not follow this HOWTO.

This HOWTO will follow a basic qmail/vpopmail/dspam setup, with additional options afterwards such as IMAP. Each additional section will also include information on setting up DSPAM for that section.

I have used the HOWTO Setup QMAIL RELAY-CTRL VPOPMAIL wiki as a base for most of the setup here.

Please refer to it if you want more information regarding setting up qmail with vpopmail.

[edit] Bugs and Trouble shooting

If you find any problems with this HOWTO or any suggestions of a better way of doing this please contact me : Bain

[edit] TODO

This is a list of things I want to add at a later stage, but is not included right now.

• chkusr patch for user checking on smtp delivery
• Anti-Virus integration

[edit] USE Flags for qmail and vpopmail

USE="apache2 maildir valias vhosts mysql"

# echo "mail-mta/qmail notlsbeforeauth" >> /etc/portage/package.use

This will fix SMTP AUTH problems using qmail smtpd

[edit] Removing other mail servers

Make sure that you unmerge the other mail handlers that may be installed, such as ssmtp, sendmail, or postfix:

# emerge -C ssmtp sendmail postfix

[edit] Install QMAIL

# emerge qmail

Once installed follow the instructions to complete the setup. These are examples. Please check your version against the version you just installed.

# emerge --config =mail-mta/qmail-1.03-r16
# ln -s /var/qmail/supervise/qmail-send /service/qmail-send
# ln -s /var/qmail/supervise/qmail-smtpd /service/qmail-smtpd

If you want to use QMAIL's pop3d add the following

# ln -s /var/qmail/supervise/qmail-pop3d /service/qmail-pop3d

Add qmail to your runlevel.

# rc-update add svscan default
# /etc/init.d/svscan start

You can change the domain that the error messages are sent from (ie. MAILER-DAEMON@localhost) by editing the /var/qmail/control/me file and putting in your preferred domain.

# nano -w /var/qmail/control/me

Delete localhost and put in domain.com and errors will now come from MAILER-DAEMON@domain.com.

[edit] Install RELAY-CTRL

Using relay-ctrl is a simple and straightforward way to allow us to send email with email clients from anywhere.

# emerge relay-ctrl
# cd /etc/tcprules.d/
# nano -w tcp.qmail-smtp

Now replace your tcp.qmail-smtp file with this one - and then change the first line to match the internal IP address of your server. Thats it. After that, we're done here.

################## START OF tcp.qmail-smtp #######################
#
# CHANGE THIS IP ADDRESS TO THE INTERNAL IP ADDRESS OF YOUR MAIL SERVER
192.168.31.50:allow,RELAYCLIENT="",RBLSMTPD=""

#-----------------------------------------------------------------
#  DONT ALLOW THESE IPS TO SEND MAIL TO US
# (Insert banned IP's here)
#
# These IP's pipe out heaps and heaps of spam
#
216.242.75.100-116:allow,RBLSMTPD="-Connections from this IP have been banned."
64.228.127.:allow,RBLSMTPD="-Connections refused due to spam from freeamateurhotties.com"
154.20.94.:allow,RBLSMTPD="-Connections refused due to spam from freeamateurhotties.com"
209.151.132.:allow,RBLSMTPD="-Connections refused due to spam from freeamateurhotties.com"
216.18.85.:allow,RBLSMTPD="-Connections refused due to spam from freeamateurhotties.com"

#-----------------------------------------------------------------
# DON'T TOUCH THIS
127.0.0.1:allow,RELAYCLIENT="",RBLSMTPD=""

#-----------------------------------------------------------------
# DON'T TOUCH THIS
:allow

# You must run the below command after editing this file and then restart the /etc/init.d/svscan service in
# order to activate the changes you make here today.
#
# tcprules /etc/tcprules.d/tcp.qmail-smtp.cdb /etc/tcprules.d/.tcp.qmail-smtp.tmp < /etc/tcprules.d/tcp.qmail-smtp
#
############# END OF FILE #################

Remeber to change the 192.168.31.50 to the IP address that will be allowed to relay with AUTH (generally the internal IP address if the server will be a firewall for your network)

Don't forget to execute that last command once you've edited this file. If you don't, these changes won't take effect. And to finish up, lets restart the whole qmail suite...

# /etc/init.d/svscan restart

[edit] Update the SMTPD Config to Allow SMTP-AUTH Using VPOPMAIL

I've tried alot of iterations on this but the easiest and most straight forward way is to completely delete the contents of your /var/qmail/control/conf-smtpd file and just replace it with this. You need not replace or tweak this file at all after putting this in.

################## START OF /var/qmail/control/conf-smtp #######################
#
TCPSERVER_OPTS="${TCPSERVER_OPTS} -R"

QMAIL_TCPSERVER_PRE="${QMAIL_TCPSERVER_PRE} envdir /etc/relay-ctrl relay-ctrl-chdir"
QMAIL_SMTP_PRE="${QMAIL_SMTP_PRE} relay-ctrl-check"

QMAIL_SMTP_AUTHHOST=$(<${QMAIL_CONTROLDIR}/me)
[ -z "${QMAIL_SMTP_POST}" ] && QMAIL_SMTP_POST=/bin/true
QMAIL_SMTP_CHECKPASSWORD="/var/vpopmail/bin/vchkpw"
QMAIL_SMTP_POST="${QMAIL_SMTP_AUTHHOST} ${QMAIL_SMTP_CHECKPASSWORD} ${QMAIL_SMTP_POST}"
#
################## END OF /var/qmail/control/conf-smtp #######################

Final touches to bring this together...

svc -t /var/qmail/supervise/qmail-smtpd
chmod u+s /var/vpopmail/bin/vchkpw

I've not done it but it was in the last howto and its said that "The following step makes sending mail a lot faster under some circumstances, and I highly recommend that you do the following if you notice delays of 30 to 45 seconds sending mail..." I've never seen any harm in it so it stays.

TCPSERVER_OPTS="-H -R -l 0" (that's lower-case L followed by zero)

(question?? should we be removing the TCPSERVER_OPTS "-R" option from conf-smtp file? If not, wouldn't it be setting that flag twice?)

If you reciving 454 Oops, unable to write pipe and auth errors when trying to send using AUTH, remove ${QMAIL_SMTP_AUTHHOST} from your QMAIL_SMTP_POST line in /var/qmail/control/conf-smtp

[edit] Install VPOPMAIL

Before installing vpopmail some things need to be explained.

For DSPAM to work it is usually run with the user dspam, but for the webui to function correctly this does not work well. For this reason we will need to run dspam as user:group vpopmail:vpopmail. The webui uses suexec to allow the instance of apache to run as vpopmail to handle the mail functions. We will for this reason need to have vpopmail:vpopmail with UID's higher then 1000. For this reason we will modify the vpopmail user and group. If you have vpopmail installed already I would suggest you uninstall vpopmail first using:

# emerge -C vpopmail

Now lets modify the user and group for vpopmail

# groupmod -g 1110 vpopmail
# usermod -u 1110 -g 1110 vpopmail

Now lets install vpopmail

# emerge =net-mail/vpopmail-5.4.6-r1

Now log into mysql as your mysql root user and pass like this...

# mysql -u root -p
  password: (enter root password here)
  ---- you'll be inside mysql at this point ----
> create database vpopmail;
> use vpopmail;
> grant select, insert, update, delete, create, drop on vpopmail.* to vpopmail@localhost identified by 'your password';
> flush privileges;
> quit

Replace the phrase 'your password' with a password for the vpopmail user.

Configure vpopmail's mysql user password

# nano /etc/vpopmail.conf

Change the password from 'secret' to the password you chose above.

If you have problems with vpopmail not accepting mail properly, please ensure that /etc/vpopmail.conf is chmod 640 and owned by root:vpopmail

# chown root:vpopmail /etc/vpopmail.conf
# chmod 640 /etc/vpopmail.conf
# chown root:vpopmail /var/vpopmail/bin/vchkpw
# chmod 4711 /var/vpopmail/bin/vchkpw

Thats it. Vpopmail is all setup.

Let gentoo know there are new binaries laying around.

# env-update && source /etc/profile

Add a domain from the commandline in a bash prompt

# vadddomain blah.com

Add a user

# vadduser user@blah.com

Delete a user

# vdeluser user@blah.com

[edit] Vpopmail and qmail

Now lets setup qmail-pop3d to work with vpopmail.

# cd /var/qmail/control
# nano -w conf-pop3

Change QMAIL_POP3_CHECKPASSWORD to reflect the following

QMAIL_POP3_CHECKPASSWORD="/var/vpopmail/bin/vchkpw"

[edit] Installing dovecot (IMAP)

I prefer using dovecot over courier. It's easer to use and much less intensive then courier. It's just a personal choice. If you want to use courier instead then look at the original HOWTO Setup QMAIL RELAY-CTRL VPOPMAIL wiki entry that this howto is based on.

Before we install dovecot we need to add vpopmail support for it.

# echo "net-mail/dovecot vpopmail" >> /etc/portage/package.use

Now install dovecot.

# emerge dovecot

next we need to edit the dovecot configs. Make a backup of the dovecot.conf file and replace it with this one

ssl_cert_file = /etc/ssl/dovecot/server.pem
ssl_key_file = /etc/ssl/dovecot/server.key
mail_location = maildir:%h/.maildir
mail_extra_groups = mail
first_valid_uid = 500
last_valid_uid = 0
disable_plaintext_auth = no
log_path = /var/log/dovecot/dovecot-err.log
info_log_path = /var/log/dovecot/dovecot.log
log_timestamp = "%b %d %H:%M:%S "
protocol imap {
}
protocol pop3 {
}
auth default {
	mechanisms = plain
	passdb vpopmail {
	}
	userdb vpopmail {
	}
	user = root
}
dict {
}
plugin {
}

Add dovecot to rc-update and start dovecot daemon

# rc-update add dovecot default

# /etc/init.d/dovecot start

You now have IMAP :)

[edit] Installing roundcube (webmail)

See HOWTO Install Roundcube

[edit] Installing DSPAM

Now that vpopmail is working lets get DSPAM working with vpopmail. First thing of course would be to install DSPAM itself so. If you have do not run KEYWORDS="~x86" you will need to set the keyword for DSPAM.

# echo "mail-filter/dspam ~x86" >> /etc/portage/package.keywords
# echo "www-apps/dspam-web ~x86" >> /etc/portage/package.keywords

This will enable ~x86 ONLY for DSPAM and the DSPAM web interface, not the whole system.

DSPAM has support for virtual users which we'll use. There are also options for large domains. Lets add support for virtual users.

# echo "mail-filter/dspam virtual-users" >> /etc/portage/package.use

Now we can emerge dspam

# emerge dspam

Once DSPAM is installed you need to run the ebuild config

# ebuild /var/db/pkg/mail-filter/dspam-3.6.1/dspam-3.6.1.ebuild config

• When prompted for a password, please enter your MySQL root password
• Select "Speed optimized database" as object database

cd /etc/mail/dspam
mysqladmin create dspamdb -p
mysql dspamdb -p < mysql_objects-speed.sql
mysql dspamdb -p < mysql_virtual_users.sql
echo "grant all privileges on dspamdb.* to vpopmail@localhost identified by 'dspam-password'" | mysql -p dspamdb

Change the dspam-password to something meaningfull

DSPAM comes witha template of text files that is used during interaction with users. This is unfortunately not in the correct place.

# mv /etc/mail/dspam/txt /var/spool/dspam/

Review these files and edit them as you see fit. Keep in mind that $u is the username of your user which in this case will be the full e-mail address as per vpopmail. below is a samples from firstrun.txt. The bold text areas have changed

To: $u
From: Anti Spam <spam@your-domain.com>
Subject: Spam Filtering is Active

To forward a spam into the system, please use the 'Forward' button in whatever email client you are using and send the spam to spam-$u.
It is not necessary to provide an explanation of the message, as it will not be opened by a human, but processed by the software.

http://your-domain.com/cgi-bin/dspam.cgi

[edit] DSPAM with vpopmail

Now lets setup DSPAM to use mysql.

# cd /etc/mail/dspam
# nano -w mysql.data

Change the file so it looks like this

/var/run/mysqld/mysqld.sock

vpopmail
vpopmail-password
dspam
true

replace vpopmail-password with the password you selected when creating your vpopmail setup.

Now we need to add vpopmail as a trusted user for dspam

# cd /etc/mail/dspam
# nano -w dspam.conf

User the Trsuted Users section add another trust for vpopmail

Trust vpopmail

Now we have to change permission on dspam to vpopmail for it to work.

# cd /etc/mail
# chown vpopmail:vpopmail -R dspam
# cd /var/spool/
# chown vpopmail:vpopmail -R dspam

So far .. so good.

now we need to give vpopmail ownership of the dspam binary

# cd /usr/bin
# chown vpopmail:vpopmail dspam

[edit] qmail, dspam and vpopmail

Here we come to the interesting part of the DSPAM installation.

What we are going to do is setup a .qmail-default file for each domain. just add the following into the .qmail-default of your domain

| /usr/bin/dspam --mode=teft --deliver=innocent --feature=noise --tokenizer=chained --user $EXT@$USER --stdout | /var/vpopmail/bin/vdelivermail '' bounce-no-mailbox

This send all mail through dspam and deliver to your user

For each user you will have to setup a .qmail-<user>-spam file. Here is an example

|/usr/bin/dspam --user user@yourdomain.dom --mode=teft --class=spam --source=error

By adding the .qmail-<user> file to my mail server, ALL email was lost for that user with these settings. ??? Any better?

[edit] DSPAM Web user interface

This is where the fun really starts.

We're going to start off by installing dspam-web

# emerge dspam-web

This will install dspam-web into /usr/share/webapps/dspam-web/

For dspam webui to work correctly you need to run it in suexec. suexec is a cow to get working and personally I want to meet the guy who wrote the specifications for this and kick him in the balls.

now to get it working.

First off lets create a home for the webui

# mkdir /var/www/localhost/virtual/your-domain.com

Next we have to setup apache to know where it is.

# nano -w /etc/apache2/vhosts.d/00_default_vhost.conf

Comment out everything then add the following

NameVirtualHost *:80
<VirtualHost *:80>
  SuexecUserGroup vpopmail vpopmail
  ServerAdmin webmaster@your-domain.com
  DocumentRoot /var/www/localhost/virtual/your-domain.com
  ServerName your-domain.com
  LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" comonvhost
  ErrorLog logs/your-domain.com-error_log
  CustomLog logs/your-domain.com-access_log common
  <Directory /var/www/localhost/virtual/your-domain.com>
    Options Indexes FollowSymLinks
  </Directory>
  <Directory /var/www/localhost/virtual/your-domain.com/cgi-bin>
    deny from all
    Options ExecCGI
    AllowOverride AuthConfig
    Order deny,allow
  </Directory>
  ScriptAlias /cgi-bin/ /var/www/localhost/virtual/your-domain.com/cgi-bin/
</VirtualHost>

Replace your directory and domain as you see fit

now lets move the content in /usr/share to /var/www so we can get the virtual host working.

# cp -r /usr/share/webapps/dspam-web/3.6.1/htdocs/* /var/www/localhost/virtual/your-domain.com
# mkdir /var/www/localhost/virtual/your-domain.com/cgi-bin
# cp -r /usr/share/webapps/dspam-web/3.6.1/hostroot/cgi-bin/* /var/www/localhost/virtual/your-domain.com/cgi-bin/

Now all the files are where they should be. Only one more thing to do. Permissions

# chown vpopmail:vpopmail -R /var/www/localhost/virtual/your-domain/

Now lets setup the dspam.cgi

# cd /var/www/localhost/virtual/your-domain.com/cgi-bin/
# nano -w configure.pl

Change $CONFIG{'LOCAL_DOMAIN'} to your domain

Remove the slash from WEB_ROOT. It should look like this

$CONFIG{'WEB_ROOT'}     = "";

hash out this line

$CONFIG{'AUTODETECT'} = 1;

Unhash the following lines

#$CONFIG{'AUTODETECT'} = 0;
#$CONFIG{'LARGE_SCALE'} = 0;
#$CONFIG{'DOMAIN_SCALE'} = 0;
#$CONFIG{'PREFERENCES_EXTENSION'} = 0;

If you set large-domain in the use settings for dspam change LARGE_SCALE=1, otherwise change DOMAIN_SCALE=1

unhashed it should look like this (without large-domain support)

$CONFIG{'AUTODETECT'} = 0;
$CONFIG{'LARGE_SCALE'} = 0;
$CONFIG{'DOMAIN_SCALE'} = 1;
$CONFIG{'PREFERENCES_EXTENSION'} = 0;

Next we setup the user authentication

# nano -w .htaccess

AuthType Basic
AuthUserFile /etc/apache2/htpasswd/dspam
AuthName "Dspam"
require valid-user
satisfy any

Now lets create the password file

# mkdir /etc/apache2/htpasswd/
# htpasswd2 -c /etc/apache2/htpasswd/dspam user@your-domain.com

This will create the password file for use with the dspam webui. For any additional users just run

# htpasswd2 /etc/apache2/htpasswd/dspam user@your-domain.com

You will now be able to log into your website and access the dspam webui.

[edit] Using mod_auth_imap for htaccess

Instead of using passwd files and htaccess we can use imap authentication to verify a user for dspam. To do this we are going to use mod_auth_imap. first grab the source from http://ben.brillat.net/projects/mod_auth_imap/ into a tmp directory.

# wget "http://ben.brillat.net/files/projects/mod_auth_imap2/mod_auth_imap2-current.tar.gz"

and untar

# tar -zxvf mod_auth_imap2-current.tar.gz

we have to patch the current version for dovecot. you can get the patch from http://www.geekware.co.za/henti/dovecot_imap.patch

# wget http://www.geekware.co.za/henti/dovecot_imap.patch

--- mod_auth_imap.c~  2005-03-04 23:11:08.000000000 +0200
+++ mod_auth_imap.c 2005-11-07 19:08:20.000000000 +0200
@@ -175,6 +175,8 @@
     }

     if (strncmp(result,"A001 OK CAPABILITY",18) != 0 &&
+       // courier returns "A001 OK Capability completed."
+       strncmp(result,"A001 OK Capability completed.",18) != 0 &&
         // Cyrus returns "A001 OK Completed"
         strncmp(result,"A001 OK Completed",17) != 0) {
         ap_log_rerror(APLOG_MARK,APLOG_WARNING|APLOG_NOERRNO,0,r,"mod_auth_imap: Server does not support imap CAPABILITY.");

Lets patch the source.

# cd mod_auth_imap2-2.0.2
# patch -p0 < ../dovecot_imap.patch

A sucessfull patch will return

patching file mod_auth_imap.c

Now lets compile the module

# apxs2 -i -a -c mod_auth_imap.c

Once compiled the module will be installed in /usr/lib/apache2/modules/

Now we need to add the module to apache for loading. We need to create a module.d file for this

# vi /etc/apache2/modules.d/99_mod_auth_imap.conf

# vim: ft=apache sw=4 ts=4
# Load the module first
LoadModule auth_imap_module                                             modules/mod_auth_imap.so

Lets backup the pam based .htaccess and create a new .htaccess file.

# mv /var/www/localhost/virtual/your-domain.com/cgi-bin/.htaccess /var/www/localhost/virtual/your-domain.com/cgi-bin/.htaccess.pam

# vi /var/www/localhost/virtual/your-domain.com/cgi-bin/.htaccess

Auth_IMAP_Enabled on
AuthName "your-domain.com"
AuthType Basic
Require valid-user
Auth_IMAP_Authoritative on
Auth_IMAP_Server localhost
Auth_IMAP_Port 143
Auth_IMAP_Log on
satisfy any

restart apache

# /etc/init.d/apache2 restart

you'll now be able to authenticate on dspam using imap aithentication.

[edit] Final touches

Now for the final touch. Currently your mail comes in to qmail/vpopmail then gets passed through dspam and delivered to your mailbox. If it is spam it gets routed to dspam quarantine. With the webui you can click on "Deliver Checked" to pass the mail as non-spam and deliver to your mailbox where it should be, and here things break. To fix it is actually very simple once you know how.

Here is what you do

# nano -w /var/vpopmail/bin/dspamdel.sh

paste this into the new file (updated)

#!/bin/bash
# "Reading" variables.

if [ "$1" == "-d" ]; then
  USER=`echo $2 | sed 's/@/ /g' | awk '{print $1}'`
  DOMAIN=`echo $2 | sed 's/@/ /g' | awk '{print $2}'`
  EXT=$USER HOST=$DOMAIN /var/vpopmail/bin/vdelivermail '' $2

else
  USER=`echo $1 | sed 's/@/ /g' | awk '{print $1}'`
  DOMAIN=`echo $1 | sed 's/@/ /g' | awk '{print $2}'`
  EXT=$USER HOST=$DOMAIN /var/vpopmail/bin/vdelivermail '' $1
fi

UPDATE : This now works while being called with or without -d. Once again I'm not 100% sure how dspam calls anything. The support and documentation is really not that realible. anyway it works better for me now, so give it a try. For those interesed this script it's basically a replacement for procmail. Since procmail -d cannot deal with virtual users in vpopmail it cannot be used for virtual delivery.

Set permissions

# chmod guo+x /var/vpopmail/bin/dspamdel.sh

Now lets tell dspam about it

# nano -w /etc/mail/dspam/dspam.conf

Change your TrustedDeliveryAgent to

TrustedDeliveryAgent "/var/vpopmail/bin/dspamdel.sh"

All done. Now your mail will be passed back to your mailbox where it should be.