Greylisting for qmail

Greylisting is a method of defending e-mail users against spam. A mail transfer agent (MTA) using greylisting will "temporarily reject" any email from a sender it does not recognize. If the mail is legitimate, the originating server will try again after a delay, and if sufficient time has elapsed, the email will be accepted.

While greylisting is not effective as in the past, it still cut a certain fraction of the total spam.

Changelog

• Apr 18, 2023
  Bai Borko showed a different approach in purging the database of jgreylist and posted his script here. Basically, instead of deleting the old records his script leaves in the database the records with trusted IPs of the last 30 days based on the qmail-smtpd logs. Have a look!

qmail-spp greylisting plugin

I introduce here how greylisting can be implemented on qmail by means of another qmail-spp plugin, which saves the data in MySQL. Having the data in MySQL is useful to measure how much spam is blocked by greylisting.

• More info here
• Author: Manuel Mausz

Download

cd /usr/local/src
wget https://manuel.mausz.at/coding/qmail-spp/greylisting/greylisting-0.5.tgz
tar xzf greylisting-0.5.tgz
cd greylisting-0.5

Compile and install

gcc -std=c99 -o /var/qmail/plugins/greylisting greylisting.c -I/usr/include -I/usr/include/mysql -I/usr/local/include/mysql -L/usr/lib/mysql -L/usr/lib64/mysql -L/usr/local/lib/mysql -L/usr/local/lib64/mysql -lmysqlclient
strip greylisting

Install the configuration files

cp mysql.cnf /var/qmail/control
chown vpopmail:vchkpw /var/qmail/control/mysql.cnf
chmod 600 /var/qmail/control/mysql.cnf
cp greylisting.config /var/qmail/control/greylisting
chown root:root /var/qmail/control/greylisting
chmod 644 /var/qmail/control/greylisting

Install the mysql database and create the mysql user:

> mysql -u root -p

CREATE USER 'greylisting'@'localhost' IDENTIFIED BY '***';
GRANT USAGE ON *.* TO 'greylisting'@'localhost' REQUIRE NONE WITH MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0;
CREATE DATABASE IF NOT EXISTS greylisting;
GRANT ALL PRIVILEGES ON greylisting.* TO 'greylisting'@'localhost';

Now install the database schema from the greylisting.sql file (note that in the same file you have examples for whitelist and blacklist queries):

mysql < greylisting.sql -D greylisting -u greylisting -p

Copy your mysql access parameters into your mysql.cnf file. For example:

cat > /var/qmail/control/mysql.cnf << __EOF__ 
[client] 
#socket=/run/mysqld/mysqld.sock 
#host=10.1.2.3   
user=greylisting 
password=secret
database=greylisting 
__EOF__

If the connection has to be done against a local mysql host, it will be via the mysql socket. If, as in my case, you have mysql in a different host, uncomment the "host" line.

Now define the "greylist" in the smtpplugin file:

# smtpplugins sample file 

# other lines here

[rcpt]
plugins/ifauthskip
plugins/greylisting

To enable greylisting you have to export this variable in your tcpserver environment. So put this in your qmail-smtpd run file

export GREYLISTING=""

or enable a tcprule in your tcp.smtp, for example:

:allow,GREYLISTING=""

Be aware that greylisting is never enabled for RELAYCLIENTS.

Finally, adjust the greylisting parameters in the /var/qmail/control/greylisting file

cat > /var/qmail/control/greylisting << __EOF__ 
mysql_default_file=control/mysql.cnf 
block_expire=2 
record_expire=2000 
record_expire_good=36 
loglevel=4 
__EOF__

block_expire=2 means that the IP will be blocked for 2 minutes, while record_expire=2000 means that after 2000 minutes it will be greylisted again.

Now install the program which will purge the database in your cronjob

cp greylisting_cleanup.sh /usr/local/sbin/greylisting_cleanup.sh
chmod +x /usr/local/sbin/greylisting_cleanup.sh

Here is a crontab example

# greylisting 
0 2 * * * /usr/local/sbin/greylisting_cleanup.sh >> /var/log/cron

If you want to use greylisting also in your submission port, it would be a good idea to switch off it in case the client have passed the authentication. In order to do this install the "skipifauth" plugin of the same author

wget https://notes.sagredo.eu/files/qmail/patches/qmail-spp/plugins/ifauthskip.c
cc -o /var/qmail/plugins/ifauthskip ifauthskip.c

and install it before greylisting in control/smtpplugin

[rcpt]
plugins/ifauthskip
plugins/greylisting

jgreylist

This is a greylist program by John Simpson. It is very good and honestly I don't know if I have to suggest this one or the previous one. It stores the data on files (have a look to the page of the author for details). He ships both a Perl and a C version of the program.

Here is how to install and configure the C program:

cd /usr/local/src
wget https://notes.sagredo.eu/files/qmail/patches/greylisting/jms/jgreylist.c
wget https://notes.sagredo.eu/files/qmail/patches/greylisting/jms/jgreylist-clean

Compile and install

cc -o /var/qmail/bin/jgreylist jgreylist.c
chmod 0750 /var/qmail/bin/jgreylist
cp jgreylist-clean /usr/local/sbin/jgreylist-clean
chmod +x /usr/local/sbin/jgreylist-clean
chown root:root /usr/local/sbin/jgreylist-clean

Define your jgreylist directory (where the data will be saved) and let qmail-smtpd to write into it

mkdir -m 0700 /var/qmail/jgreylist
chown vpopmail:vchkpw /var/qmail/jgreylist

add these variables to your qmail-smtpd run file

export JGREYLIST_DIR="/var/qmail/jgreylist" 
export JGREYLIST_LOG_SMTP=1

finally execute jgrelist before qmail-smtpd, for example

exec /usr/local/bin/softlimit -m "$SOFTLIMIT" \
/usr/local/bin/tcpserver -4 -v -R -l "$LOCAL" \
-x /home/vpopmail/etc/tcp.smtp.cdb -c "$MAXSMTPD" \
-u "$QMAILDUID" -g "$NOFILESGID" 0 25 \
/var/qmail/bin/jgreylist \
/var/qmail/bin/qmail-smtpd 2>&1

Install the "clean" program in your cronjob

# jgreylist
0 2 * * * /usr/local/sbin/jgreylist-clean >> /var/log/cron

Now restart qmail.

Bai Borko has a different approach in purging the database and posted his script here. Basically, instead of deleting the old records his script leaves in the database the records with trusted IPs of the last 30 days based on the qmail-smtpd logs. Have a look!

[rcpt]
plugins/skipifauth <-- must be /plugins/ifauthskip
plugins/greylisting

0 3 * * * /usr/local/bin/jgreylist-create-db.sh

#!/bin/bash

SMTPDLOG='/var/log/qmail/smtpd/current'
BACKUPDIR='/var/log/qmail/backup'
JGRAYDIR='/var/qmail/jgreylist'
FIELD=`echo ${JGRAYDIR}|tr -dc '/'|wc -c`
FIELD=$((FIELD+3))
TMPLIST="$(mktemp)"
TMPLIST_TMP="$(mktemp)"
DAYS=30
INTNET='192.168.0'

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/qmail/bin

grlist() {
  grep "result=accepted code=250" $1|tai64nlocal| cut -d ' ' -f1-2,20|tr '=' ' '|grep -v ${INTNET}|awk -F '.' '{print $1" "$2"."$3"."$4}'|cut -d ' ' -f1,2,5 >>$TMPLIST
}

for i in `find $BACKUPDIR -name "qmail-smtpd.*" -mtime -${DAYS}`; do 
  grlist $i
done

grlist $SMTPDLOG

cat $TMPLIST |sort -k 3 -u > $TMPLIST_TMP
mv $TMPLIST_TMP $TMPLIST
umask 077

while read line; do
  MTIME=`echo "$line"|tr -d '-'|awk -F: '{print $1$2"."$3}'|awk '{print $1$2}'`
  TMPSTR=''
  for NUM in `echo "$line"|cut -d ' ' -f3|tr '.' ' '`;do
    if [[ ${#NUM} -eq 1 ]] ; then
    NUM="00${NUM}"
  elif [[ ${#NUM} -eq 2 ]] ; then
    NUM="0${NUM}"
  fi
  TMPSTR=${TMPSTR}${NUM}/
done

FILE=`echo ${JGRAYDIR}.tmp/${TMPSTR} |rev |cut -c2- |rev`
DIR=`echo ${FILE} |cut -d / -f -${FIELD} -` 
#Make dir if not exist
if [ ! -d "$DIR" ]; then
  mkdir -p "$DIR"
fi

touch -a -m -t $MTIME $FILE
done < $TMPLIST
chown -R vpopmail:vchkpw "${JGRAYDIR}.tmp"
mv ${JGRAYDIR} "${JGRAYDIR}.4del" 
mv "${JGRAYDIR}.tmp" ${JGRAYDIR}
rm -fr "${JGRAYDIR}.4del" $TMPLIST

exec /usr/local/bin/softlimit -m "$SOFTLIMIT" \
/usr/local/bin/tcpserver -4 -v -R -l "$LOCAL" \
-x /home/vpopmail/etc/tcp.smtp.cdb -c "$MAXSMTPD" \
-u "$QMAILDUID" -g "$NOFILESGID" 0 25 \
/var/qmail/bin/jgreylist \
/var/qmail/bin/qmail-smtpd 2>&1

@400000006202eb491916bcb4 jgreylist[2379920]: 85.202.169.20: OK known
@400000006202eb581830f47c GREETDELAY from 85.202.169.20: client disconnected

@400000006203220e21823964 jgreylist[2406984]: 193.56.29.119: OK known
@400000006203221d24897e24 qmail-smtpd: read failed (hang up before quit cmd): (null) from 193.56.29.119 to (null) helo unknown

10 * * * * /usr/local/bin/jgreylist-clean-spammers.sh > /dev/null 2>&1


#!/bin/bash

SMTPDLOG='/var/log/qmail/smtpd/current'
JGRAYDIR='/var/qmail/jgreylist'
FIELD=`echo ${JGRAYDIR}|tr -dc '/'|wc -c`
FIELD=$((FIELD+3))
for i in ` grep -E "GREETDELAY .* client disconnected$|qmail-smtpd: read failed .* to \(null\) helo unknown$" $SMTPDLOG|grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}"|cut -d . -f1-3|uniq`; do
TMPSTR=''
for NUM in `echo $i|tr '.' ' '`;do
if [[ ${#NUM} -eq 1 ]] ; then
NUM="00${NUM}"
elif [[ ${#NUM} -eq 2 ]] ; then
NUM="0${NUM}"
fi
TMPSTR=${TMPSTR}${NUM}/
done
FILE=`echo ${JGRAYDIR}/${TMPSTR} |rev |cut -c2- |rev`
DIR=`echo ${FILE} |cut -d / -f -${FIELD} -`
if [ -f "$FILE" ]; then
rm -f "$FILE"
fi

if [ -d "$DIR" ] && [ -z "$(ls -A $DIR)" ]; then
rmdir $DIR
fi
done