Configuring the Sender Rewriting Scheme (SRS) on qmail

• open-spf.org
• libsrs2 site
• Marcelo Coelho's patch for qmail

SPF "breaks" email forwarding. SRS is a way to fix it. SRS is a simple way for forwarding MTAs to rewrite the sender address.

Configuration

Configure srsfilter, so that it will be called when an email for the srs user is received:

echo "| /var/qmail/bin/srsfilter" > /var/qmail/alias/.qmail-srs-default

Then create and configure a virtual domain to be used exclusively for SRS purposes. Be aware that this virtual domain should not be created by the usual vadddomain program, as it exists just to run srsfilter via the alias/.qmail-srs-default account that we created before and its definition is different from the vpopmail's virtual domains.

SRSDOMAIN=srs.mydomain.tld
echo $SRSDOMAIN:srs >> /var/qmail/control/virtualdomains

Refer to the Life With Qmail bible to understand the logic behind, expecially for what virtual domains, aliases, .qmail and extensions addresses  are concerned. An explanation is also provided below in the testing section.

Add srs.mydomain.tld to rcpthosts so that qmail-smtpd will accept mails for that domain. Do not add it to control/locals otherwise the virtualdomains file will be ignored and srsfilter will not be run.

echo $SRSDOMAIN >> /var/qmail/control/rcpthosts

Add srs.mydomain.tld in the srs_domain control file, so that srsfilter will use it in the rewritten address for all virtual hosts. Let's also create the srs_secret file, as well. It is a random string to generate and check SRS addresses.

echo $SRSDOMAIN > /var/qmail/control/srs_domain
echo "xxxxxxxxxxxxxxxxxxxxxx" > /var/qmail/control/srs_secrets

These are the only mandatory settings; look at the links above to have informations about all the other configuration parameters.

Of course we have to provide an MX record and also an SPF record like this to the newly created srs_domain in our DNS:

srs.mydomain.tld. IN TXT "v=spf1 a mx -all"

We should have already created an SPF record for the control/me domain as well. If not, let's do it now.

We can now test our SRS system.

Testing

We are going to test the ability of our SRS system to rewrite the addresses in case of a broken forwarder. Let's use a gmail.com account as the original sender, as gmail is known to be a very restrictive provider.

In what follows mydomain.tld is a virtual domain in the same server where we have configured the SRS stuff, srs.mydomain.tld is the SRS domain defined in control/srs_domain, sender@gmail.com is the original sender, while fake@remotedomain.tld is the account where the forwarder has to forward the message. Of course it will bounce back and our MTA has to perform the rewriting and notify the original sender without breaking the SPF.

Create a forwarder on our server, so that all the messages for srstest@mydomain.tld will be forwarded to fake@remotedomain.tld:

echo "&fake@remotedomain.tld" > ~vpopmail/domains/mydomain.tld/.qmail-srstest
chown vpopmail:vchkpw ~vpopmail/domains/mydomain.tld/.qmail-srstest
chmod 600 ~vpopmail/domains/mydomain.tld/.qmail-srstest

In the remotedomain.tld server, disable chkuser in your run file (comment out CHKUSER_START=ALWAYS) and set the .qmail-default file of remotedomain.tld so that it bounces the messages for not existent mailboxes

|~vpopmail/bin/vdelivermail '' bounce-no-mailbox

Since fake@remotedomain.tld is a fake recipient address, remotedomain.tld will bounce back any message to our MTA, which will inform sender@gmail.com after the rewriting.

Let's see the journey of the message around the net

This is the same as seen from our qmail logs.

1. qmail-smtpd receives and accepts the message from the original sender

2023-06-20 22:54:52.607641500 tcpserver: pid 16574 from 209.85.128.49 
2023-06-20 22:54:52.627258500 tcpserver: ok 16574 smtp.mydomain.tld:10.0.0.4:25 mail-wm1-f49.google.com:209.85.128.49::44523 
2023-06-20 22:55:13.058900500 qlogenvelope: result=accepted code=250 reason=rcptto detail=chkuser  
       helo=mail-wm1-f49.google.com  
       mailfrom=sender@gmail.com  
       rcptto=srstest@mydomain.tld  <---- this is the forwarder
       ................ 
       qp= pid=16574

2. qmail-send passes the message to the forwarder srstest@mydomain.tld

2023-06-20 22:55:19.493086500 info msg 32560286: bytes 3377 
       from <sender@gmail.com>
       qp 16615 uid 89 
2023-06-20 22:55:19.493086500 starting delivery 60: msg 32560286 
       to local mydomain.tld-srstest@mydomain.tld <---- qmail converts srstest@mydomain.tld to mydomain.tld-srstest@mydomain.tld and treats the result as mydomain.tld is a local address
2023-06-20 22:55:19.493087500 status: local 1/10 remote 0/20 
2023-06-20 22:55:19.495105500 delivery 60: success: did_0+1+0/qp_16617/

Let's see what's happening more in detail.

1. qmail opens the file control/virtualdomains looking for a line related to the recipient domain, i.e. mydomain.tld. Here is the content of that line:

   mydomain.tld:mydomain.tld

   Be aware that the two fields before and after the colon : are not a duplicate, because they have a different meaning. The one on the left is the recipient address (even though the user@ part is often omitted), while the one on the right is the user whose .qmail file will have to handle the delivery for that domain.
2. qmail prepends the field on the right of the colon : to the recipient address, which will be mydomain.tld-srstest@mydomain.tld, and treats mydomain.tld as a local address to deliver the message to.
3. qmail-local searches the user mydomain.tld (technically it's a simple user even before a domain) in the users/assign file (actually its compiled version, which is users/cdb) and it finds something like:

   +mydomain.tld-:mydomain.tld:89:89:/home/vpopmail/domains/mydomain.tld:-::

   It retrieves the home directory, which is /home/vpopmail/domains/mydomain.tld in this example.
4. Then qmail-local opens the home directory to look for the file srstest/.qmail, which will have to handle the delivery. If it doesn't find that file, it will look for the files ~mydomain.tld/.qmail-srstest and ~mydomain.tld/.qmail-default in this order.
5. In our case the file .qmail-srstest does exist and, being this the forwarder created earlier, it directs qmail-send in order to forward the message to someone else.

3. The address of the original sender is rewrited to SRS0=jiQ3=CI=gmail.com=sender@srs.mydomain.tld and the message sent to the remote server by qmail-remote

2023-06-20 22:55:19.495905500 info msg 32560301: bytes 3491 
       from <SRS0=jiQ3=CI=gmail.com=sender@srs.mydomain.tld>   <----- SPF check would be valid
       qp 16617 uid 89 
2023-06-20 22:55:19.495918500 starting delivery 61: msg 32560301 
       to remote fake@remotedomain.tld 
2023-06-20 22:55:19.495920500 status: local 0/10 remote 1/20 
2023-06-20 22:55:23.822750500 delivery 61: success: 
       <From:SRS0=jiQ3=CI=gmail.com=sender@srs.domain.tld_To:fake@remotedomain.tld>_<remoteip>_accepted_message./Remote_host_said:_25> 

4. qmail-smtpd receives the bounce (actually if the remote server has the SRS feature it will be able to bounce the message towards gmail.com, but not in this case).

2023-06-20 22:55:24.049537500 tcpserver: pid 16687 from <remoteip>
2023-06-20 22:55:24.049945500 tcpserver: ok 16687 smtp.mydomain.tld:10.0.0.4:25 <remoteip>.<remotehost>:<remoteip>::37391 
2023-06-20 22:55:44.826126500 qlogenvelope: result=accepted code=250 reason=rcptto detail=chkuser  
       helo=<remotehelo> 
       mailfrom=        <------ null sender, as it's a system msg
       rcptto=SRS0=jiQ3=CI=gmail.com=sender@srs.mydomain.tld  <------- the recipient has the srs_domain
       ................. 
       qp= pid=16687 

5. qmail-send sends the bounce to the local address SRS0=jiQ3=CI=gmail.com=sender@srs.mydomain.tld, which has the SRS domain that we bound to srsfilter

2023-06-20 22:55:51.265166500 info msg 32560286: bytes 5688 
       from <> <------- null sender, SPF is ok
       qp 16716 uid 89
2023-06-20 22:55:51.265166500 starting delivery 62: msg 32560286 
       to local srs-SRS0=jiQ3=CI=gmail.com=sender@srs.mydomain.tld <------- the srs prepend will trigger the default srs account, which in turn will run srsfilter
2023-06-20 22:55:51.265167500 status: local 1/10 remote 0/20 
2023-06-20 22:55:51.270712500 delivery 62: 
       success: srsfilter:_qp_16720/did_0+0+1/   <------- srsfilter in action... he knows what to do with that address

Now a few more details concerning what's happening behind the scene:

1. qmail receives a message for a user of the domain srs.mydomain.tld and it deliver it locally, as that domain is listed in rcpthosts.
2. qmail searches a line with the field srs.mydomain.tld on the left of the colon : in the virtualdomains file:

   srs.mydomain.tld:srs

3. The field on the right side of the colon : will be prepended to the local recipient address, which will be srs-SRS0=jiQ3=CI=gmail.com=sender@srs.mydomain.tld, with the srs prefix. This prefix represents the user whose .qmail file has to handle the delivery.
4. qmail-local opens the users/assign file (actually its compiled version users/cdb) looking for a line holding srs in the first field, but it doesn't find it, because we have no srs users (this is why we avoided to use vadddomain to create the SRS alias).
5. Since the user srs doesn't exist, it will look for an alias like alias/.qmail-srs, with no luck. Finally it finds the alias file alias/.qmail-srs-default, which will be the .qmail file that will handle the delivery.
6. The alias/.qmail-srs-default file contains the instructions to run srsfilter.
7. srsfilter is capable to "disassemble" the SRS0=jiQ3=CI=gmail.com=sender@srs.mydomain.tld address in order to retrieve the recipient address, which will be sender@gmail.com.

6. qmail-send sends the bounce to the original sender, the mailfrom is the null sender <>

2023-06-20 22:55:51.271507500 info msg 32560301: bytes 5707 
       from <>      <------ null sender
       qp 16720 uid 1000 
2023-06-20 22:55:51.271507500 starting delivery 63: msg 32560301 
       to remote sender@gmail.com 
2023-06-20 22:55:51.271508500 status: local 0/10 remote 1/20 
2023-06-20 22:55:51.915441500 delivery 63: 
       success: <From:_To:sender@gmail.com>_74.125.133.26_accepted_message./Remote_host_said:_250_2.0.0_OK__1687294551_f7-20020adff8c700000>

and gmail will show a successfull SPF check. Here his the important part of the header as seen from sender@gmail.com:

Delivered-To: sender@gmail.com
[...]
Return-Path: <>
Received: from smtp.mydomain.tld (smtp.mydomain.tld. [94.23.219.84])
        by mx.google.com with ESMTPS id v28-20020a5d591c000000b0030e57d7da1dsi2291075wrd.363.2023.06.21.00.15.28
        for <sender@gmail.com>
[...]
Received-SPF: pass (google.com: domain of postmaster@smtp.mydomain.tld designates <smtp.mydomain.tld IP> as permitted sender) client-ip=<smtp.mydomain.tld IP>;
Authentication-Results: mx.google.com;
       dkim=temperror (no key for signature) header.i=@smtp.remotedomain.tld header.s=default header.b=M9b4zh72;
       spf=pass (google.com: domain of postmaster@smtp.mydomain.tld designates <smtp.mydomain.tld IP> as permitted sender) smtp.helo=smtp.mydomain.tld
smtp.helo=smtp.mydomain.tld
[...]
Date: 20 Jun 2023 22:55:59 +0200
From: postmaster@smtp.remotedomain.tld <--- <bouncefrom>@<bouncehost>
To: sender@gmail.com
Subject: failure notice

Hi. This is the qmail-send program at smtp.remotedomain.tld. <--- <bouncehost>
I'm afraid I wasn't able to deliver your message to the following addresses.
This is a permanent error; I've given up. Sorry it didn't work out.

<fake@remotedomain.tld>:
Sorry, no mailbox here by that name. (#5.1.1)

--- Below this line is a copy of the message.

Return-Path: <SRS0=YBjh=CJ=gmail.com=sender@srs.mydomain.tld>

Note that the SPF check was done against smtp.mydomain.tld, i.e. the control/me domain, because the message to the original sender was sent with the null sender. That domain was retrieved from the HELO. This is the reason why we have to define the SPF also for the control/me domain.

The bounce that we are forwarding has been sent on behalf of postmaster@smtp.remotedomain.tld, which is derived from control files as bouncefrom and bouncehost.

echo "xxxxxxxxxxxxxxxxxxxxxx" > /var/qmail/control/srs_secret

echo "xxxxxxxxxxxxxxxxxxxxxx" > /var/qmail/control/srs_secrets