Installing a Let's Encrypt certificate for your qmail, dovecot and apache servers

Changelog

• Aug 6, 2023
  The certificates installation is now based on dehydrated. The previous documentation based on certbot will be left as is at the bottom of this page, but it won't be updated anymore.
• May 18, 2023
  added the option --key-type rsa to the certbot command, to avoid that certbot will silently default to ECDSA the private key format, which results not understandable by my openssl-1.1. In this way the format of the private key will be RSA. More info here.

• Let's Encrypt site

To enable HTTPS on your website, you need to get a certificate (a type of file) from a Certificate Authority (CA). Let’s Encrypt is a CA. In order to get a certificate for your website’s domain from Let’s Encrypt, you have to demonstrate control over the domain. With Let’s Encrypt, you do this using software that uses the ACME protocol which typically runs on your web host.

Upgrading notes

In case you have to replace a working installation of certificates based on certbot, you don't need to take any precautions as they won't be overwritten by the new ones generated by dehydrated. Therefore it is possible to run our tests on the production server itself. Of course it will be good to use a test domain while tinkering with Apache.

dehydrated installation

• github dehydrated page
• AlienBob article  (for Slackware users, but the informations are generic and suitable for all Linux distributions) 

Dehydrated is a client for signing certificates with an ACME-server (e.g. Let's Encrypt) implemented as a relatively simple (zsh-compatible) bash-script. This client supports both ACME v1 and the new ACME v2 including support for wildcard certificates!

dehydrated is a simple shell program that requires no dependencies, unlike the official certbot program, which needs a very long set of python libraries. Maintaining all this volume of programs proved to be quite time expensive, which is why I decided to switch to dehydrated.

Since your distribution is likely to provide a dehydrated package, you can install it from there. Otherwise download and install the program from the github site.

Here is a short example of a manual installation in /usr/local (modify as needed):

VERSION=0.7.1
wget https://github.com/dehydrated-io/dehydrated/releases/download/v${VERSION}/dehydrated-${VERSION}.tar.gz
tar xzf dehydrated-${VERSION}.tar.gz
cd dehydrated-${VERSION}
chown -R root:root .
cp dehydrated /usr/local/bin/
mkdir -p /usr/local/man/man1 /usr/local/share/doc/dehydrated-${VERSION}/docs/examples
cp docs/man/dehydrated.1 /usr/local/man/man1/
mkdir -p /etc/dehydrated
cp docs/examples/config /etc/dehydrated/config.new # .new just to avoid to overwrite the existing config. Rename to config
cp CHANGELOG* LICENSE* README* /usr/local/share/doc/dehydrated-${VERSION}
cp docs/*.md /usr/local/share/doc/dehydrated-${VERSION}/docs
cp docs/examples/{domains.txt,hook.sh} /usr/local/share/doc/dehydrated-${VERSION}/docs/examples

Configuration

• Config file example

Go to the /etc/dehydrated folder, where we have the configuration files and where the certificates will be installed. These are my changes to the config file:

CA="letsencrypt-test"
CHALLENGETYPE="http-01"
WELLKNOWN="/var/www/dehydrated"
KEY_ALGO=rsa
CONTACT_EMAIL=postmaster@mydomain.tld

• CA contains the URL to the Let's Encrypt server, or a parameter (in this case letsencrypt-test) that tells the program which URL to use. While testing let's leave letsencrypt-test so that the server will not be clogged. We would be banned accordingly if we exceed the limit of invalid attempts. Once everything is ok we will change it to CA="letsencrypt". More info here.
• CHALLENGETYPE="http-01" As we know, we need to prove to have control over the domains we want to get a certificate for. To do this, access will be tested to a verification file placed on our web server, in a folder where Apache must have access for each domain to be certified.
  There are also other CHALLENGETYPEs. For example dns-01 is suitable for certifying domains with wildcards such as *.sagredo.eu, but it is necessary to have programs to automatically handle the DNS server. This part is therefore not covered here.
• WELLKNOWN="/var/www/dehydrated" is the directory where dehydrated will install the ‘challenge-tokens’, which will be retrieved via http in order to validate the domain.
• KEY_ALGO=rsa specifies the algorithm to be used for the public key.

dehydrated is able to launch a program of our choice after the successful creation of the certificate (HOOK="${BASEDIR}/hook.sh"). We will dispense with this opportunity and manually create a file that will handle the post-installation tasks. In fact, I noticed that, during the creation of the certificate, dehydrated launches hook.sh several times, which is not satisfactory at all.

To use dehydrated we need to register an account:

​# dehydrated --register --accept-terms
# INFO: Using main config file /etc/dehydrated/config 
+ Generating account key... 
+ Registering account key with ACME server... 
+ Fetching account URL... 
+ Done!

Account information is saved in the /etc/dehydrated/accounts directory.

Domains configuration

• More info here

Create a domains.txt file where we will arrange the domains to be certified separated by a blank space. Domains can be entered on separate lines. Those on the same line will refer to the same certificate, whose name will be the first domain in that line.

For example

mx.mydomain.tld
mydomain.tld www.mydomain.tld webmail.mydomain.tld otherdomain.tld www.otherdomain.tld webmail.otherdomain.tld

will create two certificates mx.mydomain.tld and mydomain.tld. In what follows the first one is the certificate for the mail server, the other one for Apache sites (webmail for instance). Of course you may want to have one single certificate for everything, especially if they live in the same host.

It is also possible to write other txt files in the /etc/dehydrated/domains.d folder.

Apache configuration

In a typical case we will have to certify many domains, some of which are dedicated to the mail server, others to simple web spaces (for example webmail or the qmailadmin control panel) each with their own VirtualHost and their own DocumentRoot folder. We will make sure to save the 'challenge-tokens' in the same directory accessible to all these domains, let's say /var/www/dehydrated, which is the one referred by the WELLKNOWN parameter in the configuration file.

First, let's create this directory:

mkdir -p /var/www/dehydrated

Since we have to include the same code multiple times (one for each VirtualHost) it is convenient to automate everything by saving it in a redirect.conf file to be imported into each VirtualHost. Let's save it to the Apache configuration folder, i.e. /etc/httpd:

cat > /etc/httpd/redirect.conf << __EOF__
<IfModule mod_rewrite.c> 
  RewriteEngine On 
  RewriteCond %{REQUEST_URI} !^/.well-known/acme-challenge 
  RewriteRule ^(.*) https://%{SERVER_NAME}\$1 [R,L] 
</IfModule> 

Alias /.well-known/acme-challenge /var/www/dehydrated
<Directory /var/www/dehydrated>
  Options None 
  AllowOverride None 
  Require all granted 
</Directory>

# Define or change \${LOGDIR} to where your server saves the logs
CustomLog "\${LOGDIR}/dehydrated_access.log" combined
ErrorLog  "\${LOGDIR}/dehydrated.f2b.log"
__EOF__

By doing so, requests for files like /.well-known/acme-challenge/* will be served in the folder /var/www/dehydrated, while the rest of the traffic will be redirected (in this example) to the https address with the same domain (SERVER_NAME). As you can see, this assumes that the Apache's mod_rewrite module has been enabled.

Again for matters of order and economy of code, it will be convenient to define some variables in the general configuration file /etc/httpd.conf (or wherever you prefer). For example, we can save there the path of the file to be included and, since we're at it, the file that we will use later to import the SSL certificates and also the folder that will host the certificates:

Define REDIRECT /etc/httpd/redirect.conf
Define SSL      /etc/httpd/ssl.conf
Define CERTDIR  /etc/dehydrated/certs

So here is a schema of the virtual host related to the domain that leads to our mail server:

<VirtualHost *:80> 
  ServerName mx.mydomain.tld
  Include ${REDIRECT}
</VirtualHost>

Certificates creation

It is now the time to create our certificates:

# dehydrated -c 
# INFO: Using main config file /etc/dehydrated/config 
Processing mx.mydomain.tld 
+ Creating new directory /etc/dehydrated/certs/mx.mydomain.tld ... 
+ Signing domains... 
+ Generating private key... 
+ Generating signing request... 
+ Requesting new certificate order from CA... 
+ Received 1 authorizations URLs from the CA 
+ Handling authorization for mx.mydomain.tld 
+ 1 pending challenge(s) 
+ Deploying challenge tokens... 
+ Responding to challenge for mx.mydomain.tld authorization... 
+ Challenge is valid! 
+ Cleaning challenge tokens... 
+ Requesting certificate... 
+ Order is processing... 
+ Checking certificate... 
+ Done! 
+ Creating fullchain.pem... 
+ Done!

We should get a similar result for each requested certificate (one for each line in domains.txt). The certificates are saved in the certs folder, whose address we stored in the CERTDIR variable. Use man or even dehydrated -h to see how to delete or revoke certificates.

Once the certificate creation procedure has been successful we can set dehydrated to connect to the production address (CA="letsencrypt" in the config file).

At this point the certificates are not yet seen neither by the web-server, nor by qmail and nor by dovecot. If you are still testing and have a working certificate in production, use a test domain for the following tests with Apache.

Apache SSL configuration

As far as Apache is concerned, let's create a file with the SSL settings to import into each VirtualHost:

cat > /etc/httpd/ssl.conf << __EOF__
SSLEngine on 
SSLCertificateFile      \${CERTDIR}/\${CERTNAME}/fullchain.pem 
SSLCertificateKeyFile   \${CERTDIR}/\${CERTNAME}/privkey.pem 
SSLCertificateChainFile \${CERTDIR}/\${CERTNAME}/fullchain.pem
__EOF__

CERTNAME is the name of the certificate, to pass before including the file, as in the example below. CERTDIR is a variable that we have already defined above in the httpd.conf file.

This is an example of VirtualHost configured to host web pages in https, for example the RoundCube webmail:

<VirtualHost *:80> 
  ServerName webmail.mydomain.tld
  ServerAlias webmail.otherdomain.tld
  Include ${REDIRECT} 
</VirtualHost> 

<VirtualHost *:443> 
  # The name of the certificate is the first domain of the line in /etc/dehydrated/domains.txt
  Define CERTNAME mydomain.tld
  # SSL cannot work until the certificate is in place. Comment it out initially
  Include ${SSL} 

  ServerName webmail.mydomain.tld
  ServerAlias webmail.otherdomain.tld

  ErrorLog  ${LOGDIR}/roundcube_error.log 
  CustomLog ${LOGDIR}/roundcube_access.log combined 

  DocumentRoot ${HTDOCS}/roundcube

  <Directory ${HTDOCS}/roundcube> 
    Require all granted 
  </Directory> 
</VirtualHost>

Certificate installation on qmail and dovecot

Prepare a script that runs dehydrated and handles the certificate installation (download). Let's save it to /usr/local/bin/my_dehydrated.sh:

cat > /usr/local/bin/my_dehydrated.sh << __EOF__
#!/bin/sh 

CERTNAME=mx.mydomain.tld 
CERTDIR=/etc/dehydrated/certs/\${CERTNAME} 
QMAILDIR=/var/qmail 
QMAILCTL=/usr/local/bin/qmailctl 
DOVECOTCTL=/usr/local/bin/dovecotctl 
APACHECTL=/usr/sbin/apachectl 
DEHYDRATED=/usr/bin/dehydrated 

# cert renewal
\$DEHYDRATED -c 

if [ \$? -eq 0 ]; then 
  # qmail cert backup
  if [ ! -d "\${QMAILDIR}/control/certs_backup" ]; then 
    mkdir -p \${QMAILDIR}/control/certs_backup 
  fi 
  echo "Setting up the cert for qmail" 
  cp -p \${QMAILDIR}/control/*.pem \${QMAILDIR}/control/certs_backup/ 
  cat \${CERTDIR}/privkey.pem \${CERTDIR}/fullchain.pem > \${QMAILDIR}/control/servercert.pem 
  chown vpopmail:vchkpw \${QMAILDIR}/control/*.pem 
  chmod o-r \${QMAILDIR}/control/*.pem
  # restart qmail
  \$QMAILCTL restart 

  # restart dovecot 
  echo "Restarting dovecot" 
  \$DOVECOTCTL stop 
  sleep 5 
  \$DOVECOTCTL start 

  # restart apache 
  echo "Restarting apache" 
  \$APACHECTL -k graceful 

  exit 0 
else exit 1 
fi
__EOF__

chmod +x /usr/local/bin/my_dehydrated.sh

As we know, the qmail certificate (control/servercert.pem file) is the result of the concatenation of the private key and the fullchain. This is done by the above script. As far as qmail is concerned, we don't have to do anything else.

Regarding Dovecot, however, it is necessary to correctly set the address of the certificates in the conf.d/10-ssl.conf  file:

ssl_cert = </etc/dehydrated/certs/mx.mydomain.tld/fullchain.pem
ssl_key = </etc/dehydrated/certs/mx.mydomain.tld/privkey.pem

Letsencrypt recommends renewing certificates automatically when they have a third of their total lifetime left. The current certificate lifetime is 90 days, that means renewing 30 days before expiration. Let's set the cronjob so that it runs the script once a month:

cat > /etc/cron.d/dehydrated << __EOF__
40 2 6 * * /usr/local/bin/my_dehydrated.sh >> /var/log/cron 2>&1
__EOF__

You can now remove the old update_tmprsadh crontab line.

Installing certbot

The documentation concerning certbot is no longer updated as it has been replaced by the one based on dehydrated. It is left intact in case it might still be of help to someone.

More info:

• https://certbot.eff.org/docs/
• https://letsencrypt.org/ 

Here is how to install and configure a valid certificate from Let's Encrypt for your qmail and dovecot servers. The installation will be done by certbot.

Certbot is part of EFF’s effort to encrypt the entire Internet. Secure communication over the Web relies on HTTPS, which requires the use of a digital certificate that lets browsers verify the identity of web servers (e.g., is that really google.com?). Web servers obtain their certificates from trusted third parties called certificate authorities (CAs). Certbot is an easy-to-use client that fetches a certificate from Let’s Encrypt—an open certificate authority launched by the EFF, Mozilla, and others—and deploys it to a web server.

Installing the certificate

It needs tons of prerequisites and a python v. 3, but there must be a package for your distribution that will do everything for you. Slackware users should refer to the slackbuild from SBO here (if you don't want to bother to manually install all the dependencies, sbotools is your friend).

The certificate will be provided once you prove to be the owner of the matched domain(s). So certbot has to install an ACME challenge in a directory of your choice and then retrieve it via http. If the challenge is successfull the certificate will be installed in /etc/letsencrypt.

Create the "webroot" dir where the ACME challenge will be stored:

mkdir -p /path/to/webroot

Now set up an apache virtual domain. In this example yourdomain.tld is the domain where you are going to do the smtp connection:

<VirtualHost *:80>
 ServerName yourdomain.tld

 DocumentRoot /path/to/webroot
 <Directory /path/to/webroot>
   Require all granted
 </Directory>
</VirtualHost>

Prepare a script to install your certs via certbot as /usr/local/bin/my_certbot.sh: 

cat > /usr/local/bin/my_certbot.sh << __EOF__
#!/bin/sh
#

CERTBOT=/usr/bin/certbot
DOMAIN=mydomain.tld

\$CERTBOT certonly \\
 --webroot \\
 --webroot-path /path/to/webroot \\
 --preferred-challenges http-01 \\
 --key-type rsa \\
 -d \${DOMAIN} \\
 --email myemail@\${DOMAIN} \\
 --renew-by-default \\
 --agree-tos \\
 --text

# qmail cert
if [ ! -d "/var/qmail/control/certs_backup" ]; then
 mkdir -p /var/qmail/control/certs_backup
fi
cp -p /var/qmail/control/*.pem /var/qmail/control/certs_backup/
cat /etc/letsencrypt/live/\${DOMAIN}/privkey.pem /etc/letsencrypt/live/\${DOMAIN}/fullchain.pem > /var/qmail/control/servercert.pem
chown vpopmail:vchkpw /var/qmail/control/*.pem 
chmod o-r /var/qmail/control/*.pem
/usr/local/bin/qmailctl restart

# dovecot cert (you have to set the path inside 10-ssl.conf accordingly)
/usr/local/bin/dovecotctl restart
__EOF__

chmod +x /usr/local/bin/my_certbot.sh

mydomain.tld will be used as the certificate's name. To better understand what certbot can do:

certbot --help all

Remember to set the x flag:

chmod +x /usr/local/bin/my_certbot.sh

Now try to run it and hopefully you'll get the certificate with no errors.

Finally set up a cronjob that renew the cert once a month (the certificate remains valid for three months):

15 2 20 * * /usr/local/bin/my_certbot.sh >> /var/log/cron

Remember to disable the old update_tmprsadh crontab line.

Configuring qmail and dovecot

As far as qmail is concerned the private key and the fullchain must be merged into a single file /var/qmail/control/servercert.pem. This is done by the above script itself.

Concerning dovecot, you just have to adjust your /usr/local/dovecot/etc/dovecot/conf.d/10-ssl.conf file as follows:

#ssl_cert = </etc/ssl/certs/dovecot.pem
#ssl_key = </etc/ssl/private/dovecot.pem
ssl_cert = </etc/letsencrypt/live/yourdomain.tld/fullchain.pem
ssl_key = </etc/letsencrypt/live/yourdomain.tld/privkey.pem

Restart qmail and dovecot to enable the new certificate.

cat > /etc/cron.d/dehydrated << __EOF__
2 40 6 * * /usr/local/bin/my_dehydrated.sh >> /var/log/cron 2>&1
__EOF__

openssl s_client -starttls smtp -crlf -connect localhost:587

openssl s_client -starttls smtp -crlf -connect MY_HOST_SERVER:25

CONNECTED(00000003)
139804790581056:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:ssl/record/ssl3_record.c:332:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 179 bytes and written 351 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

openssl s_client -starttls smtp -crlf -connect MY_HOST_SERVER:25

CONNECTED(00000003)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = MY_HOST_SERVER
verify return:1
---
Certificate chain
0 s:/CN=MY_HOST_SERVER
i:/C=US/O=Let's Encrypt/CN=R3
1 s:/C=US/O=Let's Encrypt/CN=R3
i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
2 s:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIF9DCCBNygAwIBAgISBB1vdXpSWGR+hJ8POjHx8NJ5MA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMzA4MTAxNzEwMjZaFw0yMzExMDgxNzEwMjVaMB8xHTAbBgNVBAMT
SOME LINES
SOME LINES
zGH94WaJhI/53xca36KkckT3Wil4R9aAY+Oj8KFnu34KpR8sIydAICV4wlNrgb+9
j41+g+1GTKgLoVq32U5Ru1OQgzmFV+TFgSJj1KuT00sbgJXjeUj3KAET4fTHSoEB
1c97mIF8HLuqLg9as3M9EDmioyNBF0mBk0ZT7C9pIz+bXoB5wyFc3WetVp6sdDmr
oX8xcK6cQGS81EnSVI4f/c36n7wc9ot9pnqrZAP9DDAhWWT/WhSK+w==
-----END CERTIFICATE-----
subject=/CN=MY_HOST_SERVER
issuer=/C=US/O=Let's Encrypt/CN=R3
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: DH, 1024 bits
---
SSL handshake has read 5495 bytes and written 514 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : DHE-RSA-AES256-GCM-SHA384
Session-ID: A428113EAAD5E1425D95D3277C48BF80477264E0ACAA5C0DC18130FA9222CC22
Session-ID-ctx:
Master-Key: DA4FB3DB10196090A904278FF99E6153F98CA41C41D24DF503D5BE28101E8E806525D115D8B40912BEA68708CEC9528A
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 - 6f 1c 0e 0e fe e4 fb 66-0c 1f dd 62 09 b5 12 1c o......f...b....
0010 - 00 bc 64 f8 2c e0 13 18-b3 c6 c0 2c 4d 70 b6 d4 ..d.,......,Mp..
0020 - 38 a3 11 c4 7c 81 58 87-12 4a ab 82 03 d3 8a 66 8...|.X..J.....f
0030 - 6a 38 2d 97 55 4e 83 9e-bf 2d 74 8e c9 bb 75 19 j8-.UN...-t...u.
0040 - 15 d5 6b 81 21 8c 80 c9-13 c4 2d 0b 1d 1d 6f cb ..k.!.....-...o.
0050 - 8f e9 43 e0 9f fb 7b 53-96 ae 54 d3 27 8d c8 07 ..C...{S..T.'...
0060 - fc b9 7a 63 1f b5 82 60-c2 1a 95 3a 3d e4 48 35 ..zc...`...:=.H5
0070 - 8b f0 55 72 0d d2 a4 e8-5f 0a 04 05 94 b0 91 92 ..Ur...._.......
0080 - 29 18 d4 db 2d 3f d3 b2-33 29 57 20 d9 db 9d 53 )...-?..3)W ...S
0090 - 64 f4 69 7a f0 fc 26 e1-48 77 cf 50 ec 4a 4a 01 d.iz..&.Hw.P.JJ.

Start Time: 1693578519
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
250 AUTH LOGIN PLAIN CRAM-MD5

cat /etc/letsencrypt/live/${DOMAIN}/privkey.pem /etc/letsencrypt/live/${DOMAIN}/fullchain.pem > /var/qmail/control/servercert.pem

certbot certonly \
--webroot \
--webroot-path /path/to/webroot \
--preferred-challenges http-01 \
-d domain1 -d domain2 -d domain3 ....... \
--email youremail@yourdomain \
--renew-by-default \
--agree-tos \
--text

15 2 20 * * /usr/local/bin/my_certbot.sh

Traceback (most recent call last):
  File "/ usr / bin / certbot", line 5, in 
    from pkg_resources import load_entry_point
  File "/usr/lib64/python2.7/site-packages/pkg_resources/__init__.py", line 2927, in 
    @_call_aside
  File "/usr/lib64/python2.7/site-packages/pkg_resources/__init__.py", line 2913, in _call_aside
    f (* args, ** kwargs)
  File "/usr/lib64/python2.7/site-packages/pkg_resources/__init__.py", line 2940, in _initialize_master_working_set
    working_set = WorkingSet._build_master ()
  File "/usr/lib64/python2.7/site-packages/pkg_resources/__init__.py", line 635, in _build_master
    ws.require (__ requires__)
  File "/usr/lib64/python2.7/site-packages/pkg_resources/__init__.py", line 943, in require
    needed = self.resolve (parse_requirements (requirements))
  File "/usr/lib64/python2.7/site-packages/pkg_resources/__init__.py", line 829, in resolves
    raise DistributionNotFound (req, requirers)
pkg_resources.DistributionNotFound: The 'zope.interface' distribution was not found and required by certbot

/var/qmail/control/domain1.com.pem
/var/qmail/control/domain2.com.pem
/var/qmail/control/domain3.com.pem

-d domain1.tld -d domain2.tld -d domain3.tld etc.