External mailing lists

If something goes wrong, here a few resources:

Folders content

The repository contains a few folders you should be familiar with:

  • config: The main Yaml configuration files for your homebox device.
  • preseed: Docker environment to create an automatic ISO image installer for Debian.
  • install: Ansible scripts to install or test the whole server environment.
  • backup: A very useful folder that contains some important files like the passwords and certificates generated when deploying the system. This allows you to “replay” the deployment on a new server after a disaster, without loosing any information. This folder is generated automatically on the first deployment, and ignored by git
  • tests: Ansible playbooks to test the platform.
  • sandbox: Put anything you don’t want to commit here.
  • docs: This project documentation.
  • uninstall: Ansible scripts to uninstall some of the components. This allows you to test them.
  • devel: A set of containers to help setup a local development environment.

Branches

  • The current developments are done in the dev branch.
  • The master branch is kept for releases.

Starting in May 2019, the approach will be to use gitflow, from the dev branch.

Test machine

You can start to develop using a virtual machine on your workstation, for instance using KVM or VirtualBox. The environment runs on Debian Stable. You can install it using a minimalistic ISO image, called netinst.

I suggest you to use a virtual machine to test, especially with snapshots capabilities. By using snapshots, you can rollback to any stage and run the Ansible scripts again. On Linux, KVM/libvirt is the best choice, and VirtualBox might be acceptable.

For instance:

apt-get install libvirt virt-manager

Preseed

There is a preseed folder that creates an an ISO image for automatic installation. It is using Docker, and builds an automatic installer from a YAML configuration file. The preseed page give more details about this feature.

Note

This installer installs Debian only, it does not deploy the platform.

Router configuration

Unless your IP address changes too often or your port 25 is blocked, you do not need to have a fixed IP address.

However, the more you want to test, the more your router need to be configured:

  • TCP/80 (HTTP): Used to query letsencrypt for the certificates. Open it at least to obtain the SSL using LetsEncrypt. You can then close the port once you have the certificates.
  • TCP/25: SMTP, to receive and transfer emails from and to other SMTP servers on internet. You need this one only to test the reception and transmission of external emails.

Now, all the rest can be done internally, without exposing your test machine. These ports don’t need to be forwarded by your router during the development time:

  • TCP/143 and TCP/993 : IMAP and IMAPS
  • TCP/110 and TCP/995 : POP3 and POP3S
  • TCP/587 : Submission.
  • TCP/465 : SMTPS (this on is kept for compatibility with some old devices, but perhaps will be removed soon)
  • TCP/4190 : ManageSieve. Used to remotely access your mail filters, for instance with thunderbird sieve plugin.
  • TCP/443 : HTTPS access for the webmail and also Outlook autodiscover feature.
  • TCP/5222 and TCP/5269 : Jabber, clients to server and server to server implementation.
  • UDP/53 and TCP/53 : DNS Server.

Bridging your workstation

If you are using a virtual machine, it is better to use a bridge on your workstation, to transparently forward the traffic from internet.

DNS Setup

There is a script that actually updates automatically DNS entries, if you are using Gandi. For development, the simplest is to add a wilcard entry in your DNS, that would point to your virtual machine.

The initial creation of DNS records for certificate generation should take some time. So, it is better to do this before anything else.

Otherwise, you can use the DNS server implemented with Bind.

Create your hosts file

  cd config
  cp hosts-example.yml hosts.yml

The host file is in YAML format, and contains only one host, which is your homebox server.

Here an example:

all:
  hosts:
    homebox:
      ansible_host: 192.168.42.1
      ansible_user: root
      ansible_port: 22

I have actually tested with the Ansible remote user as root. However, it should be possible to run as an admin user and use sudo with little modifications.

System configuration

First, as you would do for a live environment, copy the sample configuration to create your own:

cd config
cp system-example.yml system.yml

The file is self explanatory, and inside, you will find the following block:

system:
  release: stretch
  login: true
  devel: true
  debug: true

The “debug” flag

Setting the debug flag to true will activate a lot of debug options in Dovecot, OpenLDAP, Postfix, etc… You can then filter in the console, using for instance journalctl -u postfix -u dovecot* to view postfix and dovecot logs respectively.

The “devel” flag

When you set this flag to true, various settings are changed in the development.

By default:

  • The certificates deployed are staging certificates only, which allows you to request more to LetsEncrypt.
  • The certificates are backed up on your local machine, allowing you to redeploy without asking again the same certificates, which is also faster.
  • If you want to test your system from a local computer, you will need to add the staging version of the root certificate authority. They cab be downloaded on the on the LetsEncrypt staging environment page.

As an option, without Letsencrypt:

  • The certificates can be deployed from a local ACME server using Pebble.
  • The expected server is provided as a container by a docker-compose file in the devel directory.
  • The server is for testing only and has some usage constraints which are further described in the next section.

The devel environment

To be able to develop and test locally, the devel directory provides a docker-compose file with definitions for the following containers:

  • A pebble server to act as a testing ACME server to replace Letsencrypt.
  • A challtestsrv to act as a manageable DNS server for the ACME server.
  • An apt-cacher server.
$ cd devel/
$ docker-compose up
Starting devel_pebble_1         ... done
Recreating devel_challtestsrv_1 ... done
Starting devel_apt-cacher_1     ... done
Attaching to devel_apt-cacher_1, devel_pebble_1, devel_challtestsrv_1
[…]

These containers are connected to a bridge and addressed in the subnet 10.30.50.0/24. The playbooks assume the predefined static IP addresses for the pebble and the apt-cacher servers. The server’s external_ip will be used to resolve any DNS request made by the pebble server.

The use of the pebble server in the playbooks is configured with:

system:
[…]
  devel: true
[…]

devel:
  acme_server: pebble

The Pebble ACME server is designed to create temporary CAs and certificates that can only be used for testing. A new temporary CA is created every time the server is started, and it is destroyed when it stops. The certificates role in the playbooks will install the current CA whenever they are run.

Everytime the Pebble server is started, to have a coherent system, the certificates might need to be generated again. It can be done by running the playbooks or with the help of the certificates tag:

$ cd install/
$ ansible-playbook -i ../config/hosts.yml playbooks/main.yml -t certificates

When developing or testing locally, most probably with a local domain name like example.local, there is no need for the DNS propagation checks during installation and testing:

bind:
[…]
  propagation:
    check: false
[…]

It also disables one of the opendmarc test which requires an external validating resolver.

The apt-cacher server can be used to speed up package updates on reinstalls by configuring:

system:
[…]
  apt_cacher: 10.30.50.4

Finally, to allow the use of these service the firewall should be configured with:

firewall:
  output:
    policy: deny
    rules:
      - dest: any
        port: 80,443
        comment: 'Allow web access'
      - dest: any
        proto: udp
        port: 53
        comment: 'Allow DNS requests'
      - dest: any
        proto: udp
        port: 123
        comment: 'Allow NTP requests'
      - dest: any
        proto: udp
        from_port: 68
        port: 67
        comment: 'Allow DHCP requests'
      - dest: any
        port: 25
        comment: 'Allow SMTP connections to other servers'
      - dest: any
        port: 110,995,143,993
        comment: 'Allow the retrieval of emails from other servers (POP/IMAP)'
      - dest: 10.30.50.2
        port: 14000,15000
        commit: 'Allow access to the Pebble ACME server'
      - dest: 10.30.50.3
        port: 8055
        commit: 'Allow access to the Pebble challenge Test server'
      - dest: 10.30.50.4
        port: 3142
        commit: 'Allow APT cacher access'

Setting up ansible-lint before commit

The program ansible-lint is executed by the continuous integration platform, and should be used for each commit. A hook is provided in the git-hooks folder.

The asible-lint software need to be installed on your machine. We are using the version in Debian Buster. The ansible-lint configuration file is in config/ansible-lint-default.yml.

To ensure the hook is executed before each commit, run this on your local machine:

git config --local core.hooksPath git-hooks

Development playbook

The first playbook to run is probably “dev-support.yml”. It installs some diagnostic and convenience packages on the server, to make your life easier during the development phase.

For instance, these packages are installed:

  • mc
  • telnet
  • dnsutils
  • whois
  • tmux
  • pfqueue
  • aptitude
  • man
  • less
  • vim
  • net-tools
  • file
  • swaks
  • curl
  • locate
  • colorized-logs

  • bash-completion

  • The script also configures a basic bashrc / zshrc.

  • It is also adding the LetsEncrypt staging root certificate authority to the system.

Installation playbooks

The main playbook ‘main.yml’ and includes all other playbooks, with some of them conditional, as some components are optional.

Automatic backup

Once the script has been run, the backup folder contains important files, like certificates, passwords, etc. See (see the deployment backup page for details).

Development cleanup

This playbook do the opposite of dev-support, by uninstalling the packages used for development, and restoring the bashrc to its default state. You probably want to run this script before putting your server in production.

It is also removing the LetsEncrypt staging root certificate authority from the system.

Tests / Diagnostic playbooks.

There is also a tests folder that contains test playbooks. These playbooks are running a list of system and integration tests on your development server. This is useful for diagnostic purposes and also during the development phase, to be sure nothing is broken before committing anything.

It does not replace a full test suite in a pre-production environment, but has been enough so far to catch common mistakes made in the scripts.

The following roles are run:

  • Install the development packages above,
  • Basic system tests
  • LDAP server: Binding, users list, SSL certificate, etc.
  • Home folders: Presence and permissions
  • Antivirus rspamd: Current state
  • Service opendmarc: Current state
  • SMTP certificate: presence and validity
  • Service opendkim: Current state, key validdity
  • Service postfix: Current state, certificate, emails sending and receiving
  • POP3 certificate: presence and validity
  • IMAP certificate: presence and validity
  • Service dovecot: current state, user authentication, email resolution
  • Web site for roundcube: basic access, SSL certificate test
  • Web site “autoconfig” for Thunderbird: Check the validity of the XML generated
  • Web Site “autodiscover” for Outlook: HTTPS certificate, check the validity of the XML generated
  • Antivirus tests, for instance check that an email with a virus is bounced.
  • Full text search inside attachments
  • DNS records when the DNS server is installed.

Profiling the playbook

You can profile the time taken by the whole playbook, using the Ansible profile_roles plugin:

[defaults]
retry_files_enabled = False
display_skipped_hosts = False
stdout_callback = yaml
callback_whitelist = profile_roles
roles_path = .:{{ playbook_dir }}/../../common/roles/
connection_plugins = {{ playbook_dir }}/../../common/connection-plugins/
remote_tmp = /tmp/

Then, once you have finished to run the playbook, you will see the total time. For instance:

For a full deployment:

PLAY RECAP *********************************************************************
homebox                    : ok=644  changed=394  unreachable=0    failed=0
localhost                  : ok=0    changed=0    unreachable=0    failed=0

Saturday 22 June 2019  15:18:09 +0100 (0:00:00.424)       0:18:59.596 *********
===============================================================================
dovecot --------------------------------------------------------------- 176.27s
system-prepare -------------------------------------------------------- 118.33s
postfix --------------------------------------------------------------- 112.06s
certificates ---------------------------------------------------------- 100.98s
load-defaults ---------------------------------------------------------- 81.55s
roundcube -------------------------------------------------------------- 67.77s
ldap ------------------------------------------------------------------- 66.80s
clamav ----------------------------------------------------------------- 59.39s
external-ip ------------------------------------------------------------ 39.54s
sogo ------------------------------------------------------------------- 38.57s
opendkim --------------------------------------------------------------- 27.84s
dns-server-bind -------------------------------------------------------- 27.40s
setup ------------------------------------------------------------------ 25.43s
rspamd ----------------------------------------------------------------- 24.34s
opendmarc -------------------------------------------------------------- 23.52s
packages --------------------------------------------------------------- 21.38s
website-simple --------------------------------------------------------- 17.81s
system-cleanup --------------------------------------------------------- 16.34s
user-setup ------------------------------------------------------------- 15.64s
autoconfig ------------------------------------------------------------- 14.98s
autodiscover ----------------------------------------------------------- 14.98s
remote-access ---------------------------------------------------------- 14.89s
nginx ------------------------------------------------------------------ 13.14s
imapproxy --------------------------------------------------------------- 8.58s
dns-server-bind-refresh ------------------------------------------------- 2.69s
well-known-services ----------------------------------------------------- 2.10s
dns-server-check-propagation -------------------------------------------- 1.22s
ejabberd ---------------------------------------------------------------- 0.75s
transmission ------------------------------------------------------------ 0.63s
borg-backup ------------------------------------------------------------- 0.58s
zabbix-server ----------------------------------------------------------- 0.54s
luks-remote ------------------------------------------------------------- 0.50s
fwknop-server ----------------------------------------------------------- 0.40s
privoxy ----------------------------------------------------------------- 0.31s
access-check ------------------------------------------------------------ 0.26s
backup-server ----------------------------------------------------------- 0.19s
tor --------------------------------------------------------------------- 0.18s
import-accounts --------------------------------------------------------- 0.18s
rspamd-web -------------------------------------------------------------- 0.18s
fwknop-client ----------------------------------------------------------- 0.17s
ssh-keygen -------------------------------------------------------------- 0.13s
access-report ----------------------------------------------------------- 0.12s
php-fpm ----------------------------------------------------------------- 0.08s
extra-certs ------------------------------------------------------------- 0.06s
sendxmpp ---------------------------------------------------------------- 0.05s
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
total ---------------------------------------------------------------- 1139.50s

And for an update:

PLAY RECAP *********************************************************************
homebox                    : ok=557  changed=66   unreachable=0    failed=0
localhost                  : ok=0    changed=0    unreachable=0    failed=0

Saturday 22 June 2019  14:50:39 +0100 (0:00:00.442)       0:09:57.571 *********
===============================================================================
load-defaults ---------------------------------------------------------- 85.87s
certificates ----------------------------------------------------------- 78.07s
dovecot ---------------------------------------------------------------- 54.67s
postfix ---------------------------------------------------------------- 42.22s
external-ip ------------------------------------------------------------ 36.11s
ldap ------------------------------------------------------------------- 34.02s
system-prepare --------------------------------------------------------- 27.48s
setup ------------------------------------------------------------------ 23.86s
dns-server-bind -------------------------------------------------------- 22.62s
opendkim --------------------------------------------------------------- 22.37s
rspamd ----------------------------------------------------------------- 22.30s
opendmarc -------------------------------------------------------------- 18.06s
roundcube -------------------------------------------------------------- 17.91s
user-setup ------------------------------------------------------------- 15.80s
nginx ------------------------------------------------------------------ 15.27s
sogo ------------------------------------------------------------------- 14.75s
remote-access ---------------------------------------------------------- 13.83s
website-simple ---------------------------------------------------------- 8.27s
system-cleanup ---------------------------------------------------------- 7.65s
clamav ------------------------------------------------------------------ 6.65s
autoconfig -------------------------------------------------------------- 5.68s
autodiscover ------------------------------------------------------------ 5.37s
imapproxy --------------------------------------------------------------- 4.82s
packages ---------------------------------------------------------------- 3.09s
dns-server-bind-refresh ------------------------------------------------- 1.93s
well-known-services ----------------------------------------------------- 1.35s
dns-server-check-propagation -------------------------------------------- 0.98s
ejabberd ---------------------------------------------------------------- 0.77s
transmission ------------------------------------------------------------ 0.75s
borg-backup ------------------------------------------------------------- 0.62s
zabbix-server ----------------------------------------------------------- 0.58s
luks-remote ------------------------------------------------------------- 0.53s
fwknop-server ----------------------------------------------------------- 0.43s
privoxy ----------------------------------------------------------------- 0.32s
access-check ------------------------------------------------------------ 0.28s
rspamd-web -------------------------------------------------------------- 0.24s
backup-server ----------------------------------------------------------- 0.21s
tor --------------------------------------------------------------------- 0.20s
fwknop-client ----------------------------------------------------------- 0.18s
import-accounts --------------------------------------------------------- 0.18s
access-report ----------------------------------------------------------- 0.16s
ssh-keygen -------------------------------------------------------------- 0.14s
php-fpm ----------------------------------------------------------------- 0.09s
extra-certs ------------------------------------------------------------- 0.06s
sendxmpp ---------------------------------------------------------------- 0.06s
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
total ----------------------------------------------------------------- 597.48s

Some development tools to consider

  • The fantastic tmux, mandatory IMHO: tmux github page.
  • Emacs or vim, but if you are not ready, VisualStudio code is not too bad as well, and is very well integrated in Debian / Ubuntu.
  • Test your SMTP server compliance: mxtoolbox.com.
  • DNSSEC records debugger : https://dnssec-analyzer.verisignlabs.com/
  • DNS propagation checker: https://www.whatsmydns.net/