☰

KPTree - Email Server Setup

Email Server Notes

This website content is no longer under development.

Setting up the email server on my home server was probably the most difficult task. As noted in the various recipes herein, email server setup is intricate, risky and involves significant commitment and effort to setup reliably and maintain.

My first email server was setup on an Ubuntu 14.04 virtual machine on my home server. It was based upon the setup recipe given in the blog "Beware Here Be Musings", Installing a Mailserver on Ubuntu 14.04 LTS, Part 1 and Part 2. I got the mailserver to run well after a couple of attempts. It is simple and performs well and a good starting point to learn from. Limitations include: the database is a semi-manual setup, and addition of virtual mail users must be performed manually. There is also no webmail setup instructions which generally requires integration with a web server and the associated setup.

Most recipes I found are limited in some ways. They assume use of a commercial remote server, whereas I have setup on a homeserver on a virtual machine. Few note the external setup requirements, DNS, MX, SPF and reverse DNS, which must be performed to get operating, and ensure best operating practices, a notable exception being Linode's Running a Mail Server.

A more comprehensive recipe I found was from Ex Ratione - A Mailserver on Ubuntu 16.04: Postfix, Dovecot, MySQL, Postfixadmin, Roundcube. An Apache webserver is installed for webmail and Postfix admin. A Mailserver on Ubuntu 12.04: Postfix, Dovecot, MySQL is for an earlier version recipe, but uses Horde instead of Roundcube for the web mail access. The Horde project is a groupware webmail, and has more groupware functions than Roundcube.

Email Server Setup

Basically follow the setup given in A Mailserver on Ubuntu 18.04: Postfix, Dovecot, MySQL, Postfixadmin, Roundcube, with the following comments:

  • Ignore the section "Using Amazon Web Services"
  • For "Use of example.com and mail.example.com" use kptree.net and mail.kptree.net instead
  • For "Fire up an Ubuntu 16.04 AWS Instance with a Suitable Security Group" prefer the following instead
    • Setup an Ubuntu 16.04 VM with 4 CPUs and 4G of RAM, no/disabled SWAP drive (Expect 2 CPUs and 2 GB RAM sufficient based upon text)
    • My home router needed to have the port forwarding setup to redirect inbound Internet TCP traffic to the local VM IP address for the nominated ports.
    • I never open VM SSH access ports to the Internet.....
    • The Internet ports are only redirected later in the build, when necessary for testing / implementation
  • For "Now Build a LAMP Web Server" I manually install the various LAMP install components, as I prefer to use Mariadb instead of MySQL
  • For "Configure MySQL" with Mariadb refer to SQL_MODE:

    • sudo mysql -u root -p

    SET GLOBAL sql_mode = 'ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';

    SET SESSION sql_mode = 'ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';

    SELECT @@SQL_MODE, @@GLOBAL.SQL_MODE;

  • For "Install Postfix Admin 2.93 and the MySQL Schema" the latest version of is postfixadmin-3.0, as of 2016-12-31
  • Problems with running "https://mail.example.com/postfixadmin/setup.php" failed with debug error relating to index to long. Basically indexes are restricted to 1000byte length. So the index with char(255) is ok with char length byte, however modern databases seem to default to utf8mb4 (4 bytes length) or UTF (3 bytes length) to allow for international character sets. Refer to the following links:
  • For the Configure Dovecot Section
    • The Dovecot configuration file /etc/dovecot/dovecot-sql.conf.ext
      • My vmail user uid has historically been 6004. The mail uid and gid are both 8 on default Ubuntu (Linux?) distributions.
      • I propose to install the working mail directory on an NFS drive to help limit the VM drive size, /mnt/shared/vmail.
    • Follow change directions for the Dovecot configuration file /etc/dovecot/conf.d/10-auth.conf
    • The Dovecot configuration file /etc/dovecot/10-mail.conf
      • I propose to install the working mail directory on an NFS drive to help limit the VM drive size, /mnt/shared/vmail.
      • My vmail user uid has historically been 6004. The mail uid and gid are both 8 on default Ubuntu (Linux?) distributions.
    • Note that /etc/amavis/conf.d/15-content_filter_mode directives were backslashed as follows:
      • @bypass_virus_checks_maps = (
      • \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);
      • @bypass_spam_checks_maps = (
      • \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);
    • Configuration file /etc/default/spamassassin not changed as directed, as on systemd use sudo systemctl enable spamassassin.service instead. CRON=0 changed to CRON=1.
    • Attempting to run "sudo freshclam" came up with error. I had to disable the clamav daemons to allow the update to work.
      • To find clamav related running daemons "sudo systemctl list-units -all | grep clam"
      • Stop the daemons "sudo systemctl stop clamav-daemon" and any other found daemons
      • "sudo freshclam" does not functions without error
      • Restart clamav "sudo systemctl start clamav-daemon", however a "sudo reboot" may be in order....
  • For the Configure Postfix Section no major changes (other than ensuring consistency with configuration site specifics, as above)
  • Restart Everything, and Test the Server - check /var/log/mail.log & /var/log/mail.err
  • Log showed error [postfix/smtpd[15073]: warning: SASL: Connect to private/auth failed: No such file or directory] and [postfix/smtpd[15073]: fatal: no SASL authentication mechanisms]. Upon restarting dovecot and then postfix, this error went away. (Restarting Dovecot initialises the file /var/spool/postfix/private/auth as per configuration /etc/dovecot/conf.d/10-master.conf after which this Postfix error is resolved.)
  • Added compatibility_level = 2 to /etc/postfix/main.cf concerning log message "Postfix is running with backwards-compatible default settings"

Email Client Setup Hint

A quick and dirty description to help with the email client setup, SMTP and IMAP or POP3.

IMAP Client Settings

Email address: admin@example.com

Server Type: IMAP

Server: mail.example.com

Port: 993

Username: admin@example.com

Encryption method: SSL (on dedicated port)

Authentication: Password

POP3 Client Settings

Email address: admin@example.com

Server Type: POP3

Server: mail.example.com

Port: 995

Username: admin@example.com

Encryption method: SSL (on dedicated port)

Authentication: Password

SMTP Client Settings

Server: mail.example.com

Port: 25

TICK, Server requires authentication

Encryption Method: TLS (start after connecting)

Username: admin@example.com

Auth Type: Login

Email Client Setup Hints (Gmail and Outlook)

A quick and dirty description to help with the email client setup, SMTP and IMAP or POP3 for Gmail and MSOutlook (Hotmail).

Gmail

IMAP Client Settings

server address: imap.gmail.com

user name: Your full Gmail address (e.g. me@gmail.com)

password: Your Gmail password

With Gmail 2-step authentication enabled, use an application-specific Gmail password.

IMAP port : 993

IMAP TLS/SSL required: yes

POP3 Client Settings

server address: pop.gmail.com

user name: Your full Gmail address (e.g. me@gmail.com)

password: Your Gmail password

With Gmail 2-step authentication enabled, use an application-specific Gmail password.

IMAP port : 995

IMAP TLS/SSL required: yes

SMTP Client Settings

server address: smtp.gmail.com

user name: Your full Gmail address (e.g. me@gmail.com)

password: Your Gmail password

With Gmail 2-step authentication enabled, use an application-specific Gmail password.

SMTP port (TLS): 587

SMTP TLS/SSL required: yes


MSOutlook / Hotmail

IMAP Client Settings

server address: imap-mail.outlook.com

user name: Your full email address (e.g. me@hotmail.com)

password: Your hotmail password

IMAP port : 993

IMAP TLS/SSL required: yes

POP3 Client Settings

server address: pop-mail.outlook.com

user name: Your full email address (e.g. me@hotmail.com)

password: Your hotmail password

IMAP port : 995

IMAP TLS/SSL required: yes

SMTP Client Settings

server address: smtp-mail.outlook.com

user name: Your full email address (e.g. me@hotmail.com)

password: Your hotmail password

SMTP port (TLS): 587

SMTP port (SSL): 465

SMTP TLS/SSL required: yes (STARTTLS)

Webmail Installation

Install Roundcube for Webmail

I attempted to use the instructions from Exratione A Mailserver on Ubuntu 18.04: Postfix, Dovecot, MySQL, Postfixadmin, Roundcube to install Roundcube. However the package database configuration failed. So I instead followed the instructions given in LinOxide How to Install Roundcube Webmail on Ubuntu 18.04. These instructions assume an already working LAMP stack and running email server and down load the latest install files directly from RoundCube, i.e. do not use the Unbuntu apt package system.

Some tips on the LinOxide instructions:

  • As of writing (2017-01-04) the latest version of Roundcube is 1.2.3
  • My website and mail server are already setup only to use https, so:
    • The IMAP host chosen to perform login is "ssl://sub1.example.com" and port 993 (standard IMAP secure port)
    • Similarly, The SMTP server host is ssl:sub1.example.com and port 25 (not 465). There is no SMTP username and password, just select the "Use the current IMAP username and password for SMTP authentication.
    • Personally I do not understand why you would want to use a webmail client to Gmail, Outlook, etc., as these all have there own webmail clients.
  • Instead of deleting the installer directory, disable access in apache2:

    • create new file: "/etc/apache2/sites-available/roundcube.conf" and add following:

    <Directory /var/www/html/webmail>

    Options +FollowSymLinks

    AllowOverride All

    AllowOverrideList None

    <RequireAll>

    Require all granted

    </RequireAll>

    </Directory>


    <Directory /var/www/html/webmail/installer>

    Options +FollowSymLinks

    AllowOverride None

    AllowOverrideList None

    <RequireAll>

    Require all denied

    </RequireAll>

    </Directory>

    To enable site configuration: "sudo a2ensite roundcube.conf"

  • Note to function correctly the "/var/www/html/webmail/.htaccess" file should have the <IfModule mod_php5.c> changed to <IfModule mod_php7.c>

Some additional followups:

In order to direct default traffic to the webmail page I added directive "Redirectmatch ^/$ https://mail.kptree.net/webmail" to the apaches site configuration file /etc/apache2/sites-available/sub1.example.com-ssl.conf.

Install Horde 5 for Webmail

Attempt to follow ExRatione - A Mailserver on Ubuntu 12.04: Postfix, Dovecot, MySQL 19) Install Horde 5 for Webmail, full copy with update to get working on current Ubuntu 16.04 install. However upon following this procedure it look like it is quite outdated in many ways. Upon further investigation I found this installation description that looks more up to date and functional, Howtoforge Install Horde 5 Webmail For ISPConfig On Debian Wheezy Through PEAR. The Horde site instructions Installing Horde Framework 5 and Installing Horde Groupware Webmail Edition 5 states "These are very terse instructions how to install Horde Groupware Webmail Edition and its prerequisites on a LAMP system. They are addressed to experienced administrators who know exactly what they are doing.". A last key reference is the Horde Horde Administrator's FAQ.

To summarise my feeling on Horde, basically it is too complicated to setup and use. It looks to be beyond the needs of a simple home server. At the time of writing (2017-01-03) the current version of Horde is not optimised for PHP7, which is standard with Ubuntu 16.04, which leads to further complications with the install. Exratione in subsequent later version of its installations instructions went with Roundcube for webmail interface. Furthermore some of the key features I was looking for, such as shared calendar and contact resources are included in Nextcloud. Whilst I managed to get Horde webmail running, there are still a number of issues with it that I have not been able to resolve. At this time I have decided not to proceed with using Horde. These instructions are therefore incomplete.

  1. Install as many of the needed packages as possible through apt install:

    2. "sudo apt update"

    "sudo apt install php7.0-dev php-pear php7.0-tidy php-imagick" (no php5-sasl, php-auth-sasl?)

    "sudo apt install php-geoip geoip-bin" (packages geoip-database and libgeoip1 included in php-geoip install)

    "sudo apt install php-xml-serializer php-memcache php7.0-soap php7.0-intl"

    "sudo apt install libidn11-dev libmagickwand-dev imagemagick" (libmagick++4?)

    "sudo apt install libsasl2-dev php-ssh2 php-http-webdav-server" (no libssh2-php, php-ssh2?, no libphp-jpgraph)

  2. Next update the PECL and Pear package managers and install the remaining required packages:

    3. sudo pecl channel-update pear.php.net

    sudo pear channel-update pear.php.net

    sudo pecl install lzf - with messages:

    • configuration option "php_ini" is not set to php.ini location
    • You should add "extension=lzf.so" to php.ini

    Check the Pear website for latest package versions and versions that suit you at pear Packages

    sudo pear install --alldeps channel://pear.php.net/Date_Holidays-0.21.8

    sudo pear install --alldeps channel://pear.php.net/Date_Holidays_UNO-0.1.3

    sudo pear install --alldeps channel://pear.php.net/Date_Holidays_Australia-0.2.2

    sudo pear install --alldeps channel://pear.php.net/Numbers_Words-0.18.1

    sudo pear install --alldeps channel://pear.php.net/Text_CAPTCHA-1.0.2

    sudo pear install --alldeps channel://pear.php.net/Console_GetoptPlus

    sudo pear install --alldeps channel://pear.php.net/HTTP_Request2

    sudo pear install --nodeps channel://pear.php.net/MDB2_Driver_mysql

  3. Next up is installing the Horde components. Start with these commands:

    4. sudo pear channel-discover pear.horde.org

    sudo pear install horde/Horde_role

    sudo pear run-scripts horde/Horde_role

  4. At this point, you will be prompted to enter the "Filesystem location for the base Horde application" - so enter the full path to your web root without a trailing slash, i.e. /var/www/html/horde.
  5. Next start the installation process with the command below. This will take a while to run to completion:

  6. sudo pear install -a -B --force horde/webmail

    The --force is required to allow the use of php7, otherwise this will error out the install.

    Use "pear config-show -c horde" to show where the key Horde php related files are on the system.

The following is a list of related commonly used SQL commands:

  • To confirm the status (start, stop and restart) mysql "sudo systemctl status apache2"
  • To login into mysql as root, with password prompt: "sudo mysql -u root -p"
  • Database commands:
    • To show databases: "show databases;"
    • To create database (where new database name is: dBase_NAME): "create dBase_NAME"
    • To list mySQL database sizes: "SELECT table_schema "DB Name", Round(Sum(data_length + index_length) / 1024 / 1024, 1) "DB Size in MB" FROM information_schema.tables GROUP BY table_schema;"
    • To use a database: "use databasename;"
  • To show tables in a database: "show tables;"

The following is a list of related commonly used commands and scripts:

  • Get external IP address "wget http://ipinfo.io/ip -qO -"
  • Save the current netfilter (IPTables) configuration "sudo netfilter-persistent save"
  • To check the current network hardware configuration "ip a"
  • To start (/stop /enable) the deluge daemon "sudo systemctl start deluged"
  • To start (/stop /enable) the deluge web interface daemon "sudo systemctl start deluge-web"
  • Systemd common commands (start / stop / restart / status) (enable / disable for boot control)