Atech’s Postal is an SMTP server and web management interface that’s geared towards transactional and bulk mailing (e.g. for application to user communication, and for marketing respectively). It’s quite well documented, but more importantly it’s open source (MIT license), and also seems well written – elegant, self-documenting code that’s easy to follow, useful comments, well structured. A bit of a joy really.
The Fast Server is a web server process that’s separate from the management interface server, that’s used to handle requests from click and open tracking links. However, the documentation on the Fast Server process, which is used for logging email Open and Click events, seems to be at least partially out of date, so I thought I’d dig into the code to understand and document the bits that I was unsure of.
According to the docs, the Fast Server is meant to run on a separate IP address. It binds to two ports, one for HTTP, and the other for HTTPS requests. I want to figure out what implications there might be of binding this server to the same IP address as the nginx server, but on different ports.
In this post, I’ll refer to the main Postal management interface as web_server, and the click-tracking server as fast_server. These names correspond to the names ‘web’ and ‘fast’ in the process list returned by the ‘postal status’ command.
I’ll refer to the IP addresses as primary, for the main IP address used for web_server and the SMTP server, and secondary, for the IP address required by fast_server according to the documentation. I’ll use 192.168.123.1 and 192.168.123.2 as placeholders in the text for primary and secondary IP addresses respectively.
First, the 2 IP configuration
Before that, since Linode have since kindly allocated an extra IP address to the node for me, I’ll cover what’s required for a 2 IP configuration.
The nginx config
It might be worth clarifying here that nginx binds to the primary IP simply to proxy requests to the web_server process bound to localhost:5000. On the other hand, fast_server by default binds directly to the secondary IP, ports 80 and 443.
Therefore, we need to stop nginx binding to all interfaces, as it’s configured to do by default. The change required is to change the ‘listen’ directives for the IPv4 addresses in the nginx configuration. So we need something like the following, substituting 192.168.123.1 with your primary IP : –
and, for the SSL server block : –
listen 192.168.123.1:443 ssl;
When nginx is restarted, it’s no longer bound to the second IP address, making it available for use with the click tracking ‘fast_server’ process.
The DNS Zone Configuration
The DNS configuration should include (among others) the following entries for the IP addresses (assuming this is in the example.com zone): –
postal A 192.168.123.1 track.postal A 192.168.123.2 click CNAME track.postal.example.com.
So postal.example.com will resolve to the primary IP, track.postal.example.com resolves to the secondary IP, and click.example.com resolves to the canonical track.postal.example.com (which subsequently resolves to the secondary IP).
When tracking opening of emails and the links contained within, it is click.example.com that’s embedded in the HTML of the email to provide the tracking image and rewritten links.
When the fast_server is enabled in ‘./config/postal.yml’, Postal needs to be stopped and then started (restart alone doesn’t seem to bring the fast_server process online).
From here, it’s simply a matter of adding a new ‘Domain -> Tracking Domain’ from the web interface.
Possible single-IP configuration
The config in ./spec/config/postal.yml defines fast_server’s ports as 5010 (HTTP) and 5011 (HTTPS). However they are defined as 80 and 443 in the file ./config/postal.defaults.yml, which is the base configuration file that is overlaid with ./config/postal.yml at runtime.
This suggests to that the Fast Server can at least run correctly on non-standard ports. I would guess is that links to non-standard ports are more likely to be blocked by (e.g. corporate) firewalls, or perhaps emails containing such links are seen as more spammy by some mail hosts. Whatever the reasons, the developers clearly felt that it was best to use a separate IP address for fast_server.
There is an example configuration on the GitHub project, issue 321, which binds fast_server to the standard ports 80/443, and web_server to port 8080. It reportedly runs fine like this, however one thing that I think needs to to be confirmed is whether or not CertBot (or whatever ACME client is being used) can renew the certificates when nginx is running on a non-standard port.
The Rack Interface is itself invoked by the Client class, which is instantiated and invoked in fast_server/Server.rb when a connection is made to fast_server.
Binding to SMTP and HTTP (privileged) ports
The ruby executable absolutely must have permission to bind to privileged ports. This is required for the fast_server and the SMTP server processes. Since this permission is an extended attribute of the file, upgrading the ruby package may mean the permission is lost and needs to be reinstated.
The command from Postal’s installation script is: –
sudo setcap 'cap_net_bind_service=+ep' /usr/bin/ruby2.3
What makes this an issue is that there’s nothing in the logs to show why the server processes have failed to start. The file log/rails.log only shows ‘Raven 2.1.0 configured not to capture errors’. Perhaps the configuration file ‘app/config/environments/production.rb’ can be edited to provide more useful information?
./lib/postal/message_db/message.rb -> create_load(request) is invoked by the Rack Interface class when an image URL is requested. This is Open Tracking, and ultimately results in the ‘load’ details being applied to the database.