Comparing HAProxy and NGINX as SNI proxies

Posted on August 20, 2026

Recently, I slightly misconfigured my Stalwart mailserver. Gmail sent me friendly reminders to postmaster, that it could not find a policy for MTA-STS, short for Mail Transfer Agent Strict Transport Security. This is a mechanism for a sending mailserver to determine, if STARTTLS should be used for incoming mail via a secure channel, namely HTTPS. It requests a file at https://mta-sts.<mail-domain>/.well-known/mta-sts.txt, in my case, https://mta-sts.m.filts.net/.well-known/mta-sts.txt.

The issue is, that i used NGINX to terminate HTTPS and it only has a wildcard certificate for *.filts.net, not *.m.filts.net. Those are available in Stalwart’s own certificate. Also, I had no configuration set up to proxy the sub-sub-domains, but without a valid certificate, that would not have worked anyway.

SNI Proxies

Server Name Indication (SNI) is part of the TLS connection setup to allow multiple certificates on a single IP address, by specifying the target domain name in cleartext. A SNI Proxy can use this bit of unencrypted information to pass the TCP connection unaltered to an upstream server, that then handles encryption using its own keys and certificate.

Both, HAProxy and NGINX, support SNI proxying. I already had NGINX in use, so what is the big deal? As far as I can tell, NGINX can not be configured to reverse proxy TCP connections based on SNI and also act as a normal webserver on the same listener. The way the LLMs suggested, was by annotating each virtualHosts section with something like

      listen = [
        {
          addr = "127.0.0.1";
          port = 8444;
          ssl = true;
        }
      ];

Ugly, but should work, and I don’t have that many virtual hosts defined.

HAProxy

For comparison, I wanted to see what a solution based on HAProxy would look like. HAProxy is where The PROXY protocol originated, which is accepted by Stalwart and Kanidm, and Stalwart works better using that protocol instead of X-Forwarded-For: headers. The module for HAProxy and all related changes can be viewed in this commit. It does not look too bad.

One downside is that the proxy configuration can no longer be colocated with the respective users in modules, as I was doing before. See the changes to stalwart.nix and kanidm.nix, to see what I mean.

NGINX

With a working configuration for HAProxy and NGINX upstream configuraion, I wondered, if an LLM can transform that into a configuration for NGINX only. Indeed, CLaude came up with a working solution. It even noticed, that NGINX 1.31.4+ supports the proxy protocol v2 on upstreams, which is a feature I want. That version was released yesterday!

The virtualHosts attribute sets take the listener configuration from defaultListen attribute, as before, while the streamConfig for the SNI proxy defines its own listener on public port 443.

The upside is that I don’t need a seperate process for the SNI proxy and can spare some RAM. I went with NGINX and will switch to proxy protocol v2 when 1.31.4 gets packaged in nixpkgs.