haproxy

HAProxy with Let's Encrypt: Automated HTTPS Certificate Management

Patrick de Ruiter

Patrick de Ruiter

— 10 min read
HAProxy with Let's Encrypt: Automated HTTPS Certificate Management

HTTPS is no longer optional - it's a requirement for modern web applications. This guide shows how to integrate HAProxy with Let's Encrypt for fully automated TLS certificate provisioning and renewal, giving you secure HTTPS without manual certificate management.

Overview

The integration involves:

  1. HAProxy serving HTTPS traffic and terminating TLS
  2. Certbot (Let's Encrypt client) obtaining and renewing certificates
  3. Automated certificate deployment to HAProxy
  4. HTTP-01 or DNS-01 challenge handling

Prerequisites

  • HAProxy 1.8+ (2.x recommended)
  • Certbot installed
  • Domain(s) pointing to your HAProxy server
  • Port 80 and 443 accessible from the internet

Installation

Install HAProxy

# Debian/Ubuntu
apt update
apt install haproxy

# RHEL/CentOS
yum install haproxy

Install Certbot

# Debian/Ubuntu
apt install certbot

# RHEL/CentOS
yum install certbot

# Or using snap (recommended by Let's Encrypt)
snap install --classic certbot
ln -s /snap/bin/certbot /usr/bin/certbot

HAProxy Certificate Format

HAProxy requires certificates in a specific format: the certificate chain and private key combined in a single PEM file.

-----BEGIN CERTIFICATE-----
[Your domain certificate]
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
[Intermediate certificate]
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----
[Private key]
-----END PRIVATE KEY-----

Let's Encrypt provides separate files, so we need a script to combine them.

Method 1: HTTP-01 Challenge with HAProxy

The HTTP-01 challenge requires serving a file over HTTP. We configure HAProxy to handle this.

HAProxy Configuration

# /etc/haproxy/haproxy.cfg

global
    log /dev/log local0
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    user haproxy
    group haproxy
    daemon
    
    # SSL/TLS settings
    ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384
    ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
    tune.ssl.default-dh-param 2048

defaults
    log global
    mode http
    option httplog
    option dontlognull
    timeout connect 5s
    timeout client  50s
    timeout server  50s

# HTTP Frontend - handles Let's Encrypt challenges and redirects
frontend http_frontend
    bind *:80
    
    # Let's Encrypt ACME challenge
    acl is_acme_challenge path_beg /.well-known/acme-challenge/
    use_backend letsencrypt_backend if is_acme_challenge
    
    # Redirect all other HTTP to HTTPS
    http-request redirect scheme https unless is_acme_challenge

# HTTPS Frontend
frontend https_frontend
    bind *:443 ssl crt /etc/haproxy/certs/ alpn h2,http/1.1
    
    # HSTS header
    http-response set-header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
    
    # Default backend
    default_backend web_servers

# Backend for Let's Encrypt challenges
backend letsencrypt_backend
    server certbot 127.0.0.1:8888

# Your web servers
backend web_servers
    balance roundrobin
    server web1 192.168.1.10:8080 check
    server web2 192.168.1.11:8080 check

Initial Certificate Request

For the first certificate, use standalone mode before HAProxy is running with SSL:

# Stop HAProxy temporarily
systemctl stop haproxy

# Get initial certificate
certbot certonly --standalone \
  -d example.com \
  -d www.example.com \
  --email admin@example.com \
  --agree-tos \
  --non-interactive

# Combine certificate files for HAProxy
BASE_DIR=/etc/letsencrypt/live/example.com
cat $BASE_DIR/fullchain.pem $BASE_DIR/privkey.pem > /etc/haproxy/certs/example.com.pem
chmod 600 /etc/haproxy/certs/example.com.pem

# Start HAProxy
systemctl start haproxy

Renewal with HTTP-01 Challenge

Once HAProxy is running, use webroot mode for renewals:

# Create directory for challenges
mkdir -p /var/www/letsencrypt/.well-known/acme-challenge

# Use a simple HTTP server for challenges
# Option 1: Python
cd /var/www/letsencrypt && python3 -m http.server 8888 &

# Option 2: socat (install socat package)
# Create a script to serve files

Better approach - use certbot's standalone with HAProxy's passthrough:

certbot certonly --webroot \
  -w /var/www/letsencrypt \
  -d example.com \
  -d www.example.com

Method 2: Using Certbot Standalone with HAProxy

This method temporarily stops HAProxy during renewal.

Pre and Post Hooks

# /etc/letsencrypt/cli.ini

[renewalparams]
pre-hook = systemctl stop haproxy
post-hook = /etc/letsencrypt/renewal-hooks/deploy/haproxy.sh

Deployment Script

#!/bin/bash
# /etc/letsencrypt/renewal-hooks/deploy/haproxy.sh

set -e

HAPROXY_CERT_DIR="/etc/haproxy/certs"

# Process each renewed domain
for domain in $RENEWED_DOMAINS; do
    # Find the certificate directory
    CERT_DIR="/etc/letsencrypt/live/$domain"
    
    if [ -d "$CERT_DIR" ]; then
        # Combine cert and key for HAProxy
        cat "$CERT_DIR/fullchain.pem" "$CERT_DIR/privkey.pem" > "$HAPROXY_CERT_DIR/$domain.pem"
        chmod 600 "$HAPROXY_CERT_DIR/$domain.pem"
        echo "Updated certificate for $domain"
    fi
done

# Reload HAProxy
systemctl reload haproxy || systemctl start haproxy

Make it executable:

chmod +x /etc/letsencrypt/renewal-hooks/deploy/haproxy.sh

This approach uses HAProxy's runtime API to reload certificates without restart.

HAProxy Configuration

# /etc/haproxy/haproxy.cfg

global
    log /dev/log local0
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    
    # Enable runtime API for certificate management
    ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384
    ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets

defaults
    log global
    mode http
    option httplog
    timeout connect 5s
    timeout client  50s
    timeout server  50s

# HTTP Frontend
frontend http_frontend
    bind *:80
    
    # ACME challenge path
    acl is_acme path_beg /.well-known/acme-challenge/
    use_backend acme_backend if is_acme
    
    # Redirect to HTTPS
    http-request redirect scheme https code 301 unless is_acme

# HTTPS Frontend
frontend https_frontend
    bind *:443 ssl crt /etc/haproxy/certs/ strict-sni alpn h2,http/1.1
    
    http-response set-header Strict-Transport-Security "max-age=63072000"
    
    default_backend web_servers

# ACME challenge backend
backend acme_backend
    server acme 127.0.0.1:8080

backend web_servers
    balance roundrobin
    option httpchk GET /health
    server web1 192.168.1.10:8080 check
    server web2 192.168.1.11:8080 check

Zero-Downtime Certificate Update Script

#!/bin/bash
# /usr/local/bin/haproxy-cert-update.sh

set -e

HAPROXY_CERT_DIR="/etc/haproxy/certs"
HAPROXY_SOCKET="/run/haproxy/admin.sock"

update_certificate() {
    local domain=$1
    local cert_dir="/etc/letsencrypt/live/$domain"
    local combined_cert="$HAPROXY_CERT_DIR/$domain.pem"
    
    # Combine certificates
    cat "$cert_dir/fullchain.pem" "$cert_dir/privkey.pem" > "$combined_cert"
    chmod 600 "$combined_cert"
    
    # Check if HAProxy is running with this certificate
    if echo "show ssl cert" | socat stdio "$HAPROXY_SOCKET" | grep -q "$combined_cert"; then
        # Update certificate in running HAProxy
        echo "set ssl cert $combined_cert <<" | socat stdio "$HAPROXY_SOCKET"
        cat "$combined_cert" | socat stdio "$HAPROXY_SOCKET"
        echo "" | socat stdio "$HAPROXY_SOCKET"
        echo "commit ssl cert $combined_cert" | socat stdio "$HAPROXY_SOCKET"
        echo "Certificate for $domain updated in-place"
    else
        # Certificate not yet loaded, reload HAProxy
        systemctl reload haproxy
        echo "HAProxy reloaded for new certificate $domain"
    fi
}

# Process renewed domains
for domain in $RENEWED_DOMAINS; do
    update_certificate "$domain"
done

HAProxy 2.4+ Runtime Certificate API

# List certificates
echo "show ssl cert" | socat stdio /run/haproxy/admin.sock

# Show certificate details
echo "show ssl cert /etc/haproxy/certs/example.com.pem" | socat stdio /run/haproxy/admin.sock

# Update certificate in-place (HAProxy 2.4+)
echo "set ssl cert /etc/haproxy/certs/example.com.pem <<" | socat stdio /run/haproxy/admin.sock
cat /etc/haproxy/certs/example.com.pem | socat stdio /run/haproxy/admin.sock
echo "" | socat stdio /run/haproxy/admin.sock
echo "commit ssl cert /etc/haproxy/certs/example.com.pem" | socat stdio /run/haproxy/admin.sock

Method 4: DNS-01 Challenge (Wildcard Certificates)

For wildcard certificates or when HTTP challenges aren't possible.

With Cloudflare DNS

# Install Cloudflare DNS plugin
apt install python3-certbot-dns-cloudflare

# Or with pip
pip install certbot-dns-cloudflare

Create credentials file:

# /etc/letsencrypt/cloudflare.ini
dns_cloudflare_api_token = your-api-token-here

chmod 600 /etc/letsencrypt/cloudflare.ini

Request wildcard certificate:

certbot certonly \
  --dns-cloudflare \
  --dns-cloudflare-credentials /etc/letsencrypt/cloudflare.ini \
  -d example.com \
  -d '*.example.com' \
  --email admin@example.com \
  --agree-tos

With Route53

pip install certbot-dns-route53

# Configure AWS credentials
export AWS_ACCESS_KEY_ID=your-key
export AWS_SECRET_ACCESS_KEY=your-secret

certbot certonly \
  --dns-route53 \
  -d example.com \
  -d '*.example.com'

Automated Renewal Setup

Systemd Timer

# /etc/systemd/system/certbot-renewal.service
[Unit]
Description=Certbot Renewal
After=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/bin/certbot renew --deploy-hook /etc/letsencrypt/renewal-hooks/deploy/haproxy.sh

# /etc/systemd/system/certbot-renewal.timer
[Unit]
Description=Run certbot renewal twice daily

[Timer]
OnCalendar=*-*-* 00,12:00:00
RandomizedDelaySec=43200
Persistent=true

[Install]
WantedBy=timers.target

Enable the timer:

systemctl daemon-reload
systemctl enable certbot-renewal.timer
systemctl start certbot-renewal.timer

Cron Job Alternative

# /etc/cron.d/certbot
0 0,12 * * * root certbot renew --deploy-hook /etc/letsencrypt/renewal-hooks/deploy/haproxy.sh

Multi-Domain Configuration

Separate Certificates per Domain

frontend https_frontend
    bind *:443 ssl crt /etc/haproxy/certs/example.com.pem crt /etc/haproxy/certs/example.org.pem
    
    # Route based on SNI
    use_backend example_com_servers if { ssl_fc_sni example.com }
    use_backend example_org_servers if { ssl_fc_sni example.org }

Certificate Directory

frontend https_frontend
    # Load all .pem files from directory
    bind *:443 ssl crt /etc/haproxy/certs/

HAProxy automatically matches certificates based on SNI.

Request Multiple Certificates

#!/bin/bash
# /usr/local/bin/request-certs.sh

DOMAINS=(
    "example.com www.example.com"
    "api.example.com"
    "app.example.com"
)

for domain_set in "${DOMAINS[@]}"; do
    primary=$(echo $domain_set | awk '{print $1}')
    
    certbot certonly \
        --webroot \
        -w /var/www/letsencrypt \
        $(for d in $domain_set; do echo "-d $d"; done) \
        --cert-name $primary \
        --non-interactive
done

Complete Production Setup

HAProxy Configuration

# /etc/haproxy/haproxy.cfg

global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    
    maxconn 50000
    nbthread 4
    
    # Modern SSL settings
    ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305
    ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
    ssl-default-server-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384
    ssl-default-server-options ssl-min-ver TLSv1.2 no-tls-tickets
    tune.ssl.default-dh-param 2048

defaults
    log global
    mode http
    option httplog
    option dontlognull
    option forwardfor
    option http-server-close
    timeout connect 5s
    timeout client  50s
    timeout server  50s
    timeout http-request 10s
    timeout http-keep-alive 10s

# Stats
listen stats
    bind *:8404
    stats enable
    stats uri /stats
    stats refresh 10s
    stats auth admin:changeme
    http-request use-service prometheus-exporter if { path /metrics }

# HTTP Frontend
frontend http_frontend
    bind *:80
    
    # ACME challenges
    acl is_acme path_beg /.well-known/acme-challenge/
    use_backend acme_backend if is_acme
    
    # Redirect to HTTPS with 301
    http-request redirect scheme https code 301 unless is_acme

# HTTPS Frontend
frontend https_frontend
    bind *:443 ssl crt /etc/haproxy/certs/ strict-sni alpn h2,http/1.1
    
    # Security headers
    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    http-response set-header X-Content-Type-Options "nosniff"
    http-response set-header X-Frame-Options "SAMEORIGIN"
    http-response set-header X-XSS-Protection "1; mode=block"
    
    # Log SSL info
    http-request capture req.hdr(Host) len 50
    http-request set-header X-Forwarded-Proto https
    http-request set-header X-SSL-Client-Verify %[ssl_c_verify]
    http-request set-header X-SSL-Client-DN %{+Q}[ssl_c_s_dn]
    
    # Route based on host
    use_backend api_servers if { hdr(host) -i api.example.com }
    use_backend app_servers if { hdr(host) -i app.example.com }
    default_backend web_servers

# ACME challenge backend
backend acme_backend
    server acme 127.0.0.1:8080

# Web servers
backend web_servers
    balance roundrobin
    option httpchk GET /health
    http-check expect status 200
    server web1 192.168.1.10:8080 check inter 5s
    server web2 192.168.1.11:8080 check inter 5s

# API servers
backend api_servers
    balance leastconn
    option httpchk GET /api/health
    http-check expect status 200
    server api1 192.168.1.20:8080 check inter 5s
    server api2 192.168.1.21:8080 check inter 5s

# App servers
backend app_servers
    balance roundrobin
    option httpchk GET /health
    cookie SERVERID insert indirect nocache
    server app1 192.168.1.30:8080 check cookie app1
    server app2 192.168.1.31:8080 check cookie app2

Complete Deployment Script

#!/bin/bash
# /usr/local/bin/setup-haproxy-letsencrypt.sh

set -e

DOMAINS="$@"
EMAIL="admin@example.com"
HAPROXY_CERT_DIR="/etc/haproxy/certs"
WEBROOT="/var/www/letsencrypt"

# Create directories
mkdir -p $HAPROXY_CERT_DIR
mkdir -p $WEBROOT/.well-known/acme-challenge
chown -R www-data:www-data $WEBROOT

# Create ACME challenge server
cat > /etc/systemd/system/acme-challenge.service << 'EOF'
[Unit]
Description=ACME Challenge Server
After=network.target

[Service]
Type=simple
User=www-data
WorkingDirectory=/var/www/letsencrypt
ExecStart=/usr/bin/python3 -m http.server 8080
Restart=always

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable acme-challenge
systemctl start acme-challenge

# Request certificates
for domain in $DOMAINS; do
    echo "Requesting certificate for $domain..."
    
    certbot certonly \
        --webroot \
        -w $WEBROOT \
        -d $domain \
        --email $EMAIL \
        --agree-tos \
        --non-interactive \
        --deploy-hook /etc/letsencrypt/renewal-hooks/deploy/haproxy.sh
    
    # Create HAProxy certificate
    cat /etc/letsencrypt/live/$domain/fullchain.pem \
        /etc/letsencrypt/live/$domain/privkey.pem \
        > $HAPROXY_CERT_DIR/$domain.pem
    chmod 600 $HAPROXY_CERT_DIR/$domain.pem
done

# Reload HAProxy
systemctl reload haproxy

echo "Setup complete!"

Deploy Hook Script

#!/bin/bash
# /etc/letsencrypt/renewal-hooks/deploy/haproxy.sh

set -e

HAPROXY_CERT_DIR="/etc/haproxy/certs"
HAPROXY_SOCKET="/run/haproxy/admin.sock"
LOG_FILE="/var/log/certbot-haproxy.log"

log() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> $LOG_FILE
}

update_haproxy_cert() {
    local domain=$1
    local cert_path="$HAPROXY_CERT_DIR/$domain.pem"
    local le_cert="/etc/letsencrypt/live/$domain"
    
    # Combine certificates
    cat "$le_cert/fullchain.pem" "$le_cert/privkey.pem" > "$cert_path"
    chmod 600 "$cert_path"
    
    # Try runtime update first (HAProxy 2.4+)
    if [ -S "$HAPROXY_SOCKET" ]; then
        if echo "show ssl cert $cert_path" | socat stdio "$HAPROXY_SOCKET" 2>/dev/null | grep -q "$cert_path"; then
            # Certificate exists, update in-place
            {
                echo "set ssl cert $cert_path <<"
                cat "$cert_path"
                echo ""
                echo "commit ssl cert $cert_path"
            } | socat stdio "$HAPROXY_SOCKET" 2>/dev/null
            
            if [ $? -eq 0 ]; then
                log "Updated certificate for $domain via runtime API"
                return 0
            fi
        fi
    fi
    
    # Fallback to reload
    systemctl reload haproxy
    log "Reloaded HAProxy for $domain"
}

# Process each renewed domain
for domain in $RENEWED_DOMAINS; do
    log "Processing renewal for $domain"
    update_haproxy_cert "$domain"
done

Testing and Verification

Test Certificate

# Check certificate
openssl s_client -connect example.com:443 -servername example.com < /dev/null 2>/dev/null | openssl x509 -text -noout | head -20

# Verify chain
openssl s_client -connect example.com:443 -servername example.com < /dev/null 2>/dev/null | openssl verify

# Check expiration
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null | openssl x509 -noout -dates

SSL Labs Test

curl -s "https://api.ssllabs.com/api/v3/analyze?host=example.com&publish=off&all=done"

Test Renewal

certbot renew --dry-run

Troubleshooting

Common Issues

  1. Permission denied on socket:
chmod 660 /run/haproxy/admin.sock
chown haproxy:haproxy /run/haproxy/admin.sock
  1. Certificate not found:
# Check certificate directory
ls -la /etc/haproxy/certs/

# Verify HAProxy can read them
su -s /bin/bash haproxy -c "cat /etc/haproxy/certs/example.com.pem"
  1. ACME challenge fails:
# Verify challenge server is running
curl http://localhost:8080/.well-known/acme-challenge/test

# Check HAProxy routing
curl -H "Host: example.com" http://localhost/.well-known/acme-challenge/test
  1. SNI not matching:
# Test specific SNI
curl -v --resolve example.com:443:127.0.0.1 https://example.com/

Next Steps

This post covered automated HTTPS with Let's Encrypt. In the next post, we'll explore HTTP headers debugging in HAProxy, which is essential for troubleshooting web application issues.

Summary

Key takeaways for HAProxy with Let's Encrypt:

  • Combine fullchain.pem and privkey.pem for HAProxy certificates
  • Use HTTP-01 challenges for standard domains, DNS-01 for wildcards
  • Implement zero-downtime certificate updates with HAProxy's runtime API
  • Set up automated renewal with systemd timers or cron
  • Use proper TLS settings for security (TLS 1.2+, strong ciphers)
  • Always enable HSTS for production sites

With this setup, you'll have fully automated, secure HTTPS with certificates that renew themselves indefinitely.

Read more

HAProxy Monitoring with Prometheus: Complete Observability Guide

HAProxy Monitoring with Prometheus: Complete Observability Guide

Monitoring HAProxy is essential for maintaining reliable load balancing infrastructure. Prometheus provides powerful metrics collection, alerting capabilities, and seamless Grafana integration for visualizing HAProxy performance and health. Why Prometheus for HAProxy? Prometheus offers: * Pull-based metrics - Prometheus scrapes HAProxy metrics endpoints * Time-series database - Store historical data for trend analysis

By Patrick de Ruiter