haproxy

HAProxy Data Plane API: Dynamic Configuration Management

Patrick de Ruiter

Patrick de Ruiter

— 9 min read
HAProxy Data Plane API: Dynamic Configuration Management

Managing HAProxy configuration traditionally requires editing files and reloading the service. The Data Plane API transforms HAProxy into a dynamically configurable load balancer with a RESTful interface, enabling real-time configuration changes without service interruption.

Why Data Plane API?

The Data Plane API provides:

  • Runtime configuration - Add/remove backends and servers without reloads
  • RESTful interface - Standard HTTP methods for configuration management
  • Transaction support - Atomic changes with rollback capability
  • Integration friendly - Perfect for CI/CD pipelines and orchestration tools
  • Version control - Track configuration changes over time

Installing the Data Plane API

Download and Install

# Download the latest release
DATAPLANE_VERSION="2.9.0"
wget https://github.com/haproxytech/dataplaneapi/releases/download/v${DATAPLANEAPI_VERSION}/dataplaneapi_${DATAPLANEAPI_VERSION}_linux_amd64.tar.gz

# Extract and install
tar xzf dataplaneapi_${DATAPLANEAPI_VERSION}_linux_amd64.tar.gz
sudo mv dataplaneapi /usr/local/bin/
sudo chmod +x /usr/local/bin/dataplaneapi

Configuration File

Create /etc/haproxy/dataplaneapi.yaml:

config_version: 2
name: haproxy_dataplane
mode: single

dataplaneapi:
  host: 0.0.0.0
  port: 5555
  scheme:
    - http
  
  user:
    - name: admin
      password: $6$rounds=500000$...$...  # Use mkpasswd to generate
      insecure: false
  
  transaction:
    transaction_dir: /tmp/haproxy
  
  resources:
    maps_dir: /etc/haproxy/maps
    ssl_certs_dir: /etc/haproxy/certs
    spoe_dir: /etc/haproxy/spoe
    spoe_transaction_dir: /tmp/spoe-haproxy

haproxy:
  config_file: /etc/haproxy/haproxy.cfg
  haproxy_bin: /usr/sbin/haproxy
  reload:
    reload_delay: 5
    reload_cmd: systemctl reload haproxy
    restart_cmd: systemctl restart haproxy
    status_cmd: systemctl status haproxy

Generate Password Hash

# Install mkpasswd if needed
sudo apt install whois

# Generate SHA-512 password hash
mkpasswd -m sha-512 'your-secure-password'

HAProxy Configuration for API

Add to /etc/haproxy/haproxy.cfg:

global
    log /dev/log local0
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    
    # Required for Data Plane API
    master-worker

defaults
    log global
    mode http
    option httplog
    option dontlognull
    timeout connect 5000
    timeout client 50000
    timeout server 50000

# Data Plane API Frontend
frontend dataplane_api
    bind *:5556 ssl crt /etc/haproxy/certs/api.pem
    mode http
    http-request auth unless { http_auth(api_users) }
    default_backend dataplane_api_backend

backend dataplane_api_backend
    mode http
    server dataplane 127.0.0.1:5555

userlist api_users
    user admin password $6$rounds=500000$...$...

# Application frontends and backends
frontend http_front
    bind *:80
    bind *:443 ssl crt /etc/haproxy/certs/
    default_backend web_servers

backend web_servers
    balance roundrobin
    option httpchk GET /health
    server web1 192.168.1.10:8080 check
    server web2 192.168.1.11:8080 check

Starting the Data Plane API

Systemd Service

Create /etc/systemd/system/dataplaneapi.service:

[Unit]
Description=HAProxy Data Plane API
After=network.target haproxy.service
Requires=haproxy.service

[Service]
Type=simple
ExecStart=/usr/local/bin/dataplaneapi -f /etc/haproxy/dataplaneapi.yaml
Restart=on-failure
RestartSec=5
User=haproxy
Group=haproxy

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable dataplaneapi
sudo systemctl start dataplaneapi

API Operations

Authentication

# Set credentials
export HAPROXY_API="https://haproxy.example.com:5556/v2"
export AUTH="admin:your-secure-password"

# Test connection
curl -s -u $AUTH $HAPROXY_API/info | jq .

Get Current Configuration

# Get HAProxy configuration
curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/raw | jq .

# Get specific backend
curl -s -u $AUTH "$HAPROXY_API/services/haproxy/configuration/backends/web_servers" | jq .

# List all servers in a backend
curl -s -u $AUTH "$HAPROXY_API/services/haproxy/configuration/servers?backend=web_servers" | jq .

Working with Transactions

The API uses transactions to ensure atomic configuration changes:

# Get current configuration version
VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
echo "Current version: $VERSION"

# Start a transaction
TRANSACTION=$(curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')
echo "Transaction ID: $TRANSACTION"

# Make changes within transaction (see examples below)

# Commit transaction
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/transactions/$TRANSACTION"

# Or rollback if needed
curl -s -u $AUTH -X DELETE \
  "$HAPROXY_API/services/haproxy/transactions/$TRANSACTION"

Adding a Server

# Get version and start transaction
VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
TRANSACTION=$(curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')

# Add new server
curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/configuration/servers?backend=web_servers&transaction_id=$TRANSACTION" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "web3",
    "address": "192.168.1.12",
    "port": 8080,
    "check": "enabled",
    "weight": 100
  }'

# Commit changes
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/transactions/$TRANSACTION"

Removing a Server

VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
TRANSACTION=$(curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')

# Remove server
curl -s -u $AUTH -X DELETE \
  "$HAPROXY_API/services/haproxy/configuration/servers/web3?backend=web_servers&transaction_id=$TRANSACTION"

curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/transactions/$TRANSACTION"

Modifying Server Weight

VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
TRANSACTION=$(curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')

# Update server weight
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/configuration/servers/web1?backend=web_servers&transaction_id=$TRANSACTION" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "web1",
    "address": "192.168.1.10",
    "port": 8080,
    "check": "enabled",
    "weight": 50
  }'

curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/transactions/$TRANSACTION"

Creating a New Backend

VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
TRANSACTION=$(curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')

# Create backend
curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/configuration/backends?transaction_id=$TRANSACTION" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "api_servers",
    "mode": "http",
    "balance": {"algorithm": "leastconn"},
    "httpchk": {
      "method": "GET",
      "uri": "/health"
    }
  }'

# Add servers to new backend
curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/configuration/servers?backend=api_servers&transaction_id=$TRANSACTION" \
  -H "Content-Type: application/json" \
  -d '{"name": "api1", "address": "192.168.1.20", "port": 3000, "check": "enabled"}'

curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/configuration/servers?backend=api_servers&transaction_id=$TRANSACTION" \
  -H "Content-Type: application/json" \
  -d '{"name": "api2", "address": "192.168.1.21", "port": 3000, "check": "enabled"}'

curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/transactions/$TRANSACTION"

Runtime API Operations

Some changes can be made at runtime without configuration reload:

Server State Management

# Get server runtime state
curl -s -u $AUTH "$HAPROXY_API/services/haproxy/runtime/servers?backend=web_servers" | jq .

# Disable server (drain mode)
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/runtime/servers/web1?backend=web_servers" \
  -H "Content-Type: application/json" \
  -d '{"admin_state": "drain"}'

# Set server to maintenance
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/runtime/servers/web1?backend=web_servers" \
  -H "Content-Type: application/json" \
  -d '{"admin_state": "maint"}'

# Re-enable server
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/runtime/servers/web1?backend=web_servers" \
  -H "Content-Type: application/json" \
  -d '{"admin_state": "ready"}'

Runtime Weight Adjustment

# Adjust weight without reload
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/runtime/servers/web1?backend=web_servers" \
  -H "Content-Type: application/json" \
  -d '{"operational_state": "up", "weight": 150}'

Automation Scripts

Blue-Green Deployment Script

#!/bin/bash
# blue-green-deploy.sh - Switch traffic between blue and green environments

set -e

HAPROXY_API="https://haproxy.example.com:5556/v2"
AUTH="admin:your-secure-password"
BACKEND="app_servers"

BLUE_SERVERS=("blue1:192.168.1.10:8080" "blue2:192.168.1.11:8080")
GREEN_SERVERS=("green1:192.168.1.20:8080" "green2:192.168.1.21:8080")

get_version() {
  curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version
}

start_transaction() {
  curl -s -u $AUTH -X POST \
    "$HAPROXY_API/services/haproxy/transactions?version=$1" | jq -r '.id'
}

add_server() {
  local name=$1
  local addr=$2
  local port=$3
  local txn=$4
  
  curl -s -u $AUTH -X POST \
    "$HAPROXY_API/services/haproxy/configuration/servers?backend=$BACKEND&transaction_id=$txn" \
    -H "Content-Type: application/json" \
    -d "{\"name\": \"$name\", \"address\": \"$addr\", \"port\": $port, \"check\": \"enabled\"}"
}

remove_server() {
  local name=$1
  local txn=$2
  
  curl -s -u $AUTH -X DELETE \
    "$HAPROXY_API/services/haproxy/configuration/servers/${name}?backend=$BACKEND&transaction_id=$txn"
}

commit_transaction() {
  curl -s -u $AUTH -X PUT \
    "$HAPROXY_API/services/haproxy/transactions/$1"
}

deploy_environment() {
  local target=$1
  local servers_to_add
  local servers_to_remove
  
  if [ "$target" == "green" ]; then
    servers_to_add=("${GREEN_SERVERS[@]}")
    servers_to_remove=("${BLUE_SERVERS[@]}")
  else
    servers_to_add=("${BLUE_SERVERS[@]}")
    servers_to_remove=("${GREEN_SERVERS[@]}")
  fi
  
  echo "Switching to $target environment..."
  
  VERSION=$(get_version)
  TXN=$(start_transaction $VERSION)
  
  # Add new servers
  for server in "${servers_to_add[@]}"; do
    IFS=':' read -r name addr port <<< "$server"
    echo "Adding $name ($addr:$port)"
    add_server $name $addr $port $TXN
  done
  
  # Remove old servers
  for server in "${servers_to_remove[@]}"; do
    IFS=':' read -r name addr port <<< "$server"
    echo "Removing $name"
    remove_server $name $TXN 2>/dev/null || true
  done
  
  commit_transaction $TXN
  echo "Deployment complete!"
}

case "$1" in
  blue)  deploy_environment blue ;;
  green) deploy_environment green ;;
  *)     echo "Usage: $0 {blue|green}" ;;
esac

Canary Deployment Script

#!/bin/bash
# canary-deploy.sh - Gradual traffic shifting to new version

set -e

HAPROXY_API="https://haproxy.example.com:5556/v2"
AUTH="admin:your-secure-password"
BACKEND="web_servers"

CANARY_SERVER="web-canary"
CANARY_ADDRESS="192.168.1.50"
CANARY_PORT="8080"

shift_traffic() {
  local canary_weight=$1
  local production_weight=$((100 - canary_weight))
  
  echo "Shifting traffic: Canary=$canary_weight%, Production=$production_weight%"
  
  # Use runtime API for immediate weight changes
  curl -s -u $AUTH -X PUT \
    "$HAPROXY_API/services/haproxy/runtime/servers/${CANARY_SERVER}?backend=$BACKEND" \
    -H "Content-Type: application/json" \
    -d "{\"weight\": $canary_weight}"
}

add_canary() {
  VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
  TXN=$(curl -s -u $AUTH -X POST \
    "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')
  
  curl -s -u $AUTH -X POST \
    "$HAPROXY_API/services/haproxy/configuration/servers?backend=$BACKEND&transaction_id=$TXN" \
    -H "Content-Type: application/json" \
    -d '{
      "name": "'$CANARY_SERVER'",
      "address": "'$CANARY_ADDRESS'",
      "port": '$CANARY_PORT',
      "check": "enabled",
      "weight": 0
    }'
  
  curl -s -u $AUTH -X PUT "$HAPROXY_API/services/haproxy/transactions/$TXN"
  echo "Canary server added with 0% traffic"
}

remove_canary() {
  VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
  TXN=$(curl -s -u $AUTH -X POST \
    "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')
  
  curl -s -u $AUTH -X DELETE \
    "$HAPROXY_API/services/haproxy/configuration/servers/${CANARY_SERVER}?backend=$BACKEND&transaction_id=$TXN"
  
  curl -s -u $AUTH -X PUT "$HAPROXY_API/services/haproxy/transactions/$TXN"
  echo "Canary server removed"
}

case "$1" in
  add)    add_canary ;;
  remove) remove_canary ;;
  shift)  shift_traffic $2 ;;
  *)      echo "Usage: $0 {add|remove|shift <percentage>}" ;;
esac

Python Client Library

#!/usr/bin/env python3
import requests
from requests.auth import HTTPBasicAuth
import json

class HAProxyAPI:
    def __init__(self, base_url, username, password):
        self.base_url = base_url.rstrip('/')
        self.auth = HTTPBasicAuth(username, password)
        self.session = requests.Session()
        self.session.auth = self.auth
        self.session.verify = True  # Set to False for self-signed certs
    
    def get_version(self):
        """Get current configuration version"""
        r = self.session.get(f"{self.base_url}/services/haproxy/configuration/version")
        return r.text.strip()
    
    def start_transaction(self, version=None):
        """Start a new transaction"""
        if version is None:
            version = self.get_version()
        r = self.session.post(f"{self.base_url}/services/haproxy/transactions?version={version}")
        return r.json()['id']
    
    def commit_transaction(self, txn_id):
        """Commit a transaction"""
        r = self.session.put(f"{self.base_url}/services/haproxy/transactions/{txn_id}")
        return r.status_code == 202
    
    def rollback_transaction(self, txn_id):
        """Rollback a transaction"""
        r = self.session.delete(f"{self.base_url}/services/haproxy/transactions/{txn_id}")
        return r.status_code == 204
    
    def get_backends(self):
        """List all backends"""
        r = self.session.get(f"{self.base_url}/services/haproxy/configuration/backends")
        return r.json().get('data', [])
    
    def get_servers(self, backend):
        """List servers in a backend"""
        r = self.session.get(
            f"{self.base_url}/services/haproxy/configuration/servers",
            params={'backend': backend}
        )
        return r.json().get('data', [])
    
    def add_server(self, backend, name, address, port, weight=100, check=True):
        """Add a server to a backend"""
        txn = self.start_transaction()
        
        data = {
            'name': name,
            'address': address,
            'port': port,
            'weight': weight,
            'check': 'enabled' if check else 'disabled'
        }
        
        r = self.session.post(
            f"{self.base_url}/services/haproxy/configuration/servers",
            params={'backend': backend, 'transaction_id': txn},
            json=data
        )
        
        if r.status_code in [200, 201, 202]:
            self.commit_transaction(txn)
            return True
        else:
            self.rollback_transaction(txn)
            return False
    
    def remove_server(self, backend, name):
        """Remove a server from a backend"""
        txn = self.start_transaction()
        
        r = self.session.delete(
            f"{self.base_url}/services/haproxy/configuration/servers/{name}",
            params={'backend': backend, 'transaction_id': txn}
        )
        
        if r.status_code in [200, 202, 204]:
            self.commit_transaction(txn)
            return True
        else:
            self.rollback_transaction(txn)
            return False
    
    def set_server_state(self, backend, name, state):
        """Set server runtime state (ready/drain/maint)"""
        r = self.session.put(
            f"{self.base_url}/services/haproxy/runtime/servers/{name}",
            params={'backend': backend},
            json={'admin_state': state}
        )
        return r.status_code == 200
    
    def set_server_weight(self, backend, name, weight):
        """Set server weight at runtime"""
        r = self.session.put(
            f"{self.base_url}/services/haproxy/runtime/servers/{name}",
            params={'backend': backend},
            json={'weight': weight}
        )
        return r.status_code == 200


# Usage example
if __name__ == '__main__':
    api = HAProxyAPI(
        'https://haproxy.example.com:5556/v2',
        'admin',
        'your-secure-password'
    )
    
    # List backends
    print("Backends:")
    for backend in api.get_backends():
        print(f"  - {backend['name']}")
    
    # List servers
    print("\nServers in web_servers:")
    for server in api.get_servers('web_servers'):
        print(f"  - {server['name']}: {server['address']}:{server['port']}")
    
    # Add a new server
    print("\nAdding new server...")
    api.add_server('web_servers', 'web4', '192.168.1.14', 8080)
    
    # Put server in maintenance
    print("Setting web4 to maintenance...")
    api.set_server_state('web_servers', 'web4', 'maint')

Security Best Practices

TLS Configuration

# dataplaneapi.yaml with TLS
dataplaneapi:
  host: 0.0.0.0
  port: 5555
  scheme:
    - https
  
  tls:
    tls_certificate: /etc/haproxy/certs/api.pem
    tls_key: /etc/haproxy/certs/api.key
    tls_ca: /etc/haproxy/certs/ca.pem  # For mTLS
    tls_port: 5555

Network Restrictions

frontend dataplane_api
    bind *:5556 ssl crt /etc/haproxy/certs/api.pem
    
    # Restrict to management network
    acl allowed_networks src 10.0.0.0/8 192.168.0.0/16
    http-request deny unless allowed_networks
    
    # Rate limiting
    stick-table type ip size 100k expire 30s store http_req_rate(10s)
    http-request track-sc0 src
    http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }
    
    http-request auth unless { http_auth(api_users) }
    default_backend dataplane_api_backend

Integration with CI/CD

GitLab CI Example

# .gitlab-ci.yml
stages:
  - deploy
  - traffic

variables:
  HAPROXY_API: "https://haproxy.example.com:5556/v2"

deploy-canary:
  stage: deploy
  script:
    - |
      # Add canary server
      VERSION=$(curl -s -u $HAPROXY_AUTH $HAPROXY_API/services/haproxy/configuration/version)
      TXN=$(curl -s -u $HAPROXY_AUTH -X POST \
        "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')
      
      curl -s -u $HAPROXY_AUTH -X POST \
        "$HAPROXY_API/services/haproxy/configuration/servers?backend=app_servers&transaction_id=$TXN" \
        -H "Content-Type: application/json" \
        -d '{"name": "canary", "address": "'$CANARY_IP'", "port": 8080, "weight": 10, "check": "enabled"}'
      
      curl -s -u $HAPROXY_AUTH -X PUT "$HAPROXY_API/services/haproxy/transactions/$TXN"
  only:
    - main

shift-traffic-50:
  stage: traffic
  script:
    - |
      curl -s -u $HAPROXY_AUTH -X PUT \
        "$HAPROXY_API/services/haproxy/runtime/servers/canary?backend=app_servers" \
        -H "Content-Type: application/json" \
        -d '{"weight": 50}'
  when: manual
  only:
    - main

promote-canary:
  stage: traffic
  script:
    - |
      # Shift all traffic to canary
      curl -s -u $HAPROXY_AUTH -X PUT \
        "$HAPROXY_API/services/haproxy/runtime/servers/canary?backend=app_servers" \
        -d '{"weight": 100}'
      
      # Set old servers to drain
      for server in web1 web2 web3; do
        curl -s -u $HAPROXY_AUTH -X PUT \
          "$HAPROXY_API/services/haproxy/runtime/servers/${server}?backend=app_servers" \
          -d '{"admin_state": "drain"}'
      done
  when: manual
  only:
    - main

Troubleshooting

Check API Status

# Test API connectivity
curl -v -u $AUTH $HAPROXY_API/info

# Check pending transactions
curl -s -u $AUTH $HAPROXY_API/services/haproxy/transactions | jq .

# View HAProxy stats
curl -s -u $AUTH $HAPROXY_API/services/haproxy/stats/native | jq .

Common Issues

  1. Transaction conflicts: Always use fresh version number
  2. Permission denied: Check user permissions in dataplaneapi.yaml
  3. Configuration invalid: Use GET endpoints to verify current state
  4. Reload failures: Check HAProxy configuration syntax

Next Steps

Now that you can dynamically manage HAProxy, consider:

  • Service Discovery Integration - Auto-register services from Consul or Kubernetes
  • Monitoring - Track configuration changes and performance metrics
  • GitOps - Version control your HAProxy configuration changes

The Data Plane API transforms HAProxy from a static configuration file into a dynamic, programmable load balancer that integrates seamlessly with modern infrastructure automation.

Read more

HAProxy Monitoring with Prometheus: Complete Observability Guide

HAProxy Monitoring with Prometheus: Complete Observability Guide

Monitoring HAProxy is essential for maintaining reliable load balancing infrastructure. Prometheus provides powerful metrics collection, alerting capabilities, and seamless Grafana integration for visualizing HAProxy performance and health. Why Prometheus for HAProxy? Prometheus offers: * Pull-based metrics - Prometheus scrapes HAProxy metrics endpoints * Time-series database - Store historical data for trend analysis

By Patrick de Ruiter