haproxy

HAProxy Data Plane API: Dynamic Configuration Management

Patrick de Ruiter

Patrick de Ruiter

— 9 min read
Share
HAProxy Data Plane API: Dynamic Configuration Management

Managing HAProxy configuration traditionally requires editing files and reloading the service. The Data Plane API transforms HAProxy into a dynamically configurable load balancer with a RESTful interface, enabling real-time configuration changes without service interruption.

Why Data Plane API?

The Data Plane API provides:

  • Runtime configuration - Add/remove backends and servers without reloads
  • RESTful interface - Standard HTTP methods for configuration management
  • Transaction support - Atomic changes with rollback capability
  • Integration friendly - Perfect for CI/CD pipelines and orchestration tools
  • Version control - Track configuration changes over time

Installing the Data Plane API

Download and Install

# Download the latest release
DATAPLANE_VERSION="2.9.0"
wget https://github.com/haproxytech/dataplaneapi/releases/download/v${DATAPLANEAPI_VERSION}/dataplaneapi_${DATAPLANEAPI_VERSION}_linux_amd64.tar.gz

# Extract and install
tar xzf dataplaneapi_${DATAPLANEAPI_VERSION}_linux_amd64.tar.gz
sudo mv dataplaneapi /usr/local/bin/
sudo chmod +x /usr/local/bin/dataplaneapi

Configuration File

Create /etc/haproxy/dataplaneapi.yaml:

config_version: 2
name: haproxy_dataplane
mode: single

dataplaneapi:
  host: 0.0.0.0
  port: 5555
  scheme:
    - http
  
  user:
    - name: admin
      password: $6$rounds=500000$...$...  # Use mkpasswd to generate
      insecure: false
  
  transaction:
    transaction_dir: /tmp/haproxy
  
  resources:
    maps_dir: /etc/haproxy/maps
    ssl_certs_dir: /etc/haproxy/certs
    spoe_dir: /etc/haproxy/spoe
    spoe_transaction_dir: /tmp/spoe-haproxy

haproxy:
  config_file: /etc/haproxy/haproxy.cfg
  haproxy_bin: /usr/sbin/haproxy
  reload:
    reload_delay: 5
    reload_cmd: systemctl reload haproxy
    restart_cmd: systemctl restart haproxy
    status_cmd: systemctl status haproxy

Generate Password Hash

# Install mkpasswd if needed
sudo apt install whois

# Generate SHA-512 password hash
mkpasswd -m sha-512 'your-secure-password'

HAProxy Configuration for API

Add to /etc/haproxy/haproxy.cfg:

global
    log /dev/log local0
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    
    # Required for Data Plane API
    master-worker

defaults
    log global
    mode http
    option httplog
    option dontlognull
    timeout connect 5000
    timeout client 50000
    timeout server 50000

# Data Plane API Frontend
frontend dataplane_api
    bind *:5556 ssl crt /etc/haproxy/certs/api.pem
    mode http
    http-request auth unless { http_auth(api_users) }
    default_backend dataplane_api_backend

backend dataplane_api_backend
    mode http
    server dataplane 127.0.0.1:5555

userlist api_users
    user admin password $6$rounds=500000$...$...

# Application frontends and backends
frontend http_front
    bind *:80
    bind *:443 ssl crt /etc/haproxy/certs/
    default_backend web_servers

backend web_servers
    balance roundrobin
    option httpchk GET /health
    server web1 192.168.1.10:8080 check
    server web2 192.168.1.11:8080 check

Starting the Data Plane API

Systemd Service

Create /etc/systemd/system/dataplaneapi.service:

[Unit]
Description=HAProxy Data Plane API
After=network.target haproxy.service
Requires=haproxy.service

[Service]
Type=simple
ExecStart=/usr/local/bin/dataplaneapi -f /etc/haproxy/dataplaneapi.yaml
Restart=on-failure
RestartSec=5
User=haproxy
Group=haproxy

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable dataplaneapi
sudo systemctl start dataplaneapi

API Operations

Authentication

# Set credentials
export HAPROXY_API="https://haproxy.example.com:5556/v2"
export AUTH="admin:your-secure-password"

# Test connection
curl -s -u $AUTH $HAPROXY_API/info | jq .

Get Current Configuration

# Get HAProxy configuration
curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/raw | jq .

# Get specific backend
curl -s -u $AUTH "$HAPROXY_API/services/haproxy/configuration/backends/web_servers" | jq .

# List all servers in a backend
curl -s -u $AUTH "$HAPROXY_API/services/haproxy/configuration/servers?backend=web_servers" | jq .

Working with Transactions

The API uses transactions to ensure atomic configuration changes:

# Get current configuration version
VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
echo "Current version: $VERSION"

# Start a transaction
TRANSACTION=$(curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')
echo "Transaction ID: $TRANSACTION"

# Make changes within transaction (see examples below)

# Commit transaction
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/transactions/$TRANSACTION"

# Or rollback if needed
curl -s -u $AUTH -X DELETE \
  "$HAPROXY_API/services/haproxy/transactions/$TRANSACTION"

Adding a Server

# Get version and start transaction
VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
TRANSACTION=$(curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')

# Add new server
curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/configuration/servers?backend=web_servers&transaction_id=$TRANSACTION" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "web3",
    "address": "192.168.1.12",
    "port": 8080,
    "check": "enabled",
    "weight": 100
  }'

# Commit changes
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/transactions/$TRANSACTION"

Removing a Server

VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
TRANSACTION=$(curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')

# Remove server
curl -s -u $AUTH -X DELETE \
  "$HAPROXY_API/services/haproxy/configuration/servers/web3?backend=web_servers&transaction_id=$TRANSACTION"

curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/transactions/$TRANSACTION"

Modifying Server Weight

VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
TRANSACTION=$(curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')

# Update server weight
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/configuration/servers/web1?backend=web_servers&transaction_id=$TRANSACTION" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "web1",
    "address": "192.168.1.10",
    "port": 8080,
    "check": "enabled",
    "weight": 50
  }'

curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/transactions/$TRANSACTION"

Creating a New Backend

VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
TRANSACTION=$(curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')

# Create backend
curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/configuration/backends?transaction_id=$TRANSACTION" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "api_servers",
    "mode": "http",
    "balance": {"algorithm": "leastconn"},
    "httpchk": {
      "method": "GET",
      "uri": "/health"
    }
  }'

# Add servers to new backend
curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/configuration/servers?backend=api_servers&transaction_id=$TRANSACTION" \
  -H "Content-Type: application/json" \
  -d '{"name": "api1", "address": "192.168.1.20", "port": 3000, "check": "enabled"}'

curl -s -u $AUTH -X POST \
  "$HAPROXY_API/services/haproxy/configuration/servers?backend=api_servers&transaction_id=$TRANSACTION" \
  -H "Content-Type: application/json" \
  -d '{"name": "api2", "address": "192.168.1.21", "port": 3000, "check": "enabled"}'

curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/transactions/$TRANSACTION"

Runtime API Operations

Some changes can be made at runtime without configuration reload:

Server State Management

# Get server runtime state
curl -s -u $AUTH "$HAPROXY_API/services/haproxy/runtime/servers?backend=web_servers" | jq .

# Disable server (drain mode)
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/runtime/servers/web1?backend=web_servers" \
  -H "Content-Type: application/json" \
  -d '{"admin_state": "drain"}'

# Set server to maintenance
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/runtime/servers/web1?backend=web_servers" \
  -H "Content-Type: application/json" \
  -d '{"admin_state": "maint"}'

# Re-enable server
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/runtime/servers/web1?backend=web_servers" \
  -H "Content-Type: application/json" \
  -d '{"admin_state": "ready"}'

Runtime Weight Adjustment

# Adjust weight without reload
curl -s -u $AUTH -X PUT \
  "$HAPROXY_API/services/haproxy/runtime/servers/web1?backend=web_servers" \
  -H "Content-Type: application/json" \
  -d '{"operational_state": "up", "weight": 150}'

Automation Scripts

Blue-Green Deployment Script

#!/bin/bash
# blue-green-deploy.sh - Switch traffic between blue and green environments

set -e

HAPROXY_API="https://haproxy.example.com:5556/v2"
AUTH="admin:your-secure-password"
BACKEND="app_servers"

BLUE_SERVERS=("blue1:192.168.1.10:8080" "blue2:192.168.1.11:8080")
GREEN_SERVERS=("green1:192.168.1.20:8080" "green2:192.168.1.21:8080")

get_version() {
  curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version
}

start_transaction() {
  curl -s -u $AUTH -X POST \
    "$HAPROXY_API/services/haproxy/transactions?version=$1" | jq -r '.id'
}

add_server() {
  local name=$1
  local addr=$2
  local port=$3
  local txn=$4
  
  curl -s -u $AUTH -X POST \
    "$HAPROXY_API/services/haproxy/configuration/servers?backend=$BACKEND&transaction_id=$txn" \
    -H "Content-Type: application/json" \
    -d "{\"name\": \"$name\", \"address\": \"$addr\", \"port\": $port, \"check\": \"enabled\"}"
}

remove_server() {
  local name=$1
  local txn=$2
  
  curl -s -u $AUTH -X DELETE \
    "$HAPROXY_API/services/haproxy/configuration/servers/${name}?backend=$BACKEND&transaction_id=$txn"
}

commit_transaction() {
  curl -s -u $AUTH -X PUT \
    "$HAPROXY_API/services/haproxy/transactions/$1"
}

deploy_environment() {
  local target=$1
  local servers_to_add
  local servers_to_remove
  
  if [ "$target" == "green" ]; then
    servers_to_add=("${GREEN_SERVERS[@]}")
    servers_to_remove=("${BLUE_SERVERS[@]}")
  else
    servers_to_add=("${BLUE_SERVERS[@]}")
    servers_to_remove=("${GREEN_SERVERS[@]}")
  fi
  
  echo "Switching to $target environment..."
  
  VERSION=$(get_version)
  TXN=$(start_transaction $VERSION)
  
  # Add new servers
  for server in "${servers_to_add[@]}"; do
    IFS=':' read -r name addr port <<< "$server"
    echo "Adding $name ($addr:$port)"
    add_server $name $addr $port $TXN
  done
  
  # Remove old servers
  for server in "${servers_to_remove[@]}"; do
    IFS=':' read -r name addr port <<< "$server"
    echo "Removing $name"
    remove_server $name $TXN 2>/dev/null || true
  done
  
  commit_transaction $TXN
  echo "Deployment complete!"
}

case "$1" in
  blue)  deploy_environment blue ;;
  green) deploy_environment green ;;
  *)     echo "Usage: $0 {blue|green}" ;;
esac

Canary Deployment Script

#!/bin/bash
# canary-deploy.sh - Gradual traffic shifting to new version

set -e

HAPROXY_API="https://haproxy.example.com:5556/v2"
AUTH="admin:your-secure-password"
BACKEND="web_servers"

CANARY_SERVER="web-canary"
CANARY_ADDRESS="192.168.1.50"
CANARY_PORT="8080"

shift_traffic() {
  local canary_weight=$1
  local production_weight=$((100 - canary_weight))
  
  echo "Shifting traffic: Canary=$canary_weight%, Production=$production_weight%"
  
  # Use runtime API for immediate weight changes
  curl -s -u $AUTH -X PUT \
    "$HAPROXY_API/services/haproxy/runtime/servers/${CANARY_SERVER}?backend=$BACKEND" \
    -H "Content-Type: application/json" \
    -d "{\"weight\": $canary_weight}"
}

add_canary() {
  VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
  TXN=$(curl -s -u $AUTH -X POST \
    "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')
  
  curl -s -u $AUTH -X POST \
    "$HAPROXY_API/services/haproxy/configuration/servers?backend=$BACKEND&transaction_id=$TXN" \
    -H "Content-Type: application/json" \
    -d '{
      "name": "'$CANARY_SERVER'",
      "address": "'$CANARY_ADDRESS'",
      "port": '$CANARY_PORT',
      "check": "enabled",
      "weight": 0
    }'
  
  curl -s -u $AUTH -X PUT "$HAPROXY_API/services/haproxy/transactions/$TXN"
  echo "Canary server added with 0% traffic"
}

remove_canary() {
  VERSION=$(curl -s -u $AUTH $HAPROXY_API/services/haproxy/configuration/version)
  TXN=$(curl -s -u $AUTH -X POST \
    "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')
  
  curl -s -u $AUTH -X DELETE \
    "$HAPROXY_API/services/haproxy/configuration/servers/${CANARY_SERVER}?backend=$BACKEND&transaction_id=$TXN"
  
  curl -s -u $AUTH -X PUT "$HAPROXY_API/services/haproxy/transactions/$TXN"
  echo "Canary server removed"
}

case "$1" in
  add)    add_canary ;;
  remove) remove_canary ;;
  shift)  shift_traffic $2 ;;
  *)      echo "Usage: $0 {add|remove|shift <percentage>}" ;;
esac

Python Client Library

#!/usr/bin/env python3
import requests
from requests.auth import HTTPBasicAuth
import json

class HAProxyAPI:
    def __init__(self, base_url, username, password):
        self.base_url = base_url.rstrip('/')
        self.auth = HTTPBasicAuth(username, password)
        self.session = requests.Session()
        self.session.auth = self.auth
        self.session.verify = True  # Set to False for self-signed certs
    
    def get_version(self):
        """Get current configuration version"""
        r = self.session.get(f"{self.base_url}/services/haproxy/configuration/version")
        return r.text.strip()
    
    def start_transaction(self, version=None):
        """Start a new transaction"""
        if version is None:
            version = self.get_version()
        r = self.session.post(f"{self.base_url}/services/haproxy/transactions?version={version}")
        return r.json()['id']
    
    def commit_transaction(self, txn_id):
        """Commit a transaction"""
        r = self.session.put(f"{self.base_url}/services/haproxy/transactions/{txn_id}")
        return r.status_code == 202
    
    def rollback_transaction(self, txn_id):
        """Rollback a transaction"""
        r = self.session.delete(f"{self.base_url}/services/haproxy/transactions/{txn_id}")
        return r.status_code == 204
    
    def get_backends(self):
        """List all backends"""
        r = self.session.get(f"{self.base_url}/services/haproxy/configuration/backends")
        return r.json().get('data', [])
    
    def get_servers(self, backend):
        """List servers in a backend"""
        r = self.session.get(
            f"{self.base_url}/services/haproxy/configuration/servers",
            params={'backend': backend}
        )
        return r.json().get('data', [])
    
    def add_server(self, backend, name, address, port, weight=100, check=True):
        """Add a server to a backend"""
        txn = self.start_transaction()
        
        data = {
            'name': name,
            'address': address,
            'port': port,
            'weight': weight,
            'check': 'enabled' if check else 'disabled'
        }
        
        r = self.session.post(
            f"{self.base_url}/services/haproxy/configuration/servers",
            params={'backend': backend, 'transaction_id': txn},
            json=data
        )
        
        if r.status_code in [200, 201, 202]:
            self.commit_transaction(txn)
            return True
        else:
            self.rollback_transaction(txn)
            return False
    
    def remove_server(self, backend, name):
        """Remove a server from a backend"""
        txn = self.start_transaction()
        
        r = self.session.delete(
            f"{self.base_url}/services/haproxy/configuration/servers/{name}",
            params={'backend': backend, 'transaction_id': txn}
        )
        
        if r.status_code in [200, 202, 204]:
            self.commit_transaction(txn)
            return True
        else:
            self.rollback_transaction(txn)
            return False
    
    def set_server_state(self, backend, name, state):
        """Set server runtime state (ready/drain/maint)"""
        r = self.session.put(
            f"{self.base_url}/services/haproxy/runtime/servers/{name}",
            params={'backend': backend},
            json={'admin_state': state}
        )
        return r.status_code == 200
    
    def set_server_weight(self, backend, name, weight):
        """Set server weight at runtime"""
        r = self.session.put(
            f"{self.base_url}/services/haproxy/runtime/servers/{name}",
            params={'backend': backend},
            json={'weight': weight}
        )
        return r.status_code == 200


# Usage example
if __name__ == '__main__':
    api = HAProxyAPI(
        'https://haproxy.example.com:5556/v2',
        'admin',
        'your-secure-password'
    )
    
    # List backends
    print("Backends:")
    for backend in api.get_backends():
        print(f"  - {backend['name']}")
    
    # List servers
    print("\nServers in web_servers:")
    for server in api.get_servers('web_servers'):
        print(f"  - {server['name']}: {server['address']}:{server['port']}")
    
    # Add a new server
    print("\nAdding new server...")
    api.add_server('web_servers', 'web4', '192.168.1.14', 8080)
    
    # Put server in maintenance
    print("Setting web4 to maintenance...")
    api.set_server_state('web_servers', 'web4', 'maint')

Security Best Practices

TLS Configuration

# dataplaneapi.yaml with TLS
dataplaneapi:
  host: 0.0.0.0
  port: 5555
  scheme:
    - https
  
  tls:
    tls_certificate: /etc/haproxy/certs/api.pem
    tls_key: /etc/haproxy/certs/api.key
    tls_ca: /etc/haproxy/certs/ca.pem  # For mTLS
    tls_port: 5555

Network Restrictions

frontend dataplane_api
    bind *:5556 ssl crt /etc/haproxy/certs/api.pem
    
    # Restrict to management network
    acl allowed_networks src 10.0.0.0/8 192.168.0.0/16
    http-request deny unless allowed_networks
    
    # Rate limiting
    stick-table type ip size 100k expire 30s store http_req_rate(10s)
    http-request track-sc0 src
    http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }
    
    http-request auth unless { http_auth(api_users) }
    default_backend dataplane_api_backend

Integration with CI/CD

GitLab CI Example

# .gitlab-ci.yml
stages:
  - deploy
  - traffic

variables:
  HAPROXY_API: "https://haproxy.example.com:5556/v2"

deploy-canary:
  stage: deploy
  script:
    - |
      # Add canary server
      VERSION=$(curl -s -u $HAPROXY_AUTH $HAPROXY_API/services/haproxy/configuration/version)
      TXN=$(curl -s -u $HAPROXY_AUTH -X POST \
        "$HAPROXY_API/services/haproxy/transactions?version=$VERSION" | jq -r '.id')
      
      curl -s -u $HAPROXY_AUTH -X POST \
        "$HAPROXY_API/services/haproxy/configuration/servers?backend=app_servers&transaction_id=$TXN" \
        -H "Content-Type: application/json" \
        -d '{"name": "canary", "address": "'$CANARY_IP'", "port": 8080, "weight": 10, "check": "enabled"}'
      
      curl -s -u $HAPROXY_AUTH -X PUT "$HAPROXY_API/services/haproxy/transactions/$TXN"
  only:
    - main

shift-traffic-50:
  stage: traffic
  script:
    - |
      curl -s -u $HAPROXY_AUTH -X PUT \
        "$HAPROXY_API/services/haproxy/runtime/servers/canary?backend=app_servers" \
        -H "Content-Type: application/json" \
        -d '{"weight": 50}'
  when: manual
  only:
    - main

promote-canary:
  stage: traffic
  script:
    - |
      # Shift all traffic to canary
      curl -s -u $HAPROXY_AUTH -X PUT \
        "$HAPROXY_API/services/haproxy/runtime/servers/canary?backend=app_servers" \
        -d '{"weight": 100}'
      
      # Set old servers to drain
      for server in web1 web2 web3; do
        curl -s -u $HAPROXY_AUTH -X PUT \
          "$HAPROXY_API/services/haproxy/runtime/servers/${server}?backend=app_servers" \
          -d '{"admin_state": "drain"}'
      done
  when: manual
  only:
    - main

Troubleshooting

Check API Status

# Test API connectivity
curl -v -u $AUTH $HAPROXY_API/info

# Check pending transactions
curl -s -u $AUTH $HAPROXY_API/services/haproxy/transactions | jq .

# View HAProxy stats
curl -s -u $AUTH $HAPROXY_API/services/haproxy/stats/native | jq .

Common Issues

  1. Transaction conflicts: Always use fresh version number
  2. Permission denied: Check user permissions in dataplaneapi.yaml
  3. Configuration invalid: Use GET endpoints to verify current state
  4. Reload failures: Check HAProxy configuration syntax

Next Steps

Now that you can dynamically manage HAProxy, consider:

  • Service Discovery Integration - Auto-register services from Consul or Kubernetes
  • Monitoring - Track configuration changes and performance metrics
  • GitOps - Version control your HAProxy configuration changes

The Data Plane API transforms HAProxy from a static configuration file into a dynamic, programmable load balancer that integrates seamlessly with modern infrastructure automation.

Read more