<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:media="http://search.yahoo.com/mrss/"><channel><title>Consul | A DevOps Blog</title><link>https://blog.bsdserver.nl/tag/consul/</link><description>Hands-on DevOps and homelab content: Kubernetes clusters, self-hosted services, CI/CD pipelines and infrastructure as code, by Patrick de Ruiter of We Build Your Cloud.</description><language>en</language><image><url>https://blog.bsdserver.nl/images/app-icon.svg</url><title>A DevOps Blog</title><link>https://blog.bsdserver.nl/</link></image><lastBuildDate>Wed, 18 Nov 2026 12:00:00 +0100</lastBuildDate><atom:link href="https://blog.bsdserver.nl/tag/consul/rss/" rel="self" type="application/rss+xml"/><ttl>60</ttl><item><title>HAProxy and Consul: Dynamic Service Discovery for Load Balancing</title><link>https://blog.bsdserver.nl/haproxy-and-consul-dynamic-service-discovery-for-load-balancing/</link><guid isPermaLink="false">693b765b79eb54000160a628</guid><dc:creator>Patrick de Ruiter</dc:creator><pubDate>Fri, 12 Dec 2025 01:56:43 +0000</pubDate><description>In dynamic environments where services scale up and down automatically, static HAProxy configurations become a management burden.</description><category>HAProxy</category><category>Consul</category><category>Service discovery</category><category>DevOps</category><category>Microservices</category><media:content url="https://blog.bsdserver.nl/content/images/2026/01/haproxy-and-consul-service-discovery.png" medium="image"/><content:encoded><![CDATA[<p>In dynamic environments where services scale up and down automatically, static HAProxy configurations become a management burden. Every instance you add or remove means editing <code>server</code> lines and reloading HAProxy.</p>
<p>Consul keeps a live list of which instances run and which are healthy. The HAProxy Data Plane API reads that list and keeps one HAProxy backend per service up to date. The server lists maintain themselves.</p>
<h2 id="why-consul--haproxy">Why Consul + HAProxy? <a class="anchor" data-pagefind-ignore href="#why-consul--haproxy" aria-label="Link to this section">#</a></h2>
<p>The combination provides:</p>
<ul>
<li><strong>Automatic service registration</strong> - Services announce themselves to Consul</li>
<li><strong>Health-aware routing</strong> - Only healthy instances receive traffic</li>
<li><strong>Dynamic scaling</strong> - New instances are automatically added to load balancing</li>
<li><strong>No downtime updates</strong> - Changes are applied with seamless reloads</li>
</ul>
<h3 id="when-to-use-it">When to use it <a class="anchor" data-pagefind-ignore href="#when-to-use-it" aria-label="Link to this section">#</a></h3>
<ul>
<li><strong>Use it</strong> when backends change often: autoscaling, frequent deployments, several teams deploying their own services.</li>
<li><strong>Skip it</strong> for a few static backends. A Consul cluster is three more machines to run; a static <code>haproxy.cfg</code> from Ansible is simpler.</li>
</ul>
<h3 id="data-plane-api-instead-of-consul-template">Data Plane API instead of consul-template <a class="anchor" data-pagefind-ignore href="#data-plane-api-instead-of-consul-template" aria-label="Link to this section">#</a></h3>
<p>This post uses the Data Plane API&rsquo;s built-in Consul service discovery. Earlier versions used consul-template to render <code>haproxy.cfg</code>; that still works, but you write and maintain the template yourself.</p>
<table>
	<thead>
			<tr>
					<th></th>
					<th>consul-template</th>
					<th>Data Plane API</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Backends</td>
					<td>Whatever your template writes</td>
					<td>One per service, generated</td>
			</tr>
			<tr>
					<td>Select instances by</td>
					<td>Service, tag, datacenter, metadata, KV</td>
					<td>Service name only (allowlist, denylist, regexp)</td>
			</tr>
			<tr>
					<td>Server weights</td>
					<td>From Consul or metadata</td>
					<td>Fixed (128)</td>
			</tr>
			<tr>
					<td>Routing changes</td>
					<td>Edit the template</td>
					<td>Runtime API (maps, frontends), no file editing</td>
			</tr>
	</tbody>
</table>
<p>The trade-off: fewer moving parts, less template freedom.</p>
<h2 id="architecture-overview">Architecture Overview <a class="anchor" data-pagefind-ignore href="#architecture-overview" aria-label="Link to this section">#</a></h2>
<pre tabindex="0"><code>                    ┌─────────────────────────────────┐
                    │           Consul Cluster        │
                    │  ┌───────┐ ┌───────┐ ┌───────┐ │
                    │  │Server1│ │Server2│ │Server3│ │
                    │  └───────┘ └───────┘ └───────┘ │
                    └────────────────┬────────────────┘
                                     │
              ┌──────────────────────┼──────────────────────┐
              │                      │                      │
              ▼                      ▼                      ▼
      ┌───────────────┐      ┌───────────────┐      ┌───────────────┐
      │   Service A   │      │   Service B   │      │ Load balancer │
      │ + Consul Agent│      │ + Consul Agent│      │ + Consul Agent│
      └───────────────┘      └───────────────┘      │ + Data Plane  │
              ▲                      ▲              │   API         │
              │                      │              │ + HAProxy     │
              └──────────────────────┴──────────────┴───────────────┘
                                                            ▲
                                                            │
                                                      Clients/Users
</code></pre><table>
	<thead>
			<tr>
					<th>Component</th>
					<th>Runs on</th>
					<th>Job</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Consul servers</td>
					<td>3 dedicated machines</td>
					<td>Store the catalog: services, addresses, health. Three so the cluster survives losing one.</td>
			</tr>
			<tr>
					<td>Consul agent</td>
					<td>Every service node and the load balancer</td>
					<td>Registers local services, runs their health checks, reports to the servers.</td>
			</tr>
			<tr>
					<td>Data Plane API</td>
					<td>Load balancer</td>
					<td>Polls Consul, creates a <code>consul-backend-…-&lt;service&gt;</code> backend per service, adds and removes servers, reloads HAProxy.</td>
			</tr>
			<tr>
					<td>HAProxy</td>
					<td>Load balancer</td>
					<td>Balances traffic. The frontend picks a backend by Host header from a map file.</td>
			</tr>
	</tbody>
</table>
<p>The flow: an instance starts → its agent registers it → the check passes → the Data Plane API sees it at its next poll (every few seconds) → adds a server to the service&rsquo;s backend → HAProxy reloads seamlessly. Stopping or failing an instance runs the same chain in reverse.</p>
<h2 id="setting-up-consul">Setting Up Consul <a class="anchor" data-pagefind-ignore href="#setting-up-consul" aria-label="Link to this section">#</a></h2>
<h3 id="consul-server-installation">Consul Server Installation <a class="anchor" data-pagefind-ignore href="#consul-server-installation" aria-label="Link to this section">#</a></h3>
<p>One binary for servers and agents; the configuration decides the role. Install it on all servers and service nodes.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Download Consul</span>
</span></span><span class="line"><span class="cl"><span class="nv">CONSUL_VERSION</span><span class="o">=</span><span class="s2">&#34;2.1.0&#34;</span>
</span></span><span class="line"><span class="cl">wget https://releases.hashicorp.com/consul/<span class="si">${</span><span class="nv">CONSUL_VERSION</span><span class="si">}</span>/consul_<span class="si">${</span><span class="nv">CONSUL_VERSION</span><span class="si">}</span>_linux_amd64.zip
</span></span><span class="line"><span class="cl">unzip consul_<span class="si">${</span><span class="nv">CONSUL_VERSION</span><span class="si">}</span>_linux_amd64.zip
</span></span><span class="line"><span class="cl">sudo mv consul /usr/local/bin/
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Create directories</span>
</span></span><span class="line"><span class="cl">sudo mkdir -p /etc/consul.d /var/lib/consul
</span></span><span class="line"><span class="cl">sudo useradd --system --home /var/lib/consul --shell /bin/false consul
</span></span><span class="line"><span class="cl">sudo chown -R consul:consul /etc/consul.d /var/lib/consul
</span></span></code></pre></div><h3 id="consul-server-configuration">Consul Server Configuration <a class="anchor" data-pagefind-ignore href="#consul-server-configuration" aria-label="Link to this section">#</a></h3>
<p>What the key settings do:</p>
<ul>
<li><code>bootstrap_expect = 3</code>: wait for three servers before electing a leader.</li>
<li><code>retry_join</code>: the other servers; a restarted server finds them again.</li>
<li><code>bind_addr</code>: picks the private address for cluster traffic, so one file fits all servers.</li>
<li><code>client_addr = &quot;0.0.0.0&quot;</code>: API and UI on all interfaces. Firewall port 8500.</li>
<li><code>acl</code> with <code>default_policy = &quot;deny&quot;</code>: no access without a token. Enable it from day one.</li>
<li><code>connect</code> and the gRPC ports: only for Consul Connect (see Next Steps).</li>
</ul>
<p>Create <code>/etc/consul.d/consul.hcl</code> (unique <code>node_name</code> per server):</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-hcl" data-lang="hcl"><span class="line"><span class="cl"><span class="n">datacenter</span> <span class="o">=</span> <span class="s2">&#34;dc1&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">node_name</span> <span class="o">=</span> <span class="s2">&#34;consul-server-1&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">data_dir</span> <span class="o">=</span> <span class="s2">&#34;/var/lib/consul&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">log_level</span> <span class="o">=</span> <span class="s2">&#34;INFO&#34;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">server</span> <span class="o">=</span> <span class="kt">true</span>
</span></span><span class="line"><span class="cl"><span class="n">bootstrap_expect</span> <span class="o">=</span> <span class="m">3</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">bind_addr</span> <span class="o">=</span> <span class="s2">&#34;{{ GetPrivateInterfaces | include \&#34;network\&#34; \&#34;10.0.0.0/8\&#34; | attr \&#34;address\&#34; }}&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">client_addr</span> <span class="o">=</span> <span class="s2">&#34;0.0.0.0&#34;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">ui_config</span> {
</span></span><span class="line"><span class="cl"><span class="n">  enabled</span> <span class="o">=</span> <span class="kt">true</span>
</span></span><span class="line"><span class="cl">}
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">connect</span> {
</span></span><span class="line"><span class="cl"><span class="n">  enabled</span> <span class="o">=</span> <span class="kt">true</span>
</span></span><span class="line"><span class="cl">}
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">addresses</span> {
</span></span><span class="line"><span class="cl"><span class="n">  http</span> <span class="o">=</span> <span class="s2">&#34;0.0.0.0&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">  grpc</span> <span class="o">=</span> <span class="s2">&#34;0.0.0.0&#34;</span>
</span></span><span class="line"><span class="cl">}
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">ports</span> {
</span></span><span class="line"><span class="cl"><span class="n">  http</span> <span class="o">=</span> <span class="m">8500</span>
</span></span><span class="line"><span class="cl"><span class="n">  grpc</span> <span class="o">=</span> <span class="m">8502</span>
</span></span><span class="line"><span class="cl"><span class="n">  grpc_tls</span> <span class="o">=</span> <span class="m">8503</span>
</span></span><span class="line"><span class="cl">}
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">retry_join</span> <span class="o">=</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;consul-server-1.example.com&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;consul-server-2.example.com&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;consul-server-3.example.com&#34;</span>
</span></span><span class="line"><span class="cl"><span class="p">]</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">acl</span> {
</span></span><span class="line"><span class="cl"><span class="n">  enabled</span> <span class="o">=</span> <span class="kt">true</span>
</span></span><span class="line"><span class="cl"><span class="n">  default_policy</span> <span class="o">=</span> <span class="s2">&#34;deny&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">  enable_token_persistence</span> <span class="o">=</span> <span class="kt">true</span>
</span></span><span class="line"><span class="cl">}
</span></span></code></pre></div><h3 id="consul-agent-configuration-for-service-nodes">Consul Agent Configuration (for service nodes) <a class="anchor" data-pagefind-ignore href="#consul-agent-configuration-for-service-nodes" aria-label="Link to this section">#</a></h3>
<p>Same as the server, minus the server parts. <code>client_addr = &quot;127.0.0.1&quot;</code> keeps the agent&rsquo;s API local. The load balancer gets an agent too, so the Data Plane API only ever talks to <code>127.0.0.1:8500</code>.</p>
<p>Create <code>/etc/consul.d/consul.hcl</code> on service nodes:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-hcl" data-lang="hcl"><span class="line"><span class="cl"><span class="n">datacenter</span> <span class="o">=</span> <span class="s2">&#34;dc1&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">node_name</span> <span class="o">=</span> <span class="s2">&#34;service-node-1&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">data_dir</span> <span class="o">=</span> <span class="s2">&#34;/var/lib/consul&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">log_level</span> <span class="o">=</span> <span class="s2">&#34;INFO&#34;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">server</span> <span class="o">=</span> <span class="kt">false</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">bind_addr</span> <span class="o">=</span> <span class="s2">&#34;{{ GetPrivateInterfaces | include \&#34;network\&#34; \&#34;10.0.0.0/8\&#34; | attr \&#34;address\&#34; }}&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">client_addr</span> <span class="o">=</span> <span class="s2">&#34;127.0.0.1&#34;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">retry_join</span> <span class="o">=</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;consul-server-1.example.com&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;consul-server-2.example.com&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">  <span class="s2">&#34;consul-server-3.example.com&#34;</span>
</span></span><span class="line"><span class="cl"><span class="p">]</span><span class="c1">
</span></span></span><span class="line"><span class="cl"><span class="c1">
</span></span></span><span class="line"><span class="cl"><span class="c1"># Must match the servers, otherwise the agent cannot use tokens at all
</span></span></span><span class="line"><span class="cl"><span class="k">acl</span> {
</span></span><span class="line"><span class="cl"><span class="n">  enabled</span> <span class="o">=</span> <span class="kt">true</span>
</span></span><span class="line"><span class="cl"><span class="n">  default_policy</span> <span class="o">=</span> <span class="s2">&#34;deny&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">  enable_token_persistence</span> <span class="o">=</span> <span class="kt">true</span>
</span></span><span class="line"><span class="cl">}
</span></span></code></pre></div><h3 id="systemd-service">Systemd Service <a class="anchor" data-pagefind-ignore href="#systemd-service" aria-label="Link to this section">#</a></h3>
<p>One unit for servers and agents. <code>Type=notify</code> makes units that start after Consul, like the Data Plane API, wait until it is ready.</p>
<p>Create <code>/etc/systemd/system/consul.service</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="k">[Unit]</span>
</span></span><span class="line"><span class="cl"><span class="na">Description</span><span class="o">=</span><span class="s">Consul Service Discovery Agent</span>
</span></span><span class="line"><span class="cl"><span class="na">Documentation</span><span class="o">=</span><span class="s">https://www.consul.io/</span>
</span></span><span class="line"><span class="cl"><span class="na">After</span><span class="o">=</span><span class="s">network-online.target</span>
</span></span><span class="line"><span class="cl"><span class="na">Wants</span><span class="o">=</span><span class="s">network-online.target</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">[Service]</span>
</span></span><span class="line"><span class="cl"><span class="na">Type</span><span class="o">=</span><span class="s">notify</span>
</span></span><span class="line"><span class="cl"><span class="na">User</span><span class="o">=</span><span class="s">consul</span>
</span></span><span class="line"><span class="cl"><span class="na">Group</span><span class="o">=</span><span class="s">consul</span>
</span></span><span class="line"><span class="cl"><span class="na">ExecStart</span><span class="o">=</span><span class="s">/usr/local/bin/consul agent -config-dir=/etc/consul.d/</span>
</span></span><span class="line"><span class="cl"><span class="na">ExecReload</span><span class="o">=</span><span class="s">/bin/kill -HUP $MAINPID</span>
</span></span><span class="line"><span class="cl"><span class="na">KillMode</span><span class="o">=</span><span class="s">process</span>
</span></span><span class="line"><span class="cl"><span class="na">KillSignal</span><span class="o">=</span><span class="s">SIGTERM</span>
</span></span><span class="line"><span class="cl"><span class="na">Restart</span><span class="o">=</span><span class="s">on-failure</span>
</span></span><span class="line"><span class="cl"><span class="na">RestartSec</span><span class="o">=</span><span class="s">5</span>
</span></span><span class="line"><span class="cl"><span class="na">LimitNOFILE</span><span class="o">=</span><span class="s">65536</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">[Install]</span>
</span></span><span class="line"><span class="cl"><span class="na">WantedBy</span><span class="o">=</span><span class="s">multi-user.target</span>
</span></span></code></pre></div><p>Run <code>systemctl enable --now consul</code>. <code>consul members</code> shows the cluster once three servers have joined.</p>
<h3 id="bootstrapping-acls">Bootstrapping ACLs <a class="anchor" data-pagefind-ignore href="#bootstrapping-acls" aria-label="Link to this section">#</a></h3>
<p>With <code>default_policy = &quot;deny&quot;</code> nothing works until tokens exist: agents log <code>Coordinate update blocked by ACLs</code> and <code>Service registration blocked by ACLs</code>, and the services never reach the catalog.</p>
<p>You create three tokens:</p>
<ul>
<li><strong>Bootstrap token</strong>: full access, administration only.</li>
<li><strong>Agent token</strong>: lets agents register nodes and services. Goes on every agent.</li>
<li><strong>Data Plane API token</strong>: read-only. The API only reads services, nodes and health.</li>
</ul>
<p>Once the servers have elected a leader:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Run once, on one server. Keep the SecretID safe: it is the management token.</span>
</span></span><span class="line"><span class="cl">consul acl bootstrap
</span></span><span class="line"><span class="cl"><span class="nb">export</span> <span class="nv">CONSUL_HTTP_TOKEN</span><span class="o">=</span><span class="s2">&#34;&lt;bootstrap SecretID&gt;&#34;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">cat &gt; agent-policy.hcl <span class="s">&lt;&lt;&#39;EOF&#39;
</span></span></span><span class="line"><span class="cl"><span class="s">node_prefix &#34;&#34; {
</span></span></span><span class="line"><span class="cl"><span class="s">  policy = &#34;write&#34;
</span></span></span><span class="line"><span class="cl"><span class="s">}
</span></span></span><span class="line"><span class="cl"><span class="s">service_prefix &#34;&#34; {
</span></span></span><span class="line"><span class="cl"><span class="s">  policy = &#34;write&#34;
</span></span></span><span class="line"><span class="cl"><span class="s">}
</span></span></span><span class="line"><span class="cl"><span class="s">EOF</span>
</span></span><span class="line"><span class="cl">consul acl policy create -name agent -rules @agent-policy.hcl
</span></span><span class="line"><span class="cl">consul acl token create -description <span class="s2">&#34;agent token&#34;</span> -policy-name agent
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">cat &gt; dataplaneapi-policy.hcl <span class="s">&lt;&lt;&#39;EOF&#39;
</span></span></span><span class="line"><span class="cl"><span class="s">node_prefix &#34;&#34; {
</span></span></span><span class="line"><span class="cl"><span class="s">  policy = &#34;read&#34;
</span></span></span><span class="line"><span class="cl"><span class="s">}
</span></span></span><span class="line"><span class="cl"><span class="s">service_prefix &#34;&#34; {
</span></span></span><span class="line"><span class="cl"><span class="s">  policy = &#34;read&#34;
</span></span></span><span class="line"><span class="cl"><span class="s">}
</span></span></span><span class="line"><span class="cl"><span class="s">EOF</span>
</span></span><span class="line"><span class="cl">consul acl policy create -name dataplaneapi -rules @dataplaneapi-policy.hcl
</span></span><span class="line"><span class="cl">consul acl token create -description <span class="s2">&#34;dataplaneapi&#34;</span> -policy-name dataplaneapi
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># On every agent (servers and service nodes), with the agent token&#39;s SecretID.</span>
</span></span><span class="line"><span class="cl"><span class="c1"># &#34;default&#34; is also used for services defined in /etc/consul.d and for local API calls without a token.</span>
</span></span><span class="line"><span class="cl">consul acl set-agent-token agent   <span class="s2">&#34;&lt;agent token SecretID&gt;&#34;</span>
</span></span><span class="line"><span class="cl">consul acl set-agent-token default <span class="s2">&#34;&lt;agent token SecretID&gt;&#34;</span>
</span></span></code></pre></div><p>The Data Plane API token goes into the Consul registration on the load balancer (below). Export <code>CONSUL_HTTP_TOKEN</code> before running <code>consul members</code> or <code>consul catalog</code> commands.</p>
<h2 id="registering-services-with-consul">Registering Services with Consul <a class="anchor" data-pagefind-ignore href="#registering-services-with-consul" aria-label="Link to this section">#</a></h2>
<p>Only registered, healthy services reach HAProxy. A registration gives a name, a port and a health check.</p>
<p>Two ways to register:</p>
<ul>
<li><strong>File</strong> in <code>/etc/consul.d</code>: for long-lived services managed by configuration management.</li>
<li><strong>HTTP API</strong>: for deployment tooling and short-lived instances.</li>
</ul>
<h3 id="service-registration-file">Service Registration File <a class="anchor" data-pagefind-ignore href="#service-registration-file" aria-label="Link to this section">#</a></h3>
<ul>
<li><code>tags</code> and <code>meta</code>: labels for your own tooling. The Data Plane API ignores them; it selects by service name.</li>
<li><code>success_before_passing</code> / <code>failures_before_critical</code>: stop a flapping instance from going in and out of the pool.</li>
</ul>
<p>Create <code>/etc/consul.d/web-service.hcl</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-hcl" data-lang="hcl"><span class="line"><span class="cl"><span class="k">service</span> {
</span></span><span class="line"><span class="cl"><span class="n">  name</span> <span class="o">=</span> <span class="s2">&#34;web&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">  port</span> <span class="o">=</span> <span class="m">8080</span>
</span></span><span class="line"><span class="cl"><span class="n">  tags</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&#34;http&#34;, &#34;production&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="k">meta</span> {
</span></span><span class="line"><span class="cl"><span class="n">    version</span> <span class="o">=</span> <span class="s2">&#34;1.0.0&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">    environment</span> <span class="o">=</span> <span class="s2">&#34;production&#34;</span>
</span></span><span class="line"><span class="cl">  }<span class="c1">
</span></span></span><span class="line"><span class="cl"><span class="c1">  
</span></span></span><span class="line"><span class="cl"><span class="c1">  # More than one check: use a &#34;checks&#34; list, repeated check blocks fail to parse
</span></span></span><span class="line"><span class="cl"><span class="n">  checks</span> <span class="o">=</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">    {
</span></span><span class="line"><span class="cl"><span class="n">      name</span> <span class="o">=</span> <span class="s2">&#34;HTTP Health Check&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      http</span> <span class="o">=</span> <span class="s2">&#34;http://localhost:8080/health&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      interval</span> <span class="o">=</span> <span class="s2">&#34;10s&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      timeout</span> <span class="o">=</span> <span class="s2">&#34;5s&#34;</span>
</span></span><span class="line"><span class="cl">      
</span></span><span class="line"><span class="cl">      <span class="n">success_before_passing</span> <span class="o">=</span> <span class="m">2</span>
</span></span><span class="line"><span class="cl"><span class="n">      failures_before_critical</span> <span class="o">=</span> <span class="m">3</span>
</span></span><span class="line"><span class="cl">    }<span class="p">,</span>
</span></span><span class="line"><span class="cl">    {
</span></span><span class="line"><span class="cl"><span class="n">      name</span> <span class="o">=</span> <span class="s2">&#34;TCP Port Check&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      tcp</span> <span class="o">=</span> <span class="s2">&#34;localhost:8080&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      interval</span> <span class="o">=</span> <span class="s2">&#34;5s&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      timeout</span> <span class="o">=</span> <span class="s2">&#34;2s&#34;</span>
</span></span><span class="line"><span class="cl">    }
</span></span><span class="line"><span class="cl">  <span class="p">]</span>
</span></span><span class="line"><span class="cl">}
</span></span></code></pre></div><p>Apply with <code>consul reload</code>. An instance is healthy only when all its checks pass.</p>
<h3 id="api-service-registration">API Service Registration <a class="anchor" data-pagefind-ignore href="#api-service-registration" aria-label="Link to this section">#</a></h3>
<p><code>weights</code> sets Consul&rsquo;s traffic share per health state, used by Consul DNS. The Data Plane API does not copy them: every server it creates gets weight 128.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-hcl" data-lang="hcl"><span class="line"><span class="cl"><span class="k">service</span> {
</span></span><span class="line"><span class="cl"><span class="n">  name</span> <span class="o">=</span> <span class="s2">&#34;api&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">  port</span> <span class="o">=</span> <span class="m">3000</span>
</span></span><span class="line"><span class="cl"><span class="n">  tags</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&#34;http&#34;, &#34;api&#34;, &#34;v2&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="k">weights</span> {
</span></span><span class="line"><span class="cl"><span class="n">    passing</span> <span class="o">=</span> <span class="m">100</span>
</span></span><span class="line"><span class="cl"><span class="n">    warning</span> <span class="o">=</span> <span class="m">50</span>
</span></span><span class="line"><span class="cl">  }
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="k">check</span> {
</span></span><span class="line"><span class="cl"><span class="n">    name</span> <span class="o">=</span> <span class="s2">&#34;API Health&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">    http</span> <span class="o">=</span> <span class="s2">&#34;http://localhost:3000/api/health&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">    interval</span> <span class="o">=</span> <span class="s2">&#34;10s&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">    timeout</span> <span class="o">=</span> <span class="s2">&#34;3s&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="k">header</span> {
</span></span><span class="line"><span class="cl"><span class="n">      Content-Type</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&#34;application/json&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl">    }
</span></span><span class="line"><span class="cl">  }
</span></span><span class="line"><span class="cl">}
</span></span></code></pre></div><h3 id="dynamic-registration-via-http-api">Dynamic Registration via HTTP API <a class="anchor" data-pagefind-ignore href="#dynamic-registration-via-http-api" aria-label="Link to this section">#</a></h3>
<p>Register when the instance is up; deregister before you stop it, so HAProxy drops it before the process goes away.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Register a service dynamically</span>
</span></span><span class="line"><span class="cl">curl -X PUT http://localhost:8500/v1/agent/service/register <span class="se">\
</span></span></span><span class="line"><span class="cl">  -H <span class="s2">&#34;Content-Type: application/json&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  -d <span class="s1">&#39;{
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;ID&#34;: &#34;web-1&#34;,
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;Name&#34;: &#34;web&#34;,
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;Port&#34;: 8080,
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;Tags&#34;: [&#34;http&#34;, &#34;production&#34;],
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;Check&#34;: {
</span></span></span><span class="line"><span class="cl"><span class="s1">      &#34;HTTP&#34;: &#34;http://localhost:8080/health&#34;,
</span></span></span><span class="line"><span class="cl"><span class="s1">      &#34;Interval&#34;: &#34;10s&#34;
</span></span></span><span class="line"><span class="cl"><span class="s1">    }
</span></span></span><span class="line"><span class="cl"><span class="s1">  }&#39;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Deregister a service</span>
</span></span><span class="line"><span class="cl">curl -X PUT http://localhost:8500/v1/agent/service/deregister/web-1
</span></span></code></pre></div><h2 id="the-load-balancer-haproxy-and-the-data-plane-api">The Load Balancer: HAProxy and the Data Plane API <a class="anchor" data-pagefind-ignore href="#the-load-balancer-haproxy-and-the-data-plane-api" aria-label="Link to this section">#</a></h2>
<p>The Data Plane API is HAProxy&rsquo;s management API. Here it does one job: it polls Consul and keeps one backend per service in <code>haproxy.cfg</code> up to date.</p>
<p>The setup on the load balancer:</p>
<ol>
<li>Install HAProxy and write a base <code>haproxy.cfg</code> with the frontend.</li>
<li>Install the Data Plane API.</li>
<li>Register Consul with the API.</li>
</ol>
<h3 id="installing-haproxy">Installing HAProxy <a class="anchor" data-pagefind-ignore href="#installing-haproxy" aria-label="Link to this section">#</a></h3>
<p>On the load balancer, install HAProxy from the official HAProxy packages rather than the distribution&rsquo;s older <code>haproxy</code> package:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Official HAProxy packages for Ubuntu 24.04: one PPA per branch</span>
</span></span><span class="line"><span class="cl">sudo add-apt-repository -y ppa:vbernat/haproxy-3.3
</span></span><span class="line"><span class="cl">sudo apt update
</span></span><span class="line"><span class="cl">sudo apt install -y <span class="nv">haproxy</span><span class="o">=</span>3.3.<span class="se">\*</span> curl jq socat
</span></span><span class="line"><span class="cl">haproxy -v
</span></span></code></pre></div><p>3.3 is the newest branch the PPA builds for Ubuntu 24.04, but it is a short-lived stable branch, maintained until about Q1 2027. The newest LTS branch, 3.4, is built for Ubuntu 26.04 (<code>ppa:vbernat/haproxy-3.4</code>) and for Debian 13 on <a href="https://haproxy.debian.net/" rel="noopener">haproxy.debian.net</a>, which lists the repository lines per release and branch. For a long-lived Ubuntu 24.04 install, use the 3.2 LTS branch instead (<code>ppa:vbernat/haproxy-3.2</code>, maintained until 2030).</p>
<h3 id="base-haproxy-configuration">Base HAProxy Configuration <a class="anchor" data-pagefind-ignore href="#base-haproxy-configuration" aria-label="Link to this section">#</a></h3>
<p>You write the frontend; the Data Plane API writes the backends. Do this before the API starts: from then on the API owns <code>haproxy.cfg</code> and marks it <code># Dataplaneapi managed File</code>.</p>
<ul>
<li><code>defaults consul_services</code>: settings for the generated backends (balance, timeouts, forwardfor). The registration below refers to it by name.</li>
<li><code>frontend http_front</code>: looks up the Host header in <code>hosts.map</code> and sends the request to <code>consul-backend-127.0.0.1-8500-&lt;service&gt;</code>. Unknown hosts go to <code>web</code>.</li>
<li><code>frontend stats</code>: stats page and Prometheus metrics.</li>
</ul>
<p>Create <code>/etc/haproxy/haproxy.cfg</code>:</p>
<pre tabindex="0"><code class="language-haproxy" data-lang="haproxy">global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
    stats timeout 30s
    user haproxy
    group haproxy

# Settings for the backends the Data Plane API creates from Consul
defaults consul_services
    mode http
    log global
    option httplog
    option dontlognull
    option forwardfor
    balance roundrobin
    timeout connect 5s
    timeout client 50s
    timeout server 50s

defaults
    mode http
    log global
    option httplog
    option dontlognull
    timeout connect 5s
    timeout client 50s
    timeout server 50s

frontend stats
    bind *:8404
    stats enable
    stats uri /stats
    stats refresh 10s
    http-request use-service prometheus-exporter if { path /metrics }

frontend http_front
    bind *:80
    # Host header -&gt; Consul service name (hosts.map), then its generated backend
    http-request set-var(txn.service) req.hdr(host),field(1,:),lower,map(/etc/haproxy/maps/hosts.map,web)
    use_backend consul-backend-127.0.0.1-8500-%[var(txn.service)]
</code></pre><p>And the map, <code>/etc/haproxy/maps/hosts.map</code> (host, then Consul service name):</p>
<pre tabindex="0"><code>www.example.com web
api.example.com api
</code></pre><p>Check it with <code>haproxy -c -f /etc/haproxy/haproxy.cfg</code> and restart HAProxy. Until the API has created the backends, every request gets a 503.</p>
<h3 id="backend-naming">Backend Naming <a class="anchor" data-pagefind-ignore href="#backend-naming" aria-label="Link to this section">#</a></h3>
<p>The API names each backend <code>consul-backend-&lt;consul address&gt;-&lt;consul port&gt;-&lt;service&gt;</code>. The address and port are the ones you register, so with the local agent:</p>
<table>
	<thead>
			<tr>
					<th>Consul service</th>
					<th>Backend</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td><code>web</code></td>
					<td><code>consul-backend-127.0.0.1-8500-web</code></td>
			</tr>
			<tr>
					<td><code>api</code></td>
					<td><code>consul-backend-127.0.0.1-8500-api</code></td>
			</tr>
	</tbody>
</table>
<p>Servers inside get generated names like <code>SRV_6USXG</code>. Don&rsquo;t rely on them; they move between instances.</p>
<h3 id="installing-the-data-plane-api">Installing the Data Plane API <a class="anchor" data-pagefind-ignore href="#installing-the-data-plane-api" aria-label="Link to this section">#</a></h3>
<p>One static binary from GitHub. It runs as root, because it rewrites <code>haproxy.cfg</code> and reloads HAProxy.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="nv">DPAPI_VERSION</span><span class="o">=</span><span class="s2">&#34;3.4.7&#34;</span>
</span></span><span class="line"><span class="cl">wget https://github.com/haproxytech/dataplaneapi/releases/download/v<span class="si">${</span><span class="nv">DPAPI_VERSION</span><span class="si">}</span>/dataplaneapi_<span class="si">${</span><span class="nv">DPAPI_VERSION</span><span class="si">}</span>_linux_x86_64.tar.gz
</span></span><span class="line"><span class="cl">tar xzf dataplaneapi_<span class="si">${</span><span class="nv">DPAPI_VERSION</span><span class="si">}</span>_linux_x86_64.tar.gz dataplaneapi
</span></span><span class="line"><span class="cl">sudo install -m <span class="m">0755</span> dataplaneapi /usr/local/bin/
</span></span><span class="line"><span class="cl">dataplaneapi --version
</span></span></code></pre></div><h3 id="data-plane-api-configuration">Data Plane API Configuration <a class="anchor" data-pagefind-ignore href="#data-plane-api-configuration" aria-label="Link to this section">#</a></h3>
<ul>
<li><code>host: 127.0.0.1</code>: the API is only for local use here. Nothing remote needs it.</li>
<li><code>user</code>: the API user, with a SHA-512 hash (<code>mkpasswd -m sha-512</code>, package <code>whois</code>).</li>
<li><code>reload</code>: how the API reloads HAProxy; the packaged unit does a seamless reload.</li>
</ul>
<p>Create <code>/etc/haproxy/dataplaneapi.yaml</code> (mode 0600; the API rewrites it in this form on every start):</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">config_version</span><span class="p">:</span><span class="w"> </span><span class="m">2</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">haproxy-1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">dataplaneapi</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">host</span><span class="p">:</span><span class="w"> </span><span class="m">127.0.0.1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">port</span><span class="p">:</span><span class="w"> </span><span class="m">5555</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">advertised</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">api_address</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">api_port</span><span class="p">:</span><span class="w"> </span><span class="m">0</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">scheme</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span>- <span class="l">http</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">userlist</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">userlist</span><span class="p">:</span><span class="w"> </span><span class="l">controller</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">userlist_file</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">transaction</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">transaction_dir</span><span class="p">:</span><span class="w"> </span><span class="l">/tmp/haproxy</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">user</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">admin</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">insecure</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">password</span><span class="p">:</span><span class="w"> </span><span class="l">&lt;mkpasswd -m sha-512 output&gt;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">haproxy</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">config_file</span><span class="p">:</span><span class="w"> </span><span class="l">/etc/haproxy/haproxy.cfg</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">haproxy_bin</span><span class="p">:</span><span class="w"> </span><span class="l">/usr/sbin/haproxy</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">reload</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">reload_delay</span><span class="p">:</span><span class="w"> </span><span class="m">5</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">reload_cmd</span><span class="p">:</span><span class="w"> </span><span class="l">systemctl reload haproxy</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">restart_cmd</span><span class="p">:</span><span class="w"> </span><span class="l">systemctl restart haproxy</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">reload_strategy</span><span class="p">:</span><span class="w"> </span><span class="l">custom</span><span class="w">
</span></span></span></code></pre></div><p>Create <code>/etc/systemd/system/dataplaneapi.service</code>:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="k">[Unit]</span>
</span></span><span class="line"><span class="cl"><span class="na">Description</span><span class="o">=</span><span class="s">HAProxy Data Plane API</span>
</span></span><span class="line"><span class="cl"><span class="na">After</span><span class="o">=</span><span class="s">network.target haproxy.service consul.service</span>
</span></span><span class="line"><span class="cl"><span class="na">Requires</span><span class="o">=</span><span class="s">haproxy.service</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">[Service]</span>
</span></span><span class="line"><span class="cl"><span class="na">Type</span><span class="o">=</span><span class="s">simple</span>
</span></span><span class="line"><span class="cl"><span class="na">ExecStart</span><span class="o">=</span><span class="s">/usr/local/bin/dataplaneapi -f /etc/haproxy/dataplaneapi.yaml</span>
</span></span><span class="line"><span class="cl"><span class="na">Restart</span><span class="o">=</span><span class="s">on-failure</span>
</span></span><span class="line"><span class="cl"><span class="na">RestartSec</span><span class="o">=</span><span class="s">5</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">[Install]</span>
</span></span><span class="line"><span class="cl"><span class="na">WantedBy</span><span class="o">=</span><span class="s">multi-user.target</span>
</span></span></code></pre></div><p>Run <code>systemctl enable --now dataplaneapi</code>.</p>
<h3 id="registering-consul-with-the-data-plane-api">Registering Consul with the Data Plane API <a class="anchor" data-pagefind-ignore href="#registering-consul-with-the-data-plane-api" aria-label="Link to this section">#</a></h3>
<p>One API call connects the two. The API stores it in <code>/etc/haproxy/dataplane/service_discovery/consul.json</code> and keeps polling after restarts.</p>
<table>
	<thead>
			<tr>
					<th>Field</th>
					<th>Value here</th>
					<th>What it does</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td><code>address</code>, <code>port</code></td>
					<td><code>127.0.0.1</code>, <code>8500</code></td>
					<td>The local Consul agent. Also part of the backend names.</td>
			</tr>
			<tr>
					<td><code>token</code></td>
					<td>Data Plane API token</td>
					<td>The read-only ACL token from &ldquo;Bootstrapping ACLs&rdquo;.</td>
			</tr>
			<tr>
					<td><code>retry_timeout</code></td>
					<td><code>5</code></td>
					<td>Seconds between polls. Changes reach HAProxy within this time.</td>
			</tr>
			<tr>
					<td><code>service_allowlist</code></td>
					<td><code>[&quot;web&quot;, &quot;api&quot;]</code></td>
					<td>Only these services get a backend. Without it, every service in Consul does.</td>
			</tr>
			<tr>
					<td><code>health_check_policy</code></td>
					<td><code>all</code></td>
					<td>Which instances become servers (see below). Default <code>none</code>.</td>
			</tr>
			<tr>
					<td><code>defaults</code></td>
					<td><code>consul_services</code></td>
					<td>The named <code>defaults</code> section the backends use.</td>
			</tr>
			<tr>
					<td><code>server_slots_base</code></td>
					<td><code>10</code></td>
					<td>Server lines per backend to start with. Minimum 10.</td>
			</tr>
			<tr>
					<td><code>server_slots_growth_type</code>, <code>_increment</code></td>
					<td><code>linear</code>, <code>10</code></td>
					<td>How the slots grow when they run out. Default <code>exponential</code>.</td>
			</tr>
	</tbody>
</table>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">curl -u admin -X POST -H <span class="s2">&#34;Content-Type: application/json&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  http://127.0.0.1:5555/v3/service_discovery/consul -d <span class="s1">&#39;{
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;address&#34;: &#34;127.0.0.1&#34;,
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;port&#34;: 8500,
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;enabled&#34;: true,
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;token&#34;: &#34;&lt;dataplaneapi token SecretID&gt;&#34;,
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;retry_timeout&#34;: 5,
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;service_allowlist&#34;: [&#34;web&#34;, &#34;api&#34;],
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;health_check_policy&#34;: &#34;all&#34;,
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;defaults&#34;: &#34;consul_services&#34;,
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;server_slots_base&#34;: 10,
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;server_slots_growth_type&#34;: &#34;linear&#34;,
</span></span></span><span class="line"><span class="cl"><span class="s1">    &#34;server_slots_growth_increment&#34;: 10
</span></span></span><span class="line"><span class="cl"><span class="s1">  }&#39;</span>
</span></span></code></pre></div><p>The answer contains an <code>id</code>. Change settings later with <code>PUT /v3/service_discovery/consul/&lt;id&gt;</code> and the full object. A few seconds later <code>haproxy.cfg</code> has the backends:</p>
<pre tabindex="0"><code class="language-haproxy" data-lang="haproxy">backend consul-backend-127.0.0.1-8500-web from consul_services
  server SRV_YKy8O 10.0.0.21:8080 check weight 128
  server SRV_WZpFa 127.0.0.1:80 disabled weight 128
  ...
</code></pre><h2 id="choosing-services-and-instances">Choosing Services and Instances <a class="anchor" data-pagefind-ignore href="#choosing-services-and-instances" aria-label="Link to this section">#</a></h2>
<p>Two filters decide what ends up in HAProxy: which services get a backend, and which instances of a service become servers.</p>
<h3 id="which-services">Which Services <a class="anchor" data-pagefind-ignore href="#which-services" aria-label="Link to this section">#</a></h3>
<table>
	<thead>
			<tr>
					<th>Field</th>
					<th>Use it for</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td><code>service_allowlist</code></td>
					<td>A fixed list of services to publish. The safe default.</td>
			</tr>
			<tr>
					<td><code>service_denylist</code></td>
					<td>Publish everything except a few (for example <code>consul</code> itself).</td>
			</tr>
			<tr>
					<td><code>service_name_regexp</code></td>
					<td>A naming convention, like <code>^web-</code>.</td>
			</tr>
	</tbody>
</table>
<p>There is no filter on tags, metadata or datacenter. Give instances that need their own backend their own service name, such as <code>web-blue</code> and <code>web-green</code>.</p>
<h3 id="which-instances-health_check_policy">Which Instances: health_check_policy <a class="anchor" data-pagefind-ignore href="#which-instances-health_check_policy" aria-label="Link to this section">#</a></h3>
<table>
	<thead>
			<tr>
					<th>Policy</th>
					<th>An instance becomes a server when</th>
					<th>Use it for</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td><code>none</code> (default)</td>
					<td>Always</td>
					<td>Let HAProxy&rsquo;s own <code>check</code> decide. A failing instance stays listed but marked DOWN.</td>
			</tr>
			<tr>
					<td><code>any</code></td>
					<td>At least one check passes</td>
					<td>Services with an optional extra check.</td>
			</tr>
			<tr>
					<td><code>all</code></td>
					<td>Every check passes (node checks included)</td>
					<td>The normal choice: Consul decides.</td>
			</tr>
			<tr>
					<td><code>min</code></td>
					<td>At least <code>health_check_policy_min</code> checks pass</td>
					<td>Many checks, some allowed to fail.</td>
			</tr>
	</tbody>
</table>
<p>Every server also gets HAProxy&rsquo;s own <code>check</code> (a TCP connect with these defaults). It catches instances that are healthy locally but unreachable from the load balancer.</p>
<h2 id="scaling-what-happens-on-a-change">Scaling: What Happens on a Change <a class="anchor" data-pagefind-ignore href="#scaling-what-happens-on-a-change" aria-label="Link to this section">#</a></h2>
<p>The API pre-creates server slots: real servers first, the rest as disabled placeholders (<code>127.0.0.1:80 disabled</code>). When the slots run out, it adds more.</p>
<p>What the lab showed with Data Plane API 3.4.7:</p>
<ul>
<li><strong>Scale up</strong>: the new instance is a server within one poll (<code>retry_timeout</code>).</li>
<li><strong>Scale down or failing check</strong> (policy <code>all</code>): the instance is removed at the next poll; a healed instance comes back.</li>
<li><strong>Every change is one reload</strong>, also when a free slot was used. The reload is seamless: the old worker finishes its connections.</li>
<li><strong>Changes are batched per poll</strong>: ten instances registered together caused two reloads, not ten.</li>
<li><strong>Empty service</strong>: the backend stays, with only disabled slots, so requests get a 503.</li>
</ul>
<p>Runtime changes to servers are lost at the next reload. A weight set through the runtime API went back to 128 when the next instance was added. Don&rsquo;t build on them.</p>
<h2 id="blue-green-deployments">Blue-Green Deployments <a class="anchor" data-pagefind-ignore href="#blue-green-deployments" aria-label="Link to this section">#</a></h2>
<p>Run two copies as two Consul services, <code>web-blue</code> and <code>web-green</code>, both in <code>service_allowlist</code>. The <code>hosts.map</code> entry decides which one is live. Switch it through the runtime maps API: no reload, and <code>force_sync</code> writes the map file so the switch survives a restart.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Switch www.example.com to green</span>
</span></span><span class="line"><span class="cl">curl -u admin -X PUT -H <span class="s2">&#34;Content-Type: application/json&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  <span class="s2">&#34;http://127.0.0.1:5555/v3/services/haproxy/runtime/maps/hosts.map/entries/www.example.com?force_sync=true&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  -d <span class="s1">&#39;{&#34;value&#34;:&#34;web-green&#34;}&#39;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Roll back</span>
</span></span><span class="line"><span class="cl">curl -u admin -X PUT -H <span class="s2">&#34;Content-Type: application/json&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  <span class="s2">&#34;http://127.0.0.1:5555/v3/services/haproxy/runtime/maps/hosts.map/entries/www.example.com?force_sync=true&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  -d <span class="s1">&#39;{&#34;value&#34;:&#34;web-blue&#34;}&#39;</span>
</span></span></code></pre></div><p>The same call with <code>POST .../entries</code> adds a new host route.</p>
<p>Percentage canaries don&rsquo;t fit this model: the API ignores Consul weights, and runtime weights are reset at the next reload. Use a separate host (<code>canary.example.com web-canary</code>) or a different tool for that.</p>
<h2 id="health-checks-and-service-discovery">Health Checks and Service Discovery <a class="anchor" data-pagefind-ignore href="#health-checks-and-service-discovery" aria-label="Link to this section">#</a></h2>
<p>Health checks decide who gets traffic, together with the <code>health_check_policy</code> from the registration. Too shallow, and broken instances keep receiving traffic. Too strict, and healthy ones get pulled.</p>
<h3 id="consul-health-check-types">Consul Health Check Types <a class="anchor" data-pagefind-ignore href="#consul-health-check-types" aria-label="Link to this section">#</a></h3>
<table>
	<thead>
			<tr>
					<th>Type</th>
					<th>Use it for</th>
					<th>Watch out</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>HTTP</td>
					<td>Web apps and APIs with a <code>/health</code> endpoint</td>
					<td>Make the endpoint check real dependencies</td>
			</tr>
			<tr>
					<td>TCP</td>
					<td>Services where you can only test the port</td>
					<td>A hung app that accepts connections still passes</td>
			</tr>
			<tr>
					<td>Script</td>
					<td>Custom logic</td>
					<td>Off by default: needs <code>enable_local_script_checks</code></td>
			</tr>
			<tr>
					<td>gRPC</td>
					<td>gRPC services with the standard health protocol</td>
					<td></td>
			</tr>
			<tr>
					<td>TTL</td>
					<td>Workers and batch jobs without a port</td>
					<td>The service must report in itself</td>
			</tr>
	</tbody>
</table>
<p>The example shows all five for the syntax. A real service needs one or two.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-hcl" data-lang="hcl"><span class="line"><span class="cl"><span class="k">service</span> {
</span></span><span class="line"><span class="cl"><span class="n">  name</span> <span class="o">=</span> <span class="s2">&#34;web&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">  port</span> <span class="o">=</span> <span class="m">8080</span>
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="n">checks</span> <span class="o">=</span> <span class="p">[</span><span class="c1">
</span></span></span><span class="line"><span class="cl"><span class="c1">    # HTTP check
</span></span></span><span class="line"><span class="cl">    {
</span></span><span class="line"><span class="cl"><span class="n">      name</span> <span class="o">=</span> <span class="s2">&#34;HTTP Health&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      http</span> <span class="o">=</span> <span class="s2">&#34;http://localhost:8080/health&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      method</span> <span class="o">=</span> <span class="s2">&#34;GET&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      interval</span> <span class="o">=</span> <span class="s2">&#34;10s&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      timeout</span> <span class="o">=</span> <span class="s2">&#34;5s&#34;</span>
</span></span><span class="line"><span class="cl">    }<span class="p">,</span><span class="c1">
</span></span></span><span class="line"><span class="cl"><span class="c1">    
</span></span></span><span class="line"><span class="cl"><span class="c1">    # TCP check
</span></span></span><span class="line"><span class="cl">    {
</span></span><span class="line"><span class="cl"><span class="n">      name</span> <span class="o">=</span> <span class="s2">&#34;TCP Port&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      tcp</span> <span class="o">=</span> <span class="s2">&#34;localhost:8080&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      interval</span> <span class="o">=</span> <span class="s2">&#34;5s&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      timeout</span> <span class="o">=</span> <span class="s2">&#34;2s&#34;</span>
</span></span><span class="line"><span class="cl">    }<span class="p">,</span><span class="c1">
</span></span></span><span class="line"><span class="cl"><span class="c1">    
</span></span></span><span class="line"><span class="cl"><span class="c1">    # Script check (the agent needs enable_local_script_checks = true,
</span></span></span><span class="line"><span class="cl"><span class="c1">    # otherwise registering the whole service fails)
</span></span></span><span class="line"><span class="cl">    {
</span></span><span class="line"><span class="cl"><span class="n">      name</span> <span class="o">=</span> <span class="s2">&#34;Script Check&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      args</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&#34;/usr/local/bin/health-check.sh&#34;</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="n">      interval</span> <span class="o">=</span> <span class="s2">&#34;30s&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      timeout</span> <span class="o">=</span> <span class="s2">&#34;10s&#34;</span>
</span></span><span class="line"><span class="cl">    }<span class="p">,</span><span class="c1">
</span></span></span><span class="line"><span class="cl"><span class="c1">    
</span></span></span><span class="line"><span class="cl"><span class="c1">    # gRPC check
</span></span></span><span class="line"><span class="cl">    {
</span></span><span class="line"><span class="cl"><span class="n">      name</span> <span class="o">=</span> <span class="s2">&#34;gRPC Health&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      grpc</span> <span class="o">=</span> <span class="s2">&#34;localhost:9090&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      grpc_use_tls</span> <span class="o">=</span> <span class="kt">true</span>
</span></span><span class="line"><span class="cl"><span class="n">      interval</span> <span class="o">=</span> <span class="s2">&#34;10s&#34;</span>
</span></span><span class="line"><span class="cl">    }<span class="p">,</span><span class="c1">
</span></span></span><span class="line"><span class="cl"><span class="c1">    
</span></span></span><span class="line"><span class="cl"><span class="c1">    # TTL check (service must update)
</span></span></span><span class="line"><span class="cl">    {
</span></span><span class="line"><span class="cl"><span class="n">      name</span> <span class="o">=</span> <span class="s2">&#34;TTL Check&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      ttl</span> <span class="o">=</span> <span class="s2">&#34;30s&#34;</span>
</span></span><span class="line"><span class="cl"><span class="n">      status</span> <span class="o">=</span> <span class="s2">&#34;passing&#34;</span>
</span></span><span class="line"><span class="cl">    }
</span></span><span class="line"><span class="cl">  <span class="p">]</span>
</span></span><span class="line"><span class="cl">}
</span></span></code></pre></div><h3 id="updating-ttl-check">Updating TTL Check <a class="anchor" data-pagefind-ignore href="#updating-ttl-check" aria-label="Link to this section">#</a></h3>
<p>The service calls <code>pass</code> before the TTL runs out to stay healthy. Calling <code>fail</code> takes it out on purpose, for example while draining.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Check IDs are service:&lt;service-id&gt;:&lt;n&gt;; the TTL check above is the 5th check of &#34;web&#34;</span>
</span></span><span class="line"><span class="cl"><span class="c1"># (curl -s http://localhost:8500/v1/agent/checks | jq keys lists them)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Pass TTL check</span>
</span></span><span class="line"><span class="cl">curl -X PUT http://localhost:8500/v1/agent/check/pass/service:web:5
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Fail TTL check</span>
</span></span><span class="line"><span class="cl">curl -X PUT http://localhost:8500/v1/agent/check/fail/service:web:5
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Warn TTL check</span>
</span></span><span class="line"><span class="cl">curl -X PUT http://localhost:8500/v1/agent/check/warn/service:web:5
</span></span></code></pre></div><h2 id="monitoring-and-observability">Monitoring and Observability <a class="anchor" data-pagefind-ignore href="#monitoring-and-observability" aria-label="Link to this section">#</a></h2>
<p>Compare what Consul thinks is healthy with what HAProxy actually uses. If they differ, look at the Data Plane API.</p>
<h3 id="consul-service-health-queries">Consul Service Health Queries <a class="anchor" data-pagefind-ignore href="#consul-service-health-queries" aria-label="Link to this section">#</a></h3>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># List all healthy web services</span>
</span></span><span class="line"><span class="cl">curl http://localhost:8500/v1/health/service/web?passing<span class="o">=</span><span class="nb">true</span> <span class="p">|</span> jq .
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Get service catalog</span>
</span></span><span class="line"><span class="cl">curl http://localhost:8500/v1/catalog/services <span class="p">|</span> jq .
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Watch for changes</span>
</span></span><span class="line"><span class="cl">watch -n <span class="m">2</span> <span class="s1">&#39;curl -s http://localhost:8500/v1/health/service/web | jq length&#39;</span>
</span></span></code></pre></div><h3 id="haproxy-stats-integration">HAProxy Stats Integration <a class="anchor" data-pagefind-ignore href="#haproxy-stats-integration" aria-label="Link to this section">#</a></h3>
<p>The base configuration already serves the stats page on <code>:8404/stats</code> and Prometheus metrics on <code>:8404/metrics</code>. See the <a href="https://blog.bsdserver.nl/haproxy-monitoring-with-prometheus-complete-observability-guide/">Prometheus monitoring post</a>.</p>
<h2 id="troubleshooting">Troubleshooting <a class="anchor" data-pagefind-ignore href="#troubleshooting" aria-label="Link to this section">#</a></h2>
<p>Follow the chain: registered → checks passing (per <code>health_check_policy</code>) → seen by the Data Plane API → server in the backend → <code>hosts.map</code> points at the service. The first broken link is the problem. On a fresh install it is usually ACLs: look for <code>blocked by ACLs</code> in the Consul log.</p>
<h3 id="common-issues">Common Issues <a class="anchor" data-pagefind-ignore href="#common-issues" aria-label="Link to this section">#</a></h3>
<ol>
<li><strong>No <code>consul-backend-…</code> backend at all</strong>
<ul>
<li>Is the service in <code>service_allowlist</code>?</li>
<li>Does the registration have the Data Plane API token? Without it, a default-deny Consul shows no services.</li>
<li>Check the API log: <code>journalctl -u dataplaneapi</code></li>
</ul>
</li>
<li><strong>Backend exists, requests get a 503</strong>
<ul>
<li>No instance passes the policy: every server line is a disabled slot.</li>
<li>Check the checks: <code>curl -s http://localhost:8500/v1/health/checks/web | jq .</code></li>
</ul>
</li>
<li><strong>Requests reach the wrong service</strong>
<ul>
<li>Check the map: <code>echo &quot;show map /etc/haproxy/maps/hosts.map&quot; | sudo socat - /run/haproxy/admin.sock</code></li>
</ul>
</li>
<li><strong>Hand edits to <code>haproxy.cfg</code> disappear</strong>
<ul>
<li>The Data Plane API owns the file. Change it through the API.</li>
</ul>
</li>
</ol>
<h3 id="debug-commands">Debug Commands <a class="anchor" data-pagefind-ignore href="#debug-commands" aria-label="Link to this section">#</a></h3>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># The registration, as the API sees it (the token is included: don&#39;t paste this output)</span>
</span></span><span class="line"><span class="cl">curl -s -u admin http://127.0.0.1:5555/v3/service_discovery/consul <span class="p">|</span> jq .
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Generated backends and their servers</span>
</span></span><span class="line"><span class="cl">grep -A3 <span class="s1">&#39;^backend consul-backend&#39;</span> /etc/haproxy/haproxy.cfg
</span></span><span class="line"><span class="cl"><span class="nb">echo</span> <span class="s2">&#34;show servers state consul-backend-127.0.0.1-8500-web&#34;</span> <span class="p">|</span> sudo socat - /run/haproxy/admin.sock
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Reloads caused by the API</span>
</span></span><span class="line"><span class="cl">journalctl -u haproxy --since <span class="s2">&#34;10 min ago&#34;</span> <span class="p">|</span> grep <span class="s2">&#34;Reloading HAProxy&#34;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Consul side</span>
</span></span><span class="line"><span class="cl">consul members
</span></span><span class="line"><span class="cl">consul catalog nodes -service<span class="o">=</span>web
</span></span></code></pre></div><h2 id="security-considerations">Security Considerations <a class="anchor" data-pagefind-ignore href="#security-considerations" aria-label="Link to this section">#</a></h2>
<p>Anyone who can register a service named <code>web</code> receives your web traffic. Protect Consul like the rest of your traffic path.</p>
<ul>
<li><strong>Read-only token for the API.</strong> The <code>dataplaneapi</code> policy only reads nodes and services; a KV write with it gets a 403. It is stored in <code>/etc/haproxy/dataplane/service_discovery/consul.json</code> (root, mode 0600).</li>
<li><strong>API on localhost.</strong> <code>host: 127.0.0.1</code>, a hashed password, <code>dataplaneapi.yaml</code> mode 0600. Add TLS before exposing it.</li>
<li><strong>Local Consul agent.</strong> The API talks plain HTTP to <code>127.0.0.1:8500</code>. The registration schema has a <code>mode</code> field (<code>http</code> or <code>https</code>) but no CA or client-certificate settings; not tested here.</li>
<li><strong>Narrow the agent token in production.</strong> The agent policy here can write every service; scope it per node and service.</li>
<li><strong>Encrypt Consul traffic.</strong> Gossip encryption (<code>encrypt</code>) and TLS on RPC between agents and servers.</li>
</ul>
<h2 id="automate-it">Automate it <a class="anchor" data-pagefind-ignore href="#automate-it" aria-label="Link to this section">#</a></h2>
<p>The Ansible roles and Terraform code that build this setup are on GitHub:
<a href="https://github.com/pderuiter/blog-haproxy-consul" rel="noopener">pderuiter/blog-haproxy-consul</a>. It creates a three-server Consul cluster with ACLs, a service node with the web and api services, and an HAProxy load balancer where the Data Plane API discovers the services in Consul; fill in the inventory, four token UUIDs and the API password, then run <code>ansible-playbook site.yml</code>.</p>
<h2 id="next-steps">Next Steps <a class="anchor" data-pagefind-ignore href="#next-steps" aria-label="Link to this section">#</a></h2>
<p>With Consul service discovery integrated, consider:</p>
<ul>
<li><strong>Prometheus Monitoring</strong> - Add metrics collection for HAProxy and Consul</li>
<li><strong>Consul Connect</strong> - Implement service mesh with automatic mTLS</li>
<li><strong>Data Plane API</strong> - Manage frontends and ACLs through the same API: see the <a href="https://blog.bsdserver.nl/haproxy-data-plane-api-dynamic-configuration-management/">Data Plane API post</a></li>
</ul>
<p>Consul and HAProxy together create a self-healing setup where services join and leave the load balancer pool based on their health.</p>
]]></content:encoded></item></channel></rss>