ECMP and Multipath: Load Balancing at the Routing Layer

January 16, 2026 · 7 min read

Two paths to the same destination. Same cost. Traditional routing picks one. ECMP (Equal-Cost Multi-Path) uses both, spreading traffic across available paths.

The concept is simple: multiple equal routes, traffic distributed. The implementation has nuances: how traffic is distributed, what makes routes “equal,” and why some flows always use the same path.

ECMP is simple but requires understanding to use effectively.

How ECMP Works

Without ECMP:
                    [Path A: cost 10] → (used)
Host → Router →
                    [Path B: cost 10] → (ignored)

With ECMP:
                    [Path A: cost 10] → (50% traffic)
Host → Router →
                    [Path B: cost 10] → (50% traffic)

Traffic is distributed per-flow, not per-packet. All packets for the same flow use the same path (preventing reordering).

Basic ECMP Configuration

Static Routes ECMP

configure

# Two equal-cost static routes
set protocols static route 10.0.0.0/8 next-hop 192.168.1.1
set protocols static route 10.0.0.0/8 next-hop 192.168.1.2

# VyOS automatically installs both if costs equal
commit

Verify ECMP Routes

show ip route 10.0.0.0/8

# Output:
# S>* 10.0.0.0/8 [1/0] via 192.168.1.1, eth0, weight 1, 00:05:00
#                 via 192.168.1.2, eth1, weight 1, 00:05:00

Multiple next-hops shown = ECMP active.

ECMP with BGP

Enable Multipath

configure

# Enable ECMP for eBGP
set protocols bgp address-family ipv4-unicast maximum-paths ebgp 4

# Enable ECMP for iBGP
set protocols bgp address-family ipv4-unicast maximum-paths ibgp 4

commit

BGP Path Requirements

For BGP paths to be ECMP-eligible, they must have:

Same AS_PATH length
Same origin (IGP/EGP/incomplete)
Same MED (or MED comparison disabled)
Same local preference

# Compare paths
show bgp ipv4 unicast 10.0.0.0/8

# If paths differ in AS_PATH length, not ECMP-eligible
# Path 1: AS_PATH 65001 65002 (length 2)
# Path 2: AS_PATH 65001 (length 1)  ← shorter, wins alone

Allow Multipath from Same AS

# For multiple connections to same AS
set protocols bgp address-family ipv4-unicast maximum-paths ebgp 4
set protocols bgp address-family ipv4-unicast multipath-relax

# multipath-relax: Allows ECMP even if AS_PATH differs (same length)

ECMP with OSPF

Enable OSPF ECMP

configure

# OSPF supports ECMP by default
# Configure maximum paths
set protocols ospf parameters maximum-paths 4

commit

OSPF naturally creates ECMP when multiple paths have equal cost.

Hash Algorithm

Traffic distribution uses hash of packet headers. Same hash = same path.

Hash Inputs

Default hash inputs:
- Source IP
- Destination IP
- Source port
- Destination port
- Protocol

Hash result → selects path

Configure Hash Algorithm

# VyOS uses kernel's fib_multipath_hash_policy
# 0 = Layer 3 only (src/dst IP)
# 1 = Layer 4 (src/dst IP + ports)
# 2 = Layer 3 or inner for tunnels

configure
set system sysctl parameter net.ipv4.fib_multipath_hash_policy value 1
commit

Layer 3 vs Layer 4 Hash

# Layer 3 only:
# Same src/dst IP pair always uses same path
# Different src IPs spread across paths

# Layer 4:
# Same src/dst IP but different ports can use different paths
# Better distribution for single-host scenarios

Troubleshooting ECMP

Issue: Uneven Distribution

# One path getting most traffic

# Causes:
# 1. Hash algorithm + traffic pattern = uneven
# 2. Not actually ECMP (one path preferred)
# 3. Few unique flows (small sample)

# Check if actually ECMP
show ip route 10.0.0.0/8
# Must show multiple next-hops

# Monitor per-path traffic
# Use interface counters
watch -n 1 'show interfaces ethernet eth0; show interfaces ethernet eth1'

Issue: Single Flow Always Same Path

# This is expected behavior!
# ECMP hashes per-flow, not per-packet

# Same src/dst/port always hashes to same path
# Prevents packet reordering

# For testing, use different source ports
nc -p 10001 server 80
nc -p 10002 server 80
# May use different paths

Issue: Paths Not Equal

# BGP paths not becoming ECMP

show bgp ipv4 unicast 10.0.0.0/8 bestpath

# Check what makes them unequal:
# - AS_PATH length different?
# - MED different?
# - Local preference different?

# Fix the inequality or enable multipath-relax

Issue: Route Flapping

# One path keeps appearing/disappearing

# ECMP recalculates when paths change
# Can cause flow redistribution

# Solution: Stabilize the flapping path
# Or implement dampening

Weighted ECMP

Not all paths are equal? Use weights:

configure

# Higher weight = more traffic
set protocols static route 10.0.0.0/8 next-hop 192.168.1.1 distance 1
set protocols static route 10.0.0.0/8 next-hop 192.168.1.2 distance 1

# Unfortunately, VyOS static routes don't have weight directly
# Use different administrative distance for preference (not ECMP)

# For weighted distribution, consider:
# - BGP with different link bandwidths
# - Policy routing with firewall marks

ECMP Failure Handling

When One Path Fails

# ECMP automatically removes failed path
# Traffic redistributes to remaining paths

# Flow rehashing happens:
# - Some flows move to different paths
# - Brief reordering possible during transition

BFD for Fast Failure Detection

configure

# Use BFD to quickly detect path failure
set protocols bfd peer 192.168.1.1
set protocols bfd peer 192.168.1.2

# When BFD detects failure, route withdrawn immediately
# ECMP recalculates faster than waiting for routing protocol

ECMP Best Practices

1. Match Bandwidth

# ECMP assumes equal paths
# 10G + 1G ECMP = poor utilization

# Either:
# - Use paths with equal bandwidth
# - Use weighted/unequal ECMP if available
# - Different approach (LAG, policy routing)

2. Enable Layer 4 Hash

# Better distribution for typical traffic
set system sysctl parameter net.ipv4.fib_multipath_hash_policy value 1

3. Monitor Both Paths

# Dashboard showing:
# - Traffic per path
# - Errors per path
# - ECMP route status

4. Test Failover

# Regularly test:
# 1. Disable one path
# 2. Verify traffic flows via remaining path
# 3. Re-enable path
# 4. Verify ECMP resumes

ECMP vs LAG

Feature	ECMP	LAG (Bond)
Layer	3 (routing)	2 (switching)
Protocols	Different paths, routers	Same path, one hop
Failure detection	Routing protocol	LACP
Configuration	Routing config	Interface config
Scalability	Many paths	Limited ports

# ECMP: Different next-hop routers
# LAG: Same router, bundled interfaces

# Use LAG for link aggregation to single device
# Use ECMP for path diversity across network

Checking ECMP Status

# View kernel routing table
ip route show 10.0.0.0/8

# Show with ECMP details
ip route show 10.0.0.0/8 | grep -i nexthop

# Count ECMP paths
ip route show 10.0.0.0/8 | grep -c nexthop

# Test which path a flow would take
ip route get 10.0.0.100 from 192.168.1.50

The Lesson

ECMP is simple but requires understanding to use effectively.

What ECMP gives you:

Automatic load distribution across equal paths
Redundancy (path failure → automatic reroute)
Increased aggregate bandwidth

What ECMP doesn’t give you:

Per-packet load balancing (would cause reordering)
Intelligent traffic distribution (hash-based, may be uneven)
Weighted distribution (standard ECMP is equal-cost)

Key understanding:

Paths must be truly equal (cost, metrics)
Distribution is per-flow, not per-packet
Hash algorithm determines distribution
Layer 4 hash usually gives better distribution
Uneven traffic is normal with few flows

Configure it, understand it, monitor it. ECMP works well when you know what to expect.