Berik Ashimov
Senior IT Engineer

Conntrack Deep Dive: Connection Tables, Limits, and Debugging

· 8 min read
#vyos #firewall #networking #troubleshooting

Your stateful firewall silently tracks every connection. Allowing “established, related” traffic to return requires remembering that you initiated the connection. That’s conntrack.

When conntrack table fills up, new connections fail mysteriously. No error, just dropped. The firewall has no room to track new connections, so it drops them.

Conntrack is the invisible stateful firewall engine. When it fails, everything fails.

What Conntrack Does

Client → Router → Server


1. Client sends SYN to server
2. Router creates conntrack entry: NEW
3. Server responds with SYN-ACK
4. Router updates entry: ESTABLISHED
5. Traffic flows, entry tracks state
6. Connection closes
7. Entry times out, removed

Without conntrack, the firewall can’t know that a packet from the server is a response to your request vs. unsolicited traffic.

Viewing Conntrack Table

Basic Commands

Terminal window
# Show all connections
show conntrack table


# Show IPv4 connections only
show conntrack table ipv4


# Show IPv6 connections only
show conntrack table ipv6


# Show connection count
show conntrack table statistics

Filtering Conntrack

Terminal window
# Show connections to specific IP
show conntrack table ipv4 | grep "192.168.1.100"


# Show connections by protocol
show conntrack table ipv4 | grep "tcp"
show conntrack table ipv4 | grep "udp"


# Show connections in specific state
show conntrack table ipv4 | grep "ESTABLISHED"
show conntrack table ipv4 | grep "TIME_WAIT"

Direct Conntrack Commands

Terminal window
# Using conntrack tool directly
sudo conntrack -L                    # List all
sudo conntrack -L -p tcp             # TCP only
sudo conntrack -L -s 192.168.1.100   # Source IP
sudo conntrack -L -d 8.8.8.8         # Destination IP
sudo conntrack -C                    # Count entries

Conntrack Statistics

Terminal window
# View statistics
show conntrack table statistics


# Or directly
sudo conntrack -S


# Output:
# cpu=0     found=12345 invalid=0 ignore=0 insert=0 insert_failed=0 drop=0 early_drop=0 error=0 search_restart=0
# cpu=1     found=12340 invalid=2 ignore=0 insert=0 insert_failed=0 drop=0 early_drop=0 error=0 search_restart=0


# Key metrics:
# drop: Packets dropped due to conntrack issues
# early_drop: Entries dropped to make room for new ones
# insert_failed: Failed to create new entry (table full)

Conntrack Table Size

Check Current Settings

Terminal window
# Current maximum entries
cat /proc/sys/net/netfilter/nf_conntrack_max


# Current count
cat /proc/sys/net/netfilter/nf_conntrack_count


# Hash table size
cat /proc/sys/net/netfilter/nf_conntrack_buckets

Increase Table Size

Terminal window
configure


# Increase max connections
set system conntrack table-size 262144  # Default is often 65536


# Adjust hash table size (should be ~25% of table-size)
set system conntrack hash-size 65536


commit

Calculate Requirements

Rule of thumb:
- Each connection uses ~350 bytes
- 65536 entries ≈ 22 MB RAM
- 262144 entries ≈ 90 MB RAM


# For NAT gateway with 1000 users:
# Assume 100 connections per user
# 1000 × 100 = 100,000 connections
# Set table-size to at least 150,000 (with headroom)

Conntrack Timeouts

View Current Timeouts

Terminal window
# TCP timeouts
cat /proc/sys/net/netfilter/nf_conntrack_tcp_timeout_established
cat /proc/sys/net/netfilter/nf_conntrack_tcp_timeout_time_wait
cat /proc/sys/net/netfilter/nf_conntrack_tcp_timeout_close_wait


# UDP timeouts
cat /proc/sys/net/netfilter/nf_conntrack_udp_timeout
cat /proc/sys/net/netfilter/nf_conntrack_udp_timeout_stream


# ICMP timeout
cat /proc/sys/net/netfilter/nf_conntrack_icmp_timeout

Configure Timeouts in VyOS

Terminal window
configure


# TCP timeouts
set system conntrack timeout tcp established 7200  # 2 hours (default 5 days!)
set system conntrack timeout tcp close 10
set system conntrack timeout tcp close-wait 60
set system conntrack timeout tcp fin-wait 120
set system conntrack timeout tcp last-ack 30
set system conntrack timeout tcp syn-recv 60
set system conntrack timeout tcp syn-sent 120
set system conntrack timeout tcp time-wait 120


# UDP timeouts
set system conntrack timeout udp other 30
set system conntrack timeout udp stream 180


# ICMP timeout
set system conntrack timeout icmp 30


commit

Aggressive Timeouts (For Busy NAT)

Terminal window
# For high-traffic NAT gateways with limited table size
set system conntrack timeout tcp established 3600   # 1 hour
set system conntrack timeout tcp time-wait 30       # Clear quickly
set system conntrack timeout udp other 30
set system conntrack timeout udp stream 60


# Warning: Too aggressive can break long-lived connections

Conntrack Problems

Problem 1: Table Full

Symptoms:

  • New connections fail randomly
  • Established connections work
  • Log shows “nf_conntrack: table full”
Terminal window
# Check table status
show conntrack table statistics


# Look for:
# early_drop > 0
# insert_failed > 0


# Check current vs max
cat /proc/sys/net/netfilter/nf_conntrack_count
cat /proc/sys/net/netfilter/nf_conntrack_max


# Fix: Increase size
set system conntrack table-size 524288

Problem 2: Memory Exhaustion

Large conntrack tables consume RAM:

Terminal window
# Calculate memory needed
# 262144 entries × 350 bytes ≈ 90 MB


# If short on memory, reduce table or add RAM
# Or reduce timeouts to expire entries faster

Problem 3: Stale Entries

Connections closed but entries remain:

Terminal window
# Clear specific entry
sudo conntrack -D -s 192.168.1.100 -d 8.8.8.8


# Clear all entries (dangerous in production!)
sudo conntrack -F


# Clear entries by protocol
sudo conntrack -D -p udp

Problem 4: Conntrack Not Tracking

Terminal window
# Some traffic bypasses conntrack (NOTRACK)
# Check if tracking is enabled


show firewall


# Look for notrack rules:
# set firewall ipv4 name RAW rule 10 action notrack

NOTRACK Rules

For high-bandwidth traffic that doesn’t need stateful inspection:

Terminal window
configure


# Skip tracking for specific traffic
set firewall ipv4 name RAW default-action accept
set firewall ipv4 name RAW rule 10 action notrack
set firewall ipv4 name RAW rule 10 destination address 224.0.0.0/4
set firewall ipv4 name RAW rule 10 description "Skip tracking for multicast"


# Apply to raw table
set firewall ipv4 input filter raw RAW


commit

When to Use NOTRACK

  • Multicast/broadcast traffic
  • High-bandwidth services that don’t need state
  • Traffic between trusted internal segments
  • When conntrack table is bottleneck

Risks of NOTRACK

  • No stateful filtering for that traffic
  • “Established, related” rules won’t work
  • Must use stateless rules for that traffic

Conntrack Helpers

Conntrack helpers track multi-connection protocols (FTP, SIP):

Terminal window
# View loaded helpers
lsmod | grep nf_conntrack


# Configure FTP helper
set system conntrack modules ftp


# Configure SIP helper
set system conntrack modules sip


commit

FTP Active Mode Fix

Terminal window
# FTP active mode requires helper to track data connection
set system conntrack modules ftp


# Helper allows firewall to recognize FTP data connections
# as related to control connection

Monitoring Conntrack

Watch Table Fill

Terminal window
# Monitor in real-time
watch -n 1 'cat /proc/sys/net/netfilter/nf_conntrack_count'


# Alert script
#!/bin/bash
MAX=$(cat /proc/sys/net/netfilter/nf_conntrack_max)
CURRENT=$(cat /proc/sys/net/netfilter/nf_conntrack_count)
THRESHOLD=80


USAGE=$((CURRENT * 100 / MAX))


if [ $USAGE -gt $THRESHOLD ]; then
    echo "Conntrack ${USAGE}% full (${CURRENT}/${MAX})"
fi

Prometheus Metrics

Terminal window
# Node exporter exposes conntrack metrics:
# node_nf_conntrack_entries
# node_nf_conntrack_entries_limit


# Alert when > 80% full
# expr: node_nf_conntrack_entries / node_nf_conntrack_entries_limit > 0.8

Conntrack by Connection Type

Heavy NAT Users

Terminal window
# Find IPs with most connections
sudo conntrack -L | awk '{print $5}' | cut -d= -f2 | sort | uniq -c | sort -rn | head


# Output:
# 5234 192.168.1.50
# 3421 192.168.1.51
# ...

Connection State Distribution

Terminal window
# Count by state
sudo conntrack -L | grep -o 'tcp .* [A-Z_]*' | awk '{print $NF}' | sort | uniq -c | sort -rn


# Output:
# 45234 ESTABLISHED
#  2341 TIME_WAIT
#   123 SYN_SENT

Best Practices

1. Size for Peak Load

Terminal window
# Calculate peak connections
# Add 50% headroom
set system conntrack table-size 262144  # More is safer

2. Tune Timeouts for Traffic

Terminal window
# Long-lived connections (database, SSH)
set system conntrack timeout tcp established 86400  # 24h


# Short-lived connections (HTTP)
set system conntrack timeout tcp established 3600   # 1h

3. Monitor Always

Terminal window
# Alert before table fills
# Dashboard showing:
# - Current entries
# - Max entries
# - Entries/second rate

4. NOTRACK Where Safe

Terminal window
# Reduce load by not tracking:
# - Internal trusted traffic
# - High-bandwidth transfers
# - Multicast

The Lesson

Conntrack is the invisible stateful firewall engine. When it fails, everything fails.

Every “allow established, related” rule depends on conntrack. Every NAT translation depends on conntrack. Without it, no stateful firewalling.

When conntrack table fills:

  • New connections silently fail
  • No error message to user
  • Existing connections keep working
  • Very confusing symptoms

Prevention:

  • Size table for expected load + headroom
  • Tune timeouts for your traffic patterns
  • Monitor table usage constantly
  • Alert before it fills, not after

The connection table is limited. Plan for it.