TCP MSS Clamping: When and Why to Adjust Segment Size

December 16, 2025 · 8 min read

TCP connections work. Then you add a VPN. Suddenly large transfers fail while small ones work. SSH connects, but SCP stalls. Websites load headers but not content.

The culprit: MTU mismatch. Your tunnel has overhead. Packets get fragmented or dropped. ICMP “fragmentation needed” messages get filtered. TCP never learns the path MTU.

MSS clamping fixes this by telling TCP to use smaller segments in the first place. No fragmentation needed, no ICMP required.

MTU vs MSS

MTU (Maximum Transmission Unit): Maximum IP packet size an interface can send.

Ethernet default: 1500 bytes
Includes IP header (20) and TCP header (20)

MSS (Maximum Segment Size): Maximum TCP payload size.

Announced during TCP handshake
MSS = MTU - 40 (IPv4) or MTU - 60 (IPv6)
Ethernet default MSS: 1460 bytes (IPv4)

MTU 1500:
┌─────────────────────────────────────────────────┐
│ IP Header │ TCP Header │      TCP Data         │
│   20 B    │    20 B    │       1460 B          │
└─────────────────────────────────────────────────┘
                           ↑ This is MSS

Why Tunnels Break Large Transfers

Tunnels add encapsulation overhead:

Normal Ethernet (MTU 1500):
[ IP | TCP | Data 1460 bytes ]  = 1500 bytes ✓

GRE Tunnel (24 bytes overhead):
[ Outer IP | GRE | Inner IP | TCP | Data 1460 bytes ] = 1524 bytes ✗
                                                         ↑ Exceeds MTU!

Options:

Fragment packets (slow, can fail)
Lower tunnel MTU (requires end-to-end coordination)
MSS clamping (transparent, works without coordination)

MSS Clamping in VyOS

Interface-Based Clamping

configure

# Clamp MSS on specific interface
set firewall options interface eth0 adjust-mss 1360

# For tunnel interfaces
set firewall options interface tun0 adjust-mss 1360
set firewall options interface wg0 adjust-mss 1380

commit

Calculating Correct MSS

# Formula: MSS = Tunnel_MTU - 40 (IPv4)
# Or:      MSS = Tunnel_MTU - 60 (IPv6)

# Common scenarios:
# PPPoE (MTU 1492):    MSS = 1492 - 40 = 1452
# GRE (MTU 1476):      MSS = 1476 - 40 = 1436
# IPsec (~MTU 1400):   MSS = 1400 - 40 = 1360
# WireGuard (MTU 1420): MSS = 1420 - 40 = 1380
# VXLAN (MTU 1450):    MSS = 1450 - 40 = 1410

Global MSS Clamping

# Apply to all interfaces (less targeted but simpler)
set firewall options all-interfaces adjust-mss 1360

commit

MSS Clamping by Zone/Direction

# Clamp only for traffic leaving via tunnel
set firewall options interface tun0 adjust-mss 1360

# Clamp for traffic entering from LAN toward tunnel
set firewall options interface eth1 adjust-mss 1360  # LAN interface

PPPoE Configuration

PPPoE is the most common MSS clamping scenario:

configure

# PPPoE interface setup
set interfaces ethernet eth0 pppoe 0 default-route auto
set interfaces ethernet eth0 pppoe 0 mtu 1492
set interfaces ethernet eth0 pppoe 0 name-server auto

# MSS clamping for PPPoE
set firewall options interface pppoe0 adjust-mss 1452

# Or use 'clamp-mss-to-pmtu' to auto-calculate
set firewall options interface pppoe0 adjust-mss clamp-mss-to-pmtu

commit

VPN Tunnel Configuration

IPsec

# IPsec has variable overhead depending on encryption
# ESP header + encryption padding: ~50-80 bytes

# Conservative MSS for IPsec
set firewall options interface vti0 adjust-mss 1360

# Or on LAN-facing interface for traffic going to VPN
set firewall options interface eth1 adjust-mss 1360

WireGuard

# WireGuard overhead: 60 bytes (IPv4) or 80 bytes (IPv6)
# Default WireGuard MTU: 1420

set firewall options interface wg0 adjust-mss 1380

GRE

# GRE overhead: 24 bytes (basic) to 28+ (with key/sequence)
# GRE over IPsec: even more overhead

set interfaces tunnel tun0 encapsulation gre
set interfaces tunnel tun0 mtu 1400
set firewall options interface tun0 adjust-mss 1360

Troubleshooting MSS Issues

Symptom: Large Transfers Fail

# Small files/pings work
ping -s 64 remote-host    # Works

# Large transfers fail/hang
ping -s 1400 remote-host  # Fails or hangs

# Solution: Add MSS clamping

Detecting Current MSS

# Capture TCP SYN packets to see advertised MSS
tcpdump -i eth0 'tcp[tcpflags] & (tcp-syn) != 0' -v

# Look for "mss 1460" or similar in output
# 14:32:15 IP host.port > dest.port: Flags [S], ... mss 1460 ...

Verify Clamping is Working

# Before clamping:
# Client sends: mss 1460
# After clamping to 1360:
# Router modifies: mss 1360

# Capture on LAN side
tcpdump -i eth1 'tcp[tcpflags] & (tcp-syn) != 0' -v

# Capture on tunnel side
tcpdump -i tun0 'tcp[tcpflags] & (tcp-syn) != 0' -v

# Compare MSS values

Test Path MTU

# Find actual path MTU
tracepath remote-host

# Manual test with DF bit
ping -M do -s 1372 remote-host  # Adjust size until works

Common Scenarios

Scenario 1: Site-to-Site VPN Users Can’t Access Some Sites

Problem: VPN tunnel MTU is 1400
         Client sends MSS 1460
         Large packets can't traverse tunnel

Solution:
set firewall options interface vti0 adjust-mss 1360

Scenario 2: PPPoE Users Have Random Website Issues

Problem: PPPoE MTU 1492
         Some sites have PMTUD blackhole
         They never learn about lower MTU

Solution:
set firewall options interface pppoe0 adjust-mss 1452

Scenario 3: GRE Tunnel Works for Ping, Not SCP

Problem: GRE overhead not accounted for
         Large SSH/SCP packets fragmented/dropped

Solution:
set interfaces tunnel tun0 mtu 1400
set firewall options interface tun0 adjust-mss 1360

Scenario 4: Double-Tunnel (GRE over IPsec)

Problem: Outer tunnel already reduces MTU
         Inner tunnel reduces it more
         Need very low MSS

Solution:
# Outer IPsec: MTU ~1400
# GRE inside: MTU 1400 - 24 = 1376
# MSS: 1376 - 40 = 1336

set firewall options interface tun0 adjust-mss 1336

Advanced Configuration

IPv6 MSS Clamping

# IPv6 header is 40 bytes (vs 20 for IPv4)
# MSS = MTU - 60

set firewall options interface eth0 adjust-mss6 1340

# Or combined for both protocols
set firewall options interface eth0 adjust-mss 1360
set firewall options interface eth0 adjust-mss6 1340

Asymmetric Clamping

# Different MSS for different directions
# Not directly supported, but can use firewall zones

# Traffic entering from Internet, leaving to tunnel
set firewall options interface eth0 adjust-mss 1360

# Traffic entering from tunnel, leaving to LAN
# (usually not needed - responses use same MSS)

Clamping with NAT

# MSS clamping works with NAT
# Apply before or after NAT (usually doesn't matter)

set nat source rule 100 outbound-interface eth0
set nat source rule 100 translation address masquerade

set firewall options interface eth0 adjust-mss 1360

# Both NAT and MSS modification happen

Why Not Just Lower MTU?

You could lower the MTU instead of MSS clamping:

# Option A: Lower MTU on all devices
set interfaces ethernet eth0 mtu 1400
# Requires changing MTU on ALL hosts in network
# DHCP can help but not all clients respect it

# Option B: MSS clamping
set firewall options interface eth0 adjust-mss 1360
# Only affects TCP
# Transparent to endpoints
# No client changes needed

MSS clamping advantages:

Transparent to endpoints
No client configuration needed
Only affects problematic TCP path
Works even when you don’t control endpoints

MTU change advantages:

Affects all protocols (UDP, etc.)
No packet modification needed
More “correct” solution

Best practice: Use MSS clamping for TCP-heavy tunnels. Lower MTU for UDP-heavy applications or when you control all devices.

Quick Reference

Tunnel Type	Overhead	Safe MTU	Safe MSS (IPv4)
PPPoE	8	1492	1452
GRE	24	1476	1436
IPsec ESP	50-80	1400	1360
WireGuard	60	1420	1380
VXLAN	50	1450	1410
L2TP	40	1460	1420

The Lesson

MSS clamping fixes problems MTU changes cannot.

When you don’t control the endpoints, can’t guarantee ICMP reaches them, and can’t change their MTU — MSS clamping is your only option.

The router intercepts TCP handshakes and modifies the MSS value. Endpoints never know it happened. They just use smaller segments that fit through your tunnel.

Every tunnel should have MSS clamping configured. It costs nothing when not needed and saves hours of troubleshooting when it is.

The symptoms are always vague: “Large files fail, small ones work.” The fix is always the same: calculate correct MSS, clamp it, done.