MTR, Tracepath, and PMTUD: Diagnosing Path Problems

“Can you ping it?” Yes. “Then why isn’t it working?” Because ping tests ICMP, not your application. Because one successful ping doesn’t show the packet loss happening every 30 seconds. Because ping doesn’t show which hop is the problem.

Ping is a smoke test, not a diagnostic. Real troubleshooting needs tools that show the path, measure loss over time, and identify exactly where problems occur.

Ping alone is never enough.

Understanding the Tools

ToolWhat it showsWhen to use
pingBasic reachabilityQuick test
traceroutePath to destinationFind route
mtrPath + statistics over timeFind where loss occurs
tracepathPath + MTU discoveryFind MTU issues

MTR Deep Dive

MTR combines traceroute with continuous ping statistics.

Basic MTR

Terminal window
# From VyOS operational mode
mtr 8.8.8.8
# Output:
# Host Loss% Snt Last Avg Best Wrst StDev
# 1. gateway.local 0.0% 100 1.2 1.5 0.8 3.2 0.5
# 2. isp-router.net 0.0% 100 8.3 9.1 7.2 15.3 1.8
# 3. core-router.isp 0.5% 100 12.1 13.2 11.0 45.2 5.2
# 4. ???
# 5. google-peer.net 0.0% 100 15.3 16.1 14.2 22.1 1.2
# 6. 8.8.8.8 0.0% 100 14.8 15.5 14.0 21.3 1.1

MTR Options

Terminal window
# Specify count
mtr -c 100 8.8.8.8
# Report mode (non-interactive)
mtr -r 8.8.8.8
# Wide report (show full hostnames)
mtr -rw 8.8.8.8
# Use TCP instead of ICMP
mtr -T -P 443 8.8.8.8
# Use UDP
mtr -u 8.8.8.8
# Set packet size
mtr -s 1400 8.8.8.8
# No DNS resolution (faster)
mtr -n 8.8.8.8
# Show AS numbers
mtr -z 8.8.8.8

Interpreting MTR Output

Host Loss% Snt Last Avg Best Wrst StDev
1. 192.168.1.1 0.0% 100 1.2 1.5 0.8 3.2 0.5
2. 10.0.0.1 15.0% 100 8.3 9.1 7.2 15.3 1.8 ← Problem here?
3. 172.16.0.1 15.0% 100 12.1 13.2 11.0 45.2 5.2 ← Or here?
4. 8.8.8.8 0.0% 100 14.8 15.5 14.0 21.3 1.1 ← Destination OK

Key insight: Loss at hop 2 that continues to hop 3 but clears by destination means hop 2 is rate-limiting ICMP replies, not dropping traffic. Real loss would persist to destination.

Reading Loss Patterns

Terminal window
# Pattern 1: Real loss
Hop 3: 15% loss
Hop 4: 15% loss
Hop 5: 15% loss Loss persists to destination = real problem at hop 3
# Pattern 2: ICMP rate limiting
Hop 3: 15% loss
Hop 4: 15% loss
Hop 5: 0% loss Clears at destination = hop 3 rate-limits ICMP, not real loss
# Pattern 3: Asymmetric routing
Hop 3: high latency spike
Hop 4: normal
Hop 5: normal Return path different, not a problem

MTR for Different Protocols

Terminal window
# ICMP might be filtered, try TCP
mtr -T -P 80 example.com
# Test actual service port
mtr -T -P 443 example.com
mtr -T -P 22 example.com
# UDP services
mtr -u -P 53 8.8.8.8

Traceroute Variants

Standard Traceroute

Terminal window
# ICMP traceroute
traceroute 8.8.8.8
# UDP traceroute (default on Linux)
traceroute -U 8.8.8.8
# TCP traceroute (bypasses some filters)
traceroute -T -p 443 8.8.8.8
# Don't resolve hostnames
traceroute -n 8.8.8.8

Traceroute Options

Terminal window
# Set max hops
traceroute -m 30 8.8.8.8
# Set packet size
traceroute 8.8.8.8 1400
# Wait time per probe
traceroute -w 2 8.8.8.8
# Probes per hop
traceroute -q 5 8.8.8.8
# Source interface
traceroute -i eth0 8.8.8.8

Interpreting Traceroute

Terminal window
# Normal output
1 192.168.1.1 (192.168.1.1) 1.234 ms 1.123 ms 1.345 ms
2 10.0.0.1 (10.0.0.1) 8.234 ms 8.123 ms 8.345 ms
3 * * * Hop doesn't respond
4 8.8.8.8 (8.8.8.8) 15.234 ms 15.123 ms 15.345 ms
# Stars (*) don't always mean problem
# Many routers don't respond to traceroute probes
# Final destination matters most

Tracepath and PMTUD

Path MTU Discovery finds the maximum packet size that can traverse a path without fragmentation.

Using Tracepath

Terminal window
# Tracepath discovers MTU along path
tracepath 8.8.8.8
# Output includes MTU at each hop
# 1: 192.168.1.1 1.234ms pmtu 1500
# 2: 10.0.0.1 8.234ms pmtu 1500
# 3: tunnel-endpoint 12.234ms pmtu 1400 ← MTU drops here
# 4: 8.8.8.8 15.234ms reached

Manual PMTUD

Terminal window
# Find MTU using ping with DF flag
# Start with 1500, decrease until works
# Linux/VyOS
ping -M do -s 1472 8.8.8.8 # 1472 + 28 = 1500
# If fails, decrease size
ping -M do -s 1400 8.8.8.8
ping -M do -s 1372 8.8.8.8 # For tunnels with 1400 MTU

Common MTU Values

ScenarioMTU
Ethernet1500
PPPoE1492
GRE tunnel1476
IPsec (AES)~1400
WireGuard1420
VXLAN1450

PMTUD Problems

Terminal window
# Symptoms of MTU problems:
# - Small packets work, large fail
# - SSH works, SCP hangs
# - Web pages partially load
# - TLS handshake fails
# Diagnose:
tracepath problematic-host.com
# Fix on VyOS:
# Option 1: Lower interface MTU
set interfaces ethernet eth0 mtu 1400
# Option 2: MSS clamping (better for VPN)
set firewall options interface eth0 adjust-mss 1360

Diagnosing Common Problems

Problem 1: Intermittent Packet Loss

Terminal window
# Run MTR for extended period
mtr -c 1000 destination.com
# Look for:
# - Consistent loss at specific hop
# - Loss that varies with time
# - Loss only during certain hours
# If loss at hop N continues to destination:
# Problem is at hop N or before

Problem 2: High Latency Spikes

Terminal window
# Check MTR StDev column
# High StDev = inconsistent latency
# Possible causes:
# - Congestion (check time of day)
# - Buffer bloat (test with different packet sizes)
# - Routing changes (check Wrst column)

Problem 3: Path Changes

Terminal window
# Run traceroute multiple times
for i in {1..10}; do traceroute -n 8.8.8.8; sleep 60; done
# Compare paths
# Different paths = load balancing or instability

Problem 4: Blackhole

Terminal window
# Traceroute stops at specific hop, no destination
1 192.168.1.1
2 10.0.0.1
3 172.16.0.1
4 * * *
5 * * *
# Possible causes:
# - Firewall blocking
# - Routing problem (no return path)
# - MTU blackhole (try smaller packets)
# Test with different methods:
traceroute -T -p 80 destination # TCP
traceroute -I destination # ICMP

VyOS Specific Diagnostics

Check Local Routing

Terminal window
# Verify route to destination
show ip route 8.8.8.8
# Check for multiple paths
show ip route 8.8.8.8 longer-prefixes
# BGP-specific path info
show ip bgp 8.8.8.8

Check Interface Stats

Terminal window
# Look for errors
show interfaces ethernet eth0
# Key metrics:
# - RX/TX errors
# - RX/TX drops
# - Collisions (shouldn't happen on modern networks)

Check Firewall Drops

Terminal window
# Enable logging on drop rules
set firewall ipv4 name WAN-IN rule 999 log enable
# Check logs
show log | grep DROP
# Might reveal blocked traffic

Scripting Diagnostics

Continuous Monitoring Script

/config/scripts/path-monitor.sh
#!/bin/bash
DESTINATION=$1
LOG_FILE="/var/log/path-monitor.log"
while true; do
echo "=== $(date) ===" >> $LOG_FILE
mtr -r -c 10 $DESTINATION >> $LOG_FILE
sleep 300
done

Alert on High Loss

/config/scripts/check-loss.sh
#!/bin/bash
DESTINATION="8.8.8.8"
THRESHOLD=5
LOSS=$(mtr -r -c 100 $DESTINATION | tail -1 | awk '{print $3}' | sed 's/%//')
if [ $(echo "$LOSS > $THRESHOLD" | bc) -eq 1 ]; then
echo "High loss detected: ${LOSS}%" | mail -s "Network Alert" admin@example.com
fi

Best Practices

1. Always Test Bidirectionally

Terminal window
# Problem might be in return path
# Test from both ends when possible
# From VyOS:
mtr remote-host
# From remote host:
mtr vyos-router

2. Test the Actual Service

Terminal window
# ICMP might work when TCP doesn't
mtr -T -P 443 website.com
mtr -T -P 22 server.com

3. Collect Data Over Time

Terminal window
# One-time test might miss intermittent issues
# Run extended tests during problem periods
mtr -c 500 problematic-host.com

4. Document Baseline

Terminal window
# Know what "normal" looks like
# Run MTR when everything works
# Compare during problems

The Lesson

Ping alone is never enough.

Ping tells you: “Something responded to ICMP.”

MTR tells you:

  • Which hops are on the path
  • Where packet loss occurs
  • Whether loss is real or ICMP rate-limiting
  • Latency variations and patterns

When troubleshooting:

  1. Start with MTR, not ping
  2. Run long enough to catch patterns
  3. Test with relevant protocol (TCP/UDP)
  4. Test bidirectionally
  5. Compare to known baseline

The path is usually the problem, not the endpoint. MTR shows you the path.