“Can you ping it?” Yes. “Then why isn’t it working?” Because ping tests ICMP, not your application. Because one successful ping doesn’t show the packet loss happening every 30 seconds. Because ping doesn’t show which hop is the problem.
Ping is a smoke test, not a diagnostic. Real troubleshooting needs tools that show the path, measure loss over time, and identify exactly where problems occur.
Ping alone is never enough.
Understanding the Tools
| Tool | What it shows | When to use |
|---|---|---|
| ping | Basic reachability | Quick test |
| traceroute | Path to destination | Find route |
| mtr | Path + statistics over time | Find where loss occurs |
| tracepath | Path + MTU discovery | Find MTU issues |
MTR Deep Dive
MTR combines traceroute with continuous ping statistics.
Basic MTR
# From VyOS operational modemtr 8.8.8.8
# Output:# Host Loss% Snt Last Avg Best Wrst StDev# 1. gateway.local 0.0% 100 1.2 1.5 0.8 3.2 0.5# 2. isp-router.net 0.0% 100 8.3 9.1 7.2 15.3 1.8# 3. core-router.isp 0.5% 100 12.1 13.2 11.0 45.2 5.2# 4. ???# 5. google-peer.net 0.0% 100 15.3 16.1 14.2 22.1 1.2# 6. 8.8.8.8 0.0% 100 14.8 15.5 14.0 21.3 1.1MTR Options
# Specify countmtr -c 100 8.8.8.8
# Report mode (non-interactive)mtr -r 8.8.8.8
# Wide report (show full hostnames)mtr -rw 8.8.8.8
# Use TCP instead of ICMPmtr -T -P 443 8.8.8.8
# Use UDPmtr -u 8.8.8.8
# Set packet sizemtr -s 1400 8.8.8.8
# No DNS resolution (faster)mtr -n 8.8.8.8
# Show AS numbersmtr -z 8.8.8.8Interpreting MTR Output
Host Loss% Snt Last Avg Best Wrst StDev1. 192.168.1.1 0.0% 100 1.2 1.5 0.8 3.2 0.52. 10.0.0.1 15.0% 100 8.3 9.1 7.2 15.3 1.8 ← Problem here?3. 172.16.0.1 15.0% 100 12.1 13.2 11.0 45.2 5.2 ← Or here?4. 8.8.8.8 0.0% 100 14.8 15.5 14.0 21.3 1.1 ← Destination OKKey insight: Loss at hop 2 that continues to hop 3 but clears by destination means hop 2 is rate-limiting ICMP replies, not dropping traffic. Real loss would persist to destination.
Reading Loss Patterns
# Pattern 1: Real lossHop 3: 15% lossHop 4: 15% lossHop 5: 15% loss ← Loss persists to destination = real problem at hop 3
# Pattern 2: ICMP rate limitingHop 3: 15% lossHop 4: 15% lossHop 5: 0% loss ← Clears at destination = hop 3 rate-limits ICMP, not real loss
# Pattern 3: Asymmetric routingHop 3: high latency spikeHop 4: normalHop 5: normal ← Return path different, not a problemMTR for Different Protocols
# ICMP might be filtered, try TCPmtr -T -P 80 example.com
# Test actual service portmtr -T -P 443 example.commtr -T -P 22 example.com
# UDP servicesmtr -u -P 53 8.8.8.8Traceroute Variants
Standard Traceroute
# ICMP traceroutetraceroute 8.8.8.8
# UDP traceroute (default on Linux)traceroute -U 8.8.8.8
# TCP traceroute (bypasses some filters)traceroute -T -p 443 8.8.8.8
# Don't resolve hostnamestraceroute -n 8.8.8.8Traceroute Options
# Set max hopstraceroute -m 30 8.8.8.8
# Set packet sizetraceroute 8.8.8.8 1400
# Wait time per probetraceroute -w 2 8.8.8.8
# Probes per hoptraceroute -q 5 8.8.8.8
# Source interfacetraceroute -i eth0 8.8.8.8Interpreting Traceroute
# Normal output1 192.168.1.1 (192.168.1.1) 1.234 ms 1.123 ms 1.345 ms2 10.0.0.1 (10.0.0.1) 8.234 ms 8.123 ms 8.345 ms3 * * * ← Hop doesn't respond4 8.8.8.8 (8.8.8.8) 15.234 ms 15.123 ms 15.345 ms
# Stars (*) don't always mean problem# Many routers don't respond to traceroute probes# Final destination matters mostTracepath and PMTUD
Path MTU Discovery finds the maximum packet size that can traverse a path without fragmentation.
Using Tracepath
# Tracepath discovers MTU along pathtracepath 8.8.8.8
# Output includes MTU at each hop# 1: 192.168.1.1 1.234ms pmtu 1500# 2: 10.0.0.1 8.234ms pmtu 1500# 3: tunnel-endpoint 12.234ms pmtu 1400 ← MTU drops here# 4: 8.8.8.8 15.234ms reachedManual PMTUD
# Find MTU using ping with DF flag# Start with 1500, decrease until works
# Linux/VyOSping -M do -s 1472 8.8.8.8 # 1472 + 28 = 1500
# If fails, decrease sizeping -M do -s 1400 8.8.8.8ping -M do -s 1372 8.8.8.8 # For tunnels with 1400 MTUCommon MTU Values
| Scenario | MTU |
|---|---|
| Ethernet | 1500 |
| PPPoE | 1492 |
| GRE tunnel | 1476 |
| IPsec (AES) | ~1400 |
| WireGuard | 1420 |
| VXLAN | 1450 |
PMTUD Problems
# Symptoms of MTU problems:# - Small packets work, large fail# - SSH works, SCP hangs# - Web pages partially load# - TLS handshake fails
# Diagnose:tracepath problematic-host.com
# Fix on VyOS:# Option 1: Lower interface MTUset interfaces ethernet eth0 mtu 1400
# Option 2: MSS clamping (better for VPN)set firewall options interface eth0 adjust-mss 1360Diagnosing Common Problems
Problem 1: Intermittent Packet Loss
# Run MTR for extended periodmtr -c 1000 destination.com
# Look for:# - Consistent loss at specific hop# - Loss that varies with time# - Loss only during certain hours
# If loss at hop N continues to destination:# Problem is at hop N or beforeProblem 2: High Latency Spikes
# Check MTR StDev column# High StDev = inconsistent latency
# Possible causes:# - Congestion (check time of day)# - Buffer bloat (test with different packet sizes)# - Routing changes (check Wrst column)Problem 3: Path Changes
# Run traceroute multiple timesfor i in {1..10}; do traceroute -n 8.8.8.8; sleep 60; done
# Compare paths# Different paths = load balancing or instabilityProblem 4: Blackhole
# Traceroute stops at specific hop, no destination1 192.168.1.12 10.0.0.13 172.16.0.14 * * *5 * * *
# Possible causes:# - Firewall blocking# - Routing problem (no return path)# - MTU blackhole (try smaller packets)
# Test with different methods:traceroute -T -p 80 destination # TCPtraceroute -I destination # ICMPVyOS Specific Diagnostics
Check Local Routing
# Verify route to destinationshow ip route 8.8.8.8
# Check for multiple pathsshow ip route 8.8.8.8 longer-prefixes
# BGP-specific path infoshow ip bgp 8.8.8.8Check Interface Stats
# Look for errorsshow interfaces ethernet eth0
# Key metrics:# - RX/TX errors# - RX/TX drops# - Collisions (shouldn't happen on modern networks)Check Firewall Drops
# Enable logging on drop rulesset firewall ipv4 name WAN-IN rule 999 log enable
# Check logsshow log | grep DROP
# Might reveal blocked trafficScripting Diagnostics
Continuous Monitoring Script
#!/bin/bashDESTINATION=$1LOG_FILE="/var/log/path-monitor.log"
while true; do echo "=== $(date) ===" >> $LOG_FILE mtr -r -c 10 $DESTINATION >> $LOG_FILE sleep 300doneAlert on High Loss
#!/bin/bashDESTINATION="8.8.8.8"THRESHOLD=5
LOSS=$(mtr -r -c 100 $DESTINATION | tail -1 | awk '{print $3}' | sed 's/%//')
if [ $(echo "$LOSS > $THRESHOLD" | bc) -eq 1 ]; then echo "High loss detected: ${LOSS}%" | mail -s "Network Alert" admin@example.comfiBest Practices
1. Always Test Bidirectionally
# Problem might be in return path# Test from both ends when possible
# From VyOS:mtr remote-host
# From remote host:mtr vyos-router2. Test the Actual Service
# ICMP might work when TCP doesn'tmtr -T -P 443 website.commtr -T -P 22 server.com3. Collect Data Over Time
# One-time test might miss intermittent issues# Run extended tests during problem periodsmtr -c 500 problematic-host.com4. Document Baseline
# Know what "normal" looks like# Run MTR when everything works# Compare during problemsThe Lesson
Ping alone is never enough.
Ping tells you: “Something responded to ICMP.”
MTR tells you:
- Which hops are on the path
- Where packet loss occurs
- Whether loss is real or ICMP rate-limiting
- Latency variations and patterns
When troubleshooting:
- Start with MTR, not ping
- Run long enough to catch patterns
- Test with relevant protocol (TCP/UDP)
- Test bidirectionally
- Compare to known baseline
The path is usually the problem, not the endpoint. MTR shows you the path.