The RouterOS Troubleshooting Toolkit: Torch, Sniffer, Profile, and a Method

Most network debugging fails before it starts, because the person doing it opens the config instead of looking at the traffic. Configs tell you what you intended. Packets tell you what’s happening. When the two disagree — and during an outage they always do — the packets win.

RouterOS is unusually well equipped here. A stock router ships with a live flow analyzer, a full packet capture engine that streams to Wireshark, a per-process CPU profiler, and enough counters to reconstruct almost any failure. Most people use maybe one of these, badly, under pressure. This article is the toolkit plus the order in which to reach for it.

One rule up front: form a hypothesis before you touch a tool. “The link is slow” is not a hypothesis. “Upload is fine, download stalls, so I expect drops on the WAN-facing queue” is — and it tells you exactly which counter to read.

Torch: Live Flows Per Interface

Torch is the first tool out of the bag for “what is this interface actually carrying right now.” It shows live traffic through one interface, classified by whatever fields you ask for:

# Everything crossing ether1, broken down by source, destination, port
/tool torch interface=ether1 src-address=0.0.0.0/0 dst-address=0.0.0.0/0 \
    port=any

# Who is eating the uplink -- aggregate by source only
/tool torch interface=ether1 src-address=10.10.0.0/16 dst-address=0.0.0.0/0

# Just DNS traffic
/tool torch interface=ether1 port=53 protocol=udp

The fields you specify are the aggregation keys: give it port=any and it splits flows per port; leave port out and traffic aggregates across ports. Narrow addresses (src-address=10.10.0.0/16) act as filters. IPv6 works with src-address6/dst-address6, and you can classify by VLAN ID and DSCP too.

Two caveats that bite people. Torch sees traffic before firewall filtering — a flow visible in torch may still be dropped by your own rules one step later. And on switches, hardware-offloaded bridge traffic never reaches the CPU, so torch can’t see it; that “idle” interface may be forwarding 9 Gbps in silicon.

Classic torch win: internet “feels slow,” torch on the WAN interface, sort by rate, and there’s one host pushing 800 Mbps of cloud backup at 2 PM. Thirty seconds, no config read.

Packet Sniffer: When You Need the Actual Packets

Torch tells you that a flow exists; the sniffer tells you what’s inside it. For quick looks, quick mode prints live to the terminal:

/tool sniffer quick ip-protocol=icmp
/tool sniffer quick interface=ether1 port=179

That second one — watching BGP session establishment live — has solved more peering tickets for me than any amount of /routing bgp session print.

For real analysis you want Wireshark, and RouterOS has a first-class path for that: TZSP streaming. The router encapsulates every captured packet and fires it at your workstation in real time:

/tool sniffer set filter-interface=ether1 filter-ip-address=203.0.113.10/32 \
    streaming-enabled=yes streaming-server=10.10.30.5 filter-stream=yes
/tool sniffer start
# ... reproduce the problem ...
/tool sniffer stop

On the workstation, Wireshark decodes TZSP natively — capture on your own NIC with filter udp port 37008 and you see the router’s traffic as if you had a tap. filter-stream=yes is not optional: it excludes the streaming traffic itself from capture, and without it you build a feedback loop that snowballs until something falls over.

For capture-to-file instead:

/tool sniffer set file-name=capture.pcap file-limit=5000 \
    filter-interface=ether1 filter-port=443
/tool sniffer start
/tool sniffer stop
/file print where name~"capture"
# download via Winbox/FTP/SFTP and open in Wireshark

Multiple filters combine through filter-operator-between-entries=and|or — check it before concluding your filter “doesn’t work.” And after the capture, the summary views are underrated:

/tool sniffer protocol print
/tool sniffer host print

host print sorted by rate is a poor man’s top-talkers even when you forgot to set any filter.

Tool Profile: What Is the CPU Doing

When a router is slow but the links aren’t full, profile the CPU:

/tool profile duration=10

# Per-core breakdown -- one pinned core with idle cores elsewhere
# is its own diagnosis (single-flow load can't spread across cores)
/tool profile cpu=all duration=10

Reading the classifier names is the skill:

• networking high — raw packet forwarding load. On a small box, possibly just too many pps; check for fasttrack being defeated (mangle rules, queues, and connection-untracked traffic all bypass it).
• firewall high — rule evaluation cost. Long linear chains, content= matchers, or regexp-based layer-7 rules. Restructure with jump chains keyed on interface lists.
• encrypting high — VPN crypto without hardware offload, or with a cipher the chip can’t accelerate.
• queuing high — queue tree/simple queue overhead; all queued traffic is CPU-touched.
• console / winbox / management high — someone left a torch or a monitor running in another session. It happens more than anyone admits.
• unclassified — not everything is labeled; treat it as forwarding-adjacent.

idle near zero on any single core is the number that matters. RouterOS distributes flows, not packets — one elephant flow lives on one core, and that core’s ceiling is your ceiling.

Interface Stats and Queue Drops

Errors and drops are the difference between “slow” and “broken,” and they’re free to check:

/interface print stats
/interface ethernet print stats
# per-interface detail: look for rx-fcs-error, rx-overflow,
# rx-drop / tx-drop, and pause frame counters
/interface ethernet print stats where name=ether1

# live rate on an interface
/interface monitor-traffic ether1

Rules of thumb: rx-fcs-error climbing means layer 1 — bad cable, dirty fiber, dying SFP. rx-overflow means the port is receiving faster than the box can drain it. For SFP modules, check the optics themselves:

/interface ethernet monitor sfp-sfpplus1
# sfp-rx-power / sfp-tx-power in dBm, temperature, DDM alarms

A link that flaps daily at the same hour and shows -19 dBm RX is not a config problem.

If shaping is involved, read the queues — drops there are invisible in interface error counters:

/queue simple print stats
/queue tree print stats
# "dropped" climbing on one queue = that class is hitting its ceiling

Conntrack Inspection

The connection table is a live record of every tracked flow, and it’s queryable:

/ip firewall connection tracking print
# total-entries vs max-entries: headroom check

# What is this host doing?
/ip firewall connection print where src-address~"10.10.20.31"

# Counting by type
/ip firewall connection print count-only where protocol=udp
/ip firewall connection print count-only where tcp-state=syn-received

A pile of syn-received entries is an inbound SYN flood; thousands of UDP entries from one internal host is malware or a misbehaving app; total-entries at max is why “the internet stopped” — new flows are being refused. I covered the DDoS side of conntrack exhaustion in the previous article in this series.

Ping and Traceroute, Properly

The default ping answers almost nothing interesting, because it sources from the egress interface and follows the main table. The options are the tool:

# Test the return path, not just the forward path:
# source from LAN address -- now the far end must route back to your LAN
/ping 198.51.100.1 src-address=10.10.20.1 count=5

# Path MTU discovery: find the largest un-fragmentable payload
/ping 198.51.100.1 size=1472 do-not-fragment count=3

# Is my DSCP marking surviving the path?
/ping 198.51.100.1 dscp=46 count=5

# Ping inside a VRF
/ping 203.0.113.9 routing-table=CUSTOMER-A

# Layer 2 reachability when IP is broken
/ping 6C:3B:6B:48:0E:8B interface=ether2

The src-address trick deserves emphasis: “router can ping, clients can’t” is almost always a return-route or NAT problem, and one sourced ping proves it in five seconds. Traceroute takes the same options (src-address, routing-table, dscp) and I’ll add /tool traceroute address=198.51.100.1 use-dns=yes count=1 as the fast one-shot form.

The Method

Tools without sequence produce thrash. Mine, after enough 3 AM calls to have opinions:

1. Define the failure precisely. Which source, which destination, which direction, since when? “VLAN 30 can’t reach the DNS server since the maintenance window” is workable. “Network is down” is not.
2. Establish what still works. Same VLAN, other destination? Other VLAN, same destination? Each answer halves the search space.
3. Check the boring things first. /interface print stats for down links and errors, /log print for the timeline, /system resource print for CPU and memory. Ninety seconds, catches a third of everything.
4. Follow one packet. Torch on ingress — does the traffic arrive? Torch on egress — does it leave? Firewall counters in between (/ip firewall filter print stats) — did a rule eat it? /ip route check 198.51.100.10 — does the router even agree with you about the next hop?
5. Capture when reasoning stalls. Sniffer with TZSP, watch the handshake fail in Wireshark. SYN with no SYN-ACK is a forward/filter problem; SYN-ACK with no final ACK is asymmetric return; RST from an unexpected address is NAT weirdness.
6. Change one thing. Then re-test the original failure — not a different, easier test that happens to pass.

The discipline of writing down the hypothesis before running the tool feels bureaucratic until the incident review, when “we torched ether1 to confirm traffic arrived, it did, so we moved to the filter counters” reconstructs itself from your terminal history.

Closing Thoughts

Learn these tools on a healthy network. Torch your uplink on a normal Tuesday so you know what baseline looks like; run /tool profile when nothing is wrong so you recognize when something is. The worst time to learn sniffer filter syntax is during the outage — the second worst is explaining afterward why it took four hours to find a bad patch cable that rx-fcs-error had been counting all along. Next in this series: RoMON, for the day the problem you’re troubleshooting is that you can no longer reach the router at all.