SNMP polling vs streaming telemetry (gNMI) — when to use which

Both mechanisms answer the same question — what is this device doing right now — and they answer it in opposite directions. SNMP has the collector ask; gNMI has the device tell. Choosing badly is expensive in a very concrete way: either you drown an access layer in poll traffic and still miss the event you were hired to catch, or you build a gRPC pipeline for hardware that ships no usable YANG model at all.

This is the reasoning we apply when we design a monitoring layer, and the trade-offs are worth understanding before anyone opens a purchase order.

What SNMP actually does on the wire#

SNMP is request/response over UDP port 161. The collector asks, the agent answers, and between the two the only thing the device sends unprompted is a notification. Four operations matter in practice:

• GET — one OID, one value, one round trip.
• GETNEXT — walk to the lexicographically next OID. One round trip per value, which is why naive walks are slow.
• GETBULK (v2c and later) — ask for up to max-repetitions successive values inside a single response PDU.
• TRAP / INFORM — the agent pushes a notification to UDP 162 without being asked: linkDown, bgpBackwardTransition and whatever else the MIB defines. Traps are fire-and-forget UDP that you cannot re-request; informs are acknowledged. Both are usually rate-limited on the box, and neither carries a counter series.

A realistic interface walk looks like this:

# 64-bit counters live in ifXTable; ifTable's 32-bit counters wrap far too fast
snmpbulkwalk -v2c -c "$COMMUNITY" -Cr25 10.0.0.1 IF-MIB::ifXTable

# a single high-speed counter, useful when you are debugging one port
snmpget -v3 -l authPriv -u nocpoll -a SHA -A "$AUTHPASS" \
        -x AES -X "$PRIVPASS" 10.0.0.1 IF-MIB::ifHCInOctets.436

Two details bite people repeatedly. First, ifIndex is not stable: on many platforms it is reassigned when a line card is reseated or a subinterface is deleted, so anything keyed on the raw index silently starts graphing a different port. Key on ifName or ifAlias and resolve the index on every discovery run. Second, ifTable counters are 32-bit; at 10 Gbit/s a 32-bit octet counter wraps in a handful of seconds, so ifXTable and its ifHC* variants are the only sane choice above Fast Ethernet.

Security is the other half. SNMPv2c is a community string in cleartext — acceptable inside a management VRF that nothing else can reach, indefensible anywhere else. SNMPv3 with authPriv costs a little agent CPU and removes the argument entirely.

Where polling stops scaling#

Polling has a hard structural limit: the poll cycle must finish before the next one starts. If you poll 400 devices, each with 48 ports and a dozen system OIDs, on a 60-second interval, the collector has 60 seconds to complete every request, retry every timeout and write every value. When the cycle overruns, you do not get an alert — you get gaps, and the gaps look exactly like an outage in the graph.

Three consequences follow:

• Granularity has a floor. Sub-minute polling is possible on a small fleet and unrealistic on a large one. Anything shorter-lived than the interval — a microburst filling an egress queue, a BGP session that flaps and reconverges — simply does not exist in your data.
• Timestamps are the collector's, not the device's. Two interfaces "sampled at 12:00:30" were actually read seconds apart. Rate calculations across devices inherit that jitter.
• State changes are visible only if the device traps them. A neighbour that goes down and comes back inside one interval leaves nothing in the polled data but a counter discontinuity. A trap covers the transition, but you get one unacknowledged UDP datagram at the moment the control plane is busiest — which is why the transitions that matter belong on ON_CHANGE.

What gNMI changes#

gNMI is a gRPC service carried over HTTP/2 and TLS, typically on a vendor-chosen port such as 57400 or 6030. Instead of asking repeatedly, the collector opens one long-lived Subscribe RPC and the device pushes updates for the YANG paths it asked for.

The practical differences:

• The device timestamps the data. Every update carries the device's own nanosecond timestamp, so correlation across a fleet is meaningful.
• One connection, many paths. HTTP/2 multiplexing means adding a subscription does not add a TCP session.
• Paths, not OIDs. /interfaces/interface[name=Ethernet1]/state/counters/in-octets is self-describing; 1.3.6.1.2.1.31.1.1.1.6.436 is not.
• On-change is possible. For state — BGP session state, interface oper-status, optics thresholds — the device reports the transition instead of you inferring it from two samples.

Subscription modes and what they cost#

The Subscribe RPC has three top-level modes — ONCE, POLL and STREAM. The useful one in practice is STREAM, and within it each path gets its own mode:

ON_CHANGE support is where implementations diverge most. Some platforms accept the subscription and then quietly stream at an internal sample rate; others reject the path outright. Verify per platform and per software version — never assume it from a datasheet.

A working gnmic subscription#

gnmic is the fastest way to prove a device does what its documentation claims:

# capabilities first: models, encodings, versions
gnmic -a 10.0.0.1:57400 -u telemetry -p "$PASS" --skip-verify capabilities

# a sampled counter stream, JSON-IETF encoded
gnmic -a 10.0.0.1:57400 -u telemetry -p "$PASS" --skip-verify \
      -e json_ietf subscribe \
      --path "/interfaces/interface/state/counters" \
      --stream-mode sample --sample-interval 10s

# session state, on change only
gnmic -a 10.0.0.1:57400 -u telemetry -p "$PASS" --skip-verify \
      subscribe --path "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/session-state" \
      --stream-mode on-change

--skip-verify belongs in a lab and nowhere else; in production the collector validates the device certificate, exactly as it would for any other TLS client.

For a permanent pipeline the same tool runs as a daemon with a config file, exporting to Prometheus or NATS or Kafka, from where the time series database consumes it. The shape we build for this is: gnmic collectors near the devices, a message bus for buffering, one writer into the TSDB, Grafana on top. The bus is what lets you restart the database without losing the window.

Which signals belong where#

The honest answer is that most networks need both, and the split falls out naturally:

Keep on SNMP: environmental and facility gear (UPS, PDU, CRAC units, door sensors), optical transceiver readings via ENTITY-SENSOR-MIB on platforms with no telemetry stack, older access switches, appliances that expose a private MIB and nothing else, and anything where a five-minute sample is genuinely sufficient.

Move to gNMI: BGP and IGP neighbour state, queue depth and drop counters, high-speed interface counters on the core and edge, LSP state, control-plane CPU under load, and anything where you need the transition rather than the average.

The deciding question is not "which is more modern" but "what is the shortest event I must be able to see". If the answer is longer than your poll cycle, SNMP is fine and cheaper to operate.

Migrating without a monitoring gap#

The migration that works is boring and overlapping:

1. Stand the gNMI pipeline up beside the existing poller. Nothing is switched off.
2. Normalise names first. Decide that interface_in_octets_total means one thing regardless of source, and map both SNMP and gNMI into it. Dashboards and alert rules then survive the cutover untouched.
3. Run both for at least one full change cycle and diff them. Where they disagree, the usual causes are counter reset semantics after a card reload, a SAMPLE interval that does not divide evenly into the poll interval, and mismatched units.
4. Move alerts to the new source one rule at a time, keeping the old rule as a silenced fallback until you trust the new one.
5. Only then remove OIDs from the poller — and only the ones the new pipeline actually replaced.

Failure modes we plan for#

• Cardinality explosion. Streaming a full interface subtree from a large chassis produces far more series than a poller ever did. Filter at the collector, not in the database.
• Dropped connections. A Subscribe RPC is long-lived, so gRPC keepalives and a defined reconnect and re-sync behaviour matter more than they do for stateless polling.
• Control-plane load. Telemetry is not free. A subscription with a one-second sample interval across every leaf of an aggressive subtree can affect the same CPU that runs your routing protocols.
• Silent schema drift. A software upgrade renames or moves a YANG path and the series simply stops. Alert on absence of data, not only on thresholds.
• Half-migrated alerting. The worst outcome is an alert rule that reads a metric neither pipeline is writing any more. This is why step 5 above comes last.

If you are already collecting flow data, the natural next step is correlating these counters with per-prefix volumes — that is covered in Building a NetFlow/IPFIX pipeline with Akvorado and ClickHouse, and the traffic decisions that follow from it in BGP traffic engineering with flow data.