Network operations.
Runbooks you can follow.

Pre-checks

1. Record vendor, exact model/SKU, serials, current and target release, boot/install mode, topology, management method and impacted services.
2. Separate Catalyst IOS/IOS XE from Nexus NX-OS. For firewalls and SD-WAN, confirm controller, HA and upgrade-order requirements.
3. Review official release notes, compatibility matrix, intermediate path, bootloader/ROMMON needs, known defects, licensing, feature deprecations, storage and memory.
4. Confirm approval, maintenance window, outage expectation, stakeholder notification, console/OOB access, escalation contacts and rollback deadline.
5. Capture baseline: versions, interfaces, routing neighbours/routes, HA/stack state, tunnels/VPNs, alarms, CPU/memory and application tests.
6. Back up configuration/state to approved storage and verify readability. Verify image checksum/signature against vendor-published value.

Execution controls

• Follow exact model/release vendor guide; never copy commands from a different platform.
• Use the vendor-prescribed order for stack/HA/cluster members and controller-managed devices.
• Monitor console and upgrade state. Do not interrupt power or repeatedly reboot when progress is unclear.
• Pause at approved go/no-go gates and timestamp every step.

Post-checks and rollback

• Confirm all members booted on target release, HA is healthy/synchronised and management works.
• Validate interfaces, LAGs, VLANs, routing, SD-WAN control/tunnels, VPNs, policies, monitoring and service-owner tests.
• Use only the documented, approved rollback. Downgrade/config reversal may not be supported; if uncertain, stop and escalate.

Evidence: change ID, versions, image verification, backup reference (no secrets), before/after outputs, tests, timestamps and sign-off.

Scope

• Record device/interface, peer port, circuit ID, location, VLAN/VRF/service, first-seen timestamp and business impact.
• Check whether one link, LAG member, redundant path or whole site is affected; review maintenance/recent changes.
• Preserve state before action. Do not bounce an interface just to test.

Read-only examples (verify release syntax)

Catalyst IOS/IOS XE: show interfaces status
Catalyst IOS/IOS XE: show interfaces <interface>
Catalyst IOS/IOS XE: show logging
Nexus NX-OS: show interface status
Nexus NX-OS: show interface <interface>
Nexus NX-OS: show logging

Checks

1. Check admin/oper state, last transition, speed/duplex, optic presence/diagnostics, description, LAG membership and errors.
2. Compare both ends: optic/cable support, fibre polarity, patch path, speed/duplex, autonegotiation and event timestamps.
3. Review LOS, unsupported optic, err-disable, port-security, STP protection, power and config-change logs.
4. Use approved remote hands for cable/optic checks. Change one variable at a time and label removed parts.

Stop/escalate: critical LAG/HA heartbeat/sole site path, unclear fault domain or provider-side evidence. Do not force speed/duplex or repeatedly clear protected ports. Validate stable link, LAG/STP/routing and service test after approved repair.

Baseline and commands

• Capture counters and timestamp; sample again after a known interval. Compare both ends and each LAG member. Do not clear counters before preserving evidence and approval.

Catalyst: show interfaces <interface>
Catalyst: show interfaces counters errors
Nexus: show interface <interface>
Nexus: show interface counters errors
Where supported: show etherchannel summary / show port-channel summary

Interpret patterns

• CRC/FCS/alignment: physical media, optics/cable, interference or negotiation; check both endpoints.
• Input errors/runts/giants: inspect exact counter definition, framing, MTU and physical health.
• Output/queue drops: investigate utilisation, microbursts, buffers, QoS queues and egress congestion.
• Discards: may be policy, VLAN/ACL, QoS or platform-specific; do not infer from label alone.

Remediate and validate

1. Compare speed/duplex, MTU, optic diagnostics, media and member health.
2. Correlate traffic, QoS/policers, queue stats, MAC movement, platform counters and recent changes.
3. Apply only evidence-based approved remediation; avoid blanket QoS/protection changes.
4. Repeat same sampling interval and confirm counters stop increasing and service symptoms resolve.

Escalate rapid growth, multi-site impact or suspected ASIC/line-card fault with evidence.

Confirm impact

• Record value, duration, baseline, threshold, device role, traffic and affected services. Separate spikes from sustained load.
• Correlate routing churn, storms, scans, logging bursts, scheduled jobs, polling, recent changes and hardware/environment alarms.
• Check management response, packet loss, HA state, neighbour changes and memory trend/free memory.

Read-only examples; validate platform/release

Catalyst IOS/IOS XE: show processes cpu sorted
Catalyst IOS/IOS XE: show processes memory
Nexus NX-OS: show processes cpu sort
Nexus NX-OS: show system resources
FortiGate: get system performance status
Palo Alto: show system resources

Response

• Identify process and CPU type; top process alone is not proof of root cause.
• Review logs, routing/ARP/MAC changes, session/inspection load, packet rates and crash/allocation errors.
• Preserve outputs; avoid repeated expensive polling. Do not kill processes, reboot or disable inspection without vendor guidance and approval.
• For suspected loop/storm, follow incident process and verify fault domain before isolating a port.

Close after resources return toward baseline, services/routes remain stable and alerts clear. Escalate sustained unexplained load, memory leak or crashes.

Baseline

• Identify design: active/passive, active/active, firewall cluster, Catalyst stack/StackWise Virtual, Nexus vPC, routing redundancy or SD-WAN pair.
• Record roles, peer reachability, sync, heartbeat/control/data links, monitored interfaces, last failover and current forwarding.

Read-only examples (verify exact platform)

FortiGate: get system ha status
Palo Alto: show high-availability state
Catalyst: show switch / platform-specific stack status
Nexus: show vpc brief
Nexus: show port-channel summary

Isolation

• Check heartbeat/peer-link, keepalive, peer routing, version compatibility, config sync, monitored links, power and hardware alarms.
• For vPC, distinguish peer-link from keepalive and check consistency; for firewalls, inspect state sync and HA events.
• Correlate upgrades, policy/interface changes, certificates/time and upstream routing.

Do not disable split-brain protection or change both peers at once. Confirm authoritative config, impact and rollback before role change. If peer state is ambiguous, stop/escalate. Validate roles, sync, routes/sessions and end-to-end tests.

Scope

• Record site, edge model/software, transport, segment/VPN, peer, affected control/tunnels, prefixes/apps and timestamp.
• Determine one tunnel vs one transport vs all controllers vs multi-site. Check provider work, templates/policies, certificates and NTP.

Cisco SD-WAN / Viptela

• Use approved vManage and exact-release CLI to inspect control connections, BFD, TLOCs, OMP routes, interface state and alarms.
• Check controller reachability/ports through actual underlay, source interface/VRF, DNS if used, NAT/firewall and system time.
• If underlay works but control is down, validate certificate chain/expiry, device identity, NTP and compatibility. Compare a healthy peer/site.

Aruba EdgeConnect / Silver Peak

• Use approved Orchestrator and release guide to inspect appliance, overlay, WAN labels, peer state, alarms and deployment status.
• Verify circuit, address/gateway/routes, DNS if needed, upstream ACL/NAT and peer. Avoid broad template pushes while diagnosing.

Recovery

1. Confirm WAN interface/address/gateway/route.
2. Test approved destinations/ports with source interface/VRF; ICMP failure alone does not prove service port failure.
3. Separate control-plane from data-plane; inspect MTU, NAT, ACLs, asymmetric routing, certificate/time and ISP events.
4. Fix evidenced fault or use approved controller workflow; do not clear all tunnels or reboot first.

Validate control, BFD/overlay, routes, path preference, app reachability and failover. Escalate with redacted evidence/provider ticket.

Capture exact flow

• Record source/destination, protocol/ports, direction, zone/interface, peer, VRF/virtual router/VDOM, time and app impact.
• Distinguish tunnel down, one-way traffic, intermittent loss and app-specific failure; check recent policy/NAT/route/certificate/provider changes.

FortiGate examples; validate VDOM/release

get system status
get router info routing-table all
diagnose vpn ike gateway list
diagnose vpn tunnel list

Palo Alto examples; validate context/release

show system info
show routing route
show vpn ike-sa
show vpn ipsec-sa
show session all filter source <source-ip> destination <destination-ip>

Trace the path

1. Check interface and route/next hop in correct VRF/virtual router/VDOM.
2. Use logs to verify actual rule, zones, source/destination, application/service and action.
3. Check NAT/translated addresses, selectors/proxy IDs, subnets, IKE version, proposals, DH groups, lifetimes, peer identity and certificate status—never expose secrets.
4. Check IKE and IPsec/child SA state and bidirectional counters.
5. Validate remote routing/policy; established SA alone does not prove application connectivity.

Make narrow approved changes based on evidence. Avoid broad allow rules, disabling inspection or changing multiple crypto parameters together. Clear/restart SA only with approval after evidence and impact review. Validate app, counters, route/policy/NAT, logs and monitoring.

Identify platform

• Record exact model, OS/version, interface/VLAN/VRF, topology, peer, affected hosts and change history.
• Catalyst IOS/IOS XE and Nexus NX-OS are separate command environments. Verify commands against official release reference.

Read-only examples

Catalyst IOS/IOS XE:
show version
show interfaces status
show vlan brief
show spanning-tree
show etherchannel summary
show ip interface brief
show ip route
show ip ospf neighbor
show ip bgp summary

Nexus NX-OS (verify feature/syntax):
show version
show interface status
show vlan brief
show spanning-tree
show port-channel summary
show vpc brief
show ip interface brief
show ip route
show ip ospf neighbors
show bgp sessions

VLAN / STP / LAG

• Confirm VLAN active; inspect access/trunk mode, allowed/native VLAN and peer consistency.
• Check STP root/role/state, topology changes, BPDU/root/loop guard and MAC moves. Do not disable protections.
• For LACP, verify both ends agree on mode, members, speed and trunk/access settings. For Nexus vPC, check peer-link, keepalive, consistency and orphan ports.

Routing

• Check interface/line protocol, addressing, VRF, route table, next hop and ARP/ND.
• For OSPF, inspect neighbour state, area, timers, authentication, MTU and network type as relevant.
• For BGP, inspect session, address family, routes under approved policy, filters, next-hop reachability and reset history.
• Do not clear sessions or change route policy as a shortcut; preserve state and assess blast radius.

Validate intended VLAN/LAG/neighbour, routes/path, no new STP/MAC/CRC events and approved end-to-end tests. Attach before/after evidence.