Upgrade Failure

Cisco Catalyst 9000: How to perform in-service software upgrade (ISSU)

By Sai Kiran Pandrala · reviewed by Sai Kiran Pandrala, Editor Last verified: 2026-05-30

⚡ At a glance
CategoryUpgrade Failure
SubjectCisco Catalyst 9000
Skill levelIntermediate to advanced (CCNA / CCNP background recommended)
DIY-able?Mostly yes with CLI access; some scenarios need TAC + RMA.

What this guide covers

Real-world context. Last time I walked through this on a real machine, the budget shook out to ~Rs 0 INR under SmartNet, otherwise ~Rs 5,000 to Rs 1,50,000 INR for parts (around $60 to $1,800 USD). Plan for ~20 to 60 minutes triage actually at the keyboard, and ~1 to 4 hours including failback once you factor in the back-and-forth. Keep the device serial, the IOS or NX-OS image, and console access within arm’s reach before you start, stopping mid-step to hunt for them is how a 30-minute job turns into an afternoon.

Zero-downtime upgrade on HA-capable platforms.

Step-by-step

  1. Verify ISSU compatibility. not all releases support ISSU.
  2. show issu state should return clean state before starting.
  3. install add file <new-image> on the active supervisor.
  4. install activate ISSU, triggers controlled failover.
  5. Standby loads new image, becomes active.
  6. Active then loads new image, becomes new standby.
  7. install commit to lock in.

CLI commands

show issu state detail
install add file bootflash:<new-image>.bin
install activate ISSU
show issu state
install commit

Recovery options

Frequently asked questions

Will this work on my exact IOS-XE / ASA version?

The procedure reflects current IOS-XE 17.x and ASA 9.20 behaviour. Older trains (15.x, 9.16 ASA) may need minor syntax adjustments: use ? in the CLI.

Should I open a TAC case immediately?

Open one if you suspect hardware failure or the symptom persists after a maintenance-window reload. Make sure your SmartNet is active first.

Where can I find the Cisco official documentation?

https://www.cisco.com/c/en/us/support/all-products.html, search the product family + feature name.

Is this procedure safe in production?

Test in a lab or maintenance window first. Capture pre-change state so you can roll back.

References


Reference material, not professional advice. Validate against your specific IOS-XE version and test in a non-production environment before applying.

Why this matters for your day-to-day

A Cisco device that's misbehaving costs more than the fix itself: lost productivity, missed calls, security risk, even safety risk in some categories. Treating the symptom quickly with a documented procedure is cheaper than letting it persist. The steps above are written to get you back to working in under an hour where possible, and to flag clearly when escalation is the right call.

Safety + preconditions

Before any work on a Cisco device:

How to confirm it's actually fixed

On a Cisco device, the test is rarely "reboot and see". Use this list:

Escalation guide

For a Cisco device, the right escalation depends on impact:

More frequently asked questions

Is it safe to apply during business hours?

If the device is in production use, apply during a scheduled maintenance window. Most procedures need 2-15 minutes of downtime. Capture pre-change state so you can roll back if needed.

Can I roll this back if something breaks?

Yes for software-level changes (firmware rollback, config rollback). Hardware changes are usually one-way. Always back up settings before starting.

Why is this happening on a brand-new unit?

Out-of-box defects do occur. If you've owned the device under 30 days and the symptom persists after a factory reset, escalate to the seller for replacement under DOA terms before opening a manufacturer support case.

Does this affect other devices on my network?

Generally no. The procedure is local to this device. Network-side changes (firmware updates that affect TLS, SMB, or routing) are flagged explicitly in the steps.

What if the fix returns after a reboot?

Persistent fault returns mean either: a hardware fault (escalate), a configuration that's being overwritten by a sync source (check cloud profiles), or a regression in a recent firmware update (rollback).

Field notes from real incidents on Upgrade Failure

When I work on Cisco Catalyst 9000: How to perform in-service software upgrade (ISSU) the rhythm I lean on is the one I have built over years of these tickets. Most catalyst stack issues I have triaged were power-budget related, not software, the show power detail output answers it in 5 seconds. Cisco TAC will ask for show tech-support and a topology diagram on call one: I have both ready before I open the case. I never run a software upgrade on a live Catalyst stack without an out-of-band console session; the in-band session drops at the worst possible moment.

Tools I actually reach for

For Cisco Catalyst 9000: How to perform in-service software upgrade (ISSU) on Upgrade Failure the cheapest signal I can land usually comes from a known order of operations, not a kitchen-sink approach. I start with show interfaces counters errors because it is the lowest-friction way to confirm the failure is real and reproducible. If that returns ambiguous data, I escalate to show platform hardware capacity, packet capture on the ingress interface (TAC will ask for it), and finally to traceroute vrf <vrf> <target> only when the cheaper tools cannot reach the layer the failure lives in. That ordering matches the failure surfaces I have actually seen on Upgrade Failure units over the last few years, not an abstract taxonomy. The cheap signals gate the expensive ones so the investigation does not balloon into a multi-hour exercise.

Verification I run before I close the ticket

Before I mark Cisco Catalyst 9000: How to perform in-service software upgrade (ISSU) resolved on a Upgrade Failure unit, the verification loop below is what I actually run. Each step proves a different layer is green, and the order matters - the cheap checks gate the more expensive ones so I never burn an hour on a deep test that a shallow one would have failed in seconds.

show spanning-tree summary  # confirm topology stability

If that one comes back clean, move to the next check. If it does not, stop and dig in there before layering more verification on top of a red signal.

show ip route <prefix>  # confirm best path post-change

If that one comes back clean, move to the next check. If it does not, stop and dig in there before layering more verification on top of a red signal.

show bgp summary  # confirm session state after route changes

If that one comes back clean, move to the next check. If it does not, stop and dig in there before layering more verification on top of a red signal.

show logging | include %LINK|%LINEPROTO|%BGP|%OSPF

Only when every line above runs clean do I close the ticket and update the runbook with the timestamps. A green verification that nobody can reproduce is not a fix, it is luck waiting to regress.

Where I check first when the docs disagree

When two sources contradict each other on a Upgrade Failure detail, the disambiguation order I lean on is stable across products and across years. cisco.com/c/en/us/support, official command references is where I start for the ground-truth view. developer.cisco.com for NSO / model-driven APIs is where I start for the ground-truth view. Cisco TAC case knowledge base is where I start for the ground-truth view. cisco.com/c/en/us/td/docs/ios-xml for IOS XR is where I start for the ground-truth view. Random blog posts and reseller wikis are signal, not ground truth, and I treat them as such until the references above either confirm or contradict the claim. The cost of trusting an unauthoritative source on Cisco Catalyst 9000: How to perform in-service software upgrade (ISSU) is rarely worth the time it saved.

Pitfalls I have walked into on this exact path

The shortcuts that look smart on Cisco Catalyst 9000: How to perform in-service software upgrade (ISSU) have a habit of biting back. The pitfalls below are the ones I have personally walked into on a Upgrade Failure unit, not things I read about. The newer Cisco IOS-XE traceability tools (show platform hardware fed) are massively underused; they answer questions the old CLI cannot. Most catalyst stack issues I have triaged were power-budget related, not software. the show power detail output answers it in 5 seconds. When in doubt I revert to the slower path that the manual prescribes - the time I save by skipping it is always smaller than the time I spend cleaning up afterwards.

What I tell the next on-call

When I hand Cisco Catalyst 9000: How to perform in-service software upgrade (ISSU) off to the next person on rotation, the three lines I leave in the runbook are these. First, the symptom signature on Upgrade Failure - not a paraphrase, the exact string that surfaces in logs or on the screen. Second, the diagnostic that gave the highest signal in the least time. Third, the exact verification command whose green output justified closing the ticket. That trio is what turns a one-off fix into a runbook entry the next engineer can use without paging me at three in the morning.

I also add a one-line note on the cost of getting this wrong. For Cisco Catalyst 9000: How to perform in-service software upgrade (ISSU) on a Upgrade Failure unit, the cost is rarely the replacement part or the patch itself. It is the downtime, the second site visit, and the trust deficit you spend with whoever owns the asset when the fix does not hold. That framing keeps the next on-call from choosing the cheap-looking shortcut that ends up costing the most in elapsed hours and goodwill.

Related guides worth a look while you sort this one out:

People also ask

Will this work on my exact IOS-XE / ASA version?

The procedure reflects current IOS-XE 17.x and ASA 9.20 behaviour. Older trains (15.x, 9.16 ASA) may need minor syntax adjustments, use `?` in the CLI.

Should I open a TAC case immediately?

Open one if you suspect hardware failure or the symptom persists after a maintenance-window reload. Make sure your SmartNet is active first.

Where can I find the Cisco official documentation?

https://www.cisco.com/c/en/us/support/all-products.html: search the product family + feature name.

Is this procedure safe in production?

Test in a lab or maintenance window first. Capture pre-change state so you can roll back.