Cisco Catalyst 9000: How to recover from a corrupted image during upgrade
By Sai Kiran Pandrala · reviewed by Sai Kiran Pandrala, Editor Last verified: 2026-05-30
| Category | Upgrade Failure |
|---|---|
| Subject | Cisco Catalyst 9000 |
| Skill level | Intermediate to advanced (CCNA / CCNP background recommended) |
| DIY-able? | Mostly yes with CLI access; some scenarios need TAC + RMA. |
What this guide covers
Upgrade interrupted; device has a corrupt image and won't boot the new release.
Resolve
- If at ROMmon: boot the prior image still on flash.
- If active is corrupt and standby still works (HA): force failover, then recover the active.
- Re-download the image from cisco.com.
- Verify MD5 before copying to the device.
- Reinstall with
install add file <image> activate commit.
CLI commands
rommon> dir bootflash:
rommon> boot bootflash:<previous-image>.bin
# After boot:
verify /md5 bootflash:<new-image>.bin
install add file bootflash:<new-image>.bin activate commit
Recovery options
- ROMmon tftpdnld for image-level recovery
- Install rollback within the auto-revert window
- Force-failover to a known-good standby supervisor
- Manual boot from a previous image on flash
Frequently asked questions
Will this work on my exact IOS-XE / ASA version?
The procedure reflects current IOS-XE 17.x and ASA 9.20 behaviour. Older trains (15.x, 9.16 ASA) may need minor syntax adjustments, use ? in the CLI.
Should I open a TAC case immediately?
Open one if you suspect hardware failure or the symptom persists after a maintenance-window reload. Make sure your SmartNet is active first.
Where can I find the Cisco official documentation?
https://www.cisco.com/c/en/us/support/all-products.html: search the product family + feature name.
Is this procedure safe in production?
Test in a lab or maintenance window first. Capture pre-change state so you can roll back.
Related guides
- All Cisco fix guides → /cisco/
- Cisco IOS error messages → /cisco/section/ios_error_messages.html
- Cisco troubleshooting by symptom → /cisco/section/troubleshoot_symptoms.html
References
- Cisco System Message Guide for IOS-XE / IOS
- Cisco Bug Search Tool: https://bst.cloudapps.cisco.com/bugsearch/
- Cisco Smart Software Manager: https://software.cisco.com
- Cisco TAC: https://mycase.cloudapps.cisco.com/case
Reference material, not professional advice. Validate against your specific IOS-XE version and test in a non-production environment before applying.
Why this matters for your day-to-day
A Cisco device that's misbehaving costs more than the fix itself: lost productivity, missed calls, security risk, even safety risk in some categories. Treating the symptom quickly with a documented procedure is cheaper than letting it persist. The steps above are written to get you back to working in under an hour where possible, and to flag clearly when escalation is the right call.
Isolate
A few things to confirm so the Cisco device fix goes cleanly:
- Latest firmware downloaded if you're going to update.
- Warranty + support contract status checked, opening sealed parts may void it.
- Backup of current configuration (where applicable) taken.
- Spare parts on hand if you anticipate replacement.
- Adequate workspace, lighting, and time. rushing causes regressions.
Validate
On a Cisco device, the test is rarely "reboot and see". Use this list:
- Active reproduction: trigger the original failure path on purpose.
- Indirect reproduction: do an activity that would expose the same subsystem.
- Status indicator review: every LED / display / app status should be green.
- 24-hour soak: leave the device under normal load overnight; check the next morning.
- Telemetry check: review the device or app's diagnostic log for new error entries.
When to call Cisco support instead
Escalate if:
- The same symptom returns within 24 hours of a clean fix.
- You see physical damage (burn marks, swollen battery, cracked PCB).
- The device is in warranty and a hardware replacement is the cheaper outcome.
- Repair requires specialised tools you don't own (alignment jigs, calibration software).
- Following the official path keeps the warranty intact, which matters more than the time spent.
More frequently asked questions
Is it safe to apply during business hours?
If the device is in production use, apply during a scheduled maintenance window. Most procedures need 2-15 minutes of downtime. Capture pre-change state so you can roll back if needed.
Can I roll this back if something breaks?
Yes for software-level changes (firmware rollback, config rollback). Hardware changes are usually one-way. Always back up settings before starting.
Will this void my warranty?
Applying official firmware updates and following the user manual will not affect warranty. Opening sealed components, jumping safety circuits, or using third-party parts can void warranty in most jurisdictions.
Should I update firmware first or last?
Update firmware first if a release note specifically mentions your symptom. Otherwise, finish the troubleshooting flow first, then update; that way you can isolate whether the update or the underlying fix solved it.
What if the fix returns after a reboot?
Persistent fault returns mean either: a hardware fault (escalate), a configuration that's being overwritten by a sync source (check cloud profiles), or a regression in a recent firmware update (rollback).
Field notes from real incidents on Upgrade Failure
When I work on Cisco Catalyst 9000: How to recover from a corrupted image during upgrade the rhythm I lean on is the one I have built over years of these tickets, not a stack of generic advice. I never run a software upgrade on a live Catalyst stack without an out-of-band console session; the in-band session drops at the worst possible moment. The newer Cisco IOS-XE traceability tools (show platform hardware fed) are massively underused; they answer questions the old CLI cannot.
Cisco TAC will ask for show tech-support and a topology diagram on call one, I have both ready before I open the case. Most catalyst stack issues I have triaged were power-budget related, not software: the show power detail output answers it in 5 seconds.
Tools I actually reach for
For Cisco Catalyst 9000: How to recover from a corrupted image during upgrade on Upgrade Failure the cheapest signal I can land usually comes from a known order of operations, not a kitchen-sink approach. I start with show tech-support (capture for TAC) because it is the lowest-friction way to confirm the failure is real and reproducible. If that returns ambiguous data, I escalate to show interfaces counters errors, show logging last 200, and finally to ping vrf <vrf> <target> only when the cheaper tools cannot reach the layer the failure lives in. That ordering matches the failure surfaces I have actually seen on Upgrade Failure units over the last few years, not an abstract taxonomy. The cheap signals gate the expensive ones so the investigation does not balloon into a multi-hour exercise.
Verification I run before I close the ticket
Before I mark Cisco Catalyst 9000: How to recover from a corrupted image during upgrade resolved on a Upgrade Failure unit, the verification loop below is what I actually run. Each step proves a different layer is green, and the order matters - the cheap checks gate the more expensive ones so I never burn an hour on a deep test that a shallow one would have failed in seconds.
show ip route <prefix> # confirm best path post-changeIf that one comes back clean, move to the next check. If it does not, stop and dig in there before layering more verification on top of a red signal.
show interfaces <int> | include errors|drops|CRCIf that one comes back clean, move to the next check. If it does not, stop and dig in there before layering more verification on top of a red signal.
show logging | include %LINK|%LINEPROTO|%BGP|%OSPFOnly when every line above runs clean do I close the ticket and update the runbook with the timestamps. A green verification that nobody can reproduce is not a fix, it is luck waiting to regress.
Where I check first when the docs disagree
When two sources contradict each other on a Upgrade Failure detail, the disambiguation order I lean on is stable across products and across years. developer.cisco.com for NSO / model-driven APIs is where I start for the ground-truth view. cisco.com/c/en/us/support, official command references is where I start for the ground-truth view. Cisco TAC case knowledge base is where I start for the ground-truth view. cisco.com/c/en/us/td/docs/ios-xml for IOS XR is where I start for the ground-truth view. Random blog posts and reseller wikis are signal, not ground truth, and I treat them as such until the references above either confirm or contradict the claim. The cost of trusting an unauthoritative source on Cisco Catalyst 9000: How to recover from a corrupted image during upgrade is rarely worth the time it saved.
Pitfalls I have walked into on this exact path
The shortcuts that look smart on Cisco Catalyst 9000: How to recover from a corrupted image during upgrade have a habit of biting back. The pitfalls below are the ones I have personally walked into on a Upgrade Failure unit, not things I read about. Cisco TAC will ask for show tech-support and a topology diagram on call one. I have both ready before I open the case. Most catalyst stack issues I have triaged were power-budget related, not software, the show power detail output answers it in 5 seconds. Cisco bug search tool is the cheapest sanity check before a config change: search the symptom, sort by affected releases, decide. When in doubt I revert to the slower path that the manual prescribes - the time I save by skipping it is always smaller than the time I spend cleaning up afterwards.
What I tell the next on-call
When I hand Cisco Catalyst 9000: How to recover from a corrupted image during upgrade off to the next person on rotation, the three lines I leave in the runbook are these. First, the symptom signature on Upgrade Failure - not a paraphrase, the exact string that surfaces in logs or on the screen. Second, the diagnostic that gave the highest signal in the least time. Third, the exact verification command whose green output justified closing the ticket. That trio is what turns a one-off fix into a runbook entry the next engineer can use without paging me at three in the morning.
I also add a one-line note on the cost of getting this wrong. For Cisco Catalyst 9000: How to recover from a corrupted image during upgrade on a Upgrade Failure unit, the cost is rarely the replacement part or the patch itself. It is the downtime, the second site visit, and the trust deficit you spend with whoever owns the asset when the fix does not hold. That framing keeps the next on-call from choosing the cheap-looking shortcut that ends up costing the most in elapsed hours and goodwill.
Related fixes
Related guides worth a look while you sort this one out:
- Cisco Catalyst 3850: How to recover from a corrupted image during upgrade
- Cisco ASA: How to recover from a corrupted image during upgrade
- Cisco ASR 1000: How to recover from a corrupted image during upgrade
- Cisco ISR 4000: How to recover from a corrupted image during upgrade
- Cisco Catalyst 9000: How to rollback to the previous image after a failed upgrade
- Cisco Catalyst 9000: How to verify image integrity with md5sum
People also ask
Will this work on my exact IOS-XE / ASA version?
The procedure reflects current IOS-XE 17.x and ASA 9.20 behaviour. Older trains (15.x, 9.16 ASA) may need minor syntax adjustments, use `?` in the CLI.
Should I open a TAC case immediately?
Open one if you suspect hardware failure or the symptom persists after a maintenance-window reload. Make sure your SmartNet is active first.
Where can I find the Cisco official documentation?
https://www.cisco.com/c/en/us/support/all-products.html. search the product family + feature name.
Is this procedure safe in production?
Test in a lab or maintenance window first. Capture pre-change state so you can roll back.