Upgrade Failure

Nvidia (Mellanox) SN3420: How to do an emergency image reload from the boot loader

Q: Where can I find the Nvidia (Mellanox) official documentation?

https://docs.nvidia.com/networking/ — search the product family + feature name.

By Sai Kiran Pandrala · reviewed by Sai Kiran Pandrala, Editor Last verified: 2026-05-30

⚡ At a glance

Vendor	Nvidia (Mellanox)
Operating system	Cumulus Linux / NVOS / SONiC
Category	Upgrade Failure
Skill level	Intermediate to advanced
DIY-able?	Yes with CLI access; some scenarios need Nvidia Enterprise Support + RMA.

I have run more Nvidia (Mellanox) upgrades than I can count and the only ones that hurt are the ones where I skipped image-integrity verification. Cumulus Linux / NVOS / SONiC either ships a `verify` step or expects you to checksum the file before onie-nos-install /home/cumulus/cumulus-linux-5.x.img.

On the SN3420 platform the activation phase is where you lose data-plane connectivity. Plan the change window around that window, not the full upgrade duration.

If something goes wrong, the rollback path on Cumulus Linux / NVOS / SONiC is well-trodden: but only if you saved running-config before starting. Do that now, before anything else.

What this guide covers

Do an emergency image reload from the boot loader on a Nvidia (Mellanox) SN3420 (Cumulus Linux / NVOS / SONiC).

Step-by-step

At the boot loader, configure IP, gateway, TFTP server.
Download the image.
Set the boot variable to the new image.
Reset to boot.

CLI / commands

# Boot recovery prompt: ONIE:/ #

# Verify image
nv show system

# Upgrade
onie-nos-install /home/cumulus/cumulus-linux-5.x.img

# Save / commit
nv config save

# Rollback
nv config patch (revert to last apply)

Recovery options

Boot loader recovery (ONIE:/ #)
Rollback to the previous image with nv config patch (revert to last apply)
Force failover to a known-good standby (HA platforms)

Frequently asked questions

Will this work on my specific Cumulus Linux / NVOS / SONiC version?

The procedure reflects current Cumulus Linux / NVOS / SONiC behaviour. Older releases may need minor syntax adjustments, use the CLI help (? or tab-completion) to verify.

Should I open a Nvidia Enterprise Support case immediately?

Open one if you suspect hardware failure or the symptom persists after a maintenance-window reload. Make sure your support entitlement is active first.

Where can I find the Nvidia (Mellanox) official documentation?

https://docs.nvidia.com/networking/. search the product family + feature name.

Is this procedure safe in production?

Test in a lab or maintenance window first. Capture pre-change state so you can roll back.

All Nvidia (Mellanox) fix guides → /nvidia/
All vendor guides → /vendors/

Related guides worth a look while you sort this one out:

References

Nvidia (Mellanox) support portal: https://enterprise-support.nvidia.com/
Nvidia (Mellanox) knowledge base: https://docs.nvidia.com/networking/
Nvidia (Mellanox) security advisories: https://www.nvidia.com/en-us/security/
Open a case: https://enterprise-support.nvidia.com/s/createcase

Reference material, not professional advice. Validate against your specific Cumulus Linux / NVOS / SONiC version and test in a non-production environment before applying.

Why this matters for your day-to-day

A Nvidia device that's misbehaving costs more than the fix itself: lost productivity, missed calls, security risk, even safety risk in some categories. Treating the symptom quickly with a documented procedure is cheaper than letting it persist. The steps above are written to get you back to working in under an hour where possible, and to flag clearly when escalation is the right call.

Before you start

A few things to confirm so the Nvidia device fix goes cleanly:

Latest firmware downloaded if you're going to update.
Warranty + support contract status checked, opening sealed parts may void it.
Backup of current configuration (where applicable) taken.
Spare parts on hand if you anticipate replacement.
Adequate workspace, lighting, and time: rushing causes regressions.

Verification checklist

After applying the fix on your Nvidia device, confirm:

The original symptom is no longer reproducible.
Related features (status LEDs, app sync, paired accessories) still work.
The device responds to a soft reboot without the fault returning.
Any error codes that were on display have cleared.
Documentation (your service log, the brand companion app) reflects the change.

Escalation guide

For a Nvidia device, the right escalation depends on impact:

Cosmetic / minor: log a ticket via the Nvidia app or web portal. Response 1-3 business days.
Mid-impact: phone support. Have your serial number ready.
Critical (production down, safety issue): in-person dealer / TAC visit. Bring proof of purchase.
Out of warranty: third-party repair shop with manufacturer-certified technicians.