Hardware Failure

Nvidia (Mellanox) SN2010 stack member missing: Diagnose & Fix

By Sai Kiran Pandrala · reviewed by Sai Kiran Pandrala, Editor Last verified: 2026-05-30

⚡ At a glance
VendorNvidia (Mellanox)
Operating systemCumulus Linux / NVOS / SONiC
CategoryHardware Failure
Skill levelIntermediate to advanced
DIY-able?Yes with CLI access; some scenarios need Nvidia Enterprise Support + RMA.

When a Nvidia (Mellanox) SN2010 starts misbehaving, the temptation is to reboot and hope. Resist it. Capture `nv show system` and `nv show platform environment` first; that 30-second buffer is the difference between a real root cause and another reload at 3am next week.

Cumulus Linux / NVOS / SONiC has a habit of logging the actual failing component into the system log seconds before the LED transitions. Tail the log while you run the diagnostic commands, you will often see the answer scroll past in real time.

Below is the exact sequence I run on customer gear. Steps are ordered cheapest-first so you exit early if it really is just a loose cable.

What this guide covers

Diagnose and recover from stack member missing on a Nvidia (Mellanox) SN2010.

Step-by-step

  1. Run the stack / chassis status command to see member states.
  2. Inspect the stack cables. re-seat both ends.
  3. Try replacing one stack cable at a time to identify a bad cable.
  4. Power-cycle the affected member if cables are good.
  5. If the member still doesn't rejoin, RMA it.

CLI / commands

# Verify hardware state
nv show system
nv show platform inventory
nv show platform environment

# Collect for Nvidia Enterprise Support
cl-support (Cumulus) / show techsupport (SONiC)

When to RMA

Frequently asked questions

Will this work on my specific Cumulus Linux / NVOS / SONiC version?

The procedure reflects current Cumulus Linux / NVOS / SONiC behaviour. Older releases may need minor syntax adjustments, use the CLI help (? or tab-completion) to verify.

Should I open a Nvidia Enterprise Support case immediately?

Open one if you suspect hardware failure or the symptom persists after a maintenance-window reload. Make sure your support entitlement is active first.

Where can I find the Nvidia (Mellanox) official documentation?

https://docs.nvidia.com/networking/: search the product family + feature name.

Is this procedure safe in production?

Test in a lab or maintenance window first. Capture pre-change state so you can roll back.

Related guides worth a look while you sort this one out:

References


Reference material, not professional advice. Validate against your specific Cumulus Linux / NVOS / SONiC version and test in a non-production environment before applying.

What changed recently?

Fault diagnosis on a Nvidia device goes faster when you map the symptom to a recent change:

The answer narrows the root cause to a manageable subset.

Safety + preconditions

Before any work on a Nvidia device:

Quick verification

Before you walk away from a Nvidia device fix, run through:

1. Reproduce the original trigger. does the issue reappear? 2. Check the device's status / health screen for any new alerts. 3. Confirm paired devices (app, hub, controller) reconnected. 4. Save / commit any configuration changes per the device's normal workflow. 5. Note the change in your maintenance log with date + firmware version.

When to call Nvidia support instead

Escalate if:

More frequently asked questions

What if the fix returns after a reboot?

Persistent fault returns mean either: a hardware fault (escalate), a configuration that's being overwritten by a sync source (check cloud profiles), or a regression in a recent firmware update (rollback).

Can I roll this back if something breaks?

Yes for software-level changes (firmware rollback, config rollback). Hardware changes are usually one-way. Always back up settings before starting.

Will this void my warranty?

Applying official firmware updates and following the user manual will not affect warranty. Opening sealed components, jumping safety circuits, or using third-party parts can void warranty in most jurisdictions.

Should I update firmware first or last?

Update firmware first if a release note specifically mentions your symptom. Otherwise, finish the troubleshooting flow first, then update; that way you can isolate whether the update or the underlying fix solved it.

Is it safe to apply during business hours?

If the device is in production use, apply during a scheduled maintenance window. Most procedures need 2-15 minutes of downtime. Capture pre-change state so you can roll back if needed.