how to debug DDS discovery failure across Docker bridge network on ROS 2 Jazzy
| Controller | ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2. 2026 |
|---|---|
| Category | Industrial Error Codes |
| Guide type | Procedure |
| Skill level | Beginner to intermediate field service tech |
| Time | 5 - 30 minutes including verification |
how to debug DDS discovery failure across Docker bridge network on ROS 2 Jazzy on ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 comes up often enough on the shop floor and in the OEM service bulletins that there is a stable recovery pattern. My first step on any ros fault is to read the alarm history before touching the reset button - last week the cell controller hit this exact alarm during a tool change and the recovery path is mostly known, the OEM manual just buries it under three layers of cross-referenced parameter tables.
What how to debug dds discovery failure across docker bridge network on ros 2 jazzy actually involves on ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026
On ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 when this lands in my queue the tools I lean on first are Wireshark with RTPS dissector for DDS discovery debugging, MoveIt 2 demo.launch.py with rviz2 MotionPlanning panel, Foxglove Studio with rosbridge for live topic and parameter inspection. Each of these surfaces a different layer of the fault - keep at least the first one in your fault-history notebook so the next time this happens you do not start cold.
For verification on ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026, the methods that survive contact with a real second-shift production workload are verify MoveIt planning scene via ros2 service call /get_planning_scene moveit_msgs/srv/GetPlanningScene and validate publisher and subscriber QoS profiles match via ros2 topic info /topic --verbose. Anything less than that and you are shipping on vibes.
Authoritative sources for ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 that I cross-reference before committing to a fix: discourse.ros.org, fast-dds.docs.eprosima.com, moveit.picknik.ai. OEM marketing brochures and trade-press writeups are signal, not ground truth.
The rest of this page is the structured fix path. Start with diagnose, then remediation, then the automation options so you do not have to do this by hand the next time it surfaces. Verify and safety sections at the end are the discipline that keeps the fix from regressing the next time you open the cabinet.
Diagnose first, fix second
Sixth: pin down the timing and reliability envelope on the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 cell under real working conditions. Run a long-duration sanity test by executing the failing program 10 times over 15 minutes, logging the timestamp and the result (cycle complete / alarm code / which axis or station faulted) per attempt to a notes file. Watch for the breakpoint where the cycle success rate dips below 80 percent - that is your real signal that something is wrong, not the one-off alarm that prompted the callout. If you are on a marginal supply (low ambient temp, brownout, dirty 3-phase, contaminated coolant), run the same test on a known-good supply or a sister cell before assuming the controller is the problem. Capture the breakpoint in your personal notes next to the firmware version, the parameter set, and the controller serial number - the next time this happens to a teammate, the notes are gold.
Seventh: run the dedicated diagnostic option for whichever subsystem the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 alarm points at. Drive suspected? Force a servo discharge and re-energize from the drive panel, then check the drive status LEDs for the green ready signal and the last-fault timestamp. Encoder suspected? Power down fully (lockout-tagout), check the encoder battery voltage at the back of the controller, re-home the axis on power-up. Cable suspected? Pin-check the encoder cable continuity end-to-end with a meter (EtherCAT or Profinet drop = use a cable tester, look for an LED link light at both ends). Each of these surfaces config that the controller silently inherits from a previous session, and 90 percent of "this used to work yesterday" reports trace to a stale parameter or a vibrated-loose connector. Capture the result of each step in your notes alongside the timestamp so you do not redo the discovery the next time.
Eighth: diff the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 setup against its last known good state. Ask the obvious question - what changed in the 72 hours before the fault started? Did the controller take a firmware update overnight (check the About panel for the firmware revision vs the previous version you wrote down in your notes)? Did you swap a drive, a motor, an encoder cable, or a fieldbus drop? Did you change a tool offset, a work offset, a vision job, or a recipe? Did the maintenance team push a new PM checklist, swap a lube reservoir, or change a coolant concentration? Use the in-controller audit trail (Fanuc PARAM history, KUKA KRC log, Cognex In-Sight job version) to anchor "before vs after" so you are not guessing. Cross-check the OEM service bulletin and the OEM community forum for the exact firmware revision - if a regression hit a batch of cells in the same week, the community catches it before the official bulletin admits it. Record the suspect ranking, then disprove suspects one at a time with the cheapest test first (parameter restore before drive swap, encoder battery check before encoder swap).
Field notes from real ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 callouts
For Robotics jobs I keep a battered field notebook of "what bit me on ROS 2 Platform Error Codes and how I cleared it", writing it down the first time has saved me a dozen overnight returns. My fastest sanity check after touching ROS 2 Platform Error Codes firmware is `tail rosout via ros2 run rqt_console rqt_console during fault reproduction`; if that comes back inside spec, I close the ticket and head to the next bay. In Robotics work the cost of guessing is measured in scrap and downtime, so I read the ROS 2 Platform Error Codes release notes before I touch a setpoint, every time, no exceptions.
Tools I actually reach for
For most ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 faults I start with Cyclone DDS ddsperf and Fast DDS fastdds discovery server CLI, fall back to MoveIt 2 demo.launch.py with rviz2 MotionPlanning panel, rqt_graph and rqt_console for runtime error stream view, Wireshark with RTPS dissector for DDS discovery debugging, ros2 topic echo and ros2 topic info -v for QoS introspection when Cyclone DDS ddsperf and Fast DDS fastdds discovery server CLI cannot surface the answer, and keep Foxglove Studio with rosbridge for live topic and parameter inspection handy for the cases where neither answers. That ordering is not academic - it matches the layers of the fault as they tend to surface, so the cheapest signal lands first and the heavier tooling only comes out when the simpler answer does not hold up. My muscle-memory shortcut for this is to run the first tool while the alarm screen is still open, not after I have already cycled controller power.
Verification I run before I call it fixed
Before I mark a ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 fault resolved, the verification loop below is what I actually run. Each step proves a different layer is green, and the order matters - the cheaper checks gate the more expensive ones.
trace executor latency with ros2 trace and analyze via tracetools_analysisIf that one comes back clean, move to the next check. If it does not, stop and dig in there before layering more verification on top of a red signal.
verify MoveIt planning scene via ros2 service call /get_planning_scene moveit_msgs/srv/GetPlanningSceneIf that one comes back clean, move to the next check. If it does not, stop and dig in there before layering more verification on top of a red signal.
check lifecycle node state with ros2 lifecycle get /node_name and force transitionIf that one comes back clean, move to the next check. If it does not, stop and dig in there before layering more verification on top of a red signal.
tail rosout via ros2 run rqt_console rqt_console during fault reproductionOnly when every line above runs clean do I close the loop and update my fault-history notebook with the timestamps.
Where I check first when the docs disagree
When two sources contradict each other on a ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 detail, the disambiguation order I lean on is stable. I usually check discourse.ros.org for the ground-truth view on this part of ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026. I usually check github.com for the ground-truth view on this part of ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026. I usually check moveit.picknik.ai for the ground-truth view on this part of ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026. OEM marketing brochures and trade-press writeups are signal, not ground truth, and I treat them as such until the references above either confirm or contradict the claim.
Solution-focused remediation path
If the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 controller is slow, faulting on cached errors, or HMI-locked, work the cache and parameter stack in order. Cycle controller power per the OEM lockout procedure (master disconnect off, wait 60 seconds for bus discharge, master disconnect on), reboot, and re-home the axes. Clear the local fault history (most controllers expose this under Maintenance -> Clear faults, or Setup -> Reset alarms). Re-load the saved parameter set with the OEM utility (Fanuc PARAM RESTORE, KUKA archive restore) to bypass any local parameter drift. Always capture timing before the cycle: time how long the failing cycle takes three times, write it down, then repeat after the parameter restore so the delta is provable in your notes. Decision point: managed-cell issues go through your controls engineering team for a cell-wide config push; standalone-cell issues go through the OEM diagnostic utility before you escalate to the OEM hotline.
For any ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 fault that smells like drive overcurrent or motor overload, walk the principle of least surprise chain in order. Confirm the workpiece mass and the tool inertia have not changed since the last known good cycle - "my program stopped finishing" reports often trace to a heavier blank or a longer tool that pushed the duty cycle past the drive thermal envelope. Confirm the feedrate and acceleration overrides at the HMI - many overcurrent alarms trace to an operator bumping rapid-feed to 150 percent for a "quick run." Check the coolant flow at the drive heatsink and the ambient temperature of the cabinet (a clogged filter or a failed cabinet fan raises ambient enough to trip SRVO-068 thermal alarms). Decision point: if the workpiece, feedrate, and cooling are all correct and the drive still faults overcurrent, swap the drive with a known-good sister unit to isolate drive vs motor vs cable, and capture the encoder feedback before and after the swap.
Start by sorting the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 fault into one of three buckets, because roughly 80% of cases fall here. Bucket one is electrical / drive: instantaneous overcurrent, sustained overload, drive overheat, bus undervoltage, or a phase-loss event. Bucket two is mechanical / motion: encoder battery low, absolute position lost, over-travel, hardstop hit, or a vibrated-loose cable. Bucket three is recipe / parameter / I/O: the program calls a tool that is not loaded, the work offset is wrong, a DI is mapped to a disconnected sensor, or a vision job version has drifted. Pick the bucket first, then act. Before you act, capture a baseline photo of the alarm screen plus the controller hour-meter so you can prove whether the fix actually moved the needle. Decision point: if the alarm is intermittent and the cell is under an OEM service contract, open the OEM hotline first - OEM phone support beats hours of speculative debugging on cost and on liability if the alarm recurs and trips a safety-related shutdown.
Automate this fix so you do not do it twice
Monitor + alert via ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 OEM diagnostic reports, alarm history, and plant dashboard ingestion
For the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026, the most useful long-running telemetry is the OEM diagnostic reports + alarm history shipped to a plant dashboard (Grafana with a CSV source, Ignition with a tag history, the fab MES OEE per SEMI E10, a Notion database via the API) and graphed on a single view. Pair that with synthetic monitoring (a small script that triggers the failing cycle or runs the failing test sequence every 5 minutes from at least two cells) so a fleet-level regression lights up before teammates report it. Subscribe the on-call inbox or a private Teams channel to the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 OEM service bulletin (Atom/RSS or vendor portal webhook) plus the OEM service-status handle so an open bulletin self-correlates with the synthetic failures.
# Tiny synthetic monitor - hit the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 controller health endpoint every 5 minutes
while true; do curl -s -o /dev/null -w "%{http_code} %{time_total} $(date -Iseconds)\n" \ -H "Authorization: Bearer $TOKEN" \ https://controller.plant.local/api/v1/me \ >> /var/log/ros-synth.log sleep 300
doneFleet maintenance-license + OEM token rotation via OEM admin
Rotating a maintenance access token on one ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 controller by hand is fine; rotating across a fleet of cells is how you end up with twelve different tokens, four expired ones, and an unknown blast radius across the plant. Drive rotation through the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 OEM admin SDK or REST under a service account with the rotation scope only, store the new token in a plant-wide password manager (1Password, Bitwarden, OEM secrets manager) with versioning enabled, and roll the consumer scripts one cell at a time with a health check between each. Pin the API version explicitly during rotation so a coincident OEM firmware push does not look like a rotation failure.
# Rotate the controller maintenance token (regenerate via the OEM utility, capture in 1Password)
op item create --vault Plant --category "API Credential" \ --title "ros controller token 2026-06-01" \ password="$NEW_CONTROLLER_TOKEN" notes="Rotated $(date -Iseconds)"
# Capture the old token as deprecated so cutover is reversible
op item create --vault Plant --category "API Credential" \ --title "ros controller token OLD 2026-06-01" \ password="$OLD_CONTROLLER_TOKEN" notes="Old token marked deprecated"Automate ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 parameter + I/O mapping snapshots via OEM utility or API
On the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026, regular parameter and I/O snapshots catch silent parameter drift, recipe edits, and stale safety-PLC permissions well before the cell starts faulting in prod. Pair OEM health checks (the OEM diagnostic SDK, the controller users API, the fieldbus device listing) with a license-validity check so both OEM-side and cell-side issues land in one folder. Run the scheduled task on a control-plane logger PC (a hardened IPC at the cell, a GitHub Actions runner against the cell-controller VPN, a small Linux box at the line) under a tightly scoped service account that mirrors the maintenance role.
# List cell operator roster + safety-PLC roles
curl -H "Authorization: Bearer $CONTROLLER_TOKEN" \ https://controller.plant.local/api/v1/operators \ > ros-operators.json
# List active fieldbus drops + their last-link-up timestamp
curl -H "Authorization: Bearer $CONTROLLER_TOKEN" \ https://controller.plant.local/api/v1/fieldbus_drops \ > ros-fieldbus.json
# Validate the maintenance license token itself
curl -H "Authorization: Bearer $CONTROLLER_TOKEN" \ https://controller.plant.local/api/v1/me \ > ros-me.json
Common pitfalls and what to watch for
Read-only validation before any write is the single step most ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 fixes skip, and it is the step that lets you roll back when a fix backfires. Photograph every existing parameter page (the axis parameters, the spindle parameters, the safety parameters, the I/O mapping, the recipe library), capture the failing photo in a notes entry, export the relevant log to CSV if the controller supports it (the OEM diagnostic tool fault-history export, the PMC log download), and photograph the HMI alarm history showing the failing window before any change. On ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 cells with multiple operating modes (manual jog, MDI, auto) record the firmware revision, the parameter state, and the I/O mapping in each before toggling anything, because a "fix" pushed only to manual mode is a known regression vector when auto mode has a different interlock set.
The mirror-image mistake is confusing a cell-level symptom with an OEM fault on ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026. A persistent SRVO-023 is often a workpiece-level change pushed by the production team rather than a ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 bug. A "program not loading" can be a renamed program rather than a deleted one. A "trigger not firing" is frequently a vibrated-loose sensor cable or a contaminated lens rather than an OEM-side regression.
Verify the fix worked
- Reproduce the original faulting cycle against ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 on the same cell AND a sister cell with the same recipe. If the alarm or fault code still surfaces on any cell, you have not fixed it.
- Watch for 24 to 48 hours via the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 controller alarm history + the fieldbus log + your fault-history notebook. Cached fault states and stale fieldbus link state mask slow-burn drift and intermittent fieldbus issues.
- Smoke-test under realistic load: replay the cycle against a test workpiece for at least 30 minutes at your normal production feedrate, log success / alarm and the timestamp per attempt to a notes file.
- Capture the new state in a fault-history notebook entry so the next time this happens you do not rediscover it. Note firmware revision + parameter set + I/O mapping + failing photo + verbatim alarm string + fix applied. Push to a plant-wide maintenance wiki if your plant uses one.
- If the fix involved a maintenance-token rotation or a parameter set change, commit the new token to your password manager and photograph the parameter dump for archival.
Safety, rollback, blast radius
- Test in a ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 maintenance mode or on a sister cell first before any change that touches the production cell. Snapshot the firmware revision, the parameter set, the I/O mapping, and the safety-PLC permissions before changing anything.
- Apply the principle of least surprise when granting teach-pendant access or safety-PLC permissions. Review the operator roster against the people who actually need access - extra teach pendants are extra blast radius.
- Use idempotent cycles where the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2, 2026 controller supports it (the OEM cycle-id de-dupe, external id keys on MES records) so a re-run cycle does not double-count parts or duplicate scrap records.
- Know your rollback path. Firmware rollback is a one-line OEM utility load; a maintenance-token rotation is reversible if you kept the old token in the password manager during cutover; a parameter set change is reversible only if you saved the previous archive.
- For cell-wide or plant-wide changes, line up a maintenance window with production scheduling before pushing through the OEM utility.
FAQ
References
- OEM service manual for ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2. 2026 (official service bulletins, alarm code reference, safety case)
- Controls-community forums (r/PLC, r/Robotics, r/CNC, r/Fanuc, r/KUKA, r/Cognex, r/labview, OEM community)
- In-controller diagnostic help and the ROS 2 Platform Error Codes, rclcpp/rclpy, DDS QoS, Lifecycle Nodes, MoveIt 2: 2026 firmware release notes
- OEM service-status portals and OEM hotline post-mortem reports
Related fixes
Related guides worth a look while you sort this one out:
- how to fix ROS 2 DDS QoS mismatch publisher reliability reliable vs subscriber best_effort
- how to debug AMR ROS bridge Nav2 BT navigator GoalAborted on dynamic obstacle
- how to clear Cyclone DDS dropped fragments warning under high lidar bandwidth on Humble
- how to clear ROS 2 RCLError transition is not registered on lifecycle_msgs configure
- how to debug MoveIt 2 OMPL planner FAILED to find motion plan after collision object add
- how to debug rclcpp callback group deadlock with MutuallyExclusiveCallbackGroup