Skip to content

Take input buffers in list order instead of by timestamp - #350

Merged
David-LP99 merged 3 commits into
micro-ROS:rollingfrom
Steve-Li-1998:fix/input-buffer-order
Nov 17, 2026
Merged

David-LP99 merged 3 commits into
micro-ROS:rollingfrom
Steve-Li-1998:fix/input-buffer-order

Conversation

@Steve-Li-1998

Copy link
Copy Markdown
Contributor

Description

Fixes #349.

Input buffers of one entity are ordered by timestamp. Samples that arrive within one clock tick get equal timestamps, and the strict comparison then returns the newest one. This has two effects:

  • rcl_take returns N+1 before N.
  • KEEP_LAST can overwrite a newer sample instead of the oldest.

Main changes

The allocated list is now the only source of order, with the newest item at the head.

  • rmw_uxrce_find_static_input_buffer_by_owner returns the last match instead of comparing timestamps.
  • When an entity reaches its depth, the reused buffer goes through put_memory and get_memory, which moves it to the head. It used to stay in place, so the newest sample could sit at the tail.

timestamp stays because the lifespan check needs it.

This differs from the arrival counter I proposed in #349. It needs no new field and has the same effect. Changing < to <= is not enough, because a buffer reused in place breaks the list order. With depth 2 and arrivals C, D, E sharing one timestamp, <= leaves [D, E] and takes E first.

Testing

Test Code Before After
Board jazzy at e64d79f with this change 76 swaps in 18 rounds of 30 s 0 swaps in 36 rounds of 30 s
Host rolling with this change 8 failed checks 0 failed checks
  • Board: ESP32-C6 on Zephyr 4.1.0, UDP over Wi-Fi through an access point that adds 0-200 ms of random delay in each direction. One node publishes and subscribes to one reliable topic, nominally every 20 ms. The two runs were not simultaneous, so please read this as swaps present or absent rather than as rates.
  • Host: the real memory.c and the functions from types.c, stored and taken the way on_topic and rmw_take do. It covers ties, overwrites, clock jumps, two interleaved owners and a take between stores. It uses subscription-style storage only and is not part of this PR.
  • Not tested: this exact commit on a board, the requests and replies that go through the same search, and the repository's test suite, which needs an Agent.

Backports

Could a maintainer please request backports to kilted and jazzy through Mergify, and to humble if it applies? kilted and jazzy have the same comparison, and I have not checked humble.

🤖 Generated with Claude Code

Samples that arrive within one clock tick get equal timestamps, and the
strict comparison then returned the newest one. rcl_take swapped such
samples, and KEEP_LAST could overwrite a newer sample than the oldest.
The timestamp comes from rmw_uros_epoch_nanos(), so a time sync may also
move it.

Allocated items are inserted newest first, so the oldest buffer of an
entity is its last match. Overwriting used to keep the buffer in place
and break that order, so recycle it through the pool to move it to the
head. timestamp remains for the lifespan check.

Fixes micro-ROS#349

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Hao <hao3.li@tu-dortmund.de>
@mergify

mergify Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

This pull request is merged.

@Steve-Li-1998

Copy link
Copy Markdown
Contributor Author

FYI, I used Claude Code to help with this, but I went through the change myself before opening the PR.

@David-LP99
David-LP99 self-requested a review November 17, 2026 06:47
@codecov-commenter

codecov-commenter commented Nov 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 55.64%. Comparing base (c3257d4) to head (39e1631).

Additional details and impacted files
@@             Coverage Diff             @@
##           rolling     #350      +/-   ##
===========================================
+ Coverage    54.93%   55.64%   +0.71%     
===========================================
  Files          102      103       +1     
  Lines         5918     6027     +109     
  Branches       944      943       -1     
===========================================
+ Hits          3251     3354     +103     
- Misses        2330     2333       +3     
- Partials       337      340       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@David-LP99 David-LP99 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank your for raising the topic @Steve-Li-1998,

The proposed solution looks good, please check the requested changes.

Also, I would really appreciate if you could add the host tests you mentioned to the PR as unit tests.

Once this is approached I'll request backports to the other distros.

Comment thread rmw_microxrcedds_c/src/types.c Resolved
Steve-Li-1998 and others added 2 commits November 17, 2026 11:25
Returning the buffer to the pool and getting it back only served to move
it to the head of the allocated list, which was hard to read. Add
move_to_head to the memory pool and use it instead.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: Hao <hao3.li@tu-dortmund.de>
Cover equal timestamps, overwrites, clock jumps, interleaved owners and a
take between stores, plus move_to_head on a pool of its own. The tests use
the buffer pool directly, so they need no Agent.

Without the fix, six of the seven order tests fail.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: Hao <hao3.li@tu-dortmund.de>
@Steve-Li-1998

Copy link
Copy Markdown
Contributor Author

Thanks for the review, @David-LP99.

  • Added move_to_head to memory.c and used it in place of the put_memory/get_memory pair.
  • Added the host tests as unit tests in test/test_input_buffer.cpp. They use the buffer pool directly, so they need no Agent. Without the fix, six of the seven order tests fail. There are also four tests for move_to_head itself.

I could only build this locally against jazzy, so the rolling result depends on CI, which is waiting for approval to run.

@David-LP99

Copy link
Copy Markdown
Contributor

@mergify backport kilted jazzy humble

@mergify

mergify Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

backport kilted jazzy humble

✅ Backports have been created

Details

@Steve-Li-1998

Copy link
Copy Markdown
Contributor Author

Thanks for the review and the merge, @David-LP99. While tracking down this issue I also found a few defects in the XRCE Client and Agent, each with an issue and a fix:

Would you have time to look at them as well?

David-LP99 pushed a commit that referenced this pull request Nov 19, 2026
Signed-off-by: Hao <hao3.li@tu-dortmund.de>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 122383e)
David-LP99 pushed a commit that referenced this pull request Nov 19, 2026
Signed-off-by: Hao <hao3.li@tu-dortmund.de>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 122383e)
David-LP99 pushed a commit that referenced this pull request Nov 19, 2026
Signed-off-by: Hao <hao3.li@tu-dortmund.de>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 122383e)
David-LP99 pushed a commit that referenced this pull request Nov 19, 2026
…) (#351)

Signed-off-by: Hao <hao3.li@tu-dortmund.de>
Co-authored-by: Hao <hao3.li@tu-dortmund.de>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
David-LP99 pushed a commit that referenced this pull request Nov 19, 2026
…) (#352)

Signed-off-by: Hao <hao3.li@tu-dortmund.de>
Co-authored-by: Hao <hao3.li@tu-dortmund.de>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
David-LP99 pushed a commit that referenced this pull request Nov 19, 2026
…) (#353)

Signed-off-by: Hao <hao3.li@tu-dortmund.de>
Co-authored-by: Hao <hao3.li@tu-dortmund.de>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants