Skip to content

perf(semaphore): avoid allocating queue storage for the first waiter - #415

Closed
rokadepiyush49-rgb wants to merge 1 commit into
apache:mainfrom
rokadepiyush49-rgb:perf/400-semaphore-first-waiter
Closed

rokadepiyush49-rgb wants to merge 1 commit into
apache:mainfrom
rokadepiyush49-rgb:perf/400-semaphore-first-waiter

Conversation

@rokadepiyush49-rgb

Copy link
Copy Markdown

Summary

Keep the only node of a WaitList inline so the first wait on a fresh primitive no longer allocates queue storage. A second concurrent waiter moves that node into the arena under the WaiterId its owner already holds, and the arena is then retained, so waiter order, permit accounting, and cancellation are unchanged.

A fresh zero-permit Semaphore handoff drops from two allocations (256 bytes) to one: the 64-byte lazily boxed macOS mutex, which Linux does not have. Reused handoffs stay at zero allocations.

Measured on an Apple M4 (macOS, Rust 1.94.1) with alternating runs. The host had background load, so each cell is the fastest run observed for that build. Lower is faster.

Scenario Baseline This PR Ratio
semaphore::acquire::handoff_permit 205 ms 162 ms 0.79×
semaphore::acquire::cancel_pending_acquire 186 ms 130 ms 0.70×
condvar::wait::notify_waiter 348 ms 305 ms 0.88×
event::wait::cancel_pending 352 ms 256 ms 0.73×
rwlock::write::readers_then_queued_writer/1 586 ms 518 ms 0.88×
semaphore::acquire::reused_handoff_permit 263 ms 271 ms 1.03×
semaphore::acquire::queued_owned_burst/4 238 ms 263 ms 1.11×
semaphore::acquire::queued_owned_burst/16 220 ms 231 ms 1.05×
mutex::lock::queued_handoff_cycle/4 262 ms 276 ms 1.05×
once::get_or_init::initialize_cell_waiter_batch/4 297 ms 317 ms 1.07×
event::wait::waiter_fan_out/4 239 ms 260 ms 1.09×

This is a trade-off. Across the 31 semaphore, mutex, rwlock, once, condvar, and event rows measured, single-waiter rows range from 0.70× to 1.05× and rows queueing four or more waiters range from 0.86× to 1.11×, because every node lookup now checks which storage is active.

Add a reused_handoff_permit benchmark for the reused case and regression coverage for multi-permit ordering, cancellation after a partial or full grant, a granted waiter surviving queue growth, waker replacement after growth, and missed wakeups on fresh semaphores.

Fixes #400.

Design Notes

The two storage forms share space, so WaitList grows by 8 bytes (Semaphore from 88 to 96) and WaiterId keeps its niche, leaving the acquisition futures from #394 unchanged. A separate inline field would add 48 bytes and measured slower. Storing the node in the future would undo #394.

Keep the only waiter of a WaitList inline instead of allocating arena
storage for it. A second concurrent waiter moves the node into the arena
under the same WaiterId, and the arena is then retained for reuse.
Copilot AI balanced review requested due to automatic review settings November 8, 2026 18:30

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@tisonkun

Copy link
Copy Markdown
Member

Thanks @rokadepiyush49-rgb for the clean enum-based WaitList design (keeping Semaphore growth down to +8 B) and especially for running and reporting the full 31-scenario benchmark suite transparently!

Those benchmark numbers answered the core question in #400 decisively: while fresh single-waiter cases speed up by avoiding the initial Arena allocation, every steady-state queue lookup and mutation now pays a storage-mode branch, regressing reused handoffs (1.03x) and 4–16 waiter contended workloads by 5%–11% (semaphore::acquire::queued_owned_burst/4 at 1.11x, mutex::lock::queued_handoff_cycle/4 at 1.05x, once::get_or_init::initialize_cell_waiter_batch/4 at 1.07x, event::wait::waiter_fan_out/4 at 1.09x), alongside the +8 B footprint increase on every uncontended instance.

Because Semaphore, Mutex, RwLock, Condvar, and channels are long-lived reusable primitives—already zero-allocation after warm-up and within ~1.12x of Tokio on reused handoff while keeping waiting futures Unpin (#394)—and true one-shot signaling belongs to oneshot, Completion, or WaitGroup (#421), we're keeping WaitList single-mode and closing #400 as not planned.

See #400 for the full summary. Closing this PR accordingly—thanks again for the thorough benchmark data!

@tisonkun tisonkun closed this Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reduce first-waiter overhead in Semaphore permit handoff

3 participants