Repository navigation
Conversation
02eb51f to
49225b9
Compare
d26f606 to
09e88c7
Compare
57f533d to
e42242a
Compare
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Forward complete read batches through caches and bundle offsets. Collect concurrent reads for 1 ms and merge gaps up to 4 MiB, with five downloads per group. Stream useful slices and discard gaps, preserving cache ownership, retries, and cancellation.
Use the same helper for initial opens and retries after body timeouts. Cover retry exhaustion and EOF timeouts without reopening empty ranges.
Document callback ordering and cache-hit handling. Assert storage ranges directly in mocks instead of tracking reads through shared state.
Allow up to 32 concurrent asynchronous cache lookups instead of five. Expand the test to verify 64 lookups complete in two waves while preserving result order and forwarding cache misses together.
Stop probing for extra bytes after reading requested slices. Use read_exact for Azure buffer reads and preserve short-read errors. Update tests and documentation for the changed behavior.
27da072 to
640476b
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 27da072f48
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| let num_added_bytes = self.cache.put_slice(missing_range, bytes.clone()); | ||
| read_reservation.keep(num_added_bytes); |
There was a problem hiding this comment.
Offload large-batch cache insertion from Tokio workers
For large regex/wildcard warmup batches, this loop performs one synchronous FileByteRangeCache::put_slice per requested range after the download finishes. Each call locks the cache and performs BTreeMap operations, potentially also copying overlapping buffers. With the 10k–1M-range batches targeted by this change, the entire population step runs without yielding on the leaf search's Tokio worker, delaying unrelated RPCs and I/O when several splits warm up concurrently. Move large-batch cache population onto run_cpu_intensive, following the Tokio/CPU-work guidance in AGENTS.md.
AGENTS.md reference: AGENTS.md:L87-L87
Useful? React with 👍 / 👎.
| let reader = storage | ||
| .get_slice_stream(path, download_chunk.clone()) | ||
| .await?; |
There was a problem hiding this comment.
Preserve Azure body-error retries for merged downloads
For Azure-backed splits, switching merged downloads to this stream removes retries for transient body failures after the first chunk. AzureBlobStorage::get_slice previously used get_to_bytes, whose retry callback encloses the complete download_all; get_slice_stream ends its retry callback after obtaining the first chunk. Subsequent I/O errors therefore abort the whole collected batch, and TimeoutAndRetryStorage does not recover them because it propagates body errors immediately and retries only elapsed deadlines. Preserve Azure's body-error retry behavior with a resumable retrying stream.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
🟡 Changes recommended
Cache bypasses and redundant memory reservations can cause unnecessary remote reads and search failures, while valid empty ranges can panic in debug builds and Azure streaming reads lose body-error retries.
5 open issues
What changed in this PR
This PR reduces search-warmup memory use by moving read coalescing into Quickwit’s storage layer. Merged downloads are streamed so gaps can be discarded rather than retained in cached buffers.
Changes:
- Updates Tantivy and forwards batched reads through directory, cache, and bundle wrappers.
- Adds streaming coalescing with bounded download concurrency and requested-range extraction.
- Extends stream timeouts, retry handling, S3 byte accounting, and regression coverage.
| File | Description |
|---|---|
quickwit/quickwit-storage/test-data/coalescing/requested_ranges.json |
Adds recorded postings-range fixtures. |
quickwit/quickwit-storage/test-data/coalescing/README.md |
Documents fixture provenance and reference measurements. |
quickwit/quickwit-storage/src/timeout_and_retry_storage.rs |
Adds stream deadlines and timeout-based resumption. |
quickwit/quickwit-storage/src/storage.rs |
Defines batch reads and revises buffer-fill behavior. |
quickwit/quickwit-storage/src/storage_cache/storage.rs |
Adds batch handling to the shared storage cache. |
quickwit/quickwit-storage/src/object_storage/s3_compatible_storage.rs |
Tracks streamed bytes and in-flight requests. |
quickwit/quickwit-storage/src/object_storage/azure_blob_storage.rs |
Simplifies reading into caller-provided buffers. |
quickwit/quickwit-storage/src/lib.rs |
Exports coalescing storage and batch helpers. |
quickwit/quickwit-storage/src/error.rs |
Preserves underlying I/O error kinds. |
quickwit/quickwit-storage/src/coalescing_storage/tests.rs |
Tests collection, sharing, and cancellation. |
quickwit/quickwit-storage/src/coalescing_storage/stream_tests.rs |
Tests extraction, ownership, and stream failures. |
quickwit/quickwit-storage/src/coalescing_storage/mod.rs |
Implements range collection, planning, and streaming extraction. |
quickwit/quickwit-storage/src/cache/storage_with_cache.rs |
Adds generic cached batch reads. |
quickwit/quickwit-storage/src/cache/mod.rs |
Exposes batch-cache utilities. |
quickwit/quickwit-storage/src/cache/batch_read.rs |
Adds ordered cache lookup and miss-forwarding helpers. |
quickwit/quickwit-storage/src/bundle_storage.rs |
Translates batch offsets and validates file bounds. |
quickwit/quickwit-search/src/storage_cache_layer_tests.rs |
Tests batch forwarding through storage caches. |
quickwit/quickwit-search/src/leaf.rs |
Installs coalescing storage in search setup. |
quickwit/quickwit-directories/src/storage_directory.rs |
Forwards directory batches to storage. |
quickwit/quickwit-directories/src/hot_directory.rs |
Adds hot-cache-aware batch forwarding. |
quickwit/quickwit-directories/src/debug_proxy_directory.rs |
Tests batch-read instrumentation. |
quickwit/quickwit-directories/src/caching_directory.rs |
Adds batch warmup reservations and cache insertion. |
quickwit/quickwit-directories/Cargo.toml |
Enables storage test utilities. |
quickwit/quickwit-config/src/node_config/mod.rs |
Makes timeout iterator captures explicit. |
quickwit/Cargo.toml |
Updates the Tantivy revision. |
quickwit/Cargo.lock |
Records updated Tantivy dependencies. |
LICENSE-3rdparty.csv |
Adds the new transitive dependency’s license entry. |
Review effort: Balanced
| quickwit_storage::read_batch_with_cache( | ||
| requested_ranges, | ||
| |requested_range| self.cache.get_slice(requested_range), |
| // Reserve every miss before starting I/O. Dropping these guards releases all | ||
| // reservations if any reservation or the download fails, or the future is dropped. | ||
| let mut read_reservations = Vec::with_capacity(ranges_to_fetch.len()); | ||
| for missing_range in &ranges_to_fetch { | ||
| read_reservations.push( | ||
| self.memory_reserver | ||
| .try_reserve_memory(missing_range.len() as u64)?, | ||
| ); | ||
| } | ||
| let downloaded = self | ||
| .underlying_filehandle | ||
| .read_bytes_batch_async(ranges_to_fetch.clone()) | ||
| .await?; | ||
| assert_eq!(downloaded.len(), ranges_to_fetch.len()); | ||
| for ((missing_range, bytes), read_reservation) in ranges_to_fetch | ||
| .into_iter() | ||
| .zip(&downloaded) | ||
| .zip(read_reservations) | ||
| { | ||
| let num_added_bytes = self.cache.put_slice(missing_range, bytes.clone()); | ||
| read_reservation.keep(num_added_bytes); | ||
| } | ||
| Ok(downloaded) |
| self.storage_directory | ||
| .storage | ||
| .get_slices(&self.path, requested_ranges) |
| let skip_cache = | ||
| requested_ranges.len() > super::batch_read::MAX_NUM_RANGES_FOR_CACHE_LOOKUP; | ||
| super::read_batch_with_async_cache( | ||
| requested_ranges, | ||
| |requested_range| self.cache.get(path, requested_range), |
| let reader = storage | ||
| .get_slice_stream(path, download_chunk.clone()) | ||
| .await?; |

Why
Tantivy coalesces postings reads to reduce request count. For sparse matches, a query needing only ~1 MB of postings may end up downloading most of a ~250 MB postings file. This can cause OOMs in two stages:
Streaming coalesced reads lets us extract the requested data and discard gaps as they arrive. The cache receives only the requested slices instead of retaining the merged downloads.
The batch API avoids a future per range from tantivy. We may receive one million and futures are very large.
Limitations
The streaming part only covers gaps introduced by coalescing.
A query that matches everything and downloads 250 MB without gaps still downloads and buffers a 250 MB chunk. It does not stream the postings lists. Full streaming of postings lists is future work.
What changes
StorageDirectory, within one batch and one file, merging gaps up to 2 MiB. Preserve input order, duplicates, and empty reads.Verification
quickwit-common,quickwit-storage,quickwit-directories, andquickwit-search.Full Docker/REST suites and live S3 memory/performance benchmarks were not run.