Skip to content

Avoid pool growth and unaligned mmap attempts when reading FILE buckets - #76

Open
notroj wants to merge 2 commits into
apache:trunkfrom
notroj:mmap-create-fail-alloc
Open

notroj wants to merge 2 commits into
apache:trunkfrom
notroj:mmap-create-fail-alloc

Conversation

@notroj

@notroj notroj commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Two changes for the memory growth reported in httpd PR 70167, where mod_ssl splits a FILE bucket at an offset which is not page-aligned, and the httpd child's memory then grows in proportion to the size of the file sent.

Trunk already page-aligns the offset in apr_mmap_create(), so the growth as reported does not reproduce here; it needs APR 1.7.x with APR-util 1.6.x. Both changes are written to be backported to those branches.

Avoid allocating from the pool when apr_mmap_create() fails

apr_mmap_create() allocated the apr_mmap_t before attempting the mapping, so every failed call left memory in the caller's pool. A FILE bucket retries the mmap for each read, so a persistent failure consumed pool memory for each read of the file. The structure is now allocated only once the file has been mapped, for both the Unix and Windows implementations.

The Windows change has not been compiled or tested.

Read a FILE bucket up to a page boundary before using mmap

A FILE bucket which does not start at a page-aligned offset is now read only as far as the next page boundary, so the remainder is mapped from a page-aligned offset. This applies only if:

  • mmap is enabled for the bucket;
  • the bucket is larger than the read buffer size, so could not be read in a single read anyway;
  • the remainder beyond the page boundary is at least APR_MMAP_THRESHOLD.

The page size comes from sysconf(_SC_PAGESIZE); where that is not available the behaviour is unchanged.

One cost on trunk: a bucket between the read buffer size and APR_MMAP_LIMIT at an unaligned offset was previously a single mmap, and is now one short read followed by one mmap.

Tests

  • test_mmap_create_fail (testmmap): a failed apr_mmap_create() does not allocate from the pool. The pool check uses apr_pool_num_bytes(), so it is only effective in a pool-debug build.
  • test_mmap_unaligned (testbuckets): a FILE bucket at an unaligned offset, larger than APR_MMAP_LIMIT, is read up to the page boundary and then as MMAP buckets.
  • test_mmap_fail (testbuckets): a file which cannot be mapped is read correctly via the fallback path.

The first two fail without the corresponding source change (the first in a pool-debug build). The full test suite passes on Linux x86_64.

Not covered

  • Each successfully mapped chunk still leaves one apr_mmap_t in the pool, about 14 bytes per MB read.
  • Neither change has been tested on the 1.x branches or end-to-end with httpd.

🤖 Generated with Claude Code

notroj and others added 2 commits November 14, 2026 13:43
A caller which repeatedly fails to map a file, such as a FILE bucket
being read in chunks, otherwise consumes memory from the pool for each
attempt, for the lifetime of that pool.

This is a partial fix for httpd PR 70167: it removes the memory growth
per failed call, but does not stop the FILE bucket retrying a failed
mmap for each read.

* mmap/unix/mmap.c, mmap/win32/mmap.c (apr_mmap_create): Allocate the
  apr_mmap_t only once the file has been mapped.

* test/testmmap.c (test_mmap_create_fail): New test.
  (testmmap): Run it.

PR: 70167
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A FILE bucket which does not start at a page-aligned offset in the
file is now read only as far as the next page boundary, so that the
remainder of the bucket is mapped from a page-aligned offset.  This
only applies where the bucket could not be read in a single read
regardless, and where enough of the bucket remains beyond the page
boundary to be mapped.

This is a partial fix for httpd PR 70167, where mod_ssl splits a FILE
bucket at an offset which is not page-aligned: with this change the
FILE bucket no longer attempts to map the file from such an offset.

* buckets/apr_buckets_file.c (file_align_length): New function.
  (file_bucket_read): Use it to limit the read length and skip the
  mmap attempt if the bucket does not start on a page boundary.

* test/testbuckets.c (make_pattern_file, pattern_match): New helpers.
  (test_mmap_unaligned, test_mmap_fail): New tests.
  (testbuckets): Run them.

PR: 70167
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@notroj
notroj force-pushed the mmap-create-fail-alloc branch from 239a34d to c74e993 Compare November 14, 2026 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant