[schmitch]

Fixes #797

What changed

• Control file cap. MAX_CONTROL_BYTES (crates/formats/apt/src/control.rs) is 64 KiB instead of 1 MiB. deb.rs now refuses a larger control file with "the package's control file is larger than 64 KiB", and uploads and imports turn that into a 400 (both read packages through parse_package).
• Indexes are streamed. FormatPlugin::index used to return a Vec<IndexFile>. It now hands each file to a sink (&mut dyn FnMut(IndexFile) -> Result<(), FormatError>). The host (crates/server/src/indexes/mod.rs) signs, checks and stores each file in the object store as it arrives, then drops its content, and records the generation in one transaction as before. apt renders one (component, architecture) list at a time: it builds the text, compresses it, hands over the plain, gz and xz files with their by-hash copies, and drops them before starting the next list. Memory now holds each stanza once plus one list's files, not every list's text, a copy and a by-hash clone at once.
• The file limits are checked as files arrive. The paths, content types, MAX_INDEX_FILES and MAX_INDEX_BYTES that sign_and_check used to check once everything was in memory are now checked file by file (Bounds). An overrun stops the plugin through the sink's error.
• Size check before rendering. IndexInput has a new field, max_bytes, which the host sets to MAX_INDEX_BYTES. Before making any list, apt adds up the stanza bytes of every list and doubles it, because each Packages is stored by name and by hash. If that is over max_bytes, it refuses without compressing anything.
• Packages already stored over the cap. A package uploaded while the cap was 1 MiB keeps its stored control data. If that data is longer than MAX_LISTED_CONTROL_BYTES (2 × the cap), apt leaves it out of the indexes instead of failing or bloating every generation. It stays in the pool (and in browse) until it is deleted. The limit is twice the cap because the stored control data is a canonical respelling that can be a few bytes longer than the uploaded file, so no package accepted under the new cap is ever left out. apt is new in 2.2.0, so only deployments of the 2.2.0 development line can have such packages. The format crates have no logging, so nothing is logged when a package is left out. docs/apt.md documents the rule.

Tests

• crates/server/tests/apt.rs (hosted_repositories_publish_signed_indexes): an upload whose control file is just over 64 KiB answers 400 with "larger than 64 KiB". With the cap set back to 1 MiB it answers 201 and the test fails.
• crates/formats/apt/src/deb.rs, a_control_file_over_the_cap_is_refused: a control file of exactly the cap is read, and one byte more is refused. It fails with the cap back at 1 MiB.
• crates/formats/apt/src/index.rs:
  • an_index_over_the_byte_cap_fails_before_anything_is_compressed: the sink receives no file and the error names the cap. With the size check disabled, render returns Ok and the test fails.
  • lists_are_handed_over_one_at_a_time: a sink error after the first list's six files stops rendering before the next list.
  • a_package_stored_over_the_cap_is_left_out: a package with control data over the listing limit is not listed, and one at the upload cap is.
• The existing apt tests now collect their files through the sink. The parser performance test in control.rs keeps its 1 MiB input as a local constant, since index text can be that large.
• Run locally on the branch rebased onto main after apt: a writer can break a hosted repository's indexes for everyone with packages of many components and architectures #761, apt: one package with a malformed Depends (or another relationship field) is published as is, and apt-get update then fails on every client of the repository #814 and Imports: an apt import holds every entry of the source's Packages index in memory, about 16 times the inflated index, so a source can exhaust the instance's memory #821 merged: cargo fmt --all --check; cargo clippy -p forge-format-apt -p forge-plugin-api -p forge-server --all-targets -- -D warnings; cargo test -p forge-format-apt (69 passed); cargo test -p forge-plugin-api; cargo test -p forge-server --test apt --test apt_import (13 and 3 passed); cargo xtask changelog --check.

Changelog

docs/changes/2.2.0/797.fixed.md. docs/apt.md (Uploading packages) lists the new 64 KiB limit and says what happens to older, larger packages. docs/format-plugins.md (Signed indexes) describes the streaming index contract and max_bytes.

Not fixed here

• A single list can still get as large as half of MAX_INDEX_BYTES (1 GiB), and it is held about twice while it is compressed and handed over. Reaching that takes some 8,000 packages at the cap in one list. apt: a writer can break a hosted repository's indexes for everyone with packages of many components and architectures #761 bounds the number of index files, not their size; a per-list bound would be a separate change.
• As before this change, while a generation is being made the host keeps a committed object-store handle (temp file and lease) for every file it has stored, and generation is capped at MAX_INDEX_FILES files.

🤖 Generated with Claude Code

https://claude.ai/code/session_012e6DJYtQDk1mTgJAn2noU2