[schmitch]

Fixes #798

What changed

An apt proxy revalidated each file under dists/<distribution>/ on its own clock. If the upstream has no Acquire-By-Hash (reprepro, aptly, packagecloud, many vendor repos), apt fetches Packages at its own path, and while its lists are current it revalidates only InRelease. After an upstream publish, the proxy could then serve an InRelease and a Packages from two different publishes, and apt-get update failed with "Hash Sum mismatch".

• Plugin API (crates/plugin-api): two new FormatPlugin methods, both defaulting to none.

  • file_releases(path) names the files a cached file is published with and listed in.
  • file_listed(release_path, release, path) reads the size and SHA-256 that one of those files lists for it (new type ListedFile).

• apt (crates/formats/apt/src/proxy.rs): every non-by-hash file under dists/<distribution>/ returns InRelease, then Release. InRelease and Release each return the other. The listing comes from the existing cleartext and Release::parse. Pool and by-hash paths are not touched.

• File proxy (crates/server/src/files/proxy.rs, fresh_file): besides the TTL (is_due), a cached file is now due when:

  • one of its release files was replaced (updated_at) after the file was last checked. This also covers Release.gpg drifting from Release.
  • the release file being served (the first one cached) lists it with a different size or SHA-256, and that release was checked after the file was.

  The other direction is also handled: if a file was stored after the served release was last checked and the two disagree, the release is revalidated first. This runs after a fresh fetch or a replacement too. Each check is throttled by timestamps, so an upstream that is itself inconsistent costs at most one conditional GET per change, not one per request.

  The release listing is read once per release content and cached locally. Reading it and revalidating the release both run in their own task (as fetch_stored_apart does), so request futures don't overflow the stack.

• Upstreams with by-hash indexes (Debian, Ubuntu) behave as before: apt fetches their indexes by hash, and those copies are immutable.

• Docs: docs/apt.md (proxy section) and docs/format-plugins.md (the two new methods).

No admin API changes, so no web UI screen is needed.

Tests

• New e2e test proxies_serve_indexes_consistent_with_the_release (crates/server/tests/apt.rs) against the mock mirror, with a metadata TTL of one hour:
  1. Caches InRelease, Release, Release.gpg and Packages.gz.
  2. The upstream publishes, and only InRelease is made due.
  3. Checks that the Packages.gz served then matches what the new InRelease lists, and that Release and Release.gpg follow.
  4. Checks that once everything is consistent, the upstream is asked nothing more.
  5. Reverse case: only Packages.gz is made due, and the InRelease served afterwards lists the new one.
• Without the server change the test fails on that check: the served Packages.gz is 479498e8…, 102 bytes, while the InRelease lists f79dff3d…, 104 bytes.
• Unit tests for releases and listed_in in the apt crate.
• Run locally, all passing:
  • cargo fmt --all --check
  • cargo clippy -p forge-server -p forge-plugin-api -p forge-format-apt -p forge-plugin --all-targets -- -D warnings
  • cargo test -p forge-plugin-api -p forge-format-apt -p forge-plugin
  • cargo test -p forge-server --lib
  • cargo test -p forge-server --test apt --test raw --test proxy_cache --test group_proxy --test apt_import
  • cargo xtask changelog --check

Changelog

docs/changes/2.2.0/798.fixed.md

---

Generated by Claude Code