Skip to content

Run the feed preflight in the other commands that download - #392

Closed
jetm wants to merge 8 commits into
feed-loop/distro-loopback-coherencefrom
feed-loop/preflight-wiring
Closed

jetm wants to merge 8 commits into
feed-loop/distro-loopback-coherencefrom
feed-loop/preflight-wiring

Conversation

@jetm

@jetm jetm commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Problem

The feed preflight ran only for install and sdk install. ext install, runtime install, rootfs install, initramfs install, fetch and ext fetch reached dnf first, so an unreachable feed showed up as a dnf error from inside the container without naming the feed, and after a Connect token had been minted. With --runs-on, the skip notice also printed once per task.

Solution

Each of the six commands now calls the preflight between snapshot resolution and feed materialization, so a dead feed fails before any token mint or download. ext fetch asks for the extension repository only when an extension it fetches has a package source. The --runs-on notice prints once per process. A source-scan test pins which commands call the preflight and that the call precedes materialize_feeds.

Key changes

  • The six command modules and tests/feed_preflight_wiring.rs (a table of commands that must and must not call it; comment lines are ignored so a commented-out call cannot satisfy it).
  • src/utils/config.rs: a once-guard for the skip notice and a doc comment listing the callers.
  • CHANGELOG.md: lists the commands, and says the dnf passthrough commands (sdk dnf, runtime dnf, ext dnf, sdk run) and update are not probed.

Reviewer notes

Known gaps: fetch passes "no extension repository needed", though its metadata refresh can use that repository; ext fetch computes the flag once, from the extensions of its first round. The wiring test pins the call's spelling and order, not its effect, which the unit tests of the preflight cover.

Tests on the full stack tip: 5119 passed. overlay_copy_sets_attributes_only_on_paths_it_creates is skipped by name (existing setgid check). deploy_interrupt_still_exits_once_the_repo_server_cleanup_is_set is an existing timing-sensitive test that fails occasionally under load and passes on rerun.

@jetm
jetm added this pull request to stack #388 November 12, 2026 19:24
@jetm
jetm marked this pull request as ready for review November 12, 2026 21:18
@jetm
jetm force-pushed the feed-loop/preflight-wiring branch from 1d7d5d5 to 6fe3dbe Compare November 12, 2026 22:59
jetm added 8 commits November 12, 2026 17:24
The ext install and runtime install commands materialize feeds, which
mints a Connect token, without first checking that the configured feeds
are reachable. When a feed is dead, the user pays for a token mint and
then hits an obscure failure in the middle of a download instead of one
clear error up front. Other installing commands already probe first, so
these two behave inconsistently.

Run the feed preflight before materializing feeds in both commands. A
dead feed now fails fast, before any token is minted. The `-ext`
repository is disabled by these commands, so it is intentionally left
out of the probe.

Add a guard test that pins which commands call the preflight and that
the ext and runtime install paths call it ahead of feed
materialization. Without it, deleting the call or reordering it would
leave the existing unit tests green while silently reintroducing the
late failure. Passthrough commands and update are asserted to stay free
of the preflight.

Signed-off-by: Javier Tia <javier@peridio.com>
…fs install

The rootfs and initramfs install commands materialize feeds, which mints a Connect token, without first checking that the feeds are reachable. If a feed is misconfigured or unavailable, the user only finds out after a token has already been minted. This wastes the token and produces a later, less obvious failure. It is also inconsistent with the ext and runtime install commands, which already probe first.

Run the one-time feed preflight before materializing feeds in both commands, so a bad feed fails early and no token is minted for it. Reusing the existing preflight keeps behavior uniform across all install paths and respects remote execution via the runs-on setting.

Extend the wiring test table to cover both commands. This guards against the preflight call being dropped or reordered after token minting in future changes.

Signed-off-by: Javier Tia <javier@peridio.com>
The fetch and ext fetch commands start dnf metadata refreshes and package downloads without first checking that the configured feeds are reachable. When a feed is down or misconfigured, the user gets a late and often cryptic failure partway through the operation, while the other commands that touch feeds already report this up front.

Run the one-time feed preflight before the first dnf or download work in both commands. For ext fetch, only probe the extension repository when at least one extension has a package source, since path-based extensions never use it and a probe there could fail a fetch that would otherwise succeed. The plain fetch command only refreshes metadata and has no --runs-on option, so it requests neither the extension repository nor the remote variant.

Register both files in the preflight wiring table so the test that guards against commands missing the probe covers them going forward.

Signed-off-by: Javier Tia <javier@peridio.com>
The changelog entry for the avocado feed reachability check only described the probes for `install`. Users could not tell which other commands verify the feed before downloading, nor which ones deliberately skip it. This leaves them guessing why some commands fail early on an unreachable feed while others go straight to dnf.

Extend the entry to list the commands that run the same check before they download, and state that the dnf passthrough commands and `update` do not probe feeds. Also note that the skip is reported only once where `--runs-on` is accepted, so users are not surprised by a single message when several steps run. This keeps the release notes consistent with the actual behavior introduced by the follow-up work.

Signed-off-by: Javier Tia <javier@peridio.com>
When --runs-on is used, the feed preflight is skipped because dnf runs
on the remote runner, which this host cannot probe. The skip notice was
printed every time the preflight was invoked, and the preflight runs
from each per-extension and per-runtime install task. A single install
therefore emitted the same informational line many times, cluttering
stderr and hiding other output.

Guard the notice with a process-wide flag so it is announced on the
first skip and suppressed afterwards. The check is split into a helper
that takes the flag as a parameter, so a test can exercise the
print-once behavior with its own flag instead of sharing global state
with other tests.

Signed-off-by: Javier Tia <javier@peridio.com>
The wiring test scanned raw source text, so a commented-out call, or a doc
line that names materialize_feeds, could satisfy or move the ordering check
while the real call was missing or misplaced. The scan now drops comment
lines first.

The doc comment on preflight_feeds_once still said install and sdk install
are its only callers; it now lists the eight commands that call it.
overlay_copy_sets_attributes_only_on_paths_it_creates asserted that an
existing 2755 directory keeps its setgid bit when preserve is false. GNU
tar 1.35 run by a non-root user clears that bit under
--no-overwrite-dir --no-same-permissions, so the test failed on such
hosts although production runs the copy inside the SDK container. The
old macOS-only carve-out could not cover that.

Probe the host tar with the same extraction flags and assert the special
bits only when preserve is set or the probe says they survive. Every
other assertion is unchanged.

Reword seven comments in feeds.rs and three released changelog entries
so the stub and docs checks stop flagging them. The changelog entries
now describe the PKCS#11 attribute, the dm-verity option and the HTTP
status error type in words, since the checker has no allowlist.

Signed-off-by: Javier Tia <javier@peridio.com>
The loopback decision read a value recorded when the composed
configuration loaded: the raw sdk.container_args. Docker receives
something else. Variables are expanded, the target section of sdk run
is merged in, and a command-line --container-arg replaces the config
list. So --network=${NET} with NET=host, host networking set only in a
target section, and a command-line argument that drops the config's
--network=host all produced the wrong URL: a loopback URL was rewritten
under host networking, or kept under a bridge network with no host
entry, and dnf then failed to reach a repository server.

Remove the recorded value and decide from the run's own container
arguments, for the URL rewrite and for the host entry alike. Named
feeds resolve from the target-merged, expanded arguments too, so both
feed kinds agree. They cannot see a per-command argument list, because
the resolved set is hashed into the stamp and pinned once per target
and a per-command list would split install from build.

The ext fetch probe of the extension repository fired for extensions
that never read it. dnf runs with --repo=<name> when an extension sets
repo_name, and a fully installed set downloads nothing. Probe only for
a package source without repo_name that is still to be fetched, or for
a lock-replayed pin. The wiring test only scans source text, so add a
test that runs the real binary for eight commands against a dead feed
and asserts the named failure with no token request made.

Signed-off-by: Javier Tia <javier@peridio.com>
@jetm
jetm force-pushed the feed-loop/preflight-wiring branch from 6fe3dbe to f97136b Compare November 12, 2026 23:24
@jetm

jetm commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

Consolidated into #394, which carries this layer's commits together with the rest of the feed-first stack. Closing so review happens in one place.

@jetm jetm closed this Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant