Repository navigation
ci: require matching main proofs before publication - #177
Conversation
Validate CI and native evidence for the landed source and its current run attempts before authorizing publication. Cover native test registration and results, pending counterparts, immutable evidence identity, missing metadata and validator refusal behavior.
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 5 advisory finding(s) below merit a look before merge.
Not checked: tests were not run; formal verification proved 0 of 13 changed function(s) (13 not verified).
Formal verification. PR-changed functions: 0/13 verified (0 proven, 0 may-equivalent, 0 distinguished) · 13 not verified (12 vacuous, 1 unsupported).
Not verified on this run: \_\_init\_\_ (unsupported), attempt\_jobs (vacuous: never exercised), create\_evidence (vacuous: never exercised), gate\_for\_run (vacuous: never exercised), main (vacuous: never exercised), validate\_gate (vacuous: never exercised), validate\_inventory (vacuous: never exercised), candidate\_gate (vacuous: never exercised), prepare (vacuous: never exercised), recover\_applied (vacuous: never exercised), and 3 more.
Graphify review — findings
Adds required main-branch proofs to the release gate: a new network-proof CI job runs the live PyPI hash test under one real xdist worker and derives a zero-skip inventory from the JUnit, and every Native feed leg now emits its ctest registration and JUnit, bound to that leg by native-leg. release/ready downloads the network report and evidence aggregate folds it in, so a missing or skipped proof stops release instead of passing silently; the native workflow also drops its path filter and runs on every main push, and release waits on it via workflow_run, whose name-matched trigger and attempt-suffixed artifact names are where a spoofed or re-run leg could go wrong. The new main_proofs module and the updated tests pin these contracts, including rejecting the wrong source, non-full rows, inserted ctest parallelism, and reverted proofs.
Worth a look
- Release now triggered by workflow_run of 'Native feed' (name-matched, branch filter spoofable) —
.github/workflows/release.yml:26· Escalate · medium- NOT verified (no proof, no reproducing execution) — needs human review
- Release now fires twice per main push and the two runs can race —
.github/workflows/release.yml:26· Escalate · medium- NOT verified (no proof, no reproducing execution) — needs human review
- Artifacts named by run_attempt break the native-proof job on 'Re-run failed jobs' —
.github/workflows/native-feed-post.yml:335· Escalate · medium- NOT verified (no proof, no reproducing execution) — needs human review
- Inconsistent workflow path contract between matching_main_runs and require_main_ready —
scripts/release_chain/evidence.py:108· Escalate · medium- NOT verified (no proof, no reproducing execution) — needs human review
- Missing main_proofs raises KeyError instead of a gate rejection —
scripts/release_chain/evidence.py:329· Escalate · medium- NOT verified (no proof, no reproducing execution) — needs human review
Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 478 functions depend on the 473 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
recover_applied()— 16 callers, 14 callees - worse:
publication()— 8 callers, 20 callees - worse:
gate_for_run()— 8 callers, 11 callees - worse:
candidate_gate()— 16 callers, 5 callees - worse:
create_evidence()— 9 callers, 8 callees - new:
native_aggregate()— 14 callers, 5 callees - worse:
prepare()— 3 callers, 18 callees - new:
native_inventory()— 7 callers, 7 callees - …and 19 more — each is listed as a finding
Verification — 478 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 478 function(s) in the blast radius were not formally verified this run
Docs that may be stale (advisory)
README.md§ Contributing (lines 145-169): references changed symbolsgitdocs/backtesting.md§ CLI reference (lines 26-53): references changed symbolsapidocs/provider-contract.md§ In-tree adapter skeleton (lines 32-81): references changed symbols__init__docs/providers.md§ Common market and bar API (lines 74-111): references changed symbolsfetchdocs/providers/fxmacrodata.md§ Timestamps and vintages (lines 52-86): references changed symbolssnapshotdocs/providers/fxmacrodata.md§ Constructor reference (lines 107-128): references changed symbolsapidocs/release-receipt-report.md§ Validation and remaining rollout requirement (lines 300-377): references changed symbols__init__,git,parsedocs/releasing.md§ Required gates and protected merge (lines 52-82): references changed symbolsgitnative/feed/README.md§ Serve: one producer, many runners (lines 541-617): references changed symbolssnapshot
Formal verification
Could not verify: Could not verify \_\_init\_\_.
The verifier did not have enough to check \_\_init\_\_, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
Could not verify: Could not verify attempt\_jobs.
The verifier did not have enough to check attempt\_jobs, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 264 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly FileNotFoundError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify create\_evidence.
The verifier did not have enough to check create\_evidence, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly ValueError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify gate\_for\_run.
The verifier did not have enough to check gate\_for\_run, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 247 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify main.
The verifier did not have enough to check main, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 1 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
Could not verify: Could not verify validate\_gate.
The verifier did not have enough to check validate\_gate, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 30 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify validate\_inventory.
The verifier did not have enough to check validate\_inventory, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 9 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify candidate\_gate.
The verifier did not have enough to check candidate\_gate, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 160 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly ValueError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify prepare.
The verifier did not have enough to check prepare, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 253 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly FileNotFoundError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify recover\_applied.
The verifier did not have enough to check recover\_applied, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 225 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly FileNotFoundError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify verified\_package.
The verifier did not have enough to check verified\_package, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 207 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly URLError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify publication.
The verifier did not have enough to check publication, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 253 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly ValueError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify retain.
The verifier did not have enough to check retain, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 160 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly KeyError — names the real obstacle, not a sampling gap)
· 20 grounded finding(s) anchored inline below; 7 more finding(s) on lines outside this diff (see the check run).
|
|
||
| def create_evidence(root: Path, directory: Path, needs: dict[str, Any]) -> dict[str, Any]: | ||
| require(set(needs) == NEEDS, "required CI job inventory changed") | ||
| def create_evidence( |
There was a problem hiding this comment.
create_evidence()
fans out to 8 callees (efferent coupling); 9 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return runs | ||
|
|
||
|
|
||
| def latest_main_run(repo: str, workflow: str, *, head: str) -> dict[str, Any]: |
There was a problem hiding this comment.
latest_main_run()
high coupling complexity (Ca·Ce = 12).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return max(runs, key=lambda row: row["id"]) | ||
|
|
||
|
|
||
| def gate_for_run( |
There was a problem hiding this comment.
gate_for_run()
fans out to 11 callees (efferent coupling); 8 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return cast(list[str], value) | ||
|
|
||
|
|
||
| def validate_inventory(value: Any, label: str) -> None: |
There was a problem hiding this comment.
validate_inventory()
6 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| ) | ||
|
|
||
|
|
||
| def validate_main( |
There was a problem hiding this comment.
validate_main()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
|
|
||
|
|
||
| @pytest.fixture | ||
| def complete_main_reports(native_reports, tmp_path, monkeypatch): |
There was a problem hiding this comment.
complete_main_reports()
fans out to 9 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
|
|
||
|
|
||
| @pytest.mark.parametrize("leg,missing", REQUIRED_NATIVE_OMISSIONS) | ||
| def test_validate_main_refuses_mutually_reduced_required_coverage( |
There was a problem hiding this comment.
test_validate_main_refuses_mutually_reduced_required_coverage()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| ] | ||
|
|
||
|
|
||
| def assert_native_post(workflow: dict[str, Any]) -> None: |
There was a problem hiding this comment.
assert_native_post()
high coupling complexity (Ca·Ce = 12).
Grounded coupling-delta finding (deterministic), not an LLM guess.
|
|
||
|
|
||
| @pytest.fixture | ||
| def saved_recovery_transport(ready, baseline, monkeypatch): |
There was a problem hiding this comment.
saved_recovery_transport()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| assert report == state["saved"] | ||
|
|
||
|
|
||
| def test_recovery_saved_receipt_keeps_installed_image_mismatch_refusal(saved_recovery_transport): |
There was a problem hiding this comment.
test_recovery_saved_receipt_keeps_installed_image_mismatch_refusal()
fans out to 8 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
Use an atomic cursor for concurrent producer observations and client-worker advances while preserving send-before-advance behavior and existing queue and retention locking.
Release preparation requires native and live-PyPI proof for the exact landed main commit and current workflow attempts. Pending counterparts return a neutral wait, while incomplete, failed or mismatched evidence blocks publication.
Pin required native test coverage and artifact downloads, keep CTest serial, and compare validated immutable proof identity during receipt finalization. Add missing-metadata guards and regressions for callback order, recovery and receipt replay.
Make the native server's per-client cursor atomic so producer diagnostics and client workers can safely observe and advance it, preserving send-before-advance behavior and queue and retention locking.
Validation before the native cursor edit: 1,521 cases passed across the four touched policy test modules, including the live PyPI metadata check. In each of two isolated copies, removing the waiting-Python validation call caused the first executed callback or recovery refusal case to fail after the original controls passed. The remaining mutant cases were not run.
Native validation: the cursor edit built and passed five serial ARM64 TSan CTest rows. The inverse old-cursor control also passed its 88-scenario feed harness on ARM64; inverse sensitivity: unproven, not waived. PR-tier hosted native qualification passed on test merge
87975831f7fa, bound to the reviewedad126749tree: TSan passed 5/5 CTest rows, and each of the other four native legs passed 4/4. This green is one observation, not detector power. All required hosted checks must pass before merge.Scope: native proof checks require registered and executed row names and reported results; validation of CTest WILL_FAIL and DISABLED properties remains outside this change. Existing release and report action references remain unchanged.
🤖 Generated with Claude Code