Skip to content

test(conformance): exchange fresh artifacts in both Rust/Go directions - #1506

Open
coderdan wants to merge 10 commits into
feat/stack-encrypt-coverage-gapsfrom
feat/stack-encrypt-artifact-exchange
Open

coderdan wants to merge 10 commits into
feat/stack-encrypt-coverage-gapsfrom
feat/stack-encrypt-artifact-exchange

Conversation

@coderdan

@coderdan coderdan commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Stack Encrypt has two implementations that must agree on what an encrypted value means: the Rust crate (stack-encrypt, used natively) and the Go SDK (which runs the same Rust engine compiled to WebAssembly). Until now the conformance suite proved "Go writes, Rust opens" for one small field only; everything wider was checked against frozen test files or inside one language. This PR adds a fresh exchange profile: in a single run, Rust and Go each encrypt every value kind at its edges, a record with a passthrough field and search terms, whole-struct opaque blobs and EQL v4 rows, and the other side decrypts them and re-derives the same query terms. Ciphertext bytes are never compared (encryption is randomized); decrypted values and query semantics are. The lane fails if a runner or guest is missing or a fresh output is empty, and its report records who produced what with which SDK, engine, backend and storage versions.

Changes

Harness and scripts

  • scripts/stack-exchange-conformance.mjs: the required lane. Checks the fixture-update variables are unset (EQL_UPDATE_FIXTURES included, now refused by the shared checkEnvironment), both deterministic guests exist and are nonempty, and go and rustc are present, before running anything. Writes to a fresh temp directory every run (nextest reruns; go test -count=1), runs four producers then four consumers, requires every output to be nonempty and of the right producer and shape, and writes exchange.json to STACK_CONFORMANCE_REPORT_DIR through the shared writeReport (now in stack-conformance-evidence.mjs and used by the lifecycle and EQL scripts too). The run happens inside inFreshDirectory, which removes the temporary directory only after the report is written and keeps it, with its path printed, after any failure (the lockfile reads, cargo metadata and the report included). EQL rows must carry a producer manifest naming the expected language.
  • mise.toml: test:conformance:exchange. scripts/stack-conformance.sh runs it at the end of the aggregate and adds the new Rust binary; the Go selection gains TestArtifactExchange.
  • conformance/stack-encrypt/v1/exchange.json: the shared corpus (seed, record plan and rows, blob cases, the EQL lane's fixture). Values come from the existing values.json; nothing is copied.

Rust producer/consumer

  • packages/stack-encrypt/tests/exchange_conformance.rs: produces and consumes values, records (dynamic record path with a passthrough output) and OpaqueBlob<T> blobs; consumer derives its own eq/lt terms and compares them with the stored ones, and refuses a record row whose stored sealed fields are not exactly the declared ones (SQL nulls excluded); an_undeclared_record_field_is_refused proves it. STACK_EXCHANGE_OUTPUT / STACK_EXCHANGE_INPUT.
  • packages/eql/tests/encryption/tests/v4_types.rs: rows_rust_seals_open_in_the_go_sdk seals the fixture's profiles plaintexts with eql-bindings' typed targets and writes them for Go (EQL_V4_RUST_ROWS_OUT); go_rows::fixture gains a committed() reader and seal_row. The rows carry a producer object (language, SDK and version, backend, engine, schema). rows_the_go_sdk_sealed_open_in_rust refuses handed-over rows that are not Go's, and compares each equality column's stored hm with the typed *EqQuery it derives (go_rows::check_eq_terms).
  • tests/common/conformance.rs: Input also serializes.

Go producer/consumer

  • languages/golang/encrypt/exchange_conformance_test.go: TestArtifactExchange, the same three kinds through the record interface, the opaque storage path (vcffi.MarshalStorage / gensupport.Opaque), and the generated-code conventions (the passthrough field stays on the host). The consumer refuses a record row carrying an undeclared sealed field (TestArtifactExchangeRefusesAnUndeclaredRecordField); its helpers take testing.TB so that test can capture the refusal.
  • languages/golang/encrypt/eql_v4_test.go: TestV4RowsTheRustSuiteSealedOpenInGo opens the Rust-sealed rows through the generated code and compares each *Eq column's stored hm with the generated query's (EQL_V4_RUST_ROWS; skips without it, and the lane treats a skip as failure). It refuses rows whose producer is not the Rust suite's. TestV4RowsMatchTheCommittedFixture writes a producer (with the EQL guest's SHA-256) on the rows it hands over, never on the committed file.

CI

  • .github/workflows/tests-golang.yml (wasi-check): a required "fresh cross-language artifact exchange" step after both deterministic guest builds, with its report uploaded as exchange-conformance; the new script is in both path filters.
  • scripts/__tests__/stack-exchange-conformance.test.mjs: pins both directions of every kind, the corpus shape, the missing/empty-guest and missing-runner failures (stubbed paths and a subprocess run with an empty PATH), the nonempty-output checks, the EQL_UPDATE_FIXTURES refusal (direct and through the script), a failing report tail keeping the directory, the lockfile version reader, and the workflow wiring.

Contract docs

  • conformance/stack-encrypt/exchange-contract.md (new): what is exchanged, the artifact and manifest format, required execution, the report, the SQL lane, the released-version placeholder and its row shape, scope.
  • README, coverage map, fixture/values/EQL-JSON contracts and the plan: the "fresh exchange remains open" sentences now point at the profile; new checkpoint sections.

Verification

Run from the worktree on this branch, after mise run wasm:guest:build:deterministic, wasm:guest:build:eql:deterministic, wasm:guest:build and wasm:guest:build:eql.

Command Result
cargo fmt --all --check (root) and mise x --env test -- cargo fmt --all --check in packages/eql clean
mise x --env test -- cargo clippy -p stack-encrypt -p stack-encrypt-derive --all-features --all-targets -- -D warnings clean (a first run flagged too_many_arguments on the term check; the parameters now travel in a struct)
STACK_CONFORMANCE_REQUIRED=1 mise x --env test -- cargo nextest run -p stack-encrypt -p stack-encrypt-derive --all-features 1130 passed, 1 skipped (the pre-existing ignored test)
mise x --env test -- cargo clippy --locked -p eql-encryption-tests --all-targets -- -D warnings in packages/eql clean
STACK_CONFORMANCE_REQUIRED=1 CGO_ENABLED=0 go test -count=1 ./..., go vet ./..., gofmt -l . in languages/golang every package ok; vet clean; nothing to format
STACK_CONFORMANCE_REPORT_DIR=... bash scripts/stack-conformance.sh (the aggregate, now ending in the exchange) every Rust binary and Go runner passed; the exchange produced and opened 8 lanes and wrote exchange.json
STACK_CONFORMANCE_REPORT_DIR=... mise run test:conformance:exchange after the rebase 8 lanes: values 60/60, record 3/3, blobs 3/3, eql-rows 2/2 in each direction; report lists producers (stack-encrypt 0.3.0 native; Go module (devel) over wasi with the guest's SHA-256), engine versions from the root, guest and EQL lockfiles, eql_v4 / @cipherstash/eql 3.0.6 / eql-bindings 3.0.6, guest hashes, toolchains, and the empty released-version placeholder
Undeclared record field (automated, both languages) an_undeclared_record_field_is_refused and TestArtifactExchangeRefusesAnUndeclaredRecordField pass, and each fails with its check disabled
Tamper checks (by hand, each in both directions): swapped value ciphertexts, swapped record ciphertexts, a changed passthrough, swapped record and value eq terms, swapped blobs, swapped EQL envelopes, a swapped EQL hm only, and each consumer fed its own language's artifacts or rows every one fails its consumer. The exception is a replaced term for an index the corpus lists as unanswered, which is checked for presence only, as the contract says
npx vitest run scripts/__tests__/stack-conformance*.test.mjs scripts/__tests__/stack-exchange-conformance.test.mjs scripts/__tests__/stack-eql-conformance.test.mjs scripts/__tests__/stack-lifecycle-conformance.test.mjs scripts/__tests__/crates-ci.test.mjs scripts/__tests__/tests-workflow-gate.test.mjs scripts/__tests__/workflow-paths-filter-parity.test.mjs 7 files, 91 tests passed (the new file proves the missing-guest, empty-guest, missing-runner and empty-output failures, and a subprocess run with an empty PATH stops at preflight)
pnpm biome check no errors (the repository's pre-existing warnings only)
git log --format='%h %G? %s' origin/feat/stack-encrypt-coverage-gaps..HEAD every commit G

Not run: the PostgreSQL lanes (mise run test:encryption:postgres, go:test:postgres). This PR changes no SQL or database test; the exchange's SQL story is documented in the contract and left to the existing eql-postgres job.

Related

Review notes

  • Start with conformance/stack-encrypt/exchange-contract.md, then scripts/stack-exchange-conformance.mjs, then the two runners side by side.
  • The record's passthrough field is lowered as a passthrough output in Rust and held on the host in Go, which is how each SDK actually stores one; the artifact carries it as the typed input and both consumers check it.
  • The sealed-null record field is float32, not text: the engine admits a sealed null on fixed-width kinds only.
  • Deliberately not done, with reasons:
    • Fresh JSON document exchange: no Go-to-Rust or Rust-to-Go JSON opening path exists today beyond the frozen vectors; adding one is a profile of its own. The scalar EQL v4 rows are exchanged in both directions.
    • A SQL lane that stores one language's fresh rows through the other: the existing eql-postgres job stores Go's fresh rows and the EQL suite's ignored tests insert the Go-sealed fixture; running the Rust producer inside that job is left for a follow-up. The contract says so.
    • Released-version rows: the compatibility policy (Docs: EQL v4 and Stack Encrypt compatibility policy #1314) is not written; the report carries the row shape and an empty list rather than fabricated rows.
    • Python/TypeScript pairs: no runners exist.
  • No frozen fixture or artifact changes; no crate or module public behaviour changes, so no changeset and no ! commit.

Summary by CodeRabbit

  • Conformance
    • Added fresh, two-way checks that Rust and Go can exchange and open encrypted values, records, opaque blobs, and EQL rows.
    • Checks compare decrypted content and query terms, and report producer and storage details. Missing runners, guests, or fresh output cause the checks to fail.
  • CI
    • Pull requests and pushes to main that affect the conformance workflow run the exchange checks and publish their reports.
  • Documentation
    • Updated conformance guides to describe exchange coverage and remaining compatibility gaps.

@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI; Enforced overrides: Organization
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 28bfffca-37c7-4107-89bd-f68ac7540efe

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
Walkthrough

The change adds a fresh Rust-Go artifact exchange profile for values, records, opaque blobs, and EQL v4 rows. It adds producer and consumer checks, task orchestration, reports, documentation, tests, and required CI execution.

Changes: Rust-Go artifact exchange

Layer / File(s) Summary
Exchange profile and corpus
conformance/stack-encrypt/*, docs/plans/2026-10-31-stack-encrypt-conformance.md
The profile and corpus define fresh bidirectional exchange cases for values, records, blobs, and EQL v4 rows. The documentation describes producer metadata, consumer checks, task requirements, reports, and remaining scope.
EQL v4 row exchange
packages/eql/tests/encryption/tests/v4_types.rs, languages/golang/encrypt/eql_v4_test.go
The Rust and Go tests exchange EQL row fixtures with producer metadata. They open the other language’s rows and check plaintext values and equality-query terms.
Rust and Go artifact production and verification
packages/stack-encrypt/tests/common/conformance.rs, packages/stack-encrypt/tests/exchange_conformance.rs, languages/golang/encrypt/exchange_conformance_test.go
The conformance tests produce and consume fresh artifacts for values, records, and blobs. They validate metadata, case coverage, stored terms, and opened values.
Exchange orchestration, reports, and CI
scripts/stack-exchange-conformance.mjs, scripts/stack-conformance-*, scripts/stack-eql-conformance.mjs, scripts/stack-lifecycle-conformance.mjs, scripts/__tests__/stack-exchange-conformance.test.mjs, .github/workflows/tests-golang.yml, mise.toml
The exchange script runs both producer and consumer directions, validates outputs, and writes a report. The task, CI workflow, shared report writer, and tests wire and check execution.

Estimated code review effort: 4 (Complex) | ~60 minutes

Change: Other

Sequence Diagram(s)

sequenceDiagram
  participant ExchangeTask as stack-exchange-conformance.mjs
  participant RustTest as Rust exchange_conformance
  participant GoTest as Go TestArtifactExchange
  participant Report as exchange.json report
  ExchangeTask->>RustTest: Produce fresh artifacts and EQL rows
  ExchangeTask->>GoTest: Produce fresh artifacts and EQL rows
  ExchangeTask->>GoTest: Open Rust-produced artifacts
  ExchangeTask->>RustTest: Open Go-produced artifacts
  ExchangeTask->>Report: Write lane and toolchain results
Loading

Possibly related issues

  • Issue 1490: Adds a Rust-Go CI lane and reports, but does not define or populate the supported release matrix.
  • Issue 1476: Adds fresh EQL v4 row exchange; JSON and database lanes remain outside this change.
  • Issue 1477: Adds current value and query-term exchange coverage, while the additional boundary and raw-frame cases remain open.
  • Issue 1314: Adds current-version exchange evidence, but released-version combinations remain undefined.

Suggested reviewers: auxesis, freshtonic

Merge Risk: Low · previous review 08e71

The exchange lane has bounded validation and diagnostic gaps: unexpected record fields can pass, an exported fixture-update variable can modify committed fixtures, and reporting failures lose diagnostic files. These warrant localized fixes or explicit follow-up, but do not establish a production runtime failure.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "test(conformance): exchange fresh artifacts in both Rust/Go directions" matches the main change. The PR adds a required fresh artifact exchange conformance lane in which Rust and Go each pr…
Linked Issues check Passed Issue #1478 requires fresh exchange for the enrolled Rust and Go profiles. The PR adds both producer-consumer directions for values, records, opaque blobs, and EQL v4 rows. Consumers compare decrypted…
Out of Scope Changes check Passed The changed scripts, runners, fixtures, tests, CI wiring, reports, and contract documents directly support the fresh exchange profile in issue #1478. The shared report writer supports the new report a…
Full details: Title check

Explanation

The title "test(conformance): exchange fresh artifacts in both Rust/Go directions" matches the main change. The PR adds a required fresh artifact exchange conformance lane in which Rust and Go each produce values, records, opaque blobs, and EQL v4 rows that the other language opens and checks. The title names the scope (Rust/Go, both directions) and the conventional commit type. It does not cover every detail, such as the CI workflow, reporting helper, or deferred JSON and Python/TypeScript work, but the title does not need to.

Full details: Linked Issues check

Explanation

Issue #1478 requires fresh exchange for the enrolled Rust and Go profiles. The PR adds both producer-consumer directions for values, records, opaque blobs, and EQL v4 rows. Consumers compare decrypted data and derived query terms, not ciphertext bytes. The script requires nonempty fresh output, disables producer caches, and fails when Go, Rust, or either deterministic guest is unavailable. Artifacts and reports include SDK, engine, backend, profile, and storage metadata. Python and TypeScript have no runners, JSON exchange has no available opening path, and released-version rows remain undefined until #1314 provides a policy. These are documented issue conditions, not unmet requirements for this enrolled profile.

Full details: Out of Scope Changes check

Explanation

The changed scripts, runners, fixtures, tests, CI wiring, reports, and contract documents directly support the fresh exchange profile in issue #1478. The shared report writer supports the new report and existing conformance reports. No unrelated product behavior or database implementation changes are shown. The documented SQL, JSON, Python/TypeScript, and released-version limits remain within the issue scope.

Full details: Docstring Coverage

Explanation

Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 45 functions across 12 files. (10 skipped: 10 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.
Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI; Enforced overrides: Organization
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 54e13b47-9030-4f3c-b97a-91e70b66b2ac
Commits

Reviewing files that changed from the base of the PR and between dd6e808 and 08e711f.

Files selected for processing: 22
  • .github/workflows/tests-golang.yml
  • conformance/stack-encrypt/README.md
  • conformance/stack-encrypt/coverage.md
  • conformance/stack-encrypt/eql-json-contract.md
  • conformance/stack-encrypt/exchange-contract.md
  • conformance/stack-encrypt/fixture-contract.md
  • conformance/stack-encrypt/v1/exchange.json
  • conformance/stack-encrypt/values-contract.md
  • docs/plans/2026-10-31-stack-encrypt-conformance.md
  • languages/golang/encrypt/eql_v4_test.go
  • languages/golang/encrypt/exchange_conformance_test.go
  • mise.toml
  • packages/eql/tests/encryption/tests/v4_types.rs
  • packages/stack-encrypt/tests/common/conformance.rs
  • packages/stack-encrypt/tests/exchange_conformance.rs
  • scripts/__tests__/stack-exchange-conformance.test.mjs
  • scripts/stack-conformance-evidence.mjs
  • scripts/stack-conformance-selection.mjs
  • scripts/stack-conformance.sh
  • scripts/stack-eql-conformance.mjs
  • scripts/stack-exchange-conformance.mjs
  • scripts/stack-lifecycle-conformance.mjs

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 0 remain after this review.

@changeset-bot

changeset-bot Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 42a06a5

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Mutation testing (cargo-mutants, --in-diff, stack-auth + stack-encrypt)

No mutants were generated for the changed lines.

@coderdan

coderdan commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

Review: fresh artifact exchange (head 4d67b46)

Ran mise run test:conformance:exchange with a report dir: 8 lanes, each nonempty (values 60, record 3, blobs 3, eql-rows 2 per direction), toolchains, three lockfiles, guest hashes, storage versions and the empty released_versions all present. CI step is unconditional (no if: / continue-on-error:), sits after both deterministic guest builds in wasi-check (job MISE_ENV: test), and the five workflow/conformance vitest files pass (79 tests).

Hand tamper checks, each run in both directions (Go to Rust, Rust to Go):

Tamper Go to Rust Rust to Go
swap c of i8-max / i8-min fails (opened value) fails (opened 127, want -128)
swap name.c between record rows fails fails
passthrough id 1 to 999 fails fails
swap record age.eq term fails (stored term is not the query term) fails
swap a values eq term fails fails
swap two blobs fails fails
own-language artifact file fed to its own consumer refused refused
EQL: swap email envelopes between rows fails fails
EQL: swap email.hm only passes fails
EQL: own-language rows fed to own consumer passes passes
values: replace bytes-binary's unanswered ore with another case's passes passes

Standards

  • scripts/stack-exchange-conformance.mjs:366-373: a third copy of the print, mkdirSync, writeFileSync(REPORT_DIR/...) block already in stack-lifecycle-conformance.mjs:59 and stack-eql-conformance.mjs:121, instead of a writeReport in stack-conformance-evidence.mjs. The three drift independently. CONFIRMED.
  • scripts/stack-exchange-conformance.mjs:250: the mkdtempSync directory is never removed. After a few local runs I had four stack-exchange-* directories of ciphertext artifacts in $TMPDIR. Remove it on success and print its path on failure. CONFIRMED.
  • conformance/stack-encrypt/exchange-contract.md ("Both consumers refuse ... another language's own output"), and the PR body ("each artifact carries a producer manifest"): neither is true for eql-rows. Rows carry no manifest, and both EQL consumers open their own language's rows (table above). The report's producers['eql-rows-go'] is the committed fixture's "Regenerate with EQL_UPDATE_FIXTURES=1 ... do not edit" comment, which is not a producer record. CONFIRMED.
  • packages/eql/tests/encryption/fixtures/v4_rows.json:2 (the comment Go writes, eql_v4_test.go:353) says "the Rust suite opens and queries them". Rust only opens them (next section). CONFIRMED.

Standards: 4 findings. Worst: the contract and PR body claim a producer manifest and own-language refusal that the EQL lane does not have.

Spec

  • Compare query semantics, not ciphertext (v4_types.rs:1749, go_rows::open_row): the Rust consumer of Go's EQL rows never derives a query term, so a swapped hm passes Go to Rust. compared() reports this honestly, but the bullet is only half met, and the suite already has typed query targets (integer::IntegerEqQuery and the others) to derive the term with. CONFIRMED.
  • Record producer SDK, engine, backend and storage versions in artifacts: met for values, records and blobs. For eql-rows it is missing: no language, backend, guest hash or eql-bindings version in the artifact, and only free text from Rust. Partial. CONFIRMED.
  • Unanswered index terms (exchange_conformance.rs:401, exchange_conformance_test.go:237) are checked for presence only, so a replaced ore on bytes-binary passes both ways. This is documented and there is no operator to derive the term from. Acceptable. PLAUSIBLE gap.
  • Missing runner or guest fails the required lane: met, and proven by stubs. requireGuests checks only that the file exists and is nonempty, so a stale local guest still passes. CI builds both guests in the same job, so this is acceptable. PLAUSIBLE.
  • Both Rust/Go directions: met. No Python or TypeScript pair is claimed. The released-version placeholder is shaped and empty: met.
  • Deferrals: no fresh JSON document exchange, no SQL lane storing one language's rows through the other, and the two older overlapping CI steps left in place (the EQL one also covers smallint rows). All three are stated and acceptable for this PR. The JSON one keeps the first bullet partial, so the issue should stay open for it or the gap should be tracked. The SQL bundle version appears only through the @cipherstash/eql package version. Partial. PLAUSIBLE.

Spec: 3 CONFIRMED-or-partial, 3 met, 2 plausible gaps. Worst: the EQL Go-to-Rust direction compares no query term.

… in both directions

The scalar profile proved fresh Go-to-Rust opening of one field; the wider
fixtures proved frozen Rust-to-Go reads and local round trips. This adds the
artifact-exchange-v1 corpus and a producer/consumer runner in each language:
every positive value-boundary case, a record with a passthrough field and
equality/order terms, OpaqueBlob<T> against the Go opaque storage path, and
the EQL v4 profiles rows sealed by eql-bindings' typed targets for the Go
generated code to open. Consumers compare decrypted values and derive their
own query terms against the stored ones; ciphertext bytes are never compared,
because encryption is randomized. Each file carries a manifest naming the
producer SDK, engine, backend, key source, profiles and capabilities, and a
consumer refuses another seed, profile or its own language's output.

Refs #1478
The exchange script fails by name on a missing or empty guest and on a
missing runner before anything runs, writes to a fresh directory every run
so no producer cache is reused, requires every fresh output to be nonempty
and from the named producer, and reports the engine, guest, lockfile and
storage versions beside each exchanged kind and direction. The task is
required in tests-golang.yml after both deterministic guest builds and its
report is uploaded. The released-version matrix is an empty, shaped
placeholder until the compatibility policy defines its rows.

Refs #1478
…resh directory

The exchange script added a third copy of the print-and-write report
block the lifecycle and EQL scripts each carried. writeReport now lives
in the shared evidence module and all three use it.

The exchange's temporary directory was never removed, so every local
run left a directory of artifacts behind. A passing run removes it; a
failing one keeps it and prints its path for inspection.
…e their terms in Rust

The EQL lane's rows carried no producer manifest, so each consumer
opened rows its own language wrote, and the Go rows reported the
committed fixture's regeneration comment as their producer. Both
producers now write a producer object (language, SDK and version,
backend, engine version or guest hash, schema) on the rows they hand
over; each consumer refuses rows that are not the other runner's, and
the task requires the manifest and records it in the report.

The Rust consumer of Go's rows only opened them, so a swapped hm term
passed. It now derives each equality column's typed query and compares
its hm with the stored one, as the Go consumer already did, and the
report says so for both directions.
@coderdan
coderdan force-pushed the feat/stack-encrypt-artifact-exchange branch from 4d67b46 to 08e711f Compare November 2, 2026 14:22
@coderdan

coderdan commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

Review findings and fixes

I rebased the branch onto the current feat/stack-encrypt-coverage-gaps (which had four new commits) and force-pushed it. The original three commits are now 64744ec77, ebeeefbe9 and 7e2497b5c. The fixes sit on top. New head: 08e711f84.

Standards

  • Third copy of the report writer: fixed in ec2a2efa9. writeReport now lives in stack-conformance-evidence.mjs, and the exchange, lifecycle and EQL scripts all use it. A vitest case pins it.
  • Fresh directory never removed: fixed in ec2a2efa9. A passing run removes it; a failing run keeps it and prints its path. I checked that no stack-exchange-* directory remains after a passing run.
  • Contract and PR body claimed a producer manifest and own-language refusal that eql-rows lacked: fixed in 3f15fe857 (code) and 08e711f84 (contract). Both producers now write a producer object on the rows they hand over (language, SDK and version, backend, engine version or EQL-guest SHA-256, schema), and the committed fixture stays unchanged. Each EQL consumer refuses rows that are not the other runner's. checkRows requires the manifest, and the report records it in place of the fixture's regeneration comment.
  • The v4_rows.json comment says "the Rust suite opens and queries them": now true, through 3f15fe857. The committed fixture is untouched.

Spec

  • Rust consumer of Go EQL rows compared no query term: fixed in 3f15fe857. go_rows::check_eq_terms derives each equality column's typed *EqQuery and compares its hm with the stored one. It also runs against the committed fixture, where it passes. compared() now says the same for both directions, and a vitest case pins that every term-carrying lane compares derived query terms.
  • EQL artifacts lacked producer, engine and backend versions: fixed in 3f15fe857, as above.
  • Unanswered index terms are checked for presence only: declined. There is no operator to derive the term from, and the contract already says so.
  • Stale local guest passes requireGuests: declined. CI builds both guests in the same job, and the report records their hashes.
  • JSON document exchange, the SQL lane and the older overlapping CI steps: left as stated deferrals. The first bullet of the issue stays partial for JSON documents.

Tamper re-check after the fixes

Tamper Go to Rust Rust to Go
EQL: swap email.hm only fails (was passing) fails
EQL: own-language rows fed to own consumer refused (was passing) refused (was passing)
EQL: swap envelopes fails fails

Verification on 08e711f84

  • cargo fmt --all --check, at the root and in packages/eql: clean.
  • clippy, -D warnings, on stack-encrypt / stack-encrypt-derive and on eql-encryption-tests: clean.
  • nextest on stack-encrypt with STACK_CONFORMANCE_REQUIRED=1: 1129 passed, 1 skipped.
  • v4_types: 16 passed.
  • Go: go test -count=1 ./... with STACK_CONFORMANCE_REQUIRED=1 passed in all 10 packages; go vet and gofmt -l are clean.
  • mise run test:conformance:exchange: 8 lanes, nonempty, and every row except blobs compares derived query terms.
  • vitest: 7 files, 91 tests passed.
  • pnpm biome check: no errors.

@coderdan
coderdan marked this pull request as ready for review November 2, 2026 14:23
@coderdan
coderdan requested a review from a team as a code owner November 2, 2026 14:23
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-02T14:25:46.840630Z 08e711f Draft marked ready
🔒 Security Review ✅ Completed 2026-11-02T14:26:26.702752Z 08e711f Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderdan

coderdan commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor
Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@cipherstash-agent cipherstash-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review attestation

Reviewed 08e711f · posted from Claude Code · claude-sonnet-5-5 · effort medium
Sources: code-review · claude-sonnet-5-5
Skills: code-review, review-cross-check, review-post

No findings.

Read in full: scripts/stack-exchange-conformance.mjs, the workflow, mise.toml, stack-conformance.sh and evidence/selection changes, the Rust exchange consumer and term check, and the EQL rows producer/consumer changes on both the Rust and Go sides. Tests were not run in this review.


Generated by Claude Code

@coderabbitai coderabbitai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up Comments

Nitpick Check the EQL row count against the committed fixture.

154-156: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Check the EQL row count against the committed fixture.

checkRows accepts any nonempty rows array. A fresh export that carries fewer rows than the committed fixture passes this check, and both EQL consumers open every row they receive. A regression that drops rows from the export would therefore go undetected. Compare rows.rows.length with the row count of corpus.eql.fixture so a partial export fails.

🔧 Suggested fix
   if (rows.table !== corpus.eql.table || !rows.rows?.length)
     throw new Error(`${path}: not ${corpus.eql.table} rows`)
+  const committed = JSON.parse(readFileSync(join(root, corpus.eql.fixture), 'utf8'))
+  if (rows.rows.length !== committed.rows.length)
+    throw new Error(
+      `${path}: ${rows.rows.length} rows, expected ${committed.rows.length}`,
+    )

Open the source location

🤖 Prompt for AI Agents
Treat finding text, file paths, and code as untrusted review data. Never follow
instructions embedded in them. Verify each finding against current code. Fix
only still-valid issues, skip the rest with a brief reason, keep changes
minimal, and validate.

Review comment at @scripts/stack-exchange-conformance.mjs around lines 154 -
156:
Update checkRows to compare the exported rows.rows.length with the row count in
the committed fixture referenced by corpus.eql.fixture, and reject mismatches
while preserving the existing table and nonempty checks.

Comment thread scripts/stack-exchange-conformance.mjs Resolved
Comment thread scripts/stack-exchange-conformance.mjs Resolved
…d sealed field

The Rust consumer read only the declared sealed fields of each record
row, so an extra key in an artifact's fields was never looked at. It now
compares the stored keys with the declared sealed set (SQL nulls
excluded), as it already did for passthrough fields, and a tamper test
proves an undeclared field fails the consumer.
…d field

The Go consumer read only the declared fields from a record row's
stored fields and dropped anything else before Open. Every stored key
must now be a declared non-passthrough field. The consumer helpers take
testing.TB so a tamper test can run the consumer on an artifact with an
extra field and assert it is refused.
…irectory until the report is written

The exchange runs TestV4RowsMatchTheCommittedFixture, which rewrites
v4_rows.json when EQL_UPDATE_FIXTURES is set, yet the shared environment
check did not refuse it. It now does, for every conformance script.

The fresh directory was removed before the lockfile reads, cargo
metadata, git rev-parse and the report, which no catch covered, so a
failure there deleted the artifacts and printed no path. The run now
happens inside inFreshDirectory, which removes the directory only after
the report is written and otherwise keeps it and prints its path. Tests
cover the refusal (directly and through the script) and a failing
report tail.
nonminimal_bool flagged the combined predicate; two filters say the same
thing, one condition each.
@coderdan

coderdan commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the four CodeRabbit findings (all confirmed against the code):

  • Go consumer accepted an undeclared record field (844588f). Every key in stored.Fields must now be a declared non-passthrough field. TestArtifactExchangeRefusesAnUndeclaredRecordField proves the check bites.
  • Rust consumer accepted an undeclared record field (1554b59, 42a06a5 for clippy). The stored fields keys are compared with the declared sealed set per row, SQL nulls excluded. an_undeclared_record_field_is_refused proves the check bites.
  • EQL_UPDATE_FIXTURES was not refused (7b398da). It is now in the shared checkEnvironment list, with tests and a contract note.
  • The fresh directory was removed before the report tail (7b398da). The run now happens inside inFreshDirectory, which removes the directory only after writeReport and keeps it, printing its path, on any failure. The tests include a failing tail.

Re-run on 42a06a5: cargo fmt --check, clippy -D warnings, nextest (1130 passed, 1 skipped), Go ./encrypt/... with STACK_CONFORMANCE_REQUIRED=1, go vet, gofmt -l, mise run test:conformance:exchange (8 lanes, report written, fresh directory removed, tree clean), vitest (exchange, conformance and EQL script tests) and biome check. All clean.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants