Skip to content

fix(test): harden portable offline test integrity - #694

Open
mogul wants to merge 1 commit into
mainfrom
fix/portable-test-integrity
Open

mogul wants to merge 1 commit into
mainfrom
fix/portable-test-integrity

Conversation

@mogul

@mogul mogul commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Context

Shared host/microVM mounts can lose local executable bits while Git still records
the intended mode. Offline Bats/pre-commit runs also retained a controlling
terminal after detaching stdin, allowing installer fixtures to consume a human
response through /dev/tty. macOS exposed additional fixture and provider-input
path portability defects that Linux-only checks did not catch.

These concerns cross the same test boundary: the runner executes the installer,
provider projection depends on secure pre-bootstrap path handling, and tests must
validate the Git metadata a fresh checkout receives. This PR integrates those
protections as one coherent test-integrity change.

Changes

  • Add scripts/validate-git-executable-modes, a read-only stage-0 Git-index
    check requiring intended direct entrypoints to be 100755.
  • Run the index-mode check from pre-commit and npm run validate:exec-bits.
  • Document the distinction between release-integrity validation and local
    shared-mount repair via scripts/fix-exec-bits.
  • Run offline Bats tests with explicit noninteractive installer behavior so
    consent-gated actions cannot borrow /dev/tty during pre-commit.
  • Make provider-input canonicalization, safe path traversal, materialization,
    publication, and cleanup use one resolved, supported host Python interpreter.
  • Preserve macOS native aliases and supported Windows runtime path forms while
    retaining component-by-component O_NOFOLLOW traversal.
  • Harden runner/provider fixtures against inherited host command discovery,
    terminal interaction, double-slash temporary paths, and repository-local
    TMPDIR transport staging.

Verification

macOS host

PASS  bash scripts/fix-exec-bits
PASS  npm run validate:exec-bits
PASS  ACQ_BATS_JOBS=1 bash scripts/test-acq-bats \
      test/bats/01-test-runner.bats \
      test/bats/05-install.bats \
      test/bats/147-provider-input-projection.bats \
      test/bats/148-provider-credential-bindings.bats \
      test/bats/150-exec-bits.bats
      191/191 tests passed
PASS  ShellCheck on changed runner, installer, backend, provider, and Bats files
PASS  git diff --cached --check

The run included macOS path-alias, restricted-PATH validator, terminal prompt,
provider-projection, credential, and vendored-Bats mode coverage. No host MSB,
Homebrew, VM, or credential operation was invoked by the tests.

Additional review

  • Focused Linux Bats suites passed before host validation.
  • A bounded pseudo-terminal full-suite run reached all 1,094 tests without the
    prior installer prompt or terminal hang.
  • Independent adversarial review found no blockers after remediation.

Known limitation

The full offline suite still has a pre-existing MSYS fixture failure outside this
change. This PR does not claim a clean full-suite result. Local Bash 3.2 execution
was not available; the existing CI Bash 3.2 shard remains the compatibility gate.

Security impact

  • Strengthens release integrity by checking Git-recorded executable modes instead
    of trusting mount-specific filesystem metadata.
  • Prevents offline test fixtures from reading a developer's controlling terminal
    for installer consent.
  • Preserves fail-closed provider-input path security and adds explicit failure
    when a suitable pre-bootstrap Python interpreter is unavailable.
  • No secrets, network allowlists, production deployments, or authorization policy
    are changed.

Rollback

Revert this PR. The executable gate, noninteractive test-runner behavior, and
provider-input portability changes are self-contained.

AI assistance

Implementation and review were AI-assisted. The macOS validation was performed
by the human operator. Human review is required before production use.

@mogul
mogul requested a review from a team as a code owner November 13, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant