Skip to content

Fix selected plans rehashing unselected external models - #6097

Open
mday-io wants to merge 1 commit into
SQLMesh:mainfrom
mday-io:mday/preserve-unselected-external-hashes
Open

mday-io wants to merge 1 commit into
SQLMesh:mainfrom
mday-io:mday/preserve-unselected-external-hashes

Conversation

@mday-io

@mday-io mday-io commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Why

Schema refresh during a selected plan can recompute cached fingerprints for an unselected external co-parent sourced from deployed state. When the persisted fingerprint differs from a fresh calculation, the plan can report external and downstream changes outside the selection.

How

Preserve the deployed data and metadata hashes for unselected external models in the schema-update DAG when their serialized definitions remain unchanged. Selected external edits, selected dependency schema changes and non-external models keep the existing behavior.

The regression deliberately injects inconsistent persisted data and metadata hashes. It proves this preservation invariant; it does not claim that a normal SQLMesh migration creates the mismatches.

Test Plan

  • The regression fails on current upstream without the fix.
  • Removing either hash-restoration assignment independently makes the regression fail.
  • Native and dbt selector suites passed: 48 tests.
  • Selected-plan integration cases passed: 3 tests.
  • make style passed all four hooks.
  • make fast-test passed all six phases: 2,947 passed and 13 skipped, with four workers.
  • Complete CI on the refreshed commit.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

Summary by CodeRabbit

  • Bug Fixes
    • Selecting an external model no longer incorrectly marks unchanged, unselected external models or their dependent models as modified after a schema update.
    • Changes to selected external models and their dependencies continue to be reflected when their schemas change.

@mday-io
mday-io force-pushed the mday/preserve-unselected-external-hashes branch from 32bb25f to 6731332 Compare September 25, 2026 21:26
Signed-off-by: mday-io <mdaytn@gmail.com>
@mday-io
mday-io force-pushed the mday/preserve-unselected-external-hashes branch from 6731332 to aeb4097 Compare October 31, 2026 17:00
@coderabbitai

coderabbitai Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 741f77fd-8d18-42b9-8dd9-8146c90053f7
Commits

Reviewing files that changed from the base of the PR and between 966e34d and aeb4097.

Files selected for processing: 2
  • sqlmesh/core/selector.py
  • tests/core/test_selector_native.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

After schema updates, select_models restores deployed hashes for unselected external models only when their serialized definitions remain unchanged. Native-selector tests cover fingerprint preservation and direct or indirect modification behavior after external model schema changes.

Changes: External Model Schema Refresh

Layer / File(s) Summary
Preserve deployed hashes
sqlmesh/core/selector.py
select_models records serialized definitions and deployed hashes for unselected external models before updating schemas. It restores those hashes only when a model remains present and its serialized definition is unchanged.
Selector test coverage
tests/core/test_selector_native.py
Tests cover preserved fingerprints for an unselected external coparent and direct or indirect modification behavior when a selected external model's schema changes.

Estimated code review effort: 2 (Simple) | ~15 minutes

Change: Bug fix

Merge Risk: Minimal · up to aeb40

No merge-blocking issue is established from the supplied evidence.

Security architecture risk: Low · up to aeb40

The change narrowly preserves deployed identities during selected planning. No new access or authority was identified, but interrupted or overlapping planning remains less well established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The demonstrated effect is on planning identity within the resolved target or existing fallback environment's model graph, including unselected external co-parents and descendant fingerprints. The changed path does not introduce another environment-resolution mechanism.

Trust Boundaries and Controls

  • observed — Unselected model definitions continue to come from the configured state reader, while selected definitions come from the local model collection. Preserved hashes are captured from the environment-sourced objects, not from an unselected local edit.

Resilience and Maintainability Implications

  • inferred — Refresh is not transactionally isolated from cached environment objects: cached snapshots return their model nodes directly, and schema processing mutates hashes before all work completes. An exception skips the new restoration loop. This ownership and partial-failure exposure predates the PR; the available comparison does not establish a worsened security outcome.
Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely describes the primary change: preventing selected plans from rehashing unselected external models.
Description check Passed The description includes the required Description, Test Plan, and Checklist sections. It explains the cause, implementation, scope, regression coverage, and reported test results. The unchecked CI ite…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description includes the required Description, Test Plan, and Checklist sections. It explains the cause, implementation, scope, regression coverage, and reported test results. The unchecked CI item is a non-critical remaining status item and does not make the description incomplete.

Full details: Docstring Coverage

Explanation

Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 4 functions across 2 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@mday-io
mday-io marked this pull request as ready for review October 31, 2026 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant