Skip to content

feat(standard-tests): check the cursor state of declarative sources in test_basic_read - #1270

Merged
Anatolii Yatsuk (tolik0) merged 8 commits into
mainfrom
tolik0/cdk/per-partition-state-checks
Nov 13, 2026
Merged

Anatolii Yatsuk (tolik0) merged 8 commits into
mainfrom
tolik0/cdk/per-partition-state-checks

Conversation

@tolik0

@tolik0 Anatolii Yatsuk (tolik0) commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/18410

What

test_basic_read now checks the cursor state that its read already emits, for a ConcurrentDeclarativeSource. It adds no extra read. The declarative factory builds the same cursor for a full refresh read as for an incremental sync, so this state is the one an incremental sync would emit.

Enforced checks (the test fails):

check what fails
checkpoints do not skip records emitted after them a record emitted after a state message whose resume would start past the record's cursor value, with no copy of it (same primary key, cursor value at least as high) emitted before that state. A record below the start of the slice that returned it counts from that start, because a resumed sync that requests the slice again reads it again (not under is_client_side_incremental, which drops records by value). A state past the cursor's end (a record dated in the future) bounds the resume at the end of its last slice, less the lookback window. A StateDelegatingStream is built from each state message to find the stream a resume would use; a state past api_retention_period reruns full_refresh_stream in full and flags nothing. A stream whose records cannot be matched to the read's slices is not checked (advisory note instead)
final state is not behind the newest record the final state of a ConcurrentCursor stream has no cursor value, one the cursor cannot parse, or one behind the newest record at the precision of the cursor's datetime_format
per-partition state is well formed use_global_cursor not true although the manifest sets global_substream_cursor, a states entry without a partition or cursor value, entries of the same parent stream with different partition keys (each parent of a multi-parent router may use its own partition_field), a global state without a cursor value, a lookback_window that is not a whole number of seconds between 0 and the read's duration
parent state is kept a stream with incremental_dependency has no parent_state for its parent
listed partitions have state a partition of a ListPartitionRouter has no states entry
state round trip the source cannot build its streams from its own final state: it raises, sends an HTTP request while building, or loads fewer partitions than the state holds

Advisory checks (CursorStateWarning, the test passes): records without a cursor value or with one the cursor cannot parse, and checks that could not run. Both found nothing in the fleet run. They stay advisory because the first depends on sandbox data (updated_at: null is normal) and the second points to a gap in the checker, not in the connector.

validate_state_messages: false in a basic_read entry turns these checks off, along with the state message check from 7.42.0. Such a config, and a config not listed under basic_read, reads without the hooks and with the production state throttle. ConcurrentPerPartitionCursor gets a STATE_EMIT_INTERVAL_SECONDS class attribute, so the suite can lift its 600 s state throttle for the first per-partition closes and check checkpoints there too. Failure messages give counts and field names, never record or state values, because CI logs are public.

Fleet dry run

184 connectors (177 manifest-only, 7 Python), against airbytehq/airbyte master ab1b47bce77, with this branch (d2a489d9) installed. 103 scenarios reached the checks; the rest already fail or skip test_basic_read on master (mostly expired credentials). No check crashed, and no test passed without the hooks but failed with them. 79 per-partition streams in 11 connectors ran with the lifted throttle.

check connectors flagged
checkpoints do not skip records emitted after them 1
every other check, enforced or advisory 0
connector stream(s) flagged records verdict
source-stripe customers, payment_intents, prices, plans, charges, invoices, products, setup_intents + 9 more 1,343 / 96 / 78 / 72 / 53 / 39 / 38 / 30 / ... real loss: resuming an interrupted first sync switches StateDelegatingStream to /events, which returns only the last 30 days

Compared with the first dry run (CDK at the first head of this PR, master fa95775cb59):

connector first run this run
source-lever-hiring fail on checkpoints (2 records) pass: the API returns records whose updatedAt is before updated_at_start, and the checker now counts a record from the start of its own slice
source-google-analytics-data-api fail on checkpoints (48 records) pass: yearWeek values are coarser than the read windows; same reason
source-calendly not in the population pass
source-aircall, source-slack fail before the checks pass
source-youtube-analytics pass fail: the playlist_*_a2 streams return no records, and the same without the hooks; the cursor checks found nothing
source-amazon-ads, source-nexus-datasets pass fail before the checks: 429 and "Dataset is not ready", the same without the hooks
source-fleetio, source-mntn, source-posthog pass timed out; see below

The GA4 run in the first dry run also exposed a real bug in that connector: yearWeek 202605 was parsed as 2026-05-01, which moves the state months ahead of the data in the first weeks of every year.

Not verified locally; these connectors run the checks in CI on release:

  • 20 manifest-only connectors with rotating refresh tokens, whose test secrets are single-use outside CI. Four of them have a per-partition stream with step, so the lifted checkpoint check applies: gitlab, pinterest, zendesk-support, zoom. Two use incremental_dependency: gong, pipedrive. Two use a ListPartitionRouter: greenhouse, trustpilot.
  • 10 connectors whose read timed out locally (2,700 s for manifest-only, 3,600 s for posthog): amplitude, coingecko-coins, devin-ai, fleetio, google-search-console, mailersend, microsoft-teams, mntn, posthog, rocket-chat. fleetio's sandbox has grown: its fuel_entries stream alone read more than 55,000 records in 9 minutes without the hooks. mntn and rocket-chat hang on retries (HTTP 500 and dropped connections; an SSL error on the sandbox host). posthog hangs on dropped connections in persons.
  • nasa and amazon-ads (rate limited locally), nexus-datasets (dataset not ready).
  • 4 Python sources that could not run locally: asana, freshcaller, google-ads, jina-ai-reader.

Slowest check: 5.2 s on a 277 s read (source-hubspot).

The review fixes in de7d1c5d only remove flags (retention fallbacks, multi-parent keys, unmatched slices), so this run still holds. Stripe's default config sets no api_retention_period, so it stays flagged.

Merge order

CI installs the CDK unpinned, and a CDK release ships every PR merged to main, so this PR stays in draft until this connector PR merges:

Before this PR is marked ready, a pre-release of this head runs connector CI on the 34 connectors the dry run did not cover (above), starting with gitlab, pinterest, zendesk-support and zoom, where the lifted checkpoint check applies.

lever-hiring and GA4 now pass without an opt-out, so airbytehq/airbyte#91056 (lever-hiring opt-out) and the opt-out in airbytehq/airbyte#91057 no longer gate this PR. The GA4 yearWeek/yearMonth parsing fix in #91057 stands on its own.

Conflicts with open CDK PRs

git merge-tree of this head against each PR's head, leaving out conflicts they already have with main:

No order is required; the conflicts are textual and the second PR to merge resolves them.

Testing

  • unit_tests/test/test_cursor_state_checks.py: 87 tests; unit_tests/test: 1225 passed. The checks run without network access in the tests (_no_http).
  • Mutation pass: 70 mutations in the first round (68 killed, 2 equivalent: a deepcopy of a state that is loaded only once, and >= vs > when pushing equal bounds onto a monotone stack), then 19 for the review fixes (rows 70-88: 18 killed, 1 equivalent). Two guards that survived were removed rather than kept: a redundant check_checkpoints = False (the stream has no slices left to check) and a parse guard for a slice start the cursor formatted itself. Each was applied alone to a scratch copy and run against the test file. Rows 11, 12, 25, 26, 35, 36 and 62 were re-run against the current code. Rows 89-104 cover the second review round; rows 30 and 31 no longer apply, because the final-state cursor template they mutated was removed (only StateDelegatingStream needed it, and it now builds the stream per state message). In the third review round, row 101 was corrected and rows 105-107 added: two more guards that no test failed without, and the test helper that edits the final state.
Mutations
# mutation result failing tests
0 recorder: install returns None for non-CDS -> hooks anything KILLED test_other_sources_are_not_checked
1 uninstall: keep streams hook KILLED test_basic_read_leaves_no_hooks_on_the_source
2 uninstall: keep _handle_item hook KILLED test_basic_read_leaves_no_hooks_on_the_source
3 lift only with step: always lift KILLED test_checkpoint_check_does_not_flag_a_per_partition_stream_without_step
4 lift only with step: never lift KILLED test_checkpoint_check_flags_a_per_partition_stream_with_step, test_throttle_lift_emits_more_per_partition_states_within_budget, test_lifted_closes_thin_out_after_the_first_ones
5 _has_step: ignore timedelta.max KILLED test_checkpoint_check_does_not_flag_a_per_partition_stream_without_step
6 throttle: never lift (interval stays default) KILLED test_throttle_lift_emits_more_per_partition_states_within_budget, test_lifted_closes_thin_out_after_the_first_ones
7 throttle: always lift (no budget) KILLED test_throttle_lift_emits_more_per_partition_states_within_budget, test_lifted_closes_thin_out_after_the_first_ones
8 throttle: drop MAX_LIFTED_CLOSES KILLED test_throttle_lift_emits_more_per_partition_states_within_budget
9 throttle: drop FIRST closes KILLED test_throttle_lift_emits_more_per_partition_states_within_budget
10 throttle: drop growth KILLED test_lifted_closes_thin_out_after_the_first_ones
11 _handle: do not record slices KILLED test_checkpoint_check_flags_a_per_partition_stream_with_step, test_per_partition_records_below_their_slice_start_are_not_flagged, test_records_below_their_slice_start_are_not_flagged, test_unmatched_partitions_skip_the_per_partition_checkpoint_check
12 _slice_of: partition None KILLED test_checkpoint_check_flags_a_per_partition_stream_with_step
13 _resume_start: raw state instead of slice start KILLED test_checkpoint_bound_comes_from_the_resumed_cursor, test_checkpoint_bound_of_a_state_delegating_stream_comes_from_its_incremental_stream
14 _sub_cursor: pass runtime lookback 0 -> original state no deepcopy SURVIVED
15 bound cap: no fallback to earlier bound KILLED test_bounds_past_the_cap_use_the_bound_of_an_earlier_state
16 bound cap: ignore cap KILLED test_bounds_past_the_cap_never_flag_more
17 monotone push: push every state KILLED test_a_state_that_moves_back_does_not_lower_the_bound
18 monotone push: >= instead of > SURVIVED
19 checkpoint flag: value <= bound KILLED test_a_record_at_the_resume_start_is_not_flagged
20 dedupe: no copies exemption KILLED test_checkpoint_check_does_not_flag_overlapping_slices[api_ignores_until], test_checkpoint_check_does_not_flag_overlapping_slices[no_end_option], test_checkpoint_check_compares_whole_records_without_a_primary_key
21 dedupe: copy value > instead of >= KILLED test_checkpoint_check_does_not_flag_overlapping_slices[api_ignores_until], test_checkpoint_check_does_not_flag_overlapping_slices[no_end_option], test_checkpoint_check_compares_whole_records_without_a_primary_key
22 dedupe: ignore copy position KILLED test_a_state_that_moves_back_does_not_lower_the_bound, test_a_copy_after_the_first_skipping_state_does_not_exempt_a_record
23 binary search: use last checkpoint KILLED test_a_copy_after_the_first_skipping_state_does_not_exempt_a_record
24 copy_key: no-PK uses empty list KILLED test_checkpoint_check_compares_whole_records_without_a_primary_key
25 slice count mismatch: keep the slices KILLED test_records_that_do_not_match_the_read_are_checked_by_their_own_value
26 partitioned slice count mismatch: not reported KILLED test_unmatched_partitions_skip_the_per_partition_checkpoint_check
27 scope: per-partition checkpoints always KILLED test_checkpoint_check_does_not_flag_a_per_partition_stream_without_step, test_suites_whose_read_builds_another_source_get_the_concurrent_cursor_part, test_final_state_check_skips_per_partition_streams
28 scope: per-partition checkpoints without lift KILLED test_checkpoint_check_does_not_flag_a_per_partition_stream_without_step, test_final_state_check_skips_per_partition_streams, test_lookback_window_up_to_the_read_duration_rounded_up_passes
29 scope: isinstance instead of type for ConcurrentCursor KILLED test_custom_cursor_subclasses_are_skipped
30 template: never from rebuilt KILLED test_checkpoint_bound_of_a_state_delegating_stream_comes_from_its_incremental_stream
31 template: any rebuilt cursor KILLED test_a_rebuilt_stream_without_a_concurrent_cursor_keeps_the_read_cursor
32 rebuild: no per-stream fallback KILLED test_a_final_state_the_source_cannot_load_fails, test_a_build_that_sends_http_fails
33 rebuild: no _no_http KILLED test_a_build_that_sends_http_fails
34 rebuild: HTTP reported as generic KILLED test_a_build_that_sends_http_fails
35 record safety net removed KILLED test_a_check_that_crashes_is_reported_as_advisory[add_record], test_a_crash_in_the_checkpoint_walk_keeps_the_other_checks[_resume_start]
36 state safety net removed KILLED test_a_check_that_crashes_is_reported_as_advisory[add_state], test_a_crash_in_the_checkpoint_walk_keeps_the_other_checks[_add_checkpoint]
37 report safety net removed KILLED test_a_check_that_crashes_is_reported_as_advisory[add_checkpoint_report]
38 final state: no precision formatting KILLED test_final_state_is_compared_at_the_state_precision
39 final state: <= instead of < KILLED test_checkpoint_check_flags_records_past_their_slice, test_checkpoint_check_passes_a_correct_api, test_checkpoint_check_does_not_flag_overlapping_slices[api_ignores_until]
40 final state: never flags behind KILLED test_final_state_behind_the_newest_record_fails
41 final state: missing key not flagged KILLED test_final_state_without_the_cursor_field_fails
42 final state: unparseable not flagged KILLED test_final_state_the_cursor_cannot_parse_fails
43 final state: also for partitioned KILLED test_checkpoint_check_flags_a_per_partition_stream_with_step, test_checkpoint_check_does_not_flag_a_per_partition_stream_without_step, test_suites_whose_read_builds_another_source_get_the_concurrent_cursor_part
44 cursor values: missing not counted KILLED test_missing_and_unparseable_cursor_values_are_advisory, test_global_state_is_not_required_when_no_record_has_a_cursor_value
45 cursor values: unparseable not counted KILLED test_missing_and_unparseable_cursor_values_are_advisory
46 cursor values: enforced KILLED test_missing_and_unparseable_cursor_values_are_advisory, test_basic_read_warns_on_an_advisory_cursor_state_problem
47 shape: global_at_start ignored KILLED test_global_substream_cursor_must_be_used
48 shape: entry without partition ignored KILLED test_malformed_per_partition_state_fails[entry_without_partition]
49 shape: entry without cursor ignored KILLED test_malformed_per_partition_state_fails[entry_without_cursor]
50 shape: empty partition allowed KILLED test_malformed_per_partition_state_fails[entry_without_partition]
51 shape: key sets ignored KILLED test_malformed_per_partition_state_fails[different_partition_keys]
52 shape: global state ignored KILLED test_global_state_must_hold_the_cursor
53 shape: global state checked without values KILLED test_global_state_is_not_required_when_no_record_has_a_cursor_value
54 shape: bool lookback allowed KILLED test_malformed_per_partition_state_fails[bool]
55 shape: lookback upper bound ignored KILLED test_malformed_per_partition_state_fails[lookback_longer_than_the_read], test_lookback_window_up_to_the_read_duration_rounded_up_passes
56 shape: lookback lower bound ignored KILLED test_malformed_per_partition_state_fails[negative]
57 shape: lookback floor instead of ceil KILLED test_lookback_window_up_to_the_read_duration_rounded_up_passes
58 parent: never flagged KILLED test_missing_parent_state_fails, test_parent_state_is_required_behind_a_grouping_router
59 parent: incremental_dependency ignored KILLED test_parent_state_is_not_required_without_incremental_dependency
60 parent: grouping not unwrapped KILLED test_parent_state_is_required_behind_a_grouping_router
61 coverage: never flagged KILLED test_listed_partition_without_state_fails
62 coverage: any router KILLED test_partition_coverage_is_only_checked_for_a_list_router, test_suites_whose_read_builds_another_source_get_the_concurrent_cursor_part
63 round trip: count never compared KILLED test_a_cursor_that_drops_partitions_on_load_fails
64 round trip: build errors not reported KILLED test_a_final_state_the_source_cannot_load_fails, test_a_build_that_sends_http_fails
65 suite: opt-out ignored KILLED test_basic_read_fails_on_an_enforced_cursor_state_problem
66 suite: advisory not warned KILLED test_basic_read_warns_on_an_advisory_cursor_state_problem
67 suite: enforced not asserted KILLED test_basic_read_fails_on_an_enforced_cursor_state_problem
68 suite: no uninstall KILLED test_basic_read_leaves_no_hooks_on_the_source
69 suite: recorder not passed KILLED test_basic_read_fails_on_an_enforced_cursor_state_problem, test_basic_read_warns_on_an_advisory_cursor_state_problem
70 _resume_start: None when the resumed cursor has no slice KILLED test_a_future_dated_record_does_not_silence_the_checkpoint_check[0], [5], test_a_resume_past_the_end_starts_a_lookback_window_before_the_state
71 past-end bound: no lookback window KILLED test_a_resume_past_the_end_starts_a_lookback_window_before_the_state
72 no slice-start exemption KILLED test_records_below_their_slice_start_are_not_flagged, test_per_partition_records_below_their_slice_start_are_not_flagged
73 slice-start exemption also under is_client_side_incremental KILLED test_records_below_their_slice_start_are_flagged_under_client_side_incremental, test_client_side_incremental_comes_from_the_incremental_stream_of_a_state_delegating_stream
74 client-side incremental streams: none found KILLED same two tests
75 client side read from the top-level definition only KILLED test_client_side_incremental_comes_from_the_incremental_stream_of_a_state_delegating_stream
76 partition taken from the slice for unpartitioned streams KILLED 11 tests, e.g. test_checkpoint_check_flags_records_past_their_slice
77 slice cache reused across slices KILLED test_records_below_their_slice_start_are_not_flagged, test_per_partition_records_below_their_slice_start_are_not_flagged, test_basic_read_bounds_the_lookback_window_by_the_read_duration
78 _slice_of: no slice start KILLED test_records_below_their_slice_start_are_not_flagged, test_per_partition_records_below_their_slice_start_are_not_flagged
79 a crash drops the stream from every check KILLED test_a_crash_in_the_checkpoint_walk_keeps_the_other_checks[_resume_start], [_add_checkpoint]
80 add_state keeps collecting checkpoints after a crash KILLED test_a_crash_in_the_checkpoint_walk_keeps_the_other_checks[_add_checkpoint]
81 suite: hooks installed whatever the scenario KILLED test_basic_read_installs_no_hooks_when_it_does_not_check_the_cursor_state[opted_out], [not_a_basic_read_config]
82 suite: hooks installed when opted out KILLED test_basic_read_installs_no_hooks_when_it_does_not_check_the_cursor_state[opted_out]
83 suite: reference_source=None KILLED test_basic_read_checks_a_suite_whose_read_builds_another_source, test_basic_read_warns_when_the_reference_source_cannot_build
84 suite: read_seconds = 0.0 KILLED test_basic_read_bounds_the_lookback_window_by_the_read_duration, test_basic_read_checks_a_suite_whose_read_builds_another_source
85 suite: uninstall only after a successful read KILLED test_basic_read_removes_its_hooks_when_the_read_fails
86 reference path: global_substream_cursor flags dropped KILLED test_basic_read_checks_a_suite_whose_read_builds_another_source
87 reference path: build error returns no problems KILLED test_basic_read_warns_when_the_reference_source_cannot_build
88 slice start compared with >= SURVIVED equivalent: on a tie both give the record's value
89 StateDelegatingStream: bound from the final-state stream (no per-checkpoint build) KILLED test_checkpoint_bound_of_a_state_delegating_stream_follows_its_retention_period[states_past_retention-*], test_rebuilding_a_state_delegating_stream_emits_nothing_on_the_read_queue, test_a_resume_without_a_concurrent_cursor_reads_in_full, test_rebuilds_of_a_state_delegating_stream_past_the_cap_never_flag_more
90 bound ignores a cleared state KILLED ...follows_its_retention_period[states_past_retention-per_partition]
91 build never reports a cleared state KILLED same
92 round trip compares a rebuild that cleared the state KILLED same
93 build emits on the read's queue (no repository swap) KILLED test_rebuilding_a_state_delegating_stream_emits_nothing_on_the_read_queue
94 no cap on builds KILLED test_rebuilds_of_a_state_delegating_stream_past_the_cap_never_flag_more
95 shape: any two key sets differ KILLED test_partition_keys_are_compared_per_parent[one_key_set_per_parent]
96 shape: entries without a parent field ignored KILLED test_malformed_per_partition_state_fails[different_partition_keys], test_partition_keys_are_compared_per_parent[no_parent_field]
97 shape: differing keys of one parent ignored KILLED test_partition_keys_are_compared_per_parent[different_keys_for_one_parent]
98 unmatched slices: unpartitioned stream checked by value KILLED test_records_that_do_not_match_the_read_skip_the_checkpoint_check, test_suites_whose_read_builds_another_source_skip_the_checkpoint_check
99 read on another source: checked by value KILLED test_suites_whose_read_builds_another_source_skip_the_checkpoint_check
100 StateDelegatingStream bounds shared by equal partition states KILLED test_equal_partition_states_in_different_state_messages_resume_apart
101 per-partition resume bound taken from the read's cursor (full_refresh_stream) instead of the built one KILLED test_per_partition_state_delegating_bound_comes_from_its_incremental_stream (survived until that test: the retention tests use the same definition for both sub-streams)
102 resumed stream without a concurrent cursor bounded by the read's cursor KILLED test_a_resume_without_a_concurrent_cursor_reads_in_full
103 no cap on StateDelegatingStream bounds KILLED test_bounds_past_the_cap_never_flag_more[state_delegating]
104 final-state cursor template kept (removed code) n/a 83 tests pass with or without it, so it was removed
105 shape: an entry whose cursor lacks the cursor field passes KILLED test_partition_entry_whose_cursor_lacks_the_cursor_field_fails
106 parent state: a parent_state without an incremental parent passes KILLED test_parent_state_without_the_incremental_parent_fails
107 test helper: with_final_state edits the first state message equal to the final one KILLED test_with_final_state_edits_the_last_state_message

Not checked

Streams without a ConcurrentCursor or ConcurrentPerPartitionCursor (no cursor, FinalStateCursor, custom or file-based cursors) are skipped, never failed. Sources that are not a ConcurrentDeclarativeSource are not checked. The Docker read (test_docker_image_build_and_read) is not checked either, because the checks need the source object in-process to rebuild cursors. This deviates from the issue, which expects the Docker read to get the ConcurrentCursor checks.

🤖 Generated with Claude Code

…n test_basic_read

test_basic_read now checks the cursor state the read already emits for a
ConcurrentDeclarativeSource: checkpoints that a resume would skip records
from, a final state behind the newest record, per-partition state shape,
parent state, partition coverage and a round trip through the cursor.
Cursor values and check errors are reported as CursorStateWarning.
`validate_state_messages: false` turns the checks off.
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@tolik0/cdk/per-partition-state-checks#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch tolik0/cdk/per-partition-state-checks

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

PyTest Results (Fast)

6 581 tests  +147   6 545 ✅ +148   7m 0s ⏱️ +33s
    1 suites ±  0      36 💤  -   1 
    1 files   ±  0       0 ❌ ±  0 

Results for commit 0de689c. ± Comparison against base commit 314f8fb.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

PyTest Results (Full)

6 584 tests  +147   6 548 ✅ +148   11m 37s ⏱️ -1s
    1 suites ±  0      36 💤  -   1 
    1 files   ±  0       0 ❌ ±  0 

Results for commit 0de689c. ± Comparison against base commit 314f8fb.

♻️ This comment has been updated with latest results.

…ion-state-checks

# Conflicts:
#	airbyte_cdk/test/standard_tests/source_base.py
…or state checks

- A state past the cursor's end (a record dated in the future) no longer turns the
  checkpoint check off: a cursor with no slice resumes from the end of its last state
  slice, less the lookback window.
- A record below the start of the slice that returned it counts from that start, since a
  resumed sync that requests the slice again reads it again. Streams with
  `is_client_side_incremental` still compare the record's own value.
- `test_basic_read` installs the hooks only on a read whose cursor state it checks.
- A crash while walking a stream's records or states stops only that stream's checkpoint
  check; the final-state, shape, parent, coverage and round-trip checks still run.
- Tests pin the suite wiring (read duration, reference source, hook removal after a failed
  read) and run the checks without network access.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for one soundness issue in the checkpoint model (inline, 🔴), plus three medium notes (🟠).

What's done well: no second read, resume starts computed with the CDK's own cursor code (gaps, lookback, clamping all come out right), failure messages carry no values, and the tests are real end-to-end reads. I collected and ran the 71 tests, re-applied mutations #19, #72 and #59 (all killed), and the survivors #14, #18 and #88 are equivalent. Reverting the throttle to the literal 600 fails two tests, so the production refactor is covered and behaviour-identical. No runtime, Builder, SDM or manifest-server impact; feat: matches the 7.36/7.40/7.42 precedent.

Rollout: Check: destination-motherduck is red on #1266–#1273 as well, so it's infra. Merge order with airbytehq/airbyte#91061 is as described.

…h the stream its state selects

- Build a `StateDelegatingStream` from each state message to get that checkpoint's resume
  start, instead of from the final state. A state past `api_retention_period` reruns
  `full_refresh_stream` in full, so it flags nothing. Builds are capped at 200 per stream and
  no longer put their cleared state on the read's queue.
- Skip the round-trip partition count when the build from the final state cleared it.
- Compare partition keys per parent `partition_field`, so a router with several parents that
  use different fields passes.
- Skip the checkpoint check, with an advisory `CHECK_ERRORS` line, when a stream's records
  cannot be matched to the read's slices or the read ran on another source instance, instead
  of checking each record from its own value.
- Drop the final-state cursor template, which only `StateDelegatingStream` needed.
- Tests: monkeypatch the checks module by its dotted path instead of importing it twice.
@tolik0

Anatolii Yatsuk (tolik0) commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

/prerelease

Prerelease Job Info

This job triggers the publish workflow with default arguments to create a prerelease.

Prerelease job started... Check job output.

✅ Prerelease workflow triggered successfully.

View the publish workflow run: https://github.com/airbytehq/airbyte-python-cdk/actions/runs/42871066020

@bazarnov Baz (bazarnov) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, Anatolii. It is careful work, and the two dry runs make the rollout easy to trust.

What I checked at de7d1c5d, merged locally with current main (clean merge):

  • The 83 new tests pass locally (3 runs in a row). The concurrent_partition_cursor.py change keeps the 600 s default, and reverting it to the literal fails 2 tests.
  • The fixes from the previous review hold. The retention case (late_days=5, step: P1D, api_retention_period: P30D) reports nothing, a two-parent router with different partition_fields passes, and unmatched slices only add an advisory note. Mutations 89, 90, 93, 95 and 98 are killed.
  • The throttle lift writes the cursor instance, not the class. Its counters and the interval write are under one lock on the worker threads, and uninstall runs in finally.
  • Problem texts carry only stream, field and key names, formats, counts, the read duration and exception type names.
  • On airbyte master, 211 manifest-only connectors with a basic_read entry get these checks on the next release: 214, minus chargebee, monday and stripe, which opt out in every entry. The stripe opt-out from airbytehq/airbyte#91061 is in its only basic_read entry.

Inline, by priority:

  1. with_final_state can edit an earlier, equal state message. That made the first run of the required Pytest (Fast) check fail on this head (a re-run passed). A one-line fix and a test are inline; I'd land this before merge.
  2. The per-partition StateDelegatingStream bound has no test (the row 101 claim does not hold).
  3. Two more mutations on enforced checks survive.
  4. Two wording fixes in cdk-migrations.md.

None of these changes what a connector sees, so I'm approving. Check: destination-motherduck fails here at the secret fetch (its secret has no enabled version), as it does on other open PRs such as #1267. Before merge: darynaishchenko's change request still needs her re-review, and main requires every review thread to be resolved. The rotating-token connectors that were never run (greenhouse, linear, quickbooks, salesloft, strava, trustpilot, typeform) get their first real check after release; greenhouse's job_posts hits the enforced ListPartitionRouter coverage check.

Comment thread cdk-migrations.md Resolved
Comment thread cdk-migrations.md Resolved
…ee unpinned cursor-state guards

`with_final_state` edited the first state message equal to the final one, so a
concurrent read that repeats the final state left it unedited and the test
flaked. It now edits the last equal message.

New tests pin three guards that no test failed without: a per-partition
StateDelegatingStream takes its resume bound from `incremental_stream`, a
`states` entry whose cursor lacks the cursor field fails the shape check, and a
`parent_state` missing an incremental parent fails the parent-state check.

The migration note now lists all three final-state failures and says that a
suite whose read runs on another source instance skips the checkpoint check.
@tolik0

Copy link
Copy Markdown
Contributor Author

I couldn't reproduce the greenhouse job_posts coverage failure. I ran greenhouse's manifest offline with these checks, with active=false returning nothing and with both partitions empty, and each partition still got a states entry: the CDK writes one for every partition it reads, even an empty one. On the test account the stream reads 1 record from one partition. In connector CI with the pre-release (airbytehq/airbyte#92971), test_basic_read['config_client_credentials'] passes with no cursor state problem or warning; job_posts read 1 record. config_oauth fails at the token refresh, as it does on master.

@tolik0
Anatolii Yatsuk (tolik0) marked this pull request as ready for review November 13, 2026 12:59
@tolik0
Anatolii Yatsuk (tolik0) requested a review from a team as a code owner November 13, 2026 12:59
@tolik0
Anatolii Yatsuk (tolik0) dismissed Daryna Ishchenko (darynaishchenko)’s stale review November 13, 2026 13:17

Daryna is currently OOO and won't be able to review the PR again.

@tolik0
Anatolii Yatsuk (tolik0) merged commit 608e291 into main Nov 13, 2026
28 of 29 checks passed
@tolik0
Anatolii Yatsuk (tolik0) deleted the tolik0/cdk/per-partition-state-checks branch November 13, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants