Repository navigation
Conversation
Recovery archived permanently stalled task messages but left their tasks and runs active. Treat the permanent-stall marker as exhaustion in fail_task() without inflating attempts_count, and use a separate escalation cron transaction to apply the existing fail, skip, or skip-cascade policy to marked started and queued tasks. Preserve terminal-state guards, diagnostic markers, stored queue routing, and installed recovery schedules and activation choices. Restore saved cron booleans exactly in the upgrade guidance rather than enabling every paused job; reject malformed saved state and require the snapshot to survive the SQL session. Preserve recorded enabled values in the deployment guide too. Combine the changes from dd41e5dfe and d4831bd79 onto the independent 689-terminalize-stalled branch based on main. Recalculate atlas.sum for this branch without the unrelated worker-startup migration from #696. The original commits remain recoverable at backup/689-misplaced-d4831bd79. The original implementation commit records pgTAP, worker, E2E, migration/type, and website checks; the documentation correction records rolled-back restoration checks and website lint/build. This relocation preserves their code changes but does not treat those earlier results as fresh runtime validation on the new base. Repository hooks run normally for this commit and submission. Fixes #689
🦋 Changeset detectedLatest commit: daa1753 The changes in this PR will be included in the next version bump. This PR includes changesets to release 5 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
|
View your CI Pipeline Execution ↗ for commit daa1753
💡 Verify your cache is correct by running tasks in a sandbox. Read docs ↗ ☁️ Nx Cloud last updated this comment at |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #689. Independent of the worker startup handshake in #711.
The existing recovery setup configures both cron jobs with one interval. Upgrade wiring preserves installed schedules and inactive/absent recovery choices. Historical marked queued leftovers are included; unmarked queued and terminal tasks stay unchanged.
Upgrade and deployment guidance restores recorded activation values instead of enabling everything. Saved cron values must be JSON booleans; unknown or malformed state never enables a job. Includes a core changeset, generated migration/types, pgTAP coverage, and upgrade fixtures.
Evidence
max_attempts.689-terminalize-stalled, based onmain; recalculate Atlas checksums without fix(worker): coordinate pending starts with invocation handshakes #711’s migration. Runtime suites were not rerun for this transfer. Normal commit and pre-push hooks remain enabled.Merge Danger
Door: one-way for data effects after escalation runs. Code can be reverted; completed transitions and downstream side effects cannot be undone automatically.
Blast Radius: recovery
Historical marked tasks follow the existing exhaustion policy after upgrade. In particular,
skipcan enqueue dependent business work. Inspect the documented backlog before upgrade. Pause both cron jobs and the relevant workers/callbacks when downstream execution must stop, then restore each recorded state exactly. Inactive cron jobs alone do not prevent marker-aware failure callbacks. No production migration or deployment is part of this PR submission.