Repository navigation
Conversation
… stops blocking publish() Under MySQL's default REPEATABLE READ the claim's FOR UPDATE SKIP LOCKED takes gap locks on (status, created_at); SKIP LOCKED only skips record locks. With a backlog smaller than the batch the gap runs to the end of the index, where every publish() inserts, so business transactions waited for the whole delivery and could fail with "Lock wait timeout exceeded". - Spring: okapiTransactionRunner pins READ COMMITTED when its TransactionTemplate leaves isolation at DEFAULT, using a copy so shared template beans are not mutated; explicit isolation levels are still honoured verbatim. - Exposed: ExposedTransactionRunner gains transactionIsolation, defaulting to READ_COMMITTED (null = database default). - MysqlClaimDoesNotBlockPublishersTest reproduces the lock at REPEATABLE READ (control) and proves both runners no longer block a concurrent publish. Closes #129 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
rawo
left a comment
There was a problem hiding this comment.
Review focused on transactions, deadlocks/lock waits, performance and structure. The diagnosis is right: SKIP LOCKED only skips record locks, not gap or next-key locks. READ COMMITTED is the standard fix, and the RR control test proves the reproduction. The copy-not-mutate handling of the shared TransactionTemplate is careful. One finding I'd treat as blocking, plus a few notes.
Cross-PR note (#120): #120 changes the processor to claim through RouteAwareOutboxStore.claimPending(types, limit) with FORCE INDEX (status, delivery_type, created_at, id). Under RR that query gap-locks the end of each type's range, which is again exactly where publish() of that type inserts. So #129 still applies after #120, and this PR still fixes it. But MysqlClaimDoesNotBlockPublishersTest calls the legacy claimPending(10), so once #120 lands it would stop exercising the query the processor actually runs. Whichever PR merges second should switch the test to processor.processNext() (or the route-aware claim).
…ccepts it (#129) Addresses review on #139: - Spring: pin READ COMMITTED only when the store is MysqlOutboxStore. Postgres already defaults to it, so other stores keep the template verbatim (no extra round trip, no behaviour change). Skip pinning with a startup WARN for PTMs that reject custom isolation levels (JtaTransactionManager, JpaTransactionManager without the Hibernate/EclipseLink JpaDialect); otherwise every tick would throw InvalidIsolationLevelException. - Exposed: ExposedTransactionRunner(database) uses READ COMMITTED only on MySQL/MariaDB dialects; ExposedTransactionRunner(database, level) sets it explicitly. - MysqlClaimDoesNotBlockPublishersTest drives a real OutboxProcessor tick with a blocking deliverer instead of calling claimPending directly, so it follows whatever claim query the processor runs (e.g. after #120). Adds Hibernate-JPA and DefaultJpaDialect cases. - New OutboxAutoConfigurationIsolationTest, including a JtaTransactionManager case. - CHANGELOG: behaviour-change note for Exposed overriding defaultIsolationLevel on MySQL. README: binlog_format=ROW/MIXED requirement and the JTA/JPA fallback. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Closes #129.
Problem
Under MySQL's default
REPEATABLE READ, the processor's claim (FOR UPDATE SKIP LOCKEDonidx_okapi_outbox_status_created_at) takes gap locks, becauseSKIP LOCKEDonly skips record locks. When the backlog is smaller than the batch, the gap runs to the end of the index, which is exactly where everypublish()inserts. Business transactions then wait for the entire HTTP/Kafka delivery and can fail withLock wait timeout exceeded.Fix
On MySQL/MariaDB, the scheduler (processor and purger) transaction now runs at
READ COMMITTED, which takes no gap locks. Other databases are untouched (Postgres already defaults toREAD COMMITTED).okapiTransactionRunnerpinsREAD COMMITTEDonly when the store isMysqlOutboxStoreand itsTransactionTemplateleaves isolation atDEFAULT.JtaTransactionManager(no public getter forallowCustomIsolationLevels, so always treated as unsupported) andJpaTransactionManagerwithout the Hibernate/EclipseLinkJpaDialect. Otherwise every tick would throwInvalidIsolationLevelException. The WARN points to an explicit-isolationTransactionTemplateor server-sidetransaction_isolation.ExposedTransactionRunner(database)usesREAD COMMITTEDwhendatabase.dialectis MySQL/MariaDB (resolved lazily on first use), the database default otherwise. A newExposedTransactionRunner(database, transactionIsolation)sets the level explicitly (null= database default).Behaviour changes to review
ExposedTransactionRunner(database)overrides a configuredDatabaseConfig.defaultIsolationLevelfor scheduler transactions (noted under CHANGELOG Changed).READ COMMITTEDrequiresbinlog_format=ROW(default since 5.7.7) orMIXED.Tests
MysqlClaimDoesNotBlockPublishersTest(Testcontainers MySQL) runs a realOutboxProcessor.processNext()tick with a deliverer that blocks mid-delivery, and publishes from another connection withinnodb_lock_wait_timeout=3. Going through the processor keeps it on whatever claim query the processor runs (e.g. after Claim only delivery types supported by each worker #120).REPEATABLE READrunner → the publish fails withLock wait timeout. This proves the test reproduces MySQL: processor claim gap-locks okapi_outbox under REPEATABLE READ, blocking every publish() for the duration of delivery #129.ExposedTransactionRunner→ not blockedDataSourceTransactionManager→ not blockedJpaTransactionManager→ READ COMMITTED applied, not blockedJpaTransactionManageronDefaultJpaDialect→ not pinned, ticks still runOutboxAutoConfigurationIsolationTest: auto-built template, user template at DEFAULT (copied, not mutated), explicit isolation (verbatim), non-MySQL store (verbatim), andJtaTransactionManagerover a fakeUserTransaction(premise: RC throws; okapi doesn't pin, ticks run, WARN logged once).ExposedTransactionRunnerTest: database default on non-MySQL, explicit andnullisolation../gradlew ktlintCheck build testpass locally.🤖 Generated with Claude Code