Repository navigation
Conversation
On Python 3.13+, dbm creates new files with dbm.sqlite3, whose connection is bound to the thread that opened it. PersistentDict is opened on the main thread in State.add_persistent_namespace, so the first access from an app worker thread raised "SQLite objects created in a thread can only be used in that same thread". Reopen the dbm.sqlite3 connection with check_same_thread=False; access is already serialized by PersistentDict.rlock. Other backends (gnu, ndbm, dumb) open exactly as before, and existing SQLite stores are fixed too. Also warn when an existing namespace file is in a format the running Python can't open (e.g. GDBM on a Python built without _gdbm), instead of silently starting the namespace empty. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Starting with Python 3.13,
dbmcreates new databases withdbm.sqlite3. That backend'ssqlite3connection uses the defaultcheck_same_thread=True.PersistentDictis opened on the main thread inState.add_persistent_namespace. The first time a sync app reads or writes the namespace from its worker thread, it fails with:This affects every new persistent namespace on 3.13+ when the Python build doesn't have
_gdbm. That includes the Home Assistant add-on image (python-build-standalone via uv). It also affects any SQLite store already created by 4.5.13 on 3.13.A related problem: if a namespace file exists in a format the running Python can't open (for example a GDBM file, or a GDBM-written
.pag/.dirpair, on a Python without_gdbm),dbmdoesn't fail. It creates a separate empty store next to the file, so the namespace silently comes up empty. I ran into this after the add-on moved to Python 3.13.Changes
appdaemon/utils/state.pyPersistentDict.__init__replaces thedbm.sqlite3connection with one opened on the same file withcheck_same_thread=False(and WAL mode, likedbm.sqlite3itself).PersistentDict.rlockalready serializes access, and Python'ssqlite3module is built in serialized threading mode. Other backends (gnu,ndbm,dumb) open exactly as before. File format, file names and backend selection don't change, so existing stores of every kind keep working.find_unreadable_store_files(path)sniffs the header of the store file (and of its.pagfile) for SQLite/GDBM magic. It reports files whose backend isn't available in the running Python. For a GDBM.pag, it only reports the file ifdbm.ndbmactually fails to open it.appdaemon/state.py:add_persistent_namespacelogs a warning for each such file before opening the namespace. Behaviour is otherwise unchanged (it still starts empty).tests/unit/test_persistent_dict.py: covers reads and writes from another thread (both writeback modes), concurrent writers, reopening an existingdbm.sqlite3store, an existingdbm.dumbstore still opening with its own backend, and the unreadable-file detection.Notes for review
dbm.sqlite3has no public way to passcheck_same_thread, so the fix swaps the private_Database._cxattribute. That attribute and_Database.__init__are identical in CPython 3.13, 3.14 and 3.15. The swap only runs when the shelf's database is adbm.sqlite3._Database, and the file path comes fromPRAGMA database_listinstead of rebuilding the URI. If CPython changes these internals, the new tests will fail.Testing
pytest tests/unitpasses on 3.12 (SQLite test skipped), 3.13 and 3.14 (uv / python-build-standalone builds, no_gdbm).tests/functional/test_namespaces.pyandtest_state.pypass on 3.13.ProgrammingErrorabove._gdbm: it is reported on 3.13 without_gdbm. A GDBM-compat ndbm store thatdbm.ndbmcan open is not reported.Possibly related, not changed here
remove_persistent_namespacedeletes files whosestemequalsfilepath.stem. On 3.13+ the store isns.db, so ndbm/dumb files such asns.db.db,ns.db.datorns.db.dir(stemns.db) are left behind.nstons.dbfile names means a GDBM store written by AppDaemon on ≤3.12 (filens) isn't picked up after upgrading to 3.13. The namespace starts empty and there's no warning.🤖 Generated with Claude Code