Repository navigation
Agent-memory workloads: asyncio facade, read-your-writes vector test, vector bulk I/O #306
Description
Agent memory is a write-heavy, many-small-collections workload: an agent inserts a few vectors per turn, queries right away, and several agents may share one process. This issue collects what the Python bindings need for that, apart from the engine-side parts (small-graph rebuilds, default recall), which I will take to ArcadeData/arcadedb with measurements.
What exists today: AsyncExecutor (the engine's async API, including transaction) in both packages, and the concurrency page in the docs. There is no asyncio-native API, so an agent loop written with async def has to wrap every call itself.
Proposal, in the order I would do it:
- An
asynciofacade over the existing calls (query,command, a vector search helper) that runs on a bounded thread pool and keeps one transaction per task. No engine change. It needs tests that run concurrent readers and writers on one database from several tasks, and a documented answer for both builds on what releases the GIL. - A read-your-writes test for vector indexes: insert a vector, query it in the next call, from the same task and from another task, and assert it is found. This pins the visibility semantics that agent memory depends on, for the JVM and the native package.
- Vector bulk input and output without per-row Python: pass a 2-D
numpyarray in, get the search result as arrays out. An earlier study found the engine itself at parity and the cost in Python-side materialization, so I will measure first (ab.pywith the noise gate) and only add what the numbers justify. - One docs page for agent-memory use: collection size, when the graph index pays off, what to expect right after a write.
I will attach measurements to this issue as the studies finish. Happy to change the order or drop an item if you see it differently.
Activity
tae898 commentedon Nov 12, 2026
I read the sync API, the concurrency and transactions pages and both builds before writing code for item 1. I stopped before the facade, because it needs two choices that are yours.
What I found: a transaction belongs to the thread that began it (the transactions page says so), so "one transaction per task" cannot span several
awaits on a pool: twoasyncio.to_threadcalls of one task can run on different threads. Both builds let the loop keep running during an engine call (on the concurrency page the heartbeat was at most about 90 ms late withasyncio.to_threadand 382 to 929 ms late with a blocking call on the loop), and oneDatabaseshared by several threads is documented, so a bounded pool on one database is fine. I found no way to interrupt an engine call that is already running.Choice 1, the transaction:
await adb.run_in_transaction(fn):fnis a plain function that runs on one worker thread from begin to commit, with the retriesdb.run_in_transactionhas today.async with adb.transaction():pins one worker thread to the task until the block ends. It is closer to the words "one transaction per task", but it holds a thread per open transaction, and a task that waits on another task while it holds one can deadlock the pool.I recommend (a).
Choice 2, names and cancellation: one module
arcadedb.aiowith one classAsyncDatabase(db, max_workers=8)that owns its executor (it never takes the loop's default one) and hasquery,command,run_in_transaction,find_nearest, a generalrun(fn, *args)andclose(). I would not expose cancellation in the first version: cancelling the awaiting task abandons the result, the engine call still finishes on its worker (a transaction function still commits or rolls back on its own thread), andclose()waits for running calls. The docs page would say so. The nameAsyncDatabasenext to the engine's ownAsyncExecutoris the part I would like your word on.Meanwhile I am opening a PR for item 2 only, the read-your-writes vector test on both wheels. It uses
asyncio.to_thread, so it moves to the facade by changing one helper. I will not code the facade until you reply.Test: a vector search finds the vector committed in the call before, …
Test: a vector search finds the vector committed in the call before, …
arcadedb.aio.AsyncDatabase: await query, command, run_in_transaction …
tae898 commentedon Nov 14, 2026
Status. Items 1 and 2 are done on
main: the read-your-writes test is #373 (merged 2026-11-12) andarcadedb.aio.AsyncDatabaseis #379 (merged 2026-11-12). Neither is in a release yet: both PRs were merged tomainafter26.11.1.post2, and26.11.1.post3, which is built on post2, does not carry them. I built the transaction as option (a) from my earlier comment (await adb.run_in_transaction(fn)) under the nameAsyncDatabase; that decision is not written on this thread, so I note it here.Still open: item 3 (a 2-D numpy array in and arrays out for vector search), which I will measure first with the same A/B rule before I add anything, and item 4 (a docs page on collection size and when the graph index pays off; the asyncio page and a paragraph on the vectors page exist, the sizing page does not). I keep the issue open until those two are done and the first two are in a release.