Native async create_record: a linger thread shared by the process would remove the cost of executors of 97 to 159 records #406

I found this while fixing the one-record slowdown of #393 (refs #388). The collected path of NativeAsyncExecutor.create_record has a fixed cost per executor: _start_linger starts a thread for every executor that collects a record, and each collected record goes through the lock and a copy of its properties. For a small executor that cost is larger than what the columnar frame saves, so #393 sends the first 128 records of an executor one op each, as in 26.11.1.post3 (_ASYNC_DIRECT), and collects only beyond that, with a frame only when 64 or more records are collected (_ASYNC_FRAME_MIN), so a frame appears from 192 records.

What that leaves is a band. An executor of 129 to 191 records collects a few records, hands them over one op each and pays the collection cost without getting a frame. I measured it with an untimed micro (a new executor, N records, wait_completion; the native wheel built from b77f887 against the published 26.11.1.post3 native wheel; medians in microseconds; cores 14 and 15 of the laptop):

Up to 128 records the executor is the post3 path and the ratios sit at 0.94 to 1.06. In the band the head costs 198 to 388 us more than post3 per executor (ratios 1.11 to 1.19; I ran the micro twice, the table is the first session and the ratios of the second at 96, 128, 129, 150, 191, 192 and 256 records were 96: 1.07, 128: 1.04, 129: 1.14, 150: 1.11, 191: 1.11, 192: 0.87, 256: 0.75). From 192 records the frame carries 64 or more records and the executor is faster than post3 (0.36 at 1000 records).

I expected the cost of the band to be a near-constant amount per executor, so that its relative size would fall as the direct phase grows; this micro is too noisy to show that, and it does not show the amount as constant either. With a direct phase of 96 (an earlier head, 7854838) the band was 97 to 159 records and 100 records read 1.19 (ratios at 96, 97, 100, 128 and 159 records: 96: 1.03, 97: 1.15, 100: 1.19, 128: 1.15, 159: 1.12); 128 moved it out of the 100-record case of the laptop A/B. The band is not removed, it moved, and the release notes will say so.

The bare start of a Python thread, measured next to the earlier micro of this change, was about 30 us, so I do not think the thread start explains all of the 198 us or more. Before changing anything I would split the cost: thread start, the condition and lock traffic, the copy, and the first hand-over.

My proposal is a linger thread shared by the whole process: one daemon thread, started at first use, that keeps the due times of the executors with a pending buffer. An executor then starts no thread, and the direct phase of 128 records could shrink to a few records or go away, which would also remove the band. I would measure that against this table before deciding. The data are untimed micro numbers, not a study figure; the laptop A/B of the post4 glue (the sizes 1, 100, 1k and 100k) has the study figures.

# micro: one fresh async executor, N records, wait_completion (us per executor-call): post3 against a PR 393 wheel; argv[1] = label, argv[2] = optional _ASYNC_DIRECT override, argv[3] = optional _ASYNC_FRAME_MIN override
import sys, time, tempfile, threading, statistics
import arcadedb as a
if len(sys.argv) > 2 and sys.argv[2]:
    from arcadedb._native import extras
    extras._ASYNC_DIRECT = int(sys.argv[2])
    if len(sys.argv) > 3 and sys.argv[3]:
        extras._ASYNC_FRAME_MIN = int(sys.argv[3])
d = tempfile.mkdtemp(prefix="micro-async2-")
db = a.create_database(d + "/db")
db.command("sql", "CREATE DOCUMENT TYPE T")
PC = time.perf_counter
def one(n_rec, reps):
    out = []
    for _ in range(reps):
        docs = []
        for i in range(n_rec):
            x = db.new_document("T"); x.set("a", i); x.set("s", f"s{i}"); docs.append(x)
        ex = db.async_executor()
        t0 = PC()
        for x in docs:
            ex.create_record(x)
        ex.wait_completion()
        out.append((PC() - t0) * 1e6)
    return statistics.median(out)
for _ in range(30):
    one(1, 1)
SIZES = (1, 64, 96, 127, 128, 129, 150, 191, 192, 200, 256, 500, 1000)
res = {n: one(n, 300 if n < 10 else (100 if n <= 200 else 40)) for n in SIZES}
print(sys.argv[1], "median us per executor-call (create + wait):", {k: round(v, 1) for k, v in res.items()})
db.close()

Edited 2026-11-15 19:52 KST: the table and the numbers are now for the head b77f887 (direct phase of 128, band 129 to 191, two micro sessions); the first text had the head 7854838 (direct phase of 96, band 97 to 159).