Repository navigation
Native wheel: large results are slower than the JVM wheel (to_arrow 1M about 3.4x): profile and fix #312
Description
In a timed A/B on one laptop (cores pinned, one fresh process per run, ABBA order, an A/A control that shows no difference in any figure), the native wheel is slower than the arcadedb-jvm wheel when a result is large, on the narrow table (4 columns). The ratio is the JVM wheel's time over the native wheel's time.
| figure | JVM / native | reading |
|---|---|---|
to_arrow, 1M rows |
0.298 | native about 3.4 times slower |
to_arrow, 100k rows |
0.345 | native about 2.9 times slower |
first_1M |
0.763 | native about 1.3 times slower |
The native wheel is faster on small results and lighter in memory, so this is a large-result problem, not a general one. I have only read the narrow-row summary so far; the wide-row figures are in the same study.
I do not know the cause yet. Candidates I want to rule in or out with a profile, not assume: the Arrow conversion path doing per-value work that the JVM wheel does in bulk; the Serial GC in a native image sizing its young generation too small for large allocations (the eden size stays at its initial value under allocation pressure, which I reported to GraalVM as oracle/graal#14758); and the missing profile-guided optimization, which exists only in Oracle GraalVM and cannot be used because the shipped image stays on the Community Edition (Apache 2.0).
Plan: profile to_arrow at 1M rows in the native wheel (allocation and GC logs first), try the young-generation size and heap flags the wheel passes at startup, fix any conversion cost in the code, and rerun the same figures with the same method. Done when the ratio at 100k and 1M rows is measured again after each change and the changes that help are in a release, or the remainder is shown to be inherent to ahead-of-time compilation.
Edited 2026-11-09: the study these figures come from ended WEAK by its own rule (17 other figures have an interval wider than 0.10 times the median), so I treat these numbers as indicative until the rerun of those figures has finished; I am rerunning them now. I removed the first_100k row from the first table because it is one of the 17 weak figures. The figures that remain are not among them.
Activity
tae898 commentedon Nov 9, 2026
I read the native read path against the JVM one (code only, nothing run; the released tag
26.11.1is the same commit2f28a328beas the one the study was pinned to, so the measured path is the released one). What I found, with what is read and what is still a guess.Read from the code: the native wheel has no columnar read path.
NativeResultSet.to_arrow,to_columns,to_dataframeandto_polarsall go through_drain_dicts()(database.py:333-343,:410-452), which reads JSON text rows, 2048 per crossing, into one Python dict per row and then builds each column in Python (columns.py:7-167). The JVM wheel instead gets packed typed column buffers fromColumnBatcherand wraps them withnp.frombufferandpa.array(results.py:1010-1103). This matches the study: at 1M narrow rows the JSON-row paths are at parity (to_json_listabout 1.0,iter_json_batchesabout 1.02 of the JVM time), while every columnar path is 2.2 to 3.4 times slower on native.Read from the code, separate cause:
first()pays for 256 eagerly encoded rows on native (_FIRST_BATCH = 256,database.py:40), a fixed cost that grows with the row width, not a large-result problem.Still a guess until measured: whether the JSON drain or the per-value Python loops after it dominate; Python's cyclic GC running while 1M dicts are built; and Java-side allocation under the Serial GC with the small eden of oracle/graal#14758. The plan has one cheap experiment for each (time
_drain_dictsand the column build separately, a run withgc.disable(), a run with GC logging and a larger initial heap).Candidate fixes, by size: bindings-only Python changes that need no image rebuild (infer column types with
pa.arrayand fall back onArrowInvalid, compute names once per batch, drop the per-row generator in the drain, keep a view onto the float32 tail instead of.tolist()per vector); then native columnar read frames modelled onColumnBatcher, which needs a new native entry point and an image rebuild and is a next-release item. There is prior art on the branchnative-columnar-resultsof the old working repository that was never shipped. No engine change is needed for any of it.I will not touch this before the paper deadline. I will measure first, then fix, then rerun the same figures, and I am rerunning the weak figures of the study now.
tae898 commentedon Nov 9, 2026
A design note for the native fix, so the target is the standard and not a custom format. The Arrow C Data Interface is the standard way for a native library to hand columnar data to Python without a copy: the library fills an
ArrowSchemaand anArrowArraystruct andpyarrowimports them withRecordBatch._import_from_c; the producer'sreleasecallback frees the buffers. If the native library emitted result batches this way,to_arrowwould be zero-copy andto_columns,to_dataframe,to_polarsand DuckDB would follow from it, and the same interface in the other direction would giveinsert_arrowa binary path. ADBC is the matching standard API on the Python side.The JVM wheel already moves typed column buffers (a custom packed format wrapped with
np.frombuffer), so it is close in spirit; the native wheel reads JSON rows today. Open question for a spike: thereleasecallback must be a native function pointer, which a GraalVM native image can export as an entry point, but I have not tried it. I will also look at whether the JVM wheel should move to the same interface (the Arrow Java C-data export) so both wheels share one reader.This waits until after the paper deadline. The write-side audit is running, and its result goes on this issue.
Native columns: a failed inference of any kind falls to the old rules…
Native columnar readers: infer Arrow types first, skip per-key and pe…
Native columnar readers: infer Arrow types first, skip per-key and pe…
Native: column frames for to_arrow, to_columns, to_dataframe and to_p…
tae898 commentedon Nov 13, 2026
The two reruns of the study on the released 26.11.1 pair (
ab2andab3, the same wheels in both) give valid figures for most of the large-result paths of 26.11.1, and they widen the picture beyondto_arrowon the narrow table. These are the numbers of 26.11.1, which does not contain #331 (see the edit note at the end). The ratio in the tables is native time over JVM time, so a value above 1 means the native wheel is slower. Only figures that pass the study's rule are listed (at least 10 clean pairs, a 95% interval no wider than 0.10 times the median, and an A/A control that contains 1.00 or lies outside the B/A interval), and each row names the run it comes from. 21 figures pass the rule and have one wheel more than 1.5 times slower than the other: 13 columnar, 4first(), 4 JSON reader.Columnar readers, native slower:
to_arrowab2to_arrowab2to_arrowab2to_arrowab2to_columnsab2to_columnsab3to_columnsab2to_columnsab2to_dataframeab2to_dataframeab2to_dataframeab2to_polarsab2to_polarsab3first(), native slower:firstab3firstab2firstab2firstab2JSON readers on the 128-dimension vector table, JVM slower (this goes the other way, and I have not read the cause; #314 is the neighbouring parity question for
to_json_list):iter_json_batchesab2iter_json_batchesab2to_json_listab2to_json_listab2This matches the code reading of 2026-11-09 above for 26.11.1: the native wheel had no columnar path and built the columns of all four readers from dict rows in Python, and the gap grows with the width of the row. #331 replaced that path and is in 26.11.1.post1 and 26.11.1.post2. I read the sizes as orders of magnitude and not as exact ratios, because the two nights do not always agree inside their intervals (for example
to_arrow_1Mon the narrow table, B/A 0.298 [0.291, 0.304] inab2and 0.326 [0.31, 0.339] inab3).What is not said here:
ab3ended WEAK at the round cap (an interval wider than the limit on 20 of its figures), so those figures are not reported, and the paths that are still weak arefirst,to_arrow,to_columns,to_dataframeandto_polarson some tables. I will rerun those paths as a new run on a quieter night, becauseab3lost pairs to rounds that overlapped other load. The other eight read paths are not measured on the released pair yet, so the recommended read API per wheel waits for that. These figures are the baseline of 26.11.1 and are not a statement about the current native wheel. The post1 and post2 wheels contain the columnar frames of #331, and I will measure them against the JVM wheel in a new run on the post2 pair before claiming anything about them.Edited 2026-11-14: the first version of this comment said the native wheel has no columnar read path and that no fix was started. That was true of 26.11.1 only. #331 (native column frames for
to_arrow,to_columns,to_dataframeandto_polars, decoded by the code the JVM wheel uses) was merged on 2026-11-10 and ships in 26.11.1.post1 and 26.11.1.post2, so the tables above are the baseline before that change and not the current state.tae898 commentedon Nov 13, 2026
Corrected my comment above: its tables are the figures of 26.11.1, which does not contain #331. The native column frames for
to_arrow,to_columns,to_dataframeandto_polarsshipped in 26.11.1.post1 and 26.11.1.post2, so the 2.2 to 15.6 times are the baseline before that change and not the current native wheel. I will measure the post2 wheels against the JVM wheel in a new run before claiming anything about them. What #331 left open: the first reply of a query (up to 256 rows) is still JSON,first()encodes 256 rows eagerly, and a query that fails in the middle of a frame stream has no test.tae898 commentedon Nov 14, 2026
Status, because the title and the first table of this issue still show the
26.11.1figures.Two changes to the native read path are in
26.11.1.post1and26.11.1.post2and are not in26.11.1, which is the tag every ratio on this issue was measured on: #321 (columnar readers infer the Arrow types first, one_kindper column, merged 2026-11-09) and #331 (native column frames forto_arrow,to_columns,to_dataframeandto_polars, decoded by the code the JVM wheel uses, merged 2026-11-10).Measured on a tag that contains them: nothing against the JVM wheel yet. The only figures are information figures (no verdict) of a post1 candidate against
26.11.1on the native wheel alone:to_arrowat 1M rows 2.136 s to 1.550 s,to_columns2.351 s to 1.790 s,to_dataframe2.084 s to 1.832 s. I do not read these as a native against JVM ratio.What remains: (1) the native against JVM A/B of the large-result paths on the published
26.11.1.post2pair, which is this issue's own "done when" (the ratio at 100k and 1M rows measured again after the change); (2) the first reply andfirst(), which still pay for JSON rows (#381); (3) a test for a query that fails in the middle of a frame stream. I will change the title once (1) is in, so that it no longer says 3.4 times.