Repository navigation
Conversation
…nto the vector header.
…is a multiple of 64 bits.
…is a multiple of 64 bits.
…SH to acceptt landmark count.
…pooled version for performance purposes though.
…tch and mixed landmarks.
… scoring. Also made a slight change to limit copying query info from query pool (only at landmark run boundaries) during block scoring.
… scoring. Also made a slight change to limit copying query info from query pool (only at landmark run boundaries) during block scoring.
…rainer. Netlib dependencies added.
…vecs loader to SiftLoader.
…ding payload alignment exception for recall benchmarking purposes.
…ll fixes. ITQ and multi-landmarks work.
…t. Does not work yet.
…ed vectors. Recall and perf look acceptable.
…ed layout with header interleaving. Introduced ashMaskedAdd_512.
Merge remote-tracking branch 'origin/main' into ash-dev
…to reduce conversions and memory traffic without changing training behavior. Iterations are ~2.6X faster.
The INT8 projection tests fail on one CI runner by one or two ULPs against the oracle but pass on every machine we can reproduce on. The tile and oracle assertions now name the CPU vendor, model, the flags that select the INT8 path (VNNI, AMX, AVX-512 FP16/BF16) and the oneDNN override, so a failure shows which path ran. The message is built only when an assertion fails.
Vector API lane reductions do not fix their summation order, so an interpreted and a compiled scoring of the same 1-bit block can differ by one ULP; under JDK 25 the test's exact comparison failed on that alone (e.g. 0.61249167 vs 0.6124917). It now compares within 1e-5. Against the code before the fix it still fails every run, with errors of 0.02 to 0.09.
r-devulap
reviewed
Nov 12, 2026
| return pointer ? static_cast<const Context*>(pointer)->retained_bytes.load(std::memory_order_acquire) : 0; | ||
| } | ||
|
|
||
| JV_ASH_INT8_TARGET int jvector_ash_int8_project_batch(void* pointer, const float* input, int count, |
Contributor
There was a problem hiding this comment.
With the addition of hand written VNNI kernels, this ideally belongs to the Highway module which has all the infrastructure to deal with SIMD code (compile time and runtime dispatch). Only problem is I am not sure how to cleanly separate the AMX portion which relies on onednn.
…tart FvecFileVectorValues loads the vectorization provider, possibly a native library, when its class initializes. That happened inside whichever timed test ran first and took 111 ms of a 1 s limit on an idle server, more on a busy CI runner. Initializing the class in @BeforeClass leaves each timeout covering only the read loop it guards; the limit is unchanged.
r-devulap
reviewed
Nov 12, 2026
…al step The projected value is (coarse + fine / 254) * scales. Instead of turning off floating-point contraction for the whole library, combine the two digit dot products exactly in 64-bit integers first, coarse * 254 + fine, so the remaining steps are one conversion and multiplications only. With no product added to anything there is nothing to fuse into an FMA, every build rounds the same way, and the combine loop stays free to vectorize. The test oracle uses the same expression, so the INT8 tests still compare exactly.
r-devulap
reviewed
Nov 12, 2026
| } | ||
|
|
||
| extern "C" { | ||
| void* jvector_ash_int8_create(const float* matrix, int rows, int columns, jvector_ash_fp16_result* result) { |
Contributor
There was a problem hiding this comment.
I was trying to think of ways we could untangle the SIMD portion in this file and move it Highway module. Could INT8 be modeled like the FP32 kernels? FP32 keeps the model in Java and calls a stateless Highway kernel (ash_project_batch_f32), so ISA dispatch comes from Highway. INT8 has C++ own the model and do its own ISA detection and VNNI intrinsics. If Java owned the int8 codes, scales and sums and a Highway kernel did the VNNI dot (SumOfMulQuadAccumulate maps to vpdpbusd), INT8 would benefit from all the Highway module infrastructure which we would avoid duplicating here. Java could still dispatch to oneDNN in the cases Highway doesn't cover, so the choice lives in one place. Is there a reason it can't?
A JIT deoptimization inside the measured window can make the JVM's own VarHandle linkage allocate a few hundred bytes. Require one of up to three windows to be allocation-free; a real allocation appears in every window.
A one-second timeout measured the CI machine as well as the read loop and failed under load after the test body had finished. The test channel now fails after a fixed number of read calls, which catches a spinning reader deterministically, and the EOF test also checks its call count.
A layer search expands a scored node only while it ranks among the best rerankK accepted nodes scored so far, or, with adaptive termination, within the cutoff of the rerankK-th; that bar only rises. A node that misses it when scored can never be expanded by that search, so it now goes to an unsorted deferred list instead of the candidate heap, and returns to the candidates at the next layer search, so later layers and resume() see the same candidates as before. Most scored neighbors are rejected with one comparison, as in DiskANN's bounded search list. The same top-rerankK heap drives adaptive termination, replacing its membership hash set: the visited set already keeps scored nodes unique, so the extra offer on each pop and the membership lookups are gone. Threshold searches keep every candidate. Results, scores, and visited and expanded counts are unchanged; a test compares bounded and unbounded searchers across stopping rules, filters, hierarchy and resume().
Graph views gather an expanded node's unvisited neighbors and score them in one batch when the score function supports it. ASH symmetric construction scoring gathers the neighbors' codes into one contiguous buffer and scores them in one bounded native call that reuses the pair kernel, so batch and single scores are identical.
ASH score functions now declare that they require adaptive stopping, and GraphSearcher stops adaptively whenever the score function requires it, whatever SearchScoreProvider wraps it. An ASH search therefore cannot fall back to the standard rule because an application wrapped its scores in a provider that did not ask for adaptive stopping; GraphSearcher.useAdaptiveTermination(false) remains the explicit override. A test checks plain, adaptive-requesting and application-written providers against each other and the override against a standard search.
ASH clamped its DOT_PRODUCT similarity at zero, so every estimate at or below -1 tied. That was a workaround for hierarchy descent starting from a zero floor, which now starts from negative infinity. Use (1 + dot) / 2 unclamped in Java and native, the scale exact DOT_PRODUCT scoring uses. Add TestUnnormalizedVectors: dot-product search over vectors whose lengths vary 100x must rank by the full dot product for exact, PQ and ASH scoring, by brute force and through a graph, judged against the same directions at unit length and against norm-blind ranking.
… scorer Adaptive stopping read graph scores as distances between unit-length vectors, so on vectors of other lengths it fell back to the standard rule or misjudged the margin, and it applied only to ASH. It now reads scores as true metric distances: d^2 = 1 / score - 1 for Euclidean, 1 - score for cosine, and for dot products the distance after the standard reduction of inner-product search to nearest-neighbor search, measured from an upper bound on the query's dot products, its length times the largest stored vector length. The margin, about 0.005 on the distance, means the same thing for every similarity function and every vector length, and on unit-length vectors the rule decides as before. A search that stops adaptively expands at least the nodes the standard rule would, and at most the nodes a standard search with twice the beam would. Without the cap, a relative margin admits very many candidates when distances are concentrated. The bar below which scored nodes are skipped never falls during a search, so skipped nodes stay skipped. The largest vector length comes from the vectors. ASH and PQ models record it for every vector they encode or whose codes they hold, and save it with the model; the inline and separated vector features and compaction store it in the index header, at format version 7. A length that is unknown, as for a model loaded from an older format, keeps dot-product searches on the standard rule. Searches through DefaultSearchScoreProvider.exact and forSimilarity, and all ASH searches, stop adaptively; providers built with the plain constructors keep the standard rule, and GraphSearcher.useAdaptiveTermination(false) turns adaptive stopping off.
Measured against standard stopping swept over overquery, at 1M and 10M, with ASH and PQ scores, with and without hierarchy, adaptive stopping gave less recall at the same throughput or nodes visited: raising the overquery reranks more candidates, which adaptive stopping does not. GraphSearcher therefore disables it by default, and useAdaptiveTermination is marked experimental. Searches that enable it behave as before. Graph construction keeps its behavior: the builder's searchers enable adaptive stopping, which applies where the build provider asks for it, as ASH's does.
ashkrisk
reviewed
Nov 13, 2026
| `withVersion(6)` on graph writer builders and `write(out, 6)` for compressors and compressed vectors. | ||
| Versions 6 and below still write inline vectors, NVQ, separated vectors and fused PQ exactly as before, | ||
| and 4.0.x reads them. A version 6 writer rejects fused ASH, and ASH models reject version 6. | ||
| - `OnDiskGraphIndexCompactor` always writes `CURRENT_VERSION`; it has no option to write an older version. |
Contributor
There was a problem hiding this comment.
This prevents the possibility of rollback as mentioned above for anyone who's using compaction. Compaction is a new enough feature that we might not need to care, on the other hand it seems a small enough change to add withVersion or similar support for the OnDiskGraphIndexCompactor. Does is it matter enough?
| if ((int) abi.invokeExact() != 1) | ||
| throw new UnsatisfiedLinkError("Unsupported ASH projection add-on ABI"); | ||
| create = bind(lookup, "jvector_ash_int8_create", | ||
| FunctionDescriptor.of(ADDRESS, ADDRESS, JAVA_INT, JAVA_INT, ADDRESS)); |
Contributor
There was a problem hiding this comment.
Is it possible to use jextract to generate these bindings rather than maintaining them by hand, similar to the SIMD library? Otherwise we need to manually ensure consistency between C and Java.
Comment on lines
+35
to
+36
| FP16 where the CPU supports it, falling back to FP32. INT8 projection is available on request | ||
| for x86 processors with VNNI or AMX. Native builds include the oneDNN projection engine by |
Contributor
There was a problem hiding this comment.
The default auto uses int8 projection, so it's not only enabled "on request"
| JV_ASH_INT8_TARGET float quantize(const float* input, int count, Code* output, int offset, int32_t* sum = nullptr) { | ||
| float maxabs = 0; | ||
| for (int i = 0; i < count; ++i) { | ||
| if (!std::isfinite(input[i])) throw std::invalid_argument("Nonfinite INT8 operand"); |
Contributor
There was a problem hiding this comment.
Looks like an invalid input (NaN, inf) in any of the queries causes the entire batch to fail, and the error fans out to each input query, even the valid ones. This seems undesirable, can we propagate NaNs for that particular query instead? I think the fp16 version also has a similar check which causes the same issue.
| float offset = readFloat16(in); | ||
|
|
||
| byte landmark; | ||
| if (in instanceof io.github.jbellis.jvector.disk.ByteReadable) { |
Contributor
There was a problem hiding this comment.
RandomAccessReader extends ByteReadable, so this check is always true. Similarly for other places in this file and one in AshCodes
| * Do not close from a callback or block a host executor thread needed to drain this | ||
| * service. The host must not cancel or discard already accepted drain tasks. | ||
| */ | ||
| public final class BatchingExecutor<T> implements AutoCloseable { |
Contributor
There was a problem hiding this comment.
Many of the new constructs are not marked @Experimental, for example this one. Is that intentional?
… and search-level query execution Page alignment (format 7, opt-in through withPageAlignment) places base-layer records so none crosses a page boundary: small records are packed per page, larger ones padded to whole pages when their last page is at least half full. AsymmetricHashing.setQueryOptions sets query projection precision and batch size in code; the system properties remain the defaults. BenchYAML gains construction.pageAlignment and moves queryExecution and requestsPerWorker to the search level, for any compression.
…; add an mpnet-10m example Adds a guide to building and serving indexes larger than memory, a BenchYAML example whose records fit one 4 KB page, and updates the BenchYAML guide, ASH configuration, integration, precision and batching docs, UPGRADING and the release note for the new controls. Aligns the docs with the code: the auto query projection selects INT8 first.
…-memory run config The run YAML gains a measurement section: evictIndex drops each index from the page cache before its first search, and throughput settings choose warmup queries (random, base or none), warmup passes, warming until JIT compilation settles, timed passes and the minimum seconds per pass, which replaces the jvector.bench.throughputMinSeconds property. Execution metrics add p50 and p99.9 response times and page reads per query, and now cover direct execution. The default run config is trimmed for a quick first run; run-config-larger-than-memory.yml measures cold queries on a cold index.
…eviction is available
…stead of re-encoding A non-fused disk index (for example the DiskANN-style layout, with ASH codes in memory and NVQ vectors on disk) now writes the build's ASH model and codes beside the index file during the flush. Opening the index, whether just built or reused from the index cache, loads them, and a search whose ASH settings match the stored model uses them as a deployed searcher would. Indexes without stored codes, PQ codes and mismatched settings still encode at search time. At 10M vectors the codes take about 2 s to write and under 1 s to load, where encoding them again took 12-14 s (263-284 s at 100M).
…s final Eviction works on JDK 22 as on JDK 25 (checked on Linux: no pages of the evicted file stay resident), so the cold-index test runs it from JDK 22 and the error message and benchmark guide say JDK 22+.
…nchYAML results When a search loads the ASH codes stored with a non-fused index, search_quant_time_s reports ASH.load_time_s in place of ASH.encoding_time_s, so a result row shows which of the two it did, for a cached index as well. construction.codes_write_time_s is the part of the flush spent writing the codes, and construction.codes_file_size_mb is the size of the codes file. Loading the codes after a build is now timed on its own rather than inside reopen_time_s. The benchmark guide explains when codes are stored or encoded.
…Guide A new Results section lists the fixed experiments.csv columns, and for each benchmark and named metric gives its run YAML selector, CSV key, console header and meaning. The guide's earlier paragraphs on construction and execution metrics now point to it, and How to run benchmarks and the README link to it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds ASH (Asymmetric Scalar Hashing) quantization, introduced in ASH: Asymmetric Scalar Hashing With Learned Dimensionality Reduction for High-Fidelity Vector Quantization (Tepper and Willke, CIKM 2026). ASH reduces the dimension of database vectors with a learned orthonormal projection, then quantizes each projected dimension to a few bits, while keeping queries in their original form. At the same code size it is more accurate than additive and scalar quantizers, trains and encodes quickly, and scores with SIMD lookups. This PR brings ASH to a graph index for the first time.
AsymmetricHashingtrains an ITQ or random projection;encodeAllproducesASHVectors, aCompressedVectorswith codes in compact shared pages. Graph indexes use one landmark.ASHVectorsscore construction and search.BuildScoreProvider.ashBuildScoreProviderbuilds by comparing stored codes (symmetric scoring).FusedASH(FeatureId.FUSED_ASH) stores neighbors' codes with each adjacency list and the model in the index header, at 1, 2 or 4 bits per dimension, with NVQ or full-precision reranking.QueryPreparer/PreparedQueryseparate projection from search, andBatchingExecutorqueues concurrent searches with bounded admission so that waiting queries share one batched projection.Release notes:
796.feature.md,796.enhancement.md. Documentation starts at ASH configuration and defaults and Integrating ASH queries into a database.Compatibility
CURRENT_VERSIONand the writer defaults are now 7.withVersion(6)andwrite(out, 6)keeps writing files 4.0.x reads, and a version 6 writer rejects fused ASH. DataStax Cassandra already passes its SAI-selected version to every JVector write, so it will not write version 7 until it adds an SAI version that selects it. See UPGRADING.md.OnDiskGraphIndexCompactordoes not yet compact fused ASH indexes.Dependencies
dev.ludovic.netlib:blasanddev.ludovic.netlib:lapack3.0.3 (MIT), which bringnet.sourceforge.f2j:arpack_combined_all0.1 (BSD). They are used only for ASH training, and the publishedjvectorartifact declares them. netlib calls the system BLAS and LAPACK, such as MKL or OpenBLAS, and falls back to Java implementations when none is installed, at a cost in training time.libjvector_ash_fp16-<arch>.so(19 MB on x86-64), shipped in the native JAR with its license and third-party notices. Building it needs CMake 3.13 or later;-Dnative.onednn=falsebuilds without it.Native interface
libjvector(query projection, symmetric construction scoring, neighborhood and single-vector scoring) are compatible additions under the existing handshake. Java looks each one up separately, so an older native library keeps the Java implementations; an explicitly native ASH benchmark fails instead of measuring a fallback.jvector_ash_fp16_abi, currently 1) and status codes, and no C++ exception crosses it. It loads only if present and compatible; otherwise FP16 and INT8 projection are unavailable and queries use the Highway FP32 path. See Native ABI compatibility.Validation
mvn -B clean install -DskipSIMD=false -DskipScalar=trueon JDK 22 with the native provider: 825 tests, 0 failures or errors, 6 skipped.Benchmarking
Fused ASH versus fused PQ at the same stored bytes per vector, including ASH's 5-byte header, on three large datasets. ASH has lower construction time in 6 of 6 configurations, higher recall in 12 of 12 cases and higher QPS in 22 of 24.
Construction
Total construction seconds (training, encoding, NVQ, graph build and write); lower is better.
Recall and throughput
Values: PQ → ASH. Rerank 1x and 2x rescore 10 and 20 candidates with NVQ.
PQ is marginally faster (about 4%) in the two MPNet 10M 32x cases on one core. Single-core QPS varies between graphs built from the same settings: an earlier ASH graph for that case measured 10,414 / 7,838 QPS, above PQ.
Method
One Intel Xeon 6985P-C socket (48 physical cores, two NUMA nodes), JDK 22, native provider. On-disk format 7, fused graph, degree 32, efConstruction 100, overflow 1.2, hierarchy and refinement on, NVQ reranking, search pruning off, top 10. ASH: ITQ, one landmark, D/2 projected dimensions, symmetric construction scoring, queued execution with batches of up to 8 and FP16 query projection. PQ: 8-bit subspaces sized to the same bytes, direct execution. Construction used one thread on each of 48 cores with memory interleaved across both nodes; queries ran in separate JVMs on 1 or 8 cores of one node. Construction is the mean of two builds; QPS is the mean of three samples of at least 3 s against the first build. PQ results come from an earlier run with the same machine, JDK, placement, inputs and graph parameters. Full details: ASH performance. The study can be rerun with Reproducing ASH versus PQ. During review we plan to rerun these datasets with the one-JVM recipes, run the full set of 1M datasets, and repeat both on JDK 25.