Skip to content

Fix BSI CompareValue for negative values and for foundSet columns without a value - #598

Open
Assertive-xxh wants to merge 1 commit into
RoaringBitmap:masterfrom
Assertive-xxh:fix/bsi-comparevalue-583
Open

Assertive-xxh wants to merge 1 commit into
RoaringBitmap:masterfrom
Assertive-xxh:fix/bsi-comparevalue-583

Conversation

@Assertive-xxh

Copy link
Copy Markdown

Description

Fixes the two CompareValue problems reported in #583, and a third one that the test for them turned up.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)

Changes Made

What was changed?

  • BitSliceIndexing: rewrote the per-column comparison in compareValue.
  • BitSliceIndexing and roaring64: CompareValue intersects foundSet with the existence bitmap.
  • Tests in bsi_edgecases_test.go and roaring64/bsi64_test.go.

Why was it changed?

  1. RANGE left the bit loop as soon as one bound was decided and treated the other bound as equal. With -5, 3, -100 stored, RANGE [-10, 0] returned all three columns.
  2. A column of foundSet that holds no value was compared as if it held zero, so GE 0, LT 15 and RANGE [0, 100] returned it.
  3. Not in the issue: a stored value and a bound of opposite signs and equal magnitude compared as equal, because the bound was negated before the bitwise comparison. With 100 and -100 stored, EQ -100 returned both columns and GT -100 missed 100.

How was it changed?

  • Values of opposite signs are ordered by sign alone. Values of the same sign are ordered by their remaining 63 bits, compared as unsigned, so no bound is negated (this also avoids negating MinInt64).
  • RANGE follows both bounds in one pass and stops only when both are decided.
  • roaring64 had problem 2 only, on the big-value path (a BSI with 64 planes). The int64 path already intersected.

Testing

  • TestBSICompareValueIssue583 has the cases from the issue plus the opposite-sign case.
  • TestBSICompareValueMatchesNaive (both packages) checks all six operations against a plain comparison of the stored values, over mixed-sign values including MinInt64 and MaxInt64, with and without a foundSet that contains columns holding no value.
  • All three tests fail on master and pass with this change.
  • go test, go test ./roaring64 and go test ./BitSliceIndexing pass on Windows amd64 with Go 1.24.4 and 1.25, and with GOARCH=386. The arm and arm64 builds pass. The ppc64 test binaries compile; I could not run them.

Formatting

gofmt -s -l and go tool unconvert ./... report nothing.

Fuzzing

go test -run='^$' -fuzz=FuzzSmat reports "no fuzz tests to fuzz" for me on master, so I ran what I could find: go test -tags=gofuzz -run Smat passes, and FuzzBsiStreaming ran for 60s without a failure. Neither of them calls CompareValue.

Performance Impact

The number of Contains calls per column is unchanged. A non-nil foundSet costs one extra And.

Before:

BenchmarkCompareValueRunOptimizedEndToEnd-18    4971    242927 ns/op    371658 B/op    104 allocs/op

After:

BenchmarkCompareValueRunOptimizedEndToEnd-18    5042    234665 ns/op    371663 B/op    104 allocs/op

## Breaking Changes

None. Results change only where they were wrong.

## Related Issues

Fixes #583

## Additional Notes

- Not changed: `roaring64` with a bound outside the range of the stored values, which the doc comment already says may be wrong. The `roaring64` test only uses bounds in range.
- Not changed: a BSI with more than 64 planes (#586).
- #540 rewrites the same function for speed. It is open and currently conflicts with master; this change does not build on it.
- I used an AI assistant (Claude Code) while working on this. I have read and tested the change and can answer questions about it.

Three results of CompareValue were wrong (RoaringBitmap#583):

- RANGE left the bit loop as soon as one bound was decided and treated
  the other bound as equal, so with negative values it could return
  every column.
- A stored value and a bound of opposite signs and equal magnitude
  compared as equal, because the bound was negated before the bitwise
  comparison: EQ -100 also matched 100, and GT -100 missed it.
- Columns of foundSet that hold no value were compared as if they held
  zero.

The per-column comparison now orders values of opposite signs by sign
alone and compares the remaining bits as unsigned, so no bound is
negated, and RANGE follows both bounds in one pass. foundSet is
intersected with the existence bitmap.

roaring64 only had the foundSet problem, on the big-value path.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 13, 2026 08:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants