[lemire]

Description

Fixes #528. Supersedes #595 (docs only) and #574 (mutex around Clone).

With copy-on-write enabled, some operations leave a bitmap's values unchanged but still write its needCopyOnWrite flags:

• rb.Clone() marks every container of rb as shared.
• x.Or(rb) and x.Xor(rb) mark the containers they share, when x also uses copy-on-write.

So goroutines that only clone one shared copy-on-write bitmap race with each other (#528). Static operations (Or(x1, x2), Flip, FastOr, …) read those flags, so they race with a concurrent Clone too. A lock around Clone alone (#574) leaves those races in place.

On these read-only paths, a flag only ever goes from 0 to 1. Only the bitmap's owner sets a flag back to 0, when it copies a container before modifying it, and modifying the bitmap concurrently is already forbidden. So atomic loads and stores are enough: no compare-and-swap, no mutex.

Changes Made

• needCopyOnWrite is now []uint32 instead of []bool, in both roaringArray and roaringArray64, because sync/atomic has no byte operations.
• needsCopyOnWrite(i) uses atomic.LoadUint32. setNeedsCopyOnWrite(i) loads first and stores only if the flag is still 0, so once a container is shared, later clones only read the flag. markAllAsNeedingCopyOnWrite goes through setNeedsCopyOnWrite.
• Every flag access on a path that can run on a shared bitmap now goes through those helpers. Paths that only the owner reaches (mergeBulk, insert/remove, resize, deserialization) keep plain accesses.
• In clone(), the new bitmap's flags are filled with plain stores, since no other goroutine can see it yet.
• frozenView puts the flags in its arena as uint32. The preceding arena sections all have sizes that are multiples of 8, so the flags stay aligned.
• Docs: SetCopyOnWrite, Clone (32- and 64-bit) and the README "Goroutine safety" section now say that read-only operations, including Clone, may run concurrently on a copy-on-write bitmap as long as nobody modifies it, and that each goroutine may modify its own clone.

Testing

New tests cow_concurrency_test.go and roaring64/cow_concurrency_test.go build a shared copy-on-write bitmap with array, bitmap and run containers. They run these operations on it together from many goroutines:

• Clone, and cloning then modifying the clone
• in-place x.Or, x.Xor, x.And and x.AndNot with the shared bitmap as argument
• static Or, Xor, And, AndNot, Flip, FastOr, FastAnd, HeapOr, ParOr and ParAnd
• Contains, Rank, Select, ToArray, iteration, Intersects, the cardinality functions, Equals and WriteTo

Afterwards they check that the source is unchanged and that modifying the source does not affect an earlier clone.

• On master, under -race: 39 data races reported in the root package and 33 in roaring64.
• With this PR: go test ./... and go test -race ./... pass.
• go vet reports nothing new. Cross-builds for linux/s390x (big-endian) and linux/386 succeed.

Performance Impact

A/B on Apple arm64, with a synthetic bitmap of 2048 containers (1024 bitmap, 1024 array):

benchmark	master	this PR
Clone of a copy-on-write bitmap	3.26 µs, 39.1 KB	4.47 µs, 45.3 KB
in-place x.Or(src), both copy-on-write	18.2 µs	19.6 µs
static Or, copy-on-write source	0.91 ms	0.90 ms
static Or, no copy-on-write	1.32 ms	1.23 ms (noise)
Add/Remove, no copy-on-write	100 ns	100 ns

The atomic loads themselves cost little: replacing them with plain loads saves only about 170 ns on the Clone benchmark. Most of the extra time and memory in a copy-on-write Clone comes from the 4× larger flag slice (4 bytes per container instead of 1). Packing the flags into a []uint64 bitset with atomic.OrUint64 would remove that cost. But inserting and removing at an index would then need bit shifting, so I kept []uint32 for simplicity. That could be a follow-up.

Breaking Changes

None. needCopyOnWrite is unexported, and the serialization formats are unchanged.

Related Issues

Fixes #528. Supersedes #574 and #595.