Repository navigation
Separate correlation means from the jackknife patch layout - #416
Merged
Merged
Conversation
A target b of 0.01 gives reporting measurements a common approximation policy while retaining bin_slop=1 on finer integration grids. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Tree approximations depend on the patch layout. A shared two-pass measurement keeps the full-sample means canonical while retaining patch results for covariance, including tomographic cross-pairs and aperture-mass integration. Patched text dumps cannot supply a covariance and are not reused. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
…changes RhoStat and TauStat expose catalogue mappings and output tables. Measure the identical unpartitioned catalogues once, retain patched variances and joint covariance, and reuse the means across covariance draws. Record measurement settings so cached products cannot silently supply layout-dependent means. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
…try points Reporting, fine-grid, and rho/tau jobs must use the configured target rather than silently falling back to a constructor default. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
…atched covariances Synthetic auto/cross correlations and aperture-mass and rho/tau products retain identical single-thread means while their jackknife covariances differ. A patched-means mutation fails the layout test; fine grids retain bin_slop=1. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
… covariance jobs Measurement calls retain the patch results needed for dense and derived covariances. Plotting explicitly reads the published columns and variances, at round-trip precision, so one-core figure jobs do not repeat the expensive measurement. A pinned root depth makes cached products reusable across thread counts. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
A direct producer rerun must retain a full jackknife covariance and patch results; cached columns are not a substitute for covariance. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
…he layout regression The unpatched pass has no shared jackknife centres. Verify that it occurs once, while all three covariance draws retain distinct shared layouts and fresh catalogues. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Keep b_target on the published-means pass and let TreeCorr choose its standard bin_slop for patched covariance, where the tolerance does not set the reported means. Record both configurations so caches from the shared-tolerance policy are rejected. Co-Authored-By: GPT-6 Luna <noreply@openai.com>
Apply the unpatched-means/patched-covariance policy within the current sealed-SACC measurement path. Keep SACC part reuse and derived aperture-mass covariance; do not restore unsealed xi text caches.
# Conflicts: # src/sp_validation/cosmo_val/real_space.py # src/sp_validation/tests/test_cosmo_val.py # workflow/scripts/run_2pcf.py
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JK7o9F5Ad2qR25uH6YFZZg
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Our published ξ± (and M_ap², ρ/τ) change when only the jackknife patch layout changes, by up to 0.8σ. This PR makes the published values independent of the patches. The patches are then used only for the covariance.
Why the patches leak into the means. TreeCorr groups galaxies into a tree of cells. When a pair of cells is small compared with the bin width (
bin_slop), it assigns all their galaxy pairs to one bin, using the separation of the cell centres. With patches, the tree is built inside each patch, because cells never cross patch boundaries. A different patch layout gives different cells, so different galaxy pairs are approximated, and some land in neighbouring bins. Each bin's pair count, mean separation and ξ± move with the patches. Seeding the patches makes one layout reproducible, but doesn't remove the dependence.The fix: two passes.
bin_sloptight enough (≤ 1, scaled to the bin width) that the approximation is negligible. These are the published ξ±, pair counts and separations.bin_slop, used only for the jackknife covariance.ξ±, M_ap² and the measured ρ/τ all use this, including cross-catalogue and tomographic pairs. The cache records both configurations, so older products, made with one shared pass, are not reused.
Effect on SP_v1.4.6.3 (1–250′, 20 bins, 100 patches): the difference in the means between two patch layouts drops from 0.32σ rms (0.81σ max) to 5×10⁻¹²σ, which is floating-point round-off.
Tests: synthetic layout tests give bit-identical means on one thread while the covariances differ, and reverting to patched means fails the ξ± and M_ap² regressions. Container suite, slow tests included: 537 passed (3 environment failures that also occur on the base).
Not covered here: how
bin_slopitself affects the jackknife covariance (#401). The extra unpatched pass costs about 4 min at 8 threads on v1.4.6.3.— Claude on behalf of Cail
🤖 Generated with Claude Code