Repository navigation
Update the PHZ catalog notebook to Euclid DR1 - #409
Conversation
Use the DR1 deep survey PHZ and MER tables (with the renamed interval columns), describe the four DR1 PHZ tables, and start with the catalog rather than the mosaic search. Add a comparison of the photometric redshifts with the DR1 SPE spectroscopic redshifts, using spe_z_prob and spe_z_rel to identify the reliable ones, and plot the spectrum of the brightest galaxy with a reliable spectroscopic redshift, retrieved by position with the SSA service, with its emission lines. Read the mosaic cutouts from the cloud and update the gallery description.
…tion table Widen the photometric redshift range to the one where Halpha falls in the red grism, plot only the reliable spectroscopic redshifts against the photometric ones, and retrieve the spectra of the example galaxy through euclid.objectid_spectrafile_association_dr1 and the spectrumDM service, since the DR1 spectra will not be in SSA/CAOM at release.
Use the irsacloud spectrumDM endpoint with the euclid_combspec_dr1 dataset, and build the URL in the ADQL query on the association table, as recommended by IRSA.
|
@jkrick thank you :) |
jkrick
left a comment
There was a problem hiding this comment.
Thanks for this one too! The DR1 PHZ notebook flows really well, and the new photo-z vs spec-z comparison is a great addition. I ran it against irsadev and it runs cleanly in about a minute; reading the cutout straight from S3 works nicely.
As with #408, a few of these go beyond the DR1 switch (marked from main), but since the code changed a lot it seemed like a good time to raise them. Several also match comments on #408, since the two notebooks share code; it'd be great to keep them consistent.
Before release (Nov 12)
- Please add a release-day checklist to the top of this PR conversation so one person can do the release. It should cover: removing the irsadev
Irsa.tap_url/Irsa.sia_urlcell (lines 128–132); confirming the DR1 table, SIA collection, and S3 bucket names on production; filling in the Tucci et al. reference (line 476); rerunning the notebook against production; rechecking the prose that describes specific results (items 3 and 4); and checking the Firefly output in section 6. Right now Firefly shows the table and overlay but fails to load the DR1 VIS mosaic ("could not retrieve image render data"), while the Q1 mosaic of the same field loads fine, so it should work once DR1 is public, but it's worth confirming that the image loads. - Add numpydoc-style docstrings to all functions (
bad_flux_binsandgrism_namehave comments instead).
Text that doesn't match the current output
- Section 5 text (lines 425–426) doesn't match what the notebook now shows. The brightest galaxy with a spectrum (2678621437659721934, z = 0.894) has only a blue grism spectrum. Hα is clear at 1.24 µm, but there's no visible [OIII]: it falls at the noisy blue edge, 0.95 µm. There's also no narrow spike near 1.3 µm. Please update the text to match what the plot shows, and recheck it after the release-day rerun.
- Line 352: "not every galaxy with a spectroscopic redshift has a spectrum listed in it yet" — the "yet" suggests a temporary state. Please confirm on production and reword or remove it.
Code
- Spectrum loop (lines 355–366): if no galaxy has a spectrum listed, the loop ends silently on the last galaxy and the notebook plots an empty figure. If no galaxy has a reliable redshift,
galaxyis undefined. Afor ... elsethat prints a clear message (or raises) would handle both. - Spectrum plot (lines 410–422): only mark emission lines inside the plotted wavelength range (as #408's
plot_spectradoes). The legend in the upper left currently covers the Hβ and [OIII] labels; moving it outside the axes, as in #408, fixes that. - Lines 357–359: same as #408. The ADQL query both looks up where each spectrum is stored and builds a web-service URL by gluing strings together with
||. It's easier to follow if the query returns only the plain columns and Python builds the URL. Please use the same approach in both notebooks. bad_flux_bins,grism_name, and the best-spectrum selection are copied from notebook 3. Please keep them identical to whatever #408 ends up with.- Section 3 (lines 232–238) builds an
IN (...)list of all 417 object IDs. For larger regions this list gets very long. ALEFT JOINto the SPE catalog in the section 2 query (withspe_rank = 0in theONclause) would avoid the second query and the Pythonjoin. - Small items:
- The loop variable
galaxy(line 355) could get a more specific name, to match #408. tileID(line 297) is now only printed. (from main, where it was used for a search) Remove it, or
say why it's useful.
- The loop variable
Figures
- Section 4 figure (partly from main): with the wider redshift range, the cutout now has 417 red circles and it's hard to get much out of it. Could you plot only the 81 galaxies with a reliable spectroscopic redshift, as open circles colored by
spe_z(with a colorbar), and mark the galaxy used in section 5? That ties sections 3–5 together. Use a colormap without a dark end (e.g.cool) so the circles stand out on the image.
Thanks again! No need to send this back to me for another review: once items 1–4 are done (the release-day checklist, docstrings, and the two text fixes). The rest are suggestions, so use your judgment on those, or open a follow-up issue for anything you'd rather not do in this PR.
|
Thanks @jkrick! All addressed in the latest commit:
I also checked the notebook against the two DR1 PHZ papers (Hartley et al. 2026; Enia et al. 2026) and fixed a few things: the references now point to these DR1 papers instead of the Q1 one (Tucci et al. is now cited as the Q1 PHZ paper); the description of |
Update the PHZ catalog notebook to Euclid DR1 73c2003
What changed
photo_zandclassification) and uses the deep survey PHZ and MER tables. It also notes the renamed interval columns (phz_90_int_1/_2; Q1 usedphz_90_int1/2).phz_classification = 2,phz_flags = 0, a narrow 90% PDF interval, andphz_medianbetween 0.85 and 1.85 (the range where Hα falls in the red grism).spe_rank = 0) and keeps only reliable spec-z (spe_z_prob > 0.99,spe_z_rel > 0.05), then plots photo-z against spec-z.euclid.objectid_spectrafile_association_dr1and builds the spectrumDM URLs in the ADQL query (IRSA's recipe), because DR1 spectra won't be in SSA at release. It keeps the best red and blue grism spectra and marks Hβ, [OIII] and Hα at the spec-z. This matches the approach in notebook 3.notebook_metadata.yml, and the DR1 PHZ paper (Tucci et al. 2026) added to the references.Notes for reviewers
Irsa.tap_urlandIrsa.sia_urlatirsadev. That cell has aTODOand must be removed when DR1 goes public. Please don't merge to production before then.TODOto fill in the journal and arXiv details once the paper is out.