Repository navigation
Conversation
fstagni
force-pushed
the
hs23-table-generator
branch
4 times, most recently
from
November 9, 2026 10:21
c39a4af to
7aa4049
Compare
Generates the pre-aggregated snapshot of the HEPiX HEPScore23 scores table that DIRAC bundles as src/DIRAC/WorkloadManagementSystem/Client/data/HS23scores.json: the CSV is validated, aggregated to one entry per (normalized CPU model, SMT state) with a # Meas-weighted mean of Score/Ncores, and emitted as deterministic JSON with provenance (source URL, content hash, data-as-of date). Output is stable for identical input so a plain diff in the nightly CI job of the DIRAC repository tells whether anything changed. The Python tests workflow gains a step running the generator tests and linting the script, which live with it in this repository.
fstagni
force-pushed
the
hs23-table-generator
branch
from
November 9, 2026 12:51
7aa4049 to
1fd2c16
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New feature: HS23 table generator
Script that generates the pre-aggregated HS23 (HEPScore23) scores snapshot bundled with DIRAC, plus its tests. It is run nightly by a workflow in the DIRAC repository (
hs23-table-update.yml, part of DIRACGrid/DIRAC#8870): download the HEPiX CSV, validate, aggregate to one entry per (normalized CPU model, SMT state) with a# Meas-weighted mean ofScore/Ncores, emit deterministic JSON with provenance.DIRAC opens or updates a PR whenever the derived table changed (new CPU models or score changes), never merged automatically. Context: DIRACGrid/DIRAC#8871, DIRACGrid/DIRAC#8870.
Evidence
pytest hs23-table/test_generate_hs23_table.py: 7 passed (validation/anomaly skipping, weighted mean, one-entry-per-key sorting, header fields, deterministic output, normalization kept in sync with DIRAC's).Stacked on #75 (CI fixes: modernized
Python testsworkflow, CVMFS deploy scoped to master, and a latentNameErrorfix indirac-install.pyfound by pylint 4) — merge #75 first, then this PR. The workflow changes here (a step running the generator's tests and linting its script) only work once #75's workflow fixes are in.Merge Danger
Door: two-way — new files only, nothing existing depends on them.
Blast Radius: the DIRAC nightly job starts failing (visibly) if this script moves or changes its CLI; DIRAC itself is unaffected until its bundled JSON is regenerated.