[fstagni]

New feature: CPU power from the HEPiX HS23 table

Part of #8871.

Summary

The pilot has always measured the node with DB12 at bootstrap and written the result to /LocalSite/CPUNormalizationFactor. This PR makes the HEPiX HEPScore23 (HS23) scores table the primary source of the node's CPU power, with DB12 demoted to fallback. Phase 1 records and reports only: nothing consumes the HS23 value yet (ADR: docs/adr/0001-hs23-cpu-power-from-hepix-table.md).

dirac-wms-cpu-normalization
  CPUNormalizationUnit = [units]        # ordered list, first = active unit, default ["DB12"]
  ├─ "HS23" in units → getHS23Score()   # bundled snapshot, no network at runtime
  │    getNodeInfo()                    # psutil core counts + SMT; model from /proc/cpuinfo
  │    lookupScore(entries, model, SMT) # dict lookup on normalized model
  ├─ "DB12" in units → DB12 measurement # or fallback when active=HS23 and lookup failed
  → /LocalSite/CPUNormalizationFactorHS23   (new)
  → /LocalSite/CPUNormalizationSource       (new: HS23table | DB12)
  → /LocalSite/CPUNormalizationFactor       (unchanged, DB12)

What is new

Thing	Where
Bundled, pre-aggregated HS23 snapshot: one entry per (normalized CPU model, SMT), # Meas-weighted mean of Score/Ncores, provenance header (source URL, content hash, data-as-of)	WorkloadManagementSystem/Client/data/HS23scores.json (158 entries, 48 KB, shipped in the wheel)
Table lookup: node detection, model normalization, aliases, dict lookup with safety-net pass	WorkloadManagementSystem/Client/HS23Table.py
New LocalSite options	/LocalSite/CPUNormalizationFactorHS23, /LocalSite/CPUNormalizationSource = HS23table | DB12
Unit policy (setup-global, no per-site choice)	Operations/JobScheduling/CPUNormalizationUnit — ordered list, default ["DB12"]; membership decides what is recorded, the first entry is the active unit
Snapshot refresh	nightly workflow .github/workflows/hs23-table-update.yml runs the generator from DIRACGrid/management#74 and opens/updates a PR on any derived change (new models or score changes), never merged automatically
Job parameters	CPUNormalizationFactorHS23 and CPUNormalizationSource published alongside CPUNormalizationFactor

The table is never fetched at runtime: a release pins a known table state. A CPU model newer than the bundled snapshot falls back to DB12 until the table PR is merged and released — deliberately no per-site escape hatch.

Node detection: logical/physical core counts and the SMT state come from psutil (already a hard dependency); only the CPU model name needs reading /proc/cpuinfo, as psutil does not expose it on Linux.

Deliberately not in this PR: consumers honoring the active unit (Matcher, JobAgent, Watchdog, accounting) — that is phase 2, and it must come with recalibration of the DB12-calibrated thresholds (StopSig*, MinimumTimeLeft, LHCb CPUe). With the default ["DB12"], behavior is unchanged.

Evidence

• pytest src/DIRAC/WorkloadManagementSystem/Client/test/Test_HS23Table.py: 15 passed, including a sanity check of the bundled snapshot itself (normalized keys, plausible scores) and lookups against the real data.
• test_setInitialJobParameters extended: both new parameters are published when present in ceArgs, absent otherwise.
• Generator + tests live in HS23 table generator for the DIRAC bundled snapshot management#74 (7 tests).
• The 4 failures in Test_JobWrapper.py (test_*dirac_jobexec*) fail identically on integration without this PR.

Merge Danger

Door: two-way — with the default ["DB12"] no consumer behavior changes; pilots just record one extra option in pilot.cfg.

Blast Radius: pilot bootstrap on every site; the nightly workflow is new and fails visibly if the generator CLI changes.

Companion PRs: DIRACGrid/Pilot#289 (ConfigureCPURequirements tolerates a missing DB12 value) and DIRACGrid/management#74 (generator script + tests).

[aldbr]

I haven't reviewed it yet, but from the description: is there any reason for keeping CPUNormalizationFactor instead of using CPUPower?

[aldbr]

Are you sure you want to add this here?
There is some documentation about allocated time left in https://github.com/DIRACGrid/DIRAC/blob/fb7d33f0f20426b5dd4fe5c22b50299ce65ac198/docs/source/AdministratorGuide/Systems/WorkloadManagement/allocatedTime.rst

[fstagni]

I am indeed not sure I want to keep this glossary (one of the several reasons why this is a draft).

[fstagni]

CI note: the 6 PilotWrapper check failures are not from this PR — the prepare environment step of .github/workflows/pilotWrapper.yml invokes conda directly, which is no longer on the PATH of the current ubuntu-latest runner image (ubuntu26/20261101). It fails identically on other open PRs (e.g. #8874's branch, runs from 2026-11-09 08:14), while this same branch passed on 2026-11-05 and all other PilotWrapper runs before 2026-11-09 pass. The workflow needs conda restored to the runner (or an explicit setup step, e.g. actions/setup-miniconda).