Repository navigation
Conversation
The base and local-area commands now set OMP_NUM_THREADS, OPENBLAS_NUM_THREADS, MKL_NUM_THREADS, BLIS_NUM_THREADS, VECLIB_MAXIMUM_THREADS and NUMEXPR_NUM_THREADS before numpy loads. Each gets the count its library already uses, so a host's output bytes do not change. Values already set are kept, and importing the tools changes nothing. On Linux the OpenBLAS thread count changes output bytes, so the base command now records real counts and refuses a resume with different ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copy OMP_NUM_THREADS, GOTO_NUM_THREADS and NUMEXPR_MAX_THREADS only when each is a plain positive integer. OpenBLAS and numexpr read other spellings differently, so a copied value could change their thread counts. Correct the runbook and changelog from review: name the ACS multispine staging command, cite the recorded Modal base environment, and state that setting OMP_NUM_THREADS changes scikit-learn's and torch's thread counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MICROCOSM_THREADS, when set, fills every OpenMP, BLAS, numexpr and joblib thread variable that is unset; without it the base and ACS staging commands set nothing and the release tool keeps min(os.cpu_count(), 16). The base command's resume check compares the setting and the BLAS thread count in use. These tests fail until the change that follows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The release, base and ACS multispine staging commands copy MICROCOSM_THREADS, before numpy loads, into every unset OpenMP, BLAS, numexpr and joblib thread variable, and refuse a value that is not a positive whole number. Without it the base and ACS staging commands set nothing, so each library keeps choosing its own count, and the release tool keeps its default of min(os.cpu_count(), 16). The base command records MICROCOSM_THREADS and each loaded BLAS library's thread count (threadpoolctl) in thread_environment, so a resume on a host where OpenBLAS would use a different count is refused: on Linux that count changes output bytes. threadpoolctl, already installed through scikit-learn, is now declared by microcosm-build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The commands must find the helper without their folder on the import path, the ACS local-release command applies the setting at start, a count above 64 raises numexpr's maximum, the release default keeps a blank variable, the library test needs two CPUs, and a Modal plan with an unusable MICROCOSM_THREADS is refused when it is loaded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The base, ACS staging and new ACS local-release start-up steps load the helper from their own folder, so `python -m`, `python -P` and runpy starts work again. - The ACS local-release command applies MICROCOSM_THREADS before numpy loads; before, it read the setting only when it imported the release tool partway through a run. - A count above 64 also fills NUMEXPR_MAX_THREADS, since numexpr runs one thread when asked for more than its maximum. - The release tool's default keeps a blank variable, as it always has. - A Modal plan with an unusable MICROCOSM_THREADS is refused when loaded. - threadpoolctl>=3.6, which reads numpy 2's scipy-openblas library. - The runbook and changelog name every command that reads the setting, correct scikit-learn's, torch's and MKL's rules, and say where the Linux byte measurements came from. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A MICROCOSM_THREADS value longer than int() parses must be refused with an error that names the setting, by the helper and by the Modal plan. The library test skips when joblib.cpu_count(), which also follows a CPU quota, is under two. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A MICROCOSM_THREADS value longer than int() parses is refused with an error that names the setting, by the helper and by the Modal plan. - The library test skips when joblib.cpu_count() is under two. - Comments, runbook and changelog: joblib fails on a blank LOKY_MAX_CPU_COUNT, the ACS local-release command still takes the release default when it imports the release tool, NUMEXPR_MAX_THREADS is filled above 64 and not recorded, the resume compares values as written, and blas_threads_in_use is empty only where numpy and scipy use Accelerate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR 1684's branch merged origin/main. The one conflict was the release tool's pathlib import: that branch now also imports PurePosixPath, and this branch imports Path at the top of the file for the thread setting, so the lower import keeps PurePosixPath only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #1684: this pull request targets that branch, so its diff shows only this task. It adds one thread setting,
MICROCOSM_THREADS, to the US release, base, ACS multispine staging and ACS local-release commands. When it is set, each command copies it into every library's thread variable before numpy loads. When it is not, nothing changes: the base and ACS commands set nothing, and the release tool keeps its default ofmin(os.cpu_count(), 16).The base command now records the setting and the BLAS thread count actually in use in its run configuration, so a resume on a host where OpenBLAS would use a different count is refused. On Linux that count can change output bytes.
Why one setting, and nothing set by default
--max-coresflag and thread planner of tasks 1.8 · 1 and 2.6 · 2, choose every library's thread count at once. The release tool and Modal already work this way: the release tool copiesmin(os.cpu_count(), 16)into six variables, and Modal copies the class's CPU request into four.OMP_NUM_THREADSis set) and torch's (it readsMKL_NUM_THREADSandOMP_NUM_THREADS).POPULACE_FIT_N_JOBSandPOPULACE_FIT_PREDICT_WORKERSare not set. Since Size QRF prediction workers from joblib.cpu_count() and stop comparing thread counts #1684 both followjoblib.cpu_count()when unset, andLOKY_MAX_CPU_COUNT, which the setting fills, limits that count.What changes
tools/us_build_thread_environment.py(new, standard library only)thread_setting()readsMICROCOSM_THREADS. A blank value counts as unset. A value that is not a positive whole number raisesValueErrornaming the setting.set_build_thread_environment(default_count=None)copies the count into each unset or blank one ofOMP_NUM_THREADS,OPENBLAS_NUM_THREADS,MKL_NUM_THREADS,BLIS_NUM_THREADS,VECLIB_MAXIMUM_THREADS,NUMEXPR_NUM_THREADSandLOKY_MAX_CPU_COUNT. Above 64 it also fillsNUMEXPR_MAX_THREADS, because numexpr runs one thread when given more than its maximum.LOKY_MAX_CPU_COUNT.default_countapplies only without the setting, and keeps a blank value as the release tool always has.Commands
__main__. Each loads it from its own folder, sopython -mandpython -Pstarts work.default_count=min(os.cpu_count() or 1, 16). It still runs on import, so the exact-k ladder, certification and scoring tools, which import it before numpy, read the setting too. Without the setting its values are unchanged; it now also fillsBLIS_NUM_THREADS, which no installed library reads.tools/build_us_acs_local_release.pyused to get thread values only when it imported the release tool partway through a run. It now applies the setting at start. Without the setting it behaves as before.Base command's run configuration
thread_environmentalso recordsMICROCOSM_THREADSandblas_threads_in_use: each loaded BLAS library with its thread count, read by threadpoolctl after numpy loads. The list is empty where numpy uses Accelerate (macOS).007and7, or blank and unset, differ.POPULACE_FIT_*stay recorded and not compared.thread_environmentdifferences without refusing them, so a Modal run compared with a macOS reference reportsblas_threads_in_useand does not refuse.Modal plan
tools/modal_us_stage_plan.pyrefuses a plan whoseMICROCOSM_THREADSis not a positive whole number when it is parsed. The base tool's check step loads the tool without running its start-up step, so the tool would otherwise refuse the value only after the inputs were staged.Dependency
microcosm-builddeclaresthreadpoolctl>=3.6, which scikit-learn already installs. 3.6 reads the scipy-openblas library that numpy 2 wheels bundle.uv.lockgains the dependency edge only.Docs
docs/us-modal-stage-runbook.md"Thread variables": what reads each variable, what each library does with nothing set, the Linux byte effect and where it was measured, the setting's other effects, when a change takes effect on resume, and Modal.Resume and identity effects
pyproject.tomlanduv.lock, which changed. The newthread_environmentkeys would also differ.LOKY_MAX_CPU_COUNT, which changes no imputed value.Environment variables
None is a secret, and all are optional. Wiring common to all: on Modal the runner's
tool_environmenttakes the container's environment, removes credential-like names,PYTHONHASHSEEDand runner-owned names, applies the plan'senvlast, and starts the tool withPopen(env=...). Base stage subprocesses getos.environ.copy(). Primary-QRF workers get{**os.environ, overrides}, and the overrides touch no thread variable. Route A starts commands with the operator's environment plusPYTHONUNBUFFERED.MICROCOSM_THREADS: operator shell, or a Modal plan'senv(the allowlist acceptsMICROCOSM_*). No workflow, Dockerfile, committed plan or Route A file sets it. Read by the five command files and the release tool's importers. Recorded and compared by the base command.OMP_NUM_THREADS,OPENBLAS_NUM_THREADS,MKL_NUM_THREADS: the setting, the release default, the Modal container (4 in the recordedbaserun at4b57d15a2; 2 in the check class according to the runbook), a plan'senv,.github/workflows/test.yml(1), operator shell. Recorded and compared by the base command.BLIS_NUM_THREADS: the same providers excepttest.yml; no installed library reads it. Recorded and compared.VECLIB_MAXIMUM_THREADS: the setting, the release default, operator shell. A plan may not set it. Unset in the recorded Modalbaserun. Recorded and compared.NUMEXPR_NUM_THREADS: the setting, the release default, a plan'senv, operator shell. Unset in the recorded Modalbaserun. Recorded and compared.NUMEXPR_MAX_THREADS: the setting above 64, operator shell. A plan may not set it. Not recorded.LOKY_MAX_CPU_COUNT: the setting, the release default, operator shell. A plan may not set it. Not recorded.GOTO_NUM_THREADS: operator shell only; not set by any tool, not recorded. Its effect shows inblas_threads_in_use.POPULACE_FIT_N_JOBS,POPULACE_FIT_PREDICT_WORKERS: a plan'senv(POPULACE_*) or operator shell. Not set by this change. Recorded and not compared.PYTHONHASHSEED: unchanged by this change.Unverified: what Modal sets in the
heavy,lightandacs-stagingclasses; whether the Modal container passesLOKY_MAX_CPU_COUNT,GOTO_NUM_THREADSorNUMEXPR_MAX_THREADSthrough (the receipt records only names ending in_NUM_THREADS);blas_threads_in_useon a Linux host (the tests check it on macOS, where it is empty, and with a stand-in for threadpoolctl); and the Linux byte measurements, which came from container probes on 18 November 2026 that are not stored in the repository.Test evidence
86b39fec6,78699b813,fd3f0875f).test_us_build_tool_thread_defaults.py(every case), three tests intest_us_puf_support_base_builder.pyandtest_a_plan_s_thread_setting_must_be_a_positive_whole_numberintest_us_modal_stage_plan_tool.py. The full suite was not run.main, so this stacked pull request gets no checks until it is retargeted and a commit is pushed.Tracker scope
Task 1.2 · 4's title and completed state still describe setting
POPULACE_FIT_*and filling every thread variable by default. This branch does neither, by the user's decision: one setting, nothing by default.🤖 Generated with Claude Code