Repository navigation
Conversation
…e real-model tests release
- Replace test_multi_switch_{e2e,alora,mixed_tech,realistic}.py with
test_multi_switch_real_checkpoint.py: one compose helper for the multi,
multi-alora and multi-mixed checkpoints instead of three copies. Point
vllm_generate's import, comments and MULTISWITCH_EXPLAINED.md at it.
- Pass all repos after a single --adapters flag (a repeated flag keeps only
the last list), so multi-mixed really gets rag + guardian; pipeline-parallel
equivalence now uses the shared helper instead of its own copy.
- Let the conversation adapter-map regex accept trailing fields: aLoRA entries
now carry invocation_drop_chars, and the old regex matched no aLoRA entry.
Add a CPU unit test that parses a template built by the real composer.
- Mark the real-checkpoint HF, vLLM-generate, serving, aLoRA-cache, prefix-cache
and pipeline-parallel tests release, and list them in docs/CICD.md.
test_conversation_concurrency stays out: it never composes a checkpoint.
Signed-off-by: noaa <noaa.kless@gmail.com>
noaakl
requested review from
antonpibm,
aviv1ron1,
freunda and
yairallouche
as code owners
November 17, 2026 16:56
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #167. Puts the tests gated on
GRANITE_SWITCH_E2E_MODELSinto the release tier, following the per-file audit of those tests.Changes
Merge the 4 HF real-checkpoint files into one.
test_multi_switch_{e2e,alora,mixed_tech,realistic}.py->tests/hf/test_multi_switch_real_checkpoint.py(31 tests). They overlapped and carried three near-identical compose helpers;_realisticborrowed_e2e's model. One helper now buildsmulti,multi-aloraandmulti-mixed(same output dir names, so warm reuse still works). No checks dropped.Fix: repeated
--adapterskept only the last repo list.multi-mixedwas being built from guardian only, so the "mixed LoRA + aLoRA" tests never had a mix. All repos now follow a single--adapters. Pipeline-parallel equivalence uses the shared helper instead of its own copy.Fix: conversation adapter-map regex matched no aLoRA entry. aLoRA entries in the composed template now carry
'invocation_drop_chars': Nafterinvocation_text; the regex required}right after it, so the prefix-cache and concurrency tests found 0 aLoRA adapters and bailed out. New CPU testtests/unit/test_conversation_adapter_map_parse.pyparses a template built by the real composer, so the next format change fails on PRs.Release markers on: the merged HF file,
test_multi_switch_vllm_generate,test_multi_switch_serving_e2e,test_multi_switch_alora_cache,test_conversation_prefix_cache,test_pipeline_parallelism_equivalence. Listed indocs/CICD.md.test_conversation_concurrencystays out: it never composes a checkpoint and needs one pre-built atGRANITE_SWITCH_CONV_DIR.Not in this PR: converting these files to fakes for PR runs.
Testing
Vela, 2x A100, vLLM 0.26 (
uv sync --frozen --group test), main 756218a + this branch:tests/unittests/hf/test_multi_switch.py-m releaseon the 7 filesThe failure:
test_batched_generation_matches_solo(vLLM generate), request 1 differs from solo only at the 16th and last token (653vs3392). It passed on a rerun on clean main. The compose command and the test body are unchanged by this PR (only its import path), so this looks like a batch-size near-tie flip (vLLM is not batch-invariant) rather than a regression, but one fail vs one pass doesn't prove that.Note for anyone with a persistent
GRANITE_SWITCH_E2E_DIR: deletemulti-mixedthere once. Checkpoints are reused wheneverconfig.jsonexists, so the old guardian-only build would keep being used.