Repository navigation
docs(vllm): update environment variables to match the current worker - #969
Open
lukepiette wants to merge 1 commit into
Open
lukepiette wants to merge 1 commit into
lukepiette wants to merge 1 commit into
Conversation
The page listed variables the vLLM worker no longer reads. Checked every documented variable against worker-vllm v2.30.3 (src/args_builder.py and docs/configuration.md): 37 of 89 were ignored. - Remove variables for flags vLLM dropped and for the worker's old output batching, and list them under "Removed variables" with replacements - Add STRUCTURED_OUTPUTS_CONFIG, SPECULATIVE_CONFIG, RESPONSE_ROLE, ENABLE_LOG_REQUESTS, VLLM_STARTUP_TIMEOUT, REQUEST_TIMEOUT, VLLM_EXTRA_ARGS and VLLM_CONFIG_FILE - MODEL_NAME is required and has no default; the worker does not set TENSOR_PARALLEL_SIZE automatically - Docker build arguments now match the Dockerfile - OpenAI compatibility page: drop RAW_OPENAI_OUTPUT, rename the role variable Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
lukepiette
marked this pull request as ready for review
November 14, 2026 20:42
lavanya-gunreddi
approved these changes
Nov 15, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The vLLM environment variables page documents settings the current worker ignores. I checked every variable on the page against
runpod-workers/worker-vllmv2.30.3 (src/args_builder.py,src/main.py, the Dockerfile, anddocs/configuration.md). Of 89 documented variables, 37 are not read by the worker.Why it matters: someone who sets
GUIDED_DECODING_BACKEND,SPECULATIVE_MODEL, orRAW_OPENAI_OUTPUTgets no error and no effect.Changes
STRUCTURED_OUTPUTS_CONFIG,SPECULATIVE_CONFIG(with the JSON example from the worker docs),RESPONSE_ROLE,ENABLE_LOG_REQUESTS,VLLM_STARTUP_TIMEOUT,REQUEST_TIMEOUT,VLLM_EXTRA_ARGS, andVLLM_CONFIG_FILE.MODEL_NAME(required, no default; the page saidfacebook/opt-125m) andTENSOR_PARALLEL_SIZE(the worker doesn't set it from the GPU count).VLLM_VERSION,MODEL_NAME,MODEL_REVISION,TOKENIZER_NAME,TOKENIZER_REVISION,QUANTIZATION,BASE_PATH);WORKER_CUDA_VERSIONis gone.RAW_OPENAI_OUTPUTand renames the role variable.How it was checked
A script parsed the worker's allowlist (
VALUE_FLAGS,PAIRED_BOOL_FLAGS,BOOL_FLAGS,ENV_ALIASES,REMOVED_FLAGS) and the variables the wrapper reads, then classified every variable in the page's tables. After this change, all 59 remaining variables are ones the worker reads.Not checked
Related, not in this PR
The worker's own Hub listing (
.runpod/hub.json) still offers several removed variables in the deploy form (DEFAULT_BATCH_SIZE,RAW_OPENAI_OUTPUT,SPECULATIVE_MODEL,OPENAI_RESPONSE_ROLE, and others). That needs a fix inrunpod-workers/worker-vllm.🤖 Generated with Claude Code