Skip to content

docs(vllm): update environment variables to match the current worker - #969

Open
lukepiette wants to merge 1 commit into
mainfrom
docs/vllm-env-vars-current
Open

lukepiette wants to merge 1 commit into
mainfrom
docs/vllm-env-vars-current

Conversation

@lukepiette

Copy link
Copy Markdown
Contributor

What

The vLLM environment variables page documents settings the current worker ignores. I checked every variable on the page against runpod-workers/worker-vllm v2.30.3 (src/args_builder.py, src/main.py, the Dockerfile, and docs/configuration.md). Of 89 documented variables, 37 are not read by the worker.

Why it matters: someone who sets GUIDED_DECODING_BACKEND, SPECULATIVE_MODEL, or RAW_OPENAI_OUTPUT gets no error and no effect.

Changes

  • Removed variables for flags vLLM dropped (the worker logs a warning and ignores them) and for the worker's old output batching. They move to a "Removed variables" table with the replacement for each.
  • Added STRUCTURED_OUTPUTS_CONFIG, SPECULATIVE_CONFIG (with the JSON example from the worker docs), RESPONSE_ROLE, ENABLE_LOG_REQUESTS, VLLM_STARTUP_TIMEOUT, REQUEST_TIMEOUT, VLLM_EXTRA_ARGS, and VLLM_CONFIG_FILE.
  • Corrected MODEL_NAME (required, no default; the page said facebook/opt-125m) and TENSOR_PARALLEL_SIZE (the worker doesn't set it from the GPU count).
  • Docker build arguments now match the Dockerfile (VLLM_VERSION, MODEL_NAME, MODEL_REVISION, TOKENIZER_NAME, TOKENIZER_REVISION, QUANTIZATION, BASE_PATH); WORKER_CUDA_VERSION is gone.
  • OpenAI compatibility page: drops RAW_OPENAI_OUTPUT and renames the role variable.
  • Worker version in the intro: v2.30.1 to v2.30.3.

How it was checked

A script parsed the worker's allowlist (VALUE_FLAGS, PAIRED_BOOL_FLAGS, BOOL_FLAGS, ENV_ALIASES, REMOVED_FLAGS) and the variables the wrapper reads, then classified every variable in the page's tables. After this change, all 59 remaining variables are ones the worker reads.

Not checked

  • Default values for the rows I kept. They weren't re-verified against vLLM v0.32.0 one by one.
  • Nothing was deployed. This is from source, not from a running worker.

Related, not in this PR

The worker's own Hub listing (.runpod/hub.json) still offers several removed variables in the deploy form (DEFAULT_BATCH_SIZE, RAW_OPENAI_OUTPUT, SPECULATIVE_MODEL, OPENAI_RESPONSE_ROLE, and others). That needs a fix in runpod-workers/worker-vllm.

🤖 Generated with Claude Code

The page listed variables the vLLM worker no longer reads. Checked every
documented variable against worker-vllm v2.30.3 (src/args_builder.py and
docs/configuration.md): 37 of 89 were ignored.

- Remove variables for flags vLLM dropped and for the worker's old output
  batching, and list them under "Removed variables" with replacements
- Add STRUCTURED_OUTPUTS_CONFIG, SPECULATIVE_CONFIG, RESPONSE_ROLE,
  ENABLE_LOG_REQUESTS, VLLM_STARTUP_TIMEOUT, REQUEST_TIMEOUT,
  VLLM_EXTRA_ARGS and VLLM_CONFIG_FILE
- MODEL_NAME is required and has no default; the worker does not set
  TENSOR_PARALLEL_SIZE automatically
- Docker build arguments now match the Dockerfile
- OpenAI compatibility page: drop RAW_OPENAI_OUTPUT, rename the role variable

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@mintlify

mintlify Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
runpod-docs 🟢 Ready View Preview Nov 14, 2026, 8:04 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

This branch was successfully deployed

1 active deployment
staging — 211a826e Deployed Nov 14, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants