Skip to content

[Router] Add routerSpec.tolerations to router Deployment#1006

Draft
Arkaniad wants to merge 1 commit into
vllm-project:mainfrom
Arkaniad:router-tolerations
Draft

[Router] Add routerSpec.tolerations to router Deployment#1006
Arkaniad wants to merge 1 commit into
vllm-project:mainfrom
Arkaniad:router-tolerations

Conversation

@Arkaniad

Copy link
Copy Markdown

Summary

Add support for routerSpec.tolerations in the Helm chart, enabling the router deployment to schedule onto tainted nodes (e.g., dedicated node pools).

Motivation

The router deployment template currently renders routerSpec.affinity and routerSpec.nodeSelectorTerms but silently drops routerSpec.tolerations. The serving engine deployment already supports servingEngineSpec.tolerations, so the two deployment schemas are asymmetric.

When deploying the router to a dedicated node pool with taints (e.g., for larger root volumes to accommodate the 5.4 GiB router image), there was no way to add the matching toleration to the router pod.

Changes

  • helm/values.yaml: Add routerSpec.tolerations field with helm-docs comment
  • helm/templates/deployment-router.yaml: Render tolerations block using {{- with .Values.routerSpec.tolerations }}
  • helm/README.md: Add routerSpec.tolerations row to Router Configuration table
  • helm/tests/deployment-router_test.yaml: Add test cases for tolerations rendering
  • helm/values.schema.json: Add tolerations property to routerSpec schema

Testing

  • helm template with default values: no tolerations block rendered (correct)
  • helm template with routerSpec.tolerations set: tolerations block rendered correctly
  • Added helm-unittest test cases to verify both scenarios

Example Usage

routerSpec:
  tolerations:
    - key: dedicated
      operator: Equal
      value: router
      effect: NoSchedule

Signed-off-by: Arkaniad arkaniad@users.noreply.github.com

- Add routerSpec.tolerations field to values.yaml (mirrors servingEngineSpec.tolerations)
- Render tolerations block in deployment-router.yaml template
- Add routerSpec.tolerations row to README Router Configuration table
- Add helm-unittest test cases for tolerations rendering
- Update values.schema.json with tolerations property

This enables deploying the router to tainted nodes (e.g., dedicated node pools)
by specifying tolerations in the Helm values.

Signed-off-by: Arkaniad <arkaniad@users.noreply.github.com>
Signed-off-by: Rhea Danzey <rdanzey@element.io>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for configuring Kubernetes tolerations on the router pod via routerSpec.tolerations. It includes updates to the Helm chart templates, values schema, default values with examples, documentation, and corresponding unit tests to verify correct rendering. There are no review comments, so no further feedback is provided.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant