[RFE] Support context window sizing (max-seq-len) in recommend tool

[natoscott]

Add max_seq_len parameter to ConfigIQ /recommend tool so teams can compare GPU costs across different context tiers without using the CLI.

Field teams need to answer: "If we serve GLM at 64k instead of 1M tokens, how many fewer GPUs do we need?"

Right now:

• AIC CLI already supports --max-seq-len to constrain KV cache allocation
• REST API and ConfigIQ webapp don't have this parameter
• Teams are stuck running CLI multiple times to compare context tiers

Change

1. Add max_seq_len to REST API

   • Just pass it through to cli_recommend() — it's already supported there

2. Add input fields to ConfigIQ /recommend page

   • For agg mode: single "Max sequence length (tokens)" field
   • For disagg mode: two fields — one for prefill workers, one for decode workers (optional, can override per-worker if needed)
   • Help text: "Constrains KV cache allocation. Default is input + output sequence length (not the model's max_position_embeddings). Set lower to model serving at specific context tiers."

The model's max is a hard upper limit, but the default allocation is just what's needed for your typical workload. This way you can size KV cache for 64k context without the model knowing it supports 1M.

Notes

• CLI already has this: aiconfigurator cli recommend --isl 4000 --osl 500 --max-seq-len 64000
• For disagg, CLI supports --prefill-max-seq-len and --decode-max-seq-len for per-worker tuning
• Python SDK already supports both patterns
• Just needs REST wrapper + ConfigIQ UI

[natoscott]

Re the REST API change:

EstimateRequest already has the pattern for per-worker disagg support (prefill_tp_size, decode_tp_size, etc.). Should follow the same pattern for max_seq_len:

RecommendRequest and EstimateRequest should add:

• max_seq_len: int | None — applies to both agg and disagg (default: isl + osl)
• prefill_max_seq_len: int | None — override for prefill workers in disagg (optional)
• decode_max_seq_len: int | None — override for decode workers in disagg (optional)

This mirrors the CLI (--max-seq-len, --prefill-max-seq-len, --decode-max-seq-len) and existing EstimateRequest pattern.