fix(Endpoints CLI): Ensure compatibility when deploying with specific model configs - #591
blainekasten wants to merge 2 commits into
Conversation
Broly Security ScanNote ✅ Clean scan Note Re-scan this PR anytime with
|
| fallback = await _resolve_explicit_config_for_public_model( | ||
| config, | ||
| public_model_name=public_model.name, | ||
| reference_model_id=public_model.base_model_id or _profile_model_id(candidate_profiles[0]), |
There was a problem hiding this comment.
why candidate_profiles[0]?
model with per-quantization profiles can point at different model ids (BF16 → ml_pub, FP8 → ml_fp8), so when base_model_id is unset we only search configs under whichever profile the API happened to list first. A legitimate LoRA config whose reference model is ml_fp8 would come back empty and the user gets "Config X is not valid for model Y" — the same misleading error we're fixing, just in a narrower case. Could we prefer public_model.base_model (the resource path, like resolve_model_reference does) and/or query the distinct profile model ids rather than just the first?
No description provided.