diff --git a/docs/hop-user-manual/modules/ROOT/pages/metadata-types/ai-provider.adoc b/docs/hop-user-manual/modules/ROOT/pages/metadata-types/ai-provider.adoc index a94b19b5387..a4f5e0ee991 100644 --- a/docs/hop-user-manual/modules/ROOT/pages/metadata-types/ai-provider.adoc +++ b/docs/hop-user-manual/modules/ROOT/pages/metadata-types/ai-provider.adoc @@ -40,7 +40,8 @@ The same pattern applies to every other secret in the project: never hard-code i See xref:hop-gui/perspective-ai-advisor.adoc#secrets-and-personal-information[Secrets and personal information]. The xref:pipeline/transforms/languagemodelchat.adoc[Language model chat] transform can optionally select a named AI Provider. -The provider's type, base URL, API key, chat model, timeout and temperature then take precedence over the transform's inline values; an inline value is only used when the provider leaves that field empty. +The provider's type, base URL, API key, chat model, timeout, temperature, max output tokens and thinking then take precedence over the transform's inline values; an inline value is only used when the provider leaves that field empty. +With an Ollama provider, the context size always comes from the provider (16384 when it is empty), and an Anthropic provider without *Max output tokens* sends 4096 rather than the transform's value. In particular, a temperature set on the provider overrides the temperature on every transform that uses it, so leave *Temperature* empty here if transforms need their own. Input/output mapping, mock mode, proxy and retries stay on the transform. See xref:pipeline/transforms/languagemodelchat.adoc#ai-provider-precedence[AI Provider and inline settings] for the details. diff --git a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/languagemodelchat.adoc b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/languagemodelchat.adoc index dd32da23819..e8bc9c1e046 100644 --- a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/languagemodelchat.adoc +++ b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/languagemodelchat.adoc @@ -98,15 +98,21 @@ When *AI Provider* is set, the transform starts from its own settings and then c | Timeout | The provider's *Timeout (seconds)* if it resolves to a whole number, otherwise the inline timeout. +| Max New Tokens +| The provider's *Max output tokens* if it resolves to a positive whole number, otherwise the inline *Max New Tokens*. An Anthropic provider without one sends 4096, not the inline value. + +| Context Window Size (Ollama) +| The provider's *Context size (tokens)* if it resolves to a positive whole number, otherwise 16384. The inline *Context Window Size* is not used. + | Thinking (Ollama) | The provider's *Thinking* if it is *Off* or *On*, otherwise the inline *Thinking*. |=== -Variables in provider fields are resolved first, so apart from *Temperature*, a field holding a variable that resolves to an empty value counts as empty. Empty provider fields fall back to the inline values, not to the defaults listed for each provider type on the xref:metadata-types/ai-provider.adoc[AI Provider] page. +Variables in provider fields are resolved first, so apart from *Temperature*, a field holding a variable that resolves to an empty value counts as empty. Empty provider fields fall back to the inline values, not to the defaults listed for each provider type on the xref:metadata-types/ai-provider.adoc[AI Provider] page. The exceptions are *Max New Tokens* for Anthropic and *Context Window Size* for Ollama, described above. The inline fallbacks come from the provider's model type, not from the *Model API* selected on the transform. With an Anthropic provider, for example, an empty provider *Model name* falls back to the inline Anthropic model name, even when the transform itself was set up for OpenAI. -All other options always come from the transform: mock mode, parallelism and the input and output fields, and the model-specific options such as Top P, Top K, maximum tokens, response format, proxy, retries and logging. The model-specific ones are read from the transform's fields for the provider's model type. +All other options always come from the transform: mock mode, parallelism and the input and output fields, and the model-specific options such as Top P, Top K, response format, proxy, retries and logging. The model-specific ones are read from the transform's fields for the provider's model type. To tune the temperature per transform while sharing one provider, leave *Temperature* empty on the provider and set it on each transform.