Skip to content

Add OllamaLiteClient for zero-dependency Ollama access#6626

Open
vijaytank wants to merge 1 commit into
spring-projects:mainfrom
vijaytank:feature/add-ollama-lite-client
Open

Add OllamaLiteClient for zero-dependency Ollama access#6626
vijaytank wants to merge 1 commit into
spring-projects:mainfrom
vijaytank:feature/add-ollama-lite-client

Conversation

@vijaytank

Copy link
Copy Markdown

Problem

Users running Ollama from CLI tools, edge utilities, or lightweight apps hit several compounding issues with the full Spring AI Ollama stack:

Solution

OllamaLiteClient is a zero-external-dependency alternative that uses only java.net.http.HttpClient (JDK 11+). It provides a single blocking generate(model, prompt) method targeting /api/generate, with proper input validation and safe JSON character escaping.

It lives alongside the existing OllamaApi in the org.springframework.ai.ollama.api package — users who need the full Spring stack (streaming, observability, retry, auto-configuration) should continue using OllamaApi + OllamaChatModel.

Users running Ollama in CLI tools, edge utilities, and lightweight
apps face real pain with the full Spring AI Ollama stack:

- The WebClient/RestClient transitive dependency graph adds
  significant classpath weight for scenarios that only need
  a single blocking generate() call (spring-projects#5838).
- WebClient read timeouts fire immediately for slow/large-prompt
  Ollama responses with no straightforward configuration path
  (spring-projects#3573).
- Reactive cancellation signals do not cleanly terminate the
  underlying Ollama inference thread, causing GPU resource leaks
  (spring-projects#2351, spring-projects#2944).

OllamaLiteClient solves these by using only java.net.http.HttpClient
(JDK 11+, zero external dependencies). It targets the /api/generate
endpoint with a simple blocking generate(model, prompt) call, proper
input validation, and safe JSON escaping. It lives alongside the
existing OllamaApi so users can choose the right tool for their needs.

See spring-projects#5838
See spring-projects#3573
See spring-projects#2351
See spring-projects#2944

Signed-off-by: Vijay Tank <vijaytank132@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants