fix: make Ask output token budget configurable

[bferanmi806-sketch]

Description

Make the Ask/Q&A output token budget configurable so intermediate and final answers are not silently truncated at the strategy step's smaller budget.

• Structured strategy generation remains fixed at 2000 tokens.
• Intermediate answers and final answers use OPEN_NOTEBOOK_ASK_MAX_TOKENS.
• The default is 8192 tokens.
• Malformed, zero, and negative values safely fall back to 8192 with a warning.
• Added focused regression tests and configuration documentation.
• No API, frontend, provider, dependency, or database behavior changed.

Related Issue

Fixes #1221

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (bug or feature that would cause existing functionality to not work as expected)
• Documentation update
• Code refactoring (no functional changes)
• Performance improvement
• Test coverage improvement

How Has This Been Tested?

• Tested locally with Docker
• Tested locally with development setup
• Added new unit tests
• Existing tests pass (uv run pytest)
• Manual testing performed (describe below)

Test Details:

• uv sync - completed successfully.
• uv run pytest tests/test_ask_token_budget.py -v - 9 passed, 1 dependency deprecation warning.
• uv run pytest tests/ -v - 611 passed, 3 failed, 49 errors, 2 dependency warnings. The failures and errors are pre-existing Windows path, proxy-environment, and temp-directory permission issues; the same representative failures reproduce on clean upstream a7de90d38aaf18ee85fd661854d35c11e44613e2.
• uv run ruff check . - passed.
• uv run ruff format --check open_notebook/graphs/ask.py tests/test_ask_token_budget.py - 2 files already formatted.
• uv run python -m mypy . - no issues found in 131 source files.
• git diff --check - passed.

Design Alignment

Which design principles does this PR support? (See VISION.md)

• Privacy First
• Simplicity Over Features
• API-First Architecture
• Multi-Provider Flexibility
• Extensibility Through Standards
• Async-First for Performance

Explanation:

The change adds one narrow, process-level configuration helper instead of expanding the request/API contract. It preserves provider-agnostic model selection and passes the budget through the existing per-call Esperanto configuration path.

Checklist

Code Quality

• My code follows PEP 8 style guidelines (Python)
• My code follows TypeScript best practices (Frontend; no frontend changes)
• I have added type hints to my code (Python)
• I have added JSDoc comments where appropriate (no TypeScript changes)
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas (the helper is self-documenting and has no complex undocumented logic)
• My changes generate no new warnings or errors

Testing

• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes (focused tests pass; unrelated baseline failures are documented above)
• I ran linting: make ruff or ruff check . --fix
• I ran type checking: make lint or uv run python -m mypy .

Documentation

• I have updated the relevant documentation in /docs (if applicable)
• I have added/updated docstrings for new/modified functions
• I have updated the API documentation (no API changes)
• I have added comments to complex logic (no complex logic was added)

Database Changes

• I have created migration scripts for any database schema changes (no database changes)
• Migration includes both up and down scripts
• Migration has been tested locally

Breaking Changes

• This PR includes breaking changes
• I have documented the migration path for users
• I have updated MIGRATION.md (if applicable)

Screenshots (if applicable)

Not applicable; this is a backend-only change.

Additional Context

The strategy step remains deliberately small because it produces a bounded structured Strategy object with at most five searches. The two prose-generation stages have different output requirements and therefore use the configurable budget independently.

Pre-Submission Verification

Before submitting, please verify:

• I have read CONTRIBUTING.md
• I have read VISION.md
• This PR addresses approved Issue [Bug] Ask (Q&A) answers are silently truncated due to hardcoded max_tokens=2000 in graphs/ask.py — especially visible in non-English languages #1221 claimed by bferanmi806-sketch
• I have not included unrelated changes
• My PR title follows conventional commits format: fix: make Ask output token budget configurable

---

Thank you for contributing to Open Notebook!

[cubic-dev-ai]

All reported issues were addressed across 5 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>