📄 feat: Add Opt-In DOCX Extraction Profile

[lia-by-librechat]

Summary

Build the first opt-in service-side slice of LibreChat's document-parsing consolidation: a versioned POST /v1/extract profile for DOCX using the Python AnyDoc 0.1.3 binding. This matches the AnyDoc version in Marco's LibreChat PR #14701 and includes its 1,803-byte structured DOCX fixture for real contract tests.

This does not switch LibreChat callers, change RAG /text or ingestion, re-embed collections, or enable OCR or reranking. The existing upload, inspection and persistence paths remain authoritative until caller-specific parity is proven.

Mechanism

explicitly enabled, authenticated multipart request
  -> DOCX/profile check -> bounded admission (2 active, 6 queued by default)
  -> bounded temporary staging -> killable Python child
  -> ZIP decompression guard -> AnyDoc -> typed Markdown + completeness/provenance
  -> IPC cap, reaping, temp cleanup before releasing the slot

• Refuse invalid archives and zip bombs before native parsing. Cap actual decompressed bytes (25 MiB per entry, 100 MiB total, 4,096 entries), staged input (15 MiB), and serialized output (15 MiB on both sides of IPC).
• Return partial completeness for DOCX with embedded artwork. Do not claim partial text meets strict content-inspection policy. Return only stable error codes, never native exceptions or document contents.
• Fail closed without JWT_SECRET and a verified user id, even where the legacy API permits unauthenticated requests. Feature disabled by default: the route is not registered on startup, so it does not parse multipart bodies or appear in OpenAPI when off.
• On cancellation or timeout, kill and reap a child before deleting its file or granting its slot to another parse. Do not load the optional native binding into the API process.
• Document installation, opt-in flag, error taxonomy, limits, and handoff in EXTRACTION.md. The wheel is separately pinned in requirements.extraction.txt and installed in CI. Current default Docker images do not include it; an opt-in deployment must install it into a custom image.

Verification

Check	Result
Non-container Python suite, including new extraction API and existing /text, retrieval, authorization, ingestion, loader tests	324 passed, 6 skipped, 35 integration deselected
Native AnyDoc fixture	Exact Markdown/table, completeness, parser version verified over HTTP
Negative and lifecycle tests	Auth/feature gate, MIME/profile, corrupt/empty archive, zip bomb, entry count, staged input, output/IPC limits, crash and malformed response, queue saturation, timeout, cancellation/retry
Black 24.4.0, compilation, pip check, git diff --check	Passed
PostgreSQL integration locally	Attempted. With the reaper disabled, container started but its published localhost port was unreachable from this worker network namespace; fixture setup failed before any SQL assertions. CI runs the real container lane.

Installation note: full lite requirements plus test requirements still conflict at httpx via the existing Google GenAI pin. For local verification, installed the repo-pinned lite packages without that conflicting provider, plus test_requirements.txt and the new optional wheel. No existing dependency pins changed.

Known limits / next slice

• The HTTP framework may spool multipart input before route-level staging/size checks. Configure a reverse-proxy ingress body limit for internet-facing deployments. When enabled, parser concurrency bounds requests after FastAPI has parsed their multipart bodies. A true pre-multipart admission/size gate is future work.
• The main FastAPI application still initializes its normal vector store and embeddings at startup; this is not yet a standalone parsing-only service. No performance or cost improvement has been measured.
• The new route uses the current RAG JWT identity contract. Coordinate later with the strict-scoped auth migration in RAG PR 🔒 fix: close /query and /query_multiple authorization holes; add /v1/embeddings and /v1/rerank #318 before switching LibreChat traffic.
• No PDF page accounting or OCR parity, no LibreChat client changes, no reranking, no schema/data migrations. The next slice is a behind-a-flag DOCX LibreChat client with content-inspection and old/new service contract tests. Feature-off behavior leaves existing users on their previous path.
• No browser, TypeScript, real inference provider, or performance benchmark was run, since this PR does not change those surfaces.

[lia-by-librechat]

Ready for review at pushed head a59e002cc71fc761873d6ce0957038967efb065a.

This is a disabled-by-default DOCX document-v1 extraction endpoint using the pinned AnyDoc Python binding. It leaves /text, RAG ingestion, authentication for existing routes, and LibreChat callers unchanged. The new native-worker test fixture comes from Marco's AnyDoc PR #14701.

Verification on this head: 323 non-container tests passed, 6 skipped; Black, compilation, pip compatibility, and diff checks passed. Container-backed PostgreSQL verification was attempted but blocked by an inaccessible localhost port from the sandbox; the CI lane will exercise it.

Subsystem review covered parser refusal before fallback, bounded input/output and admission, media-omission signaling, auth gates, worker crash/timeout/cancellation and temp cleanup, legacy /text, and mixed-version deployment. Remaining known gaps are called out in the PR description: multipart bodies spool before the route limit, the main app still needs embedding/vector startup, strict-scoped JWT migration is separate, and no LibreChat caller has cut over.

[lia-by-librechat]

Ready for review at new pushed head 15d84b85443f8cf4e23d083c4af6b135ce3f5344.

Follow-up to the first working pass: the opt-in extraction router is now absent at startup unless RAG_EXTRACTION_API_ENABLED=true. Returning 404 inside a registered multipart route still allowed the framework to parse/spool the upload; the new head closes that default-off exposure. A cold-start test checks both flag states and confirms that the native AnyDoc module stays unloaded in the API process. The existing /text test still exercises the production application.

The real HTTP DOCX/worker contract checks pass (21 tests). The complete non-container suite has been dispatched on this head. The prior head passed GitHub CI including its configured database lane, but that CI result does not cover this follow-up commit; the new CI run is the authoritative one. The sandbox cannot reach its own Docker-published localhost port, so local SQL assertions remain unverified.