Skip to content

feat(ai-lakera-guard): scan LLM responses (direction output/both, non-streaming + streaming)#13606

Open
janiussyafiq wants to merge 8 commits into
apache:masterfrom
janiussyafiq:feat/ai-lakera-guard-pr2
Open

feat(ai-lakera-guard): scan LLM responses (direction output/both, non-streaming + streaming)#13606
janiussyafiq wants to merge 8 commits into
apache:masterfrom
janiussyafiq:feat/ai-lakera-guard-pr2

Conversation

@janiussyafiq

@janiussyafiq janiussyafiq commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Description

PR-2 of ai-lakera-guard, following the input-scanning MVP (#13570). Adds response (output) scanning for non-streaming and streaming (SSE) traffic. Back-compatible: direction still defaults to input.

  • Schema: direction extended to input/output/both; adds response_failure_message.
  • Response path (lua_body_filter, the same dispatch ai-aliyun-content-moderation uses):
    • Non-streaming: scans ctx.var.llm_response_text; a flagged response is replaced with a provider-compatible deny.
    • Streaming: buffers the SSE response, scans the assembled completion once, then releases it verbatim (clean) or replaces it with a deny SSE terminated by [DONE] (flagged). Because the stream's 200/text/event-stream headers are already committed when buffering begins, a streamed block is delivered as the deny bodydeny_code does not apply to streams.
  • Docs (en + zh) and tests added.

Which issue(s) this PR fixes:

Part of #13291.

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible

@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. enhancement New feature or request labels Jun 25, 2026
Comment thread apisix/plugins/ai-lakera-guard.lua Outdated
Comment thread apisix/plugins/ai-lakera-guard.lua
Comment thread apisix/plugins/ai-lakera-guard.lua
Comment thread apisix/plugins/ai-lakera-guard.lua
Comment thread apisix/plugins/ai-lakera-guard.lua Outdated
… mode and fail-open support; update documentation and tests
…sts for fail-open behavior; update documentation
nic-6443
nic-6443 previously approved these changes Jun 26, 2026
AlinsRan
AlinsRan previously approved these changes Jun 26, 2026
membphis
membphis previously approved these changes Jun 26, 2026
@AlinsRan AlinsRan self-requested a review June 26, 2026 02:33
Comment thread apisix/plugins/ai-lakera-guard.lua
…for blocking and clean responses; add tests for output direction
@AlinsRan AlinsRan self-requested a review June 26, 2026 05:39
…prove fail-closed behavior; add tests for streaming scenarios
@janiussyafiq janiussyafiq dismissed stale reviews from membphis, AlinsRan, and nic-6443 via ab6cc45 June 26, 2026 07:26
@dosubot dosubot Bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants