Repository navigation
Add AI-assisted triage to the admin Review center - #1711
Merged
Paul Lizer (paullizer) merged 31 commits intoOct 8, 2026
Merged
Paul Lizer (paullizer) merged 31 commits into
Paul Lizer (paullizer) merged 31 commits into
Conversation
…restricted screen Layer 1 of 3 of the Admin Review Center stack. - Warn user executes when the safety reviewer saves the review, without an approval request, and is audited in the activity log. Saving again does not resend it. Suspend and Block still need a second eligible reviewer. - Executed warnings must be acknowledged: new user-scoped routes GET /api/safety/warnings/pending and POST /api/safety/warnings/<id>/acknowledge (owner-only, idempotent, ETag-conditional), a V2 dialog fed by a bootstrap pending count, and the acknowledgment state in the admin review and the user's Violations tab. - Escalate can no longer be chosen; legacy records keep it, labelled "Escalated (legacy)". escalate_count stays in the stats JSON. - Approved suspensions and blocks store the notice the user was sent in settings.access.notice. - user_required answers restricted users with a structured 403 (error: access_restricted) for API calls and redirects pages to /v2/access-restricted or /access-restricted. New login-only routes serve those pages and GET /api/v2/access-restriction (caller's own data only), exempt from the Terms of Use gate. - Version 0.261.296. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds the fix and feature notes, updates the safety review and approval request guides and the Content Safety settings page, records the release notes for 0.261.296, lists the new logging tags, and regenerates the docs app surface inventory for the classic Access restricted page. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ser test Render the real classic templates and scripts behind a closed API: no Escalate choice, the legacy label, the second-reviewer help text, the warning acknowledgment line, Blocked statistics, and the localized restore time on the classic Access restricted page. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…restricted branch The base now uses 0.261.296, so this branch moves to 0.261.297: config.py, its tests, docs and a new release-notes section above the base's 0.261.296 entry. Both route-inventory coverage lines are kept. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A reviewer can warn about the same violation again. The V2 store hid any warning whose violation it had already acknowledged, so a newer warning never showed in a tab that stayed open. It now keys each warning on the violation and its send time. The acknowledge route also takes the issued_at of the warning the user read. When the violation now holds a newer warning, it answers 409 safety_warning_replaced and records nothing, and the dialog reads the pending warnings again to show the newer one. A body without issued_at keeps the previous behaviour. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Two overlapping saves with Warn user, such as a double-click or two reviewers, both passed the already-sent check, so each sent a notification and only one was recorded. A save now claims the violation with a write conditional on the ETag it read (action_request_status 'sending') before anything is sent; the other save sends nothing and gets 409 safety_warning_in_progress. The outcome is written on the claimed version. While a claim is fresh, other saves and deletes wait, and 'sending' never counts as a warning to acknowledge. A claim older than five minutes reads as a failed send and can be retried once. A send whose claim was lost is audited and reported as safety_warning_not_recorded instead of overwriting another writer. The classic Save Review button is disabled while its request is in flight. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds list search, filters and server-resolved names, select-all-matching ids, dashboard windows on the feedback and safety stats, bulk review endpoints and an approvals summary. Denied and expired remediation requests now unlock their violation, re-saving an applied suspension or block no longer requests it again, and review writes are ETag-conditional. Exports take the list filters. Version 0.261.298. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…hboard Extracts the Approvals rail into CategoryRailPage and the Control Center tiles, charts and data tables into DashboardParts. Approvals gains a Dashboard category and a Safety remediation category, Group requests now lists group requests only, and its list reads status, type and who-it-is-for filters from the address. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
One Review center entry opens a rail of each section's dashboard, workbench and queues. Workbenches list records beside their detail, with search, address filters, multi-select and bulk actions; records open in full-page editors with Back and a dirty guard. Unchecked chat content moves to its own page with sequential bulk recheck. Old routes redirect. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Covers bulk caps and per-item results, select-all-matching ids, stats windows with legacy fields kept, list search by display name, the approvals summary's visibility, denied and expired requests unlocking violations, unchanged restrictions not requested twice, and ETag-conditional review writes. Moves the V2 page source checks to the Review center files and adds the new routes to the policy inventories. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Covers rail access per role with the not-available states and old-route redirects, dashboard figures opening filtered workbenches, click and Shift+click selection with per-record bulk reports, select-all-matching and the counted delete confirmation, the editor's dirty guard, save return and conflict reload, the violation editor's notice and suspension presets, sequential unchecked chat rechecks, and the Approvals Dashboard and Safety remediation categories. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds the feature and fix notes, a Review center guide, and updates the feedback, safety violation, chat recheck and approval request guides, the release notes for 0.261.298 and the logging tag inventory. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
POST /api/admin/review/feedback/assist and /api/admin/review/safety/assist analyze one record for the editor's draft or triage up to ten, storing each suggestion on its record as ai_suggestion. Records are read on the server by id and shown to the model under request-local handles with no user ids, emails or names. The reply is validated against a strict schema with one correction round, and policy is enforced on the server: no Escalate, no remediation for AI-generated findings, no weakening of an applied remediation. A content-filter refusal is retried one record at a time. Suggestions carry a fingerprint of the reviewable fields, so a pending one reads stale only when the record itself changes. Bulk update operations accept suggestion_id to apply a pending suggestion through the normal save, credited in the audit log, and dismiss_suggestion dismisses one. Lists and ids take ai=pending; feedback gains a theme field, filter and dashboard breakdown. Users never see suggestions or themes about themselves. New settings enable_admin_review_ai_assistant (off) and admin_review_ai_guidance live with the reviewer permissions; the guidance never reaches browsers. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Core tests cover request checks, identity-free views, the correction round, handle checks, policy, content-filter isolation, locked records, the limiter refund and the suggestion lifecycle. Route probes run the real routes offline with a scripted model and a real limiter. A policy test pins the decorators, the toggle check and the section's own container. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A Suspend or Block save created its approval request and then wrote the request onto the violation without checking what had changed meanwhile. A warning claimed during that window was overwritten and later recorded as executed over the pending request, and two overlapping restriction saves left the first approval approvable but unlinked. - Record a new request only on the violation's remediation state as it was read, and refuse a fresh warning claim. When the write is refused, withdraw the approval (withdraw_approval_request) so nothing stays approvable, and return 409 record_changed. - The approval executor refuses a request its violation no longer awaits (safety_log_awaits_request). - A save or archive that names an etag is written on that version or refused with record_changed, never retried onto a newer one. Without an etag, a retry merges only the fields the request changed. This covers safety and feedback PATCH, archive and bulk update/archive. - Archive refuses a violation whose warning is being sent. - Re-request messages name the "Request this ... again" control. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The classic safety review never sent reissue, so after a request was denied, expired or failed, saving the same Suspend or Block returned "No new suspension was requested" and dropped the new date and message. - Classic review: add a "Request this suspension/block again" control, shown when the violation already records that action and no request is waiting. It sends reissue: true. The help text describes where the last request stands, the notification and restore time are only sent when something will be requested, and the restore time must be in the future. - Review center editor: the same rule and wording through offersRestrictionReissue and existingRestrictionText. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… center The feedback and safety editors get an Ask AI panel that analyzes the record, fills the unsaved draft with the suggested review, marks each changed field and undoes it; a suggestion stored by triage saves through the bulk route with its suggestion id. The workbenches offer Triage with AI for checked records, ten per request with progress, waits for the rate limit and Cancel, and report each record's outcome. Each section gets an AI suggestions queue listing what every suggestion changes and why, flagging stale and locked rows, with inline notification edits, Approve selected, Approve all ready (suspensions and blocks excluded and never ticked by select all) and Dismiss, confirming how many users are warned and how many requests are created. The Feedback dashboard adds a themes breakdown and the workbench a Theme filter. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ullizer/simplechat into paullizer-admin-review-ai-assist # Conflicts: # application/single_app/route_backend_feedback.py # application/single_app/route_backend_safety.py # functional_tests/test_support/safety_review_harness.py
…n from the queue A suggestion that repeats the suspension or block its violation already records is labelled as already on the violation, still left out of Approve all, and approving it updates the review only; the queue never sends reissue. The confirmation and the report count these separately from new requests, and a record_changed, suggestion_stale or suggestion_not_pending refusal reads in the reviewer's terms on its row. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ssist is off Bulk update operations that carry a suggestion_id and dismiss_suggestion operations are refused per operation with review_assistant_disabled (403) while enable_admin_review_ai_assistant is off; the rest of the request runs. Stored suggestions stay on their records and are offered again when the assistant is turned back on. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Like the other bulk actions, a Triage with AI run leaves checked only the records that got no suggestion, including any it never sent after a cancel or stop, so they can be tried again or reviewed by hand. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The guidance is shown on the admin settings page, so 'never sent to browsers' was not accurate; it is sent to the model with each request and never to the Review center. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds the feature note, Review center, feedback and safety guide sections, the Security settings rows and troubleshooting, the features catalog entry, the regenerated docs inventory, the REVIEW_ASSIST logging tag and the 0.261.299 release notes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
This was referenced Oct 8, 2026
…s' saves working across suggestion writes Cross-record prompt injection: a triage request's records are grouped by the user each one is about and every group is its own model call, with its own handles, so text one user wrote can't steer what the model writes for another user's record. The first group always runs; groups that can't start in time, or whose call fails after others were answered, come back 'deferred' for the browser to send again. As a backstop, text a record's user can read (feedback analysis notes, action taken and response; a violation's notes and notification) is refused when it repeats 40+ characters of another record in the request and not of its own, and is refused rather than cut when too long. The prompt says which fields the user reads. /ids now returns each record's owner so the browser can keep a user's records together. Suggestion writes vs open editors: the lists and record reads return a fingerprint of the reviewable fields, which now also covers every remediation state field (request id and status, send claim, warning notification, issue and acknowledgment). A save whose etag is stale but whose fingerprint still matches goes ahead on the fresh read, with every guard and a write conditional on that version; a real change, claim, new request or acknowledgment is still refused with 409 record_changed. Bulk suggestion operations: a read failure, or any unexpected error, fails only that operation with operation_failed (500) and is logged; the rest of the request still runs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…r, and save editors with their fingerprint The AI suggestions queue shows, under 'Visible to the user', the whole text each suggestion saves that its record's user can read, marks cleared fields, labels the editable notification the same way, and the approval confirmation counts reviews that save such text. Dismissals name only the suggestion id. Triage chunks keep each user's records together, using the owners the page and 'Select all matching' report, and records the server answers 'deferred' are sent again next, with a guard against resending a chunk that made no progress. The editors and queue approvals send the record's fingerprint with its etag. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… suggestion writes Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…is always false CodeQL (js/trivial-conditional) flagged `while (!settled)` in runTriage: every path that set `settled = true` broke out at once, and the only `continue` (after a rate-limit wait) left it false, so the condition was always true and the loop really ran until a `break`, bounded only by `waits < 3`. Not a logic bug, but it hid the retry rule. `triageRetryWait` now decides whether an answer is worth waiting out (a 429, or a 503 other than the limiter's own, with a wait within the limit), and the send loop's condition is that decision plus the REVIEW_TRIAGE_MAX_WAITS (3) cap. Behaviour is unchanged. New node tests pin the cap (one try and three waits, then stop), the limiter and too-long-wait exceptions, a waited-out 503, Cancel during a wait and before the first send, and that a full run leaves nothing unprocessed; mutating the cap or the limiter exception fails them. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CodeQL flagged js/xss-through-dom in the classic safety review: the action select's value, which is DOM text, was written into data-action attributes on the notification and restore-time fields. The select is now read only through readSelectedAction(), which returns the matching constant from ACTION_VALUES (None, WarnUser, SuspendUser, BlockUser and the legacy Escalate). An empty or unknown value reads as None, as an empty one did before. The remediation fields and the save both use it, so only those constants reach the data attributes and the request. Every value the select can hold is in the list, so the review behaves the same. A browser test checks that a forged option reaches neither the data attributes nor the save, while a chosen action is kept as it is. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…o paullizer-admin-review-ai-assist
Paul Lizer (paullizer)
merged commit Oct 8, 2026
1316cd9
into
microsoft:paullizer-react-v2-ui
11 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
r1…) with no record/user/conversation ids, names or emails, emails/GUIDs in text replaced, text truncated, plus a server-computed prior-violation count and optional org guidance. One model call never covers two users' records: groups that don't fit in the request's time come backdeferredand the browser resends them. Text a user can read (feedback analysis notes/action/response, violation notes/notification) is refused, never cut, when too long, and refused when it repeats 40+ characters of another record in the request. A content-filter refusal for a group is retried per record. Telemetry is content-free; 60 requests per 10 minutes per reviewer, one at a time.ai_suggestion(pending/applied/dismissed, shown as stale) using conditional writes and a fingerprint of the reviewable fields plus, for violations, the full request/warning state (not_etag, which changes when the suggestion is stored). Lists and record reads returnetag+fingerprint; editors send both, so a save across a stored/applied/dismissed suggestion still goes ahead on the fresh read (all guards, write conditional on that version), while a real edit, claim, new request or acknowledgment still gets409 record_changed. Bulkupdateacceptssuggestion_id(refused withsuggestion_stale/suggestion_not_pending), new bulk opdismiss_suggestion, list/ids filterai=pending,/idsreturnsowners; a suggestion operation that can't read its record fails alone withoperation_failed. Applies and dismissals are credited in the admin activity log. Feedback reviews also gain a Theme with a list filter and a dashboard breakdown.Notes for reviewers
enable_admin_review_ai_assistant(default off) andadmin_review_ai_guidance(≤ 2,000 chars). Bootstrap exposes only the boolean; the guidance is stripped bysanitize_settings_for_user()and never sent to the Review center.POST /api/admin/review/feedback/assistandPOST /api/admin/review/safety/assistuse the same section decorators as their record routes, plus the toggle (403 review_assistant_disabled). While the toggle is off, bulk suggestion operations are refused per operation with the same code; other operations in the request still run.354a7681,46793704,b8dccb97): per-user model calls withdeferred+ browser resend and owner-grouped chunks; the copied-text backstop; full user-visible text in the queue; fingerprint-tolerant saves across suggestion writes; per-operation failure handling in bulk suggestion ops. One Layer 2 assertion (test_review_center_bulk_and_dashboards.py, exact/idsbody) now includes the additiveownersfield.f635cd63):js/trivial-conditionalflaggedwhile (!settled)inrunTriage(lib/reviewSuggestions.ts): everysettled = truewas followed bybreak, so the condition was always true. Not a behaviour bug; the retry rule is now the loop condition (triageRetryWait+REVIEW_TRIAGE_MAX_WAITS= 3), with new node tests for the wait cap, the limiter and too-long-wait exceptions, a waited-out 503, and Cancel before and during a wait.ac6e7139(classicadmin-safety-violations.jsaction select and its UI test only) in2690fd0e.WorkflowAssistModelover a fake client).Linked issue
No tracking issue: the requester declined one for this work. Stacked on #1709 and #1710.
Release Notes & Latest Features
Is this visible to end users?
Is this admin-facing (Admin Settings, governance, deployment, config)?
Should this become a Latest Feature card?
Screenshot needed for the card?
Version bump
application/single_app/config.pyVERSIONthird segment bumped, or not needed because this is docs-only (0.261.298 → 0.261.299; unchanged for the review and CodeQL fixes, same unreleased PR)deployers/version.txtbumped, or not needed becausedeployers/was not changedTesting / validation
The full app import fails on this machine (pyOpenSSL), so routes were exercised through Layer 1's
functional_tests/test_support/safety_review_harness.pyand fresh-process offline import probes with a fake model invoker. All results below are for head2690fd0e.python -m pytest functional_tests/test_review_assist_core.py functional_tests/test_review_assist_routes.py functional_tests/route_tests/test_review_assist_policy.py functional_tests/route_tests/test_route_blueprint_policy_inventory.py functional_tests/route_tests/test_route_unauthenticated_policy_contract.py functional_tests/route_tests/test_route_policy_test_coverage.py functional_tests/test_review_center_bulk_and_dashboards.py functional_tests/test_safety_remediation_approval_state.py functional_tests/test_approvals_dashboard_stats.py functional_tests/test_safety_warning_send_claim.py functional_tests/test_v2_admin_review_pages.py -p no:cacheprovider -q→ passed, 132 passed (core 47; assist routes 14, normal and-OPython; assist policy 6; blueprint inventory 14; unauthenticated contract 9; policy coverage 3; Layer 1/2 regressions 39).node functional_tests/test_v2_review_assist_logic.mjs→ passed (includes the new triage retry tests; mutating the wait cap or the limiter exception makes it fail)node functional_tests/test_v2_review_center_logic.mjs→ passednpm run typecheck(inapplication/v2_ui) → passednpm run build(inapplication/v2_ui) → passed (only the existing chunk-size warning)python -m pytest ui_tests/test_v2_review_center_ai_assist.py ui_tests/test_v2_review_center.py ui_tests/test_v2_approvals_dashboard.py ui_tests/test_v2_approvals_and_terms_pages.py ui_tests/test_classic_safety_review_and_access_restricted.py -p no:cacheprovider -q→ passed, 35 passed (11 AI-assist tests; 7 classic safety tests including Layer 2's forged-option test; strict fixtures fail on unexpected requests)python functional_tests/test_docs_app_surface_coverage.py→ passed, 7/7python functional_tests/test_docs_site_quality.py→ passed, 6/6python scripts/check_xss_sinks.py --base-sha 6570decdb --head-sha HEAD <changed files>→ passed (38 files);python scripts/check_broken_access_control.py --base-sha 6570decdb --head-sha HEAD <changed files>→ passed (20 files)git diff --check 6570decdb HEAD→ passed2690fd0e: CodeQL success (no warnings or errors; 2 notices from Layer 1'sab55a8f8: a cyclic import atfunctions_authentication.py:766and mixed tuple lengths inget_pending_safety_warnings), plus Analyze (python, javascript-typescript, actions), syntax, swagger-route, xss-sink, broken-access-control, malicious-PR review, branch-flow and CLA all passing.functional_tests/test_v2_admin_security_parity.py::test_every_v1_security_field_is_claimed(chat_content_settings_present),functional_tests/test_v2_admin_settings_schema.py::test_defaults_match_their_field_type(m365_trusted_download_hosts),functional_tests/test_logging_tag_standardization.py(workflow/editor-assist tags;[REVIEW_ASSIST]is standard and documented),functional_tests/test_docs_link_integrity.py(2/5; existing release-notes andexplanation/links, none added here),functional_tests/test_docs_release_notes_integrity.py(generated release-notes pages stale since before this stack).test_profile_and_admin_review_tabs(3),test_v2_stats_parity,test_v2_conversation_deep_link, bell UI teardown 404s, 12 agent/MCP editor UI tests, and thetest_v2_sidebar_conversation_scroll.pyphone-drawer flake.Documentation
docs/explanation/release_notes.md, v0.261.299)docs/explanation/features/ADMIN_REVIEW_AI_ASSISTANT.md; guidesadmin-review-center.md,admin-review-feedback.md,admin-review-safety-violations.md;docs/admin/security.md;docs/_data/features.yml;docs/_data/app_surface.yml;docs/reference/logging-tags.md)Security checklist
@swagger_route(security=get_auth_security())(decorator order asserted byfunctional_tests/route_tests/test_review_assist_policy.py)sanitize_settings_for_user()(bootstrap sends only the boolean; the guidance is stripped)