feat(agent): verify "saved to Drive" is real before completing (#17)#246
Merged
Conversation
…#17) A research Project's checkpoint claimed "saved the report to Google Drive" but only wrote a local .md — a false completion (the demo's Google-Doc payoff was fiction). Now the agent can't finish a Drive/Google-Doc deliverable unless a real doc was actually produced. - codec_gdoc_verify.py: wants_gdoc(expected_output) detects a Drive/Google-Doc deliverable; find_doc_url() looks for a real docs.google.com/document/... link in the checkpoint history; verify_deliverable() returns (ok, reason); doc_exists() is a best-effort live Drive lookup that returns None (not False) when OAuth is unavailable, so a missing setup never wedges a checkpoint that did produce a URL. Pure/stdlib except that one optional live call — fully unit-testable. - codec_agent_runner: at the checkpoint_done gate, if the checkpoint wanted a Google Doc but no real doc URL is in the history, inject ONE nudge ("[DELIVERABLE NOT DONE] ... create the real doc with the google_docs skill") instead of completing. Nudge state is seeded from history so a resume doesn't re-nudge, and it fires at most once so an unreachable OAuth can't loop forever. The agent already has google_docs (create_google_doc exists, styled + tables). Verified: 72 tests pass (8 verifier + 2 runner-gate new). The exact bug is covered — a "save to Drive" checkpoint that only produced a local .md is refused and nudged; a checkpoint with a real docs.google.com URL completes. ruff clean. NEEDS LIVE OAUTH (flagged): actually creating the doc + the live doc_exists check require Google credentials on the machine; the gating logic works regardless. Live-test: run a Project whose deliverable is "save the report to a Google Doc" — it now won't mark done on a local .md; it creates the real doc and returns its URL. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A research Project claimed 'saved to Google Drive' but only wrote a local .md. New codec_gdoc_verify + a checkpoint_done gate refuse that false completion: if the deliverable wants a Google Doc but no real docs.google.com URL is in the history, the agent is nudged once to actually create it (via the existing google_docs skill). 72 tests. Actual doc creation needs live Google OAuth (flagged); the gate logic works regardless.