A skill for filming a prototype (sp-create-video), and sp- in front of every skill

AGENTS.md

@@ -8,7 +8,7 @@ holds only *data*.
 Code, shipped to every install:
 
 `skills/` holds `clone-prototype`, `new-ui-mock`, `prototype-canvas`,
-`define-product` and `brand-kit`.
+`define-product`, `brand-kit` and `sp-scene-video`.
 `.claude/skills/` and `.agents/skills/` are symlinks to it, so this checkout
 loads the same tree an install does.
 

README.md

@@ -168,7 +168,7 @@ folder added after boot appears on its own.
 
 ## The workflow
 
-Five skills, in `skills/` (which `.claude/skills/` and `.agents/skills/`
+Six skills, in `skills/` (which `.claude/skills/` and `.agents/skills/`
 symlink to, so this checkout loads what an install does):
 
 | Skill | Use it for |
@@ -178,6 +178,7 @@ symlink to, so this checkout loads what an install does):
 | **prototype-canvas** | Running and operating the canvas: boards, `layout.json`, the `window.snapCanvas` bridge, annotated-screenshot review, the force-refresh. |
 | **define-product** | Working out what the product is before anything is drawn: an interview, one or two questions at a time, problem before solution, gaps left TBD rather than invented. It writes the project's `PRD.md`, which the canvas shows as the first tab, with the screen inventory new-ui-mock designs from. |
 | **brand-kit** | Collecting a product's own brand and promotional material -- the company's own brand or press kit, store listings, verified social accounts, the newsroom -- and turning it into the image rows of a canvas folder, each asset carrying its source and whether the company published it. |
+| **sp-scene-video** | Filming the prototype: a live-action clip of a real person using it, in an office or a lift or on the street, with the interface kept pixel-exact rather than redrawn. An animated board becomes the reference video a generative model has to keep; a green plate and a corner-pin composite are there for when it will not. |
 
 The rule the whole thing is built around: **every colour and every metric in
 a cloned artboard traces to a measurement.** Grid the reference image, look

RELEASE-NOTES.md

@@ -19,6 +19,12 @@ The app, its skills and the toolkit carry one version.
 
 Everything below is on `main` and reaches no install until a version is cut.
 
+- **A new skill films the prototype.** `sp-scene-video` turns a board into a
+  live-action clip of someone using the product -- an office, a lift, a walk
+  down a corridor -- with your interface on the phone rather than a redrawn
+  imitation of it. It needs a 火山方舟 (Volcengine Ark) key of your own, and
+  each take costs a few yuan, so it shoots one and shows you before going on.
+
 ## v1.6.2
 
 2026-09-24. Start a project from an app or an idea, keep what you paste on a

docs/2026-09-25-scene-video.md

@@ -0,0 +1,77 @@
+# Filming a prototype: what the takes taught, and what shipped
+
+2026-09-25. Over 10 and 11 September a session on `notion-purchase-motion`
+tried to put the cloned Notion purchase sheet into a real scene: a person in
+an office, the phone in their hand, the flow playing on it. Thirteen paid
+takes later it worked. `skills/sp-scene-video` is that session, kept.
+
+## The model will redraw an interface it is only shown
+
+The first take sent the three replica screens as reference images and asked
+for them back pixel for pixel. What came back was a convincing office, a
+convincing phone, and a purchase sheet that was *nearly* the one we drew:
+right layout, right blue, invented words. No amount of 不要自行发明任何界面
+changed that. A model given pictures of a UI composes a UI.
+
+What changed it was giving it a **video** of the interface instead, as
+`reference_video`, with the prompt saying the screen is the one thing that
+must survive untouched — 每一帧内容、文字、颜色、动画节奏都必须和参考视频
+完全相同，不要重新绘制. Then it kept the pixels and painted a room around
+them. That inversion is the skill's whole thesis, and why the skill's first
+phase is building an animated board rather than writing a prompt.
+
+The reference video is board 19, `19-purchase-sheet-motion.html`: the same
+`gen.py`, one CSS timeline, the flow playing. `scripts/frames.mjs` scrubs it
+frame by frame through CDP — animations paused, `animation-delay: -Ts` —
+because a screen recorder drops frames under load and never says so.
+
+## Ark, not fal, and not Replicate
+
+fal.ai was first: a key, a payment, a working `make.py`. It never produced a
+clip. Replicate listed `bytedance/seedance-2.5` at $0.9676 per second of
+output. 火山方舟 is ByteDance's own API for its own model, and after 企业实名
+认证 a 720p ten-second take with audio cost about ¥11.
+
+So the shipped script talks to Ark and only Ark. A provider flag for a path
+that has never returned a video would be untested code in every install, and
+`ark.py` is 120 lines precisely because it does one thing. The cost of that
+choice is real: the skill needs a PRC company to sign up. That is written in
+its `compatibility` line rather than worked around.
+
+## The green plate is kept, as the rescue route
+
+Before the reference video worked, the answer was going to be compositing:
+have the model shoot a phone with a chroma-key green screen and six tracking
+crosses, then corner-pin the real frames back onto it. That produced good
+clips — a hand in a lift, the sheet perfect on the glass, the thumb on top of
+it — and `scripts/composite.py` is that code: convex hull of the key, four
+edges picked by orientation so a thumb biting a corner cannot steal one,
+homography refined on the crosses, the plate's own luma multiplied back over
+the UI so the reflections survive.
+
+It ships because the guarantee is different in kind: reference video gives a
+screen that is *nearly* verbatim, a composite gives one that is exactly
+verbatim. It ships as the fallback because it costs a day and constrains the
+shot to a locked camera. It is run through `uv run --with
+opencv-python-headless`, so the toolkit keeps its two dependencies.
+
+## Two things that cost an evening
+
+Ark refuses a `data:` URI on `video_url`, so the reference video needs a URL
+its servers can fetch. A `cloudflared` quick tunnel does it, but it returned
+`530` for hours: it has to run outside the agent's sandbox, and the user's
+VPN answered DNS with fake 198.18.x addresses, so its edge SRV lookup failed.
+Pinning `--protocol http2 --edge <ip>:7844` fixed it. Both are in
+`references/reference-video.md` next to a `curl` check, because submitting a
+take against an unreachable video still costs money.
+
+And a poll that dies is not a task that died. An SSL timeout killed the runner
+mid-generation and the clip had to be fetched by task id afterwards. `ark.py`
+now retries the poll and prints the id first.
+
+## What is not here
+
+No mp4 is committed anywhere. Takes, plates and frames are derived, large, and
+belong in the project's `scratch/video/`, which is already ignored. The only
+artefact of a shoot that belongs in a canvas folder is the motion board, which
+is a board like any other and shows on the canvas.

skills/README.md

@@ -1,6 +1,6 @@
 # skills
 
-The five skills the app ships and links into each agent, one directory each, in the Agent Skills
+The six skills the app ships and links into each agent, one directory each, in the Agent Skills
 format (`SKILL.md` with YAML frontmatter, plus `references/` loaded on
 demand). This is the real directory; `.claude/skills/` and `.agents/skills/`
 are symlinks to it, so this checkout loads exactly what an install does.
@@ -15,6 +15,7 @@ a user's own folder of the same name, so keep it.
 | `prototype-canvas` | Running and operating the canvas: boards, `layout.json`, the `window.snapCanvas` bridge, annotated-screenshot review, the force-refresh. |
 | `define-product` | Interviewing the user about what their product is for, and writing it down as the project's `PRD.md`, which the canvas shows as a tab. |
 | `brand-kit` | Collecting a product's own brand and promotional material -- press kit, store listings, social, newsroom -- and turning it into the `images` rows of a canvas folder. |
+| `sp-scene-video` | Filming a board: a live-action clip of someone using the product, with the interface kept pixel-exact. The motion board, the reference video, Seedance 2.5 on Ark, and the green-plate composite. |
 
 Rules for editing one:
 

skills/sp-scene-video/SKILL.md

@@ -0,0 +1,168 @@
+---
+name: sp-scene-video
+description: Film a prototype. Turn a canvas board into a live-action clip of a real person using the product -- in an office, a lift, on the street -- with the interface itself kept pixel-exact rather than redrawn. Covers the animated motion board, rendering it to a reference video, generating the take with Seedance 2.5 on 火山方舟 (Volcengine Ark), the prompt patterns that hold a screen verbatim and keep subtitles out, reviewing a take frame by frame, and the green-plate composite for when the model will not hold the interface. Use when asked for a demo video, a launch or marketing clip, a video mockup, an app-in-the-wild or UGC-style shot, or any video of a prototype being used by a person.
+license: Apache-2.0
+compatibility: Requires python3, ffmpeg, Google Chrome, and node or bun, plus a 火山方舟 (Volcengine Ark) API key with Doubao-Seedance-2.5 enabled -- an account with 企业实名认证. The green-plate composite additionally needs uv, which fetches opencv on demand.
+metadata:
+  managed-by: super-prototyping
+---
+
+# Scene video
+
+A board shows the product. This puts the product in the world: someone in an
+office takes out their phone, your screens are on it, their thumb moves the
+flow along, the room sounds like a room.
+
+**The model paints the world. It never paints the interface.** Every technique
+here exists to enforce that one line. A generative model shown a screenshot
+will redraw it — right colours, right shapes, invented words — and a demo
+video with invented words in it is worse than no video.
+
+Boards come from `canvases/<slug>/`; see `prototype-canvas` for running the
+canvas and its `references/layout.md` for the folder rules. `sp root` prints
+the tree that ships the scripts below:
+
+```bash
+KIT="$(sp root)"
+B=canvases/<slug>; V="$B/scratch/video"; mkdir -p "$V/out"
+```
+
+`sp` not found? Run `sh <prototype-canvas skill dir>/scripts/install.sh`
+(Windows: `install.ps1`). A `[super-prototyping:notice]` line on a tool's
+stderr carries its own rule: finish the step, then do what it says.
+
+Everything this skill makes is derived and large, so all of it lives in
+`$V/` — plates, frames, takes, contact sheets — and `scratch/` is already
+gitignored. Never commit an mp4. The one thing that belongs in the folder
+proper is the motion board, because it is a board.
+
+---
+
+## Spending rule
+
+Every take is the user's money: roughly ¥11 at 720p/10s, ¥74 at 1080p/15s,
+minutes each. Before the first submission say what you are about to shoot,
+at what resolution, and how many takes you expect. Then shoot **one**, show
+it, and ask. Never loop takes unattended, and never re-submit a failed one
+without saying what you changed.
+
+The key lives in `$V/.ark_key` (chmod 600, never printed, never echoed into a
+log) or in `ARK_API_KEY`. If there is no key yet, stop and ask: the account
+needs 企业实名认证 and a balance, which is the user's to do.
+
+---
+
+## 1. Pick the route
+
+| Route | The screen comes out | Use it when |
+|---|---|---|
+| **Reference video** | Near-verbatim: the real pixels, softened by the codec | Default. Start here every time. |
+| Reference images only | Redrawn. Plausible layout, invented text | The phone is small in frame and unreadable anyway |
+| Green plate + composite | Exactly your pixels, no compromise | The reference video has been tried and the model still rewrites the UI |
+
+The routes are not exclusive: a reference video *plus* a full-resolution still
+as a reference image is the combination that reads sharpest.
+
+---
+
+## 2. Make the interface move
+
+The reference video is a board of your own, animating: `NN-<flow>-motion.html`
+in the canvas folder, one CSS timeline, emitted by the same `gen.py` as every
+other board. Then frames, then an mp4:
+
+```bash
+node "$KIT/skills/sp-scene-video/scripts/frames.mjs" "$B/19-flow-motion.html" \
+  --fps 24 --seconds 10 -o "$V/out/ui" --scale 3          # bun runs it too
+ffmpeg -y -framerate 24 -i "$V/out/ui/f%04d.png" -c:v libx264 -crf 16 \
+  -pix_fmt yuv420p "$V/out/mockup.mp4"
+ffmpeg -y -i "$V/out/mockup.mp4" -ss 6 -frames:v 1 "$V/ref-phone-hd.png"
+```
+
+The still is the second half of the trick: a reference video carries the
+motion, a full-resolution frame of it carries the sharpness, and a take given
+both comes back readable instead of merely correct.
+
+Watch `mockup.mp4` before going further. Pacing that feels fine scrubbing a
+board often turns out to be twice too fast in real time, and every later step
+inherits it.
+
+**`references/reference-video.md`** — the motion board's constraints, the
+`--knots` re-timing, getting the mp4 somewhere Ark can fetch it (a `data:`
+URI is refused), and what a take costs. Read it before the first take.
+
+---
+
+## 3. Write the prompt
+
+Into `$V/prompt.txt`, in Chinese, four blocks: 画质与风格, 场景与人物,
+屏幕契约, 声音与负面清单. The middle one is the product; the last one is why
+the clip has no subtitles, no music and no voice-over.
+
+**`references/prompting.md`** — the four blocks, the screen contract that
+stops the redraw, the three camera choices (locked POV, free direction, the
+UGC look that actually reads as real), thumb choreography, and a table from
+the failure you are looking at to the sentence that fixes it. Read it before
+writing a prompt, and again after any take comes back wrong.
+
+---
+
+## 4. Shoot a take
+
+`$U` below is the public base for `mockup.mp4`, from the reference above.
+Ark fetches the video from there; it refuses an inline one.
+
+```bash
+python3 "$KIT/skills/sp-scene-video/scripts/ark.py" \
+  --prompt-file "$V/prompt.txt" --video "$U/mockup.mp4" \
+  --image "$V/ref-phone-hd.png" --res 720p --dur 10 --tag walk -o "$V/out"
+```
+
+It submits, polls, downloads `walk-1.mp4` and writes `walk-1.json` beside it
+with the prompt, the seed and the usage. That JSON is the only account of what
+made a clip; keep it, and quote the seed when you report.
+
+Takes run for minutes, so run it in the background against a log and check
+back rather than holding the session:
+
+```bash
+python3 ... > "$V/out/walk-1.log" 2>&1 & disown
+```
+
+Add `--dry` to print the request without sending it — do that once, before the
+first real submission of a session, and read back what you are about to buy.
+
+---
+
+## 5. Read the take before believing it
+
+```bash
+ffmpeg -v error -i "$V/out/walk-1.mp4" -vf "select=not(mod(n\,24)),tile=5x2" \
+  -frames:v 1 "$V/out/walk-1-sheet.png"
+ffmpeg -v error -i "$V/out/walk-1.mp4" -vf "crop=iw/3:ih:iw/3:0,scale=iw*2:ih*2" \
+  -ss 6 -frames:v 1 "$V/out/walk-1-zoom.png"
+```
+
+Look at the sheet, then at the zoom, as images. Answer these before showing
+it to anyone:
+
+- Is the text on the screen **your** text, word for word? Read it, do not
+  glance at it. This is the failure mode of the whole exercise.
+- Does the interface change on the frame the thumb lands?
+- Any subtitle, caption, watermark or invented logo anywhere in frame?
+- Does the phone drift, zoom or roll when it was meant to be locked?
+- Does the audio have music or a voice in it? There should be a room.
+
+Report what you saw, including the seed and what the take cost. A take that
+fails the first question is not a draft to polish — it is the wrong route, and
+the answer is step 1 again.
+
+---
+
+## 6. When it still will not hold the interface
+
+**`references/compositing.md`** — the green-plate route: the plate images with
+their tracking markers, shooting a locked plate, and `composite.py`, which
+corner-pins your rendered frames back onto the screen so the pixels are
+exactly yours and the thumb stays on top. It is a day of work and it is the
+only way to be certain, so spend a take or two on the prompt first.