Run Dream-1, take over with the leader arms when it needs help, and record the corrected attempts. Train ACT on those recordings, then repeat the correction loop.
Follow these chapters in order:
- Set up the rig and check human control.
- Run Dream-1 and record corrections.
- Train ACT on the corrected takes.
To collect human-only demonstrations instead, see the alternative recording path.
You need two Metal leaders, two Metal followers, three USB cameras, Linux hardware
hosts, Python 3.12, uv, and a LiveKit server. ACT training also needs a CUDA
machine. Dream-1 needs a Dream Labs API key.
| Chapter | Purpose |
|---|---|
| 00. Overview | Roles and vocabulary |
| 01. Setup | Connect the rig and check human control |
| 02. Dream-1 | Run, correct, record, and convert |
| 03. ACT | Train, run, correct, and retrain |
Read these after completing the walkthrough, or when you need to change the code:
| Chapter | Purpose |
|---|---|
| 04. Shared schema | Cameras, state, and actions |
| 05. Robot loop | Publish observations and apply actions |
| 06. Operator loop | Send human actions and mark takes |
| 07. Recording | Align actions with observations |
| 08. Handoff | Switch between human and policy |
| 09. Reference | Settings, alternatives, and troubleshooting |
Metal arms have no brakes. Start the robot with the followers resting: that is
where they park. p parks them (walk back, switch off) or powers them up, and the
robot parks before every exit. A fault never switches the motors off.
The runnable peers are in src/; policies are in src/policies/.
All peers share portal.yaml.
For checks without hardware, see test/README.md.