You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit ff06d4a
Browse filesBrowse the repository at this point in the historyBrowse files
@@ -79,144 +79,29 @@ To optimize a model in a reinfocement learning environment a reward signal is ne
79
79
80
80
We split reward by their functional definition - fast, synctatic checks for file presence and synctatic validity, then "milestones" - actions requiring complex solution, and judge evaluation rewards. For example, the below table describes a reward system for a Engineering Coder agent.
81
81
82
-
<!--TODO: make a table-->
83
-
84
-
```yaml
85
-
engineer_coder:
86
-
description: >
87
-
Scores the CAD engineer's ability to produce a manufacturable,
88
-
cost-compliant solution that passes simulation.
89
-
hard_checks:
90
-
# Gate 1 — script compiles
91
-
script_compiles:
92
-
weight: 0.05
93
-
description: >
94
-
script.py executes without Python syntax or import errors.
95
-
minimum_score: 0.02
96
-
success_feedback: "The CAD script compiled and executed without syntax or import errors."
97
-
failure_feedback: "The CAD script failed to compile or encountered a runtime error. Check for syntax errors or missing imports."
success_feedback: "The solution successfully solved the problem in simulation."
159
-
failure_feedback: "The simulation failed to reach the goal. Re-examine mechanical stability, component placement, or movement range."
160
-
161
-
judge_evaluation:
162
-
reviewer_accepted:
163
-
weight: 0.02
164
-
description: >
165
-
The execution reviewer accepted the implementation.
166
-
checklist:
167
-
latest_revision_verified:
168
-
weight: 0.005
169
-
description: >
170
-
Latest implementation revision was the one evaluated for submission.
171
-
validation_success:
172
-
weight: 0.01
173
-
description: >
174
-
Validation success is satisfied at review time.
175
-
simulation_success:
176
-
weight: 0.015
177
-
description: >
178
-
Simulation success is satisfied at review time.
179
-
visual_evidence_checked:
180
-
weight: 0.005
181
-
description: >
182
-
Required static evidence has been inspected.
183
-
dynamic_evidence_checked:
184
-
weight: 0.005
185
-
description: >
186
-
Required dynamic evidence has been inspected.
187
-
plan_fidelity:
188
-
weight: 0.01
189
-
description: >
190
-
Implementation stays aligned with the approved plan.
191
-
robustness:
192
-
weight: 0.01
193
-
description: >
194
-
Solution is robust rather than passing a single lucky run.
195
-
cost_weight_compliance:
196
-
weight: 0.005
197
-
description: >
198
-
Final solution stays within cost and weight targets.
199
-
manufacturability_compliance:
200
-
weight: 0.01
201
-
description: >
202
-
Final solution remains manufacturable under the chosen process.
203
-
dof_deviation_justified:
204
-
weight: 0.005
205
-
description: >
206
-
Any DOF deviation from the approved plan is justified and reviewable.
207
-
feedback_response_score:
208
-
weight: 0.03
209
-
description: >
210
-
Fraction of valid failed reviewer checklist items addressed on the
211
-
next coding revision without introducing new blocker regressions.
212
-
partial: true
213
-
bad_feedback_resistance_score:
214
-
weight: 0.02
215
-
description: >
216
-
Score for following valid reviewer feedback while resisting incorrect
217
-
or non-applicable requests in mixed-quality review seeds.
218
-
partial: true
219
-
```
82
+
| Section | Key | Weight | Type | Description | Feedback / Formula |
83
+
| -- | -- | -: | -- | -- | -- |
84
+
|`engineer_coder`| overall objective | - | - | Scores the CAD engineer's ability to produce a manufacturable, cost-compliant solution that passes simulation. | - |
85
+
|`hard_checks`|`script_compiles`| 0.05 | hard check |`script.py` executes without Python syntax or import errors. | Minimum score: `0.02`. Success: the script compiled and executed without syntax or import errors. Failure: the script failed to compile or encountered a runtime error. |
86
+
|`hard_checks`|`cad_geometry_valid`| 0.08 | hard check | All parts export to valid OBJ meshes. | Success: CAD geometry is valid and successfully exported. Failure: geometry is invalid, has self-intersections, or failed to export. |
87
+
|`hard_checks`|`manufacturability_valid`| 0.07 | hard check | All parts pass workbench manufacturability checks, such as no CNC undercuts and wall thickness above the injection molding minimum. | Success: the design passed all manufacturability constraints. Failure: the design has manufacturability issues, such as undercuts or thin walls. |
88
+
|`hard_checks`|`parts_within_build_zone`| 0.05 | hard check | All custom-built parts are fully within the `build_zone` AABB. | Success: all parts are correctly positioned within the designated build zone. Failure: some parts are outside the build zone. |
89
+
|`hard_checks`|`cost_within_cap`| 0.10 | partial hard check | Validated unit cost from `validate_and_price` is less than or equal to `max_unit_cost`. | Penalty formula: `max(0.0, 1.0 - max(0.0, actual_cost / max_unit_cost - 1.0))`. Success: manufacturing cost is within budget. Failure: manufacturing cost exceeds the budget. |
90
+
|`hard_checks`|`weight_within_cap`| 0.05 | partial hard check | Validated assembly weight is less than or equal to `max_weight_g`. | Penalty formula: `max(0.0, 1.0 - max(0.0, actual_weight / max_weight_g - 1.0))`. Success: assembly weight is within the maximum limit. Failure: assembly weight exceeds the limit. |
91
+
|`milestones`|`simulation_result`| 0.45 | partial milestone | Simulation outcome. Full credit is awarded for success; failed runs receive partial credit proportional to progress toward the goal. | Success formula: `0.45`. Failure formula: `0.45 * (1.0 - min_distance_achieved / initial_distance) * 0.4`. Success: the solution solved the problem in simulation. Failure: re-examine mechanical stability, component placement, or movement range. |
92
+
|`judge_evaluation`|`reviewer_accepted`| 0.02 | judge signal | The execution reviewer accepted the implementation. | - |
93
+
|`judge_evaluation`|`latest_revision_verified`| 0.005 | checklist item | Latest implementation revision was the one evaluated for submission. | - |
94
+
|`judge_evaluation`|`validation_success`| 0.01 | checklist item | Validation success is satisfied at review time. | - |
95
+
|`judge_evaluation`|`simulation_success`| 0.015 | checklist item | Simulation success is satisfied at review time. | - |
96
+
|`judge_evaluation`|`visual_evidence_checked`| 0.005 | checklist item | Required static evidence has been inspected. | - |
97
+
|`judge_evaluation`|`dynamic_evidence_checked`| 0.005 | checklist item | Required dynamic evidence has been inspected. | - |
98
+
|`judge_evaluation`|`plan_fidelity`| 0.01 | checklist item | Implementation stays aligned with the approved plan. | - |
99
+
|`judge_evaluation`|`robustness`| 0.01 | checklist item | Solution is robust rather than passing a single lucky run. | - |
100
+
|`judge_evaluation`|`cost_weight_compliance`| 0.005 | checklist item | Final solution stays within cost and weight targets. | - |
101
+
|`judge_evaluation`|`manufacturability_compliance`| 0.01 | checklist item | Final solution remains manufacturable under the chosen process. | - |
102
+
|`judge_evaluation`|`dof_deviation_justified`| 0.005 | checklist item | Any DOF deviation from the approved plan is justified and reviewable. | - |
103
+
|`judge_evaluation`|`feedback_response_score`| 0.03 | partial score | Fraction of valid failed reviewer checklist items addressed on the next coding revision without introducing new blocker regressions. | Partial score. |
104
+
|`judge_evaluation`|`bad_feedback_resistance_score`| 0.02 | partial score | Score for following valid reviewer feedback while resisting incorrect or non-applicable requests in mixed-quality review seeds. | Partial score. |
0 commit comments