Skip to content

Tighten hypothesis handling in pony-debug using Zeller's Scientific Debugging #82

Description

@SeanTAllen

The investigation loop (checkpoint 4) has the right shape — hypothesis, prediction, experiment, observe — but could borrow more structure from Zeller's Scientific Debugging framework (Why Programs Fail, 2005).

Two things Zeller formalizes that the current skill leaves loose:

Explicit refinement branching. When predictions hold, refine the hypothesis — narrow it toward a more specific cause. When they don't, replace it with an alternate built from what the evidence actually showed. The current skill addresses replacement on refutation but not refinement on partial confirmation. Agents tend to either keep investigating a confirmed-but-broad hypothesis without narrowing, or discard it entirely when one prediction fails.

Debugging logbook. A cumulative log of each iteration's hypothesis, prediction, experiment, and outcome — not just a per-iteration artifact, but a running document that shows the full chain of reasoning and makes the narrowing visible. The current "artifact per iteration" instruction doesn't emphasize accumulation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions