Golden Rules for Harness Profile Development
Golden rules are human-specified design constraints that govern failure-driven harness profile evolution, preventing test-set contamination and direct answer encoding. These constraints enforce three core requirements:
- Minimal Reusable Control: Prioritize minimal, reusable procedural interventions rather than maximal, rigid workflows.
- Semantic Routing: Route execution, state transitions, and verification checks based strictly on visible task semantics and observable workspace evidence, never on task identifiers, names, hashes, hidden tests, or verifier code.
- Evaluation Integrity: Strictly separate development feedback from frozen evaluation, ensuring that candidate runbooks are frozen prior to held-out benchmark evaluation.
Golden rules govern the meta-level evolution of the control profile and are distinct from the operational checks evaluated during an agent execution.
0
1
Tags
Prep Sessions
Long-Horizon Agent Reliability: Stateful Scaffolding and Runtime Verification @ University of Michigan - Ann Arbor
Ch.3 State Persistence and Continuous Optimization - Long-Horizon Agent Reliability: Stateful Scaffolding and Runtime Verification @ University of Michigan - Ann Arbor
Failure-Driven Harness Optimization and Procedural Memory - Long-Horizon Agent Reliability: Stateful Scaffolding and Runtime Verification @ University of Michigan - Ann Arbor
Related
Failure-Driven Harness Optimization
Golden Rules for Harness Profile Development
Control Drift in Multi-Task Harness Optimization
Sparse Routing in Shared Agent Runbooks
Multi-Task Harness Abstraction via Hyper-Agent
Procedural Practices in Agent Harnesses
Single-Task Harness Adaptation Loop
Selective Filtering in Harness Learning
Procedural-Memory Gap in Autonomous Agents
Which entities are responsible for proposing targeted modifications to runtime controls after an execution failure?
True or False: Proposed harness modifications take effect immediately in production without requiring review or regression testing against existing tasks.
In failure-driven harness optimization, how are postmortem findings applied to prevent future agent execution failures without retraining or modifying underlying model weights?
Golden Rules for Harness Profile Development
Control Drift in Multi-Task Harness Optimization
Multi-Task Harness Abstraction via Hyper-Agent
Learn After
Under the golden rules for harness profile development, which requirement prohibits conditioning state transitions or verification checks on task names, task hashes, or hidden test code?
Golden rules act as operational checks evaluated during an agent's execution run.
Name the three core requirements enforced by golden rules during failure-driven harness profile evolution.