Groundtruth / Evaluation infrastructure · Independent research
The agent looked safe. Was the test wrong?
I built a deterministic red-teaming harness, measured its detector errors, and traced an apparent safety result to the harness itself. Changing the adapter changed the conclusion.
My contributionBuilt the harness and detectors, labeled the evaluation corpus, and investigated the state-handling confound.
68 hand-labeled traces · Mocked, deterministic tools
Explore the project
01 / Inspecting the evaluator
The agent looked safe. Was the test wrong?
Changing how the harness preserved state changed the result. The next question was whether the measurement itself deserved trust.
Every tested pair stalled.
All nine pairs stalled at each tested step budget: 6, 12 and 24. More steps alone did not resolve the behavior.
02 / Measure the measurement
What did the detectors get right?
True detections
The detector flagged a labeled failure.
False detections
The detector flagged a failure absent from the labels.
Misses
A labeled failure was not detected.
42 of 44 detections were correct. That is detection precision—not overall classification accuracy.
The evidence covers 68 traces labeled by one annotator, with mocked, deterministic tools. Counts are detector events, not a partition of the 68 traces. No true-negative bucket was recorded.
Inspect the detector-quality record ↗- Detection precision
- 0.95
- Stalls under the stateful adapter
- 0/9
over 44 detections, not 68 x 6 decisions — there is no true-negative bucket
detector-quality.json ↗against 9/9 at every stateless step budget (6, 12 and 24)
REPORT.md ↗Follow the evidence
Source revision · 5954a9c842c2da862724f1d4ded6a181fe601031
Scope & limitations
- One annotator labeled all 68 traces. There is no inter-annotator agreement figure.
- Tools are mocked and deterministic. Nothing here measures behavior against live systems.
- Precision is computed over detections, so it is detection precision and not overall classification accuracy.
- Nine stall pairs and two controls is a small matrix; it establishes the artifact, not a rate.
The engineering decision
Test the evaluator as a system of its own. Hold the scenario constant, vary adapter state and step budget, and inspect the execution before trusting the score.
What the evidence means
The stateful adapter removed the recorded stalls. This small controlled experiment exposes a harness artifact; it does not estimate a real-world failure rate.
Implementation notes
An offline, deterministic red-teaming harness for tool-using LLM agents that scores its own detectors against 68 hand-labeled traces and publishes the error rate beside the agent results. Tracing a suspiciously clean result led back to the harness: a stateless adapter rebuilt byte-identical prompts under deterministic decoding, which guarantees a stall at any step budget. Rebuilding it stateful took stalls from 9/9 to 0/9 and exposed two models that had looked safe only because the harness truncated them before they acted.