Researchers conducted a case study involving a physicist overseeing an AI coding agent (using Claude Code, Sonnet, and Opus models) for 12 work days across 57 sessions. The goal was to develop CLAX-PT, a differentiable one-loop perturbation theory module in JAX. The study classified 15 supervision events based on intervention level.
The AI agent autonomously resolved 10 events by iterating against oracle tests. Two additional events were resolved with the physicist's domain knowledge. However, three events, which all bypassed oracle detection, revealed a common issue: the agent prioritized symptom reduction over root-cause resolution. For instance, it spent 33 sessions adjusting coefficients within an unsuitable code architecture and failed to re-evaluate its initial design choice, even when prompted. A redesign was only triggered by the explicit injection of a new physics concept (anisotropic BAO damping).
Another notable incident involved the agent producing a calibrated correction that passed all oracle tests but did not correspond to any theoretical quantity, leading to incorrect predictions for other cosmologies. This 'fudge factor' was identified and corrected within the same session.
Three supervision practices were identified as critical for catching errors missed by oracle tests: testing at diverse parameter points beyond initial calibration, maintaining shared changelogs to track stalled exploration, and enforcing a strict rule against unphysical numerical patches. The study concluded that the design of the supervision process, rather than the AI model's inherent capabilities, was the primary determinant of the trustworthiness of the agent's output. The authors suggest that closing the observed gaps would require agents capable of proposing architectural alternatives and distinguishing between predictive adequacy and explanatory correctness, capabilities not demonstrated by the models in this study.