Skip to content
← All TILs
TIL

TIL : having a budget model review an expensive model's code

model-routinginference-costworkflow

Opus 5 rewrote an E2E Behat suite to replace fixed sleep() calls with bounded polling: 23 files, +1235/−100. I had the diff reviewed by DeepSeek v4 Flash through oh-my-pi, two agents in parallel — one on the PHP, one on the shell scripts.

26 minutes, 193 billed calls, $0.064. Eight findings, none blocking.

The sharpest one fits in three lines:

# run-by-family.sh, run under `set -uo pipefail`
BEHAT_ORDER=()
behat "${BEHAT_ORDER[@]}"   # older bash: "BEHAT_ORDER[@]: unbound variable"

The default invocation, with no argument, dies before the first test on any bash older than 4.4. My machine runs 5.2: it could never show up here.

The second finding is the same kind. A currentPartner variable set in only one of the four variants of a Behat step. Result: a race guard silently early-returns — it protects nothing, and never says so.

Neither is a logic bug. They’re the gaps only a reader with no memory of the writing goes looking for: the model that wrote the code knows what it meant to do, and re-reads its intent rather than its text.

Hence the move I’m keeping: expensive model to write, budget model to review, and never the same one twice. One measured run on a diff I knew — not a benchmark.

Further reading