Notes on Agentic Engineering: Where the Human Still Matters

Simon Willison recently started documenting Agentic Engineering Patterns — practices for getting good results out of coding agents like Claude Code and Codex, where the agent both writes and executes code, iterating largely on its own. This page is my running companion to that idea, told from the other side of the loop: short case studies from my own projects where a human in the loop still made the difference.

Each entry describes what the agent did, where its frame of reasoning fell short, and what the intervention cost — or saved.

• May 13, 2026

  How "take ours" silently overwrote five coworkers' PRs

  Context. I was rebasing a long-lived feature branch onto main with a coding agent. The branch had 14 commits — partly debug code, partly an intentional rollback of an architectural change a coworker and I had built earlier. Main had moved on with 356 commits since the branch was cut.

  The bug. Pulling main produced five merge conflicts. The branch had reshaped a feature one way; main had reshaped it differently. The same files had been edited in incompatible directions on both sides.

  The instruction. Faced with merge vs. rebase, I told the agent, "let's go with the simplest option." It picked merge.

  What the agent was about to do. When the merge produced five conflicts, the agent reused the same framing on its own: "the branch represents the desired rollback target, so the branch's side wins." It ran git checkout --ours on every conflict, committed the merge, and was about to ask me to push. The diff looked plausible: all the rollback edits were intact, git status was clean. From inside the merge, nothing seemed wrong.

  Why it didn't notice. The "branch is the rollback target" reasoning is true for the architectural pieces — the part the branch was explicitly designed to undo. It's false for every other line in those files. --ours doesn't know the difference between "an edit the branch made" and "an edit main made that the branch never saw." It treats every conflicted hunk as a single binary choice: branch wins or main wins. But these files had been touched by many people on main since the branch was cut — five different coworkers, roughly nine distinct features. None of their work appeared in the branch's diff because the branch had never seen it. --ours discarded all of it without a single warning.

  The human intervention. I paused and said, "if you're removing code I wrote, that's fine. If you're removing other people's code, that's not." That one sentence reframed the whole task. The agent ran git log -S and git blame against main for each removed chunk, attributed every deletion to an author, and found work from five coworkers silently nuked across the five files. The fix was to redo the merge: take main's version of each conflicted file (git checkout --theirs), then surgically re-apply only the rollback edits that belonged to me. The surgical pass took roughly twenty minutes — much longer than the first attempt, but every preserved coworker contribution made it justified.

  What it would have cost otherwise. Roughly nine landed features from five engineers, gone. They'd have noticed when their dashboards stopped working, their API routes returned 401s, or their newer evaluation columns disappeared. Best case: angry messages and a revert. Worst case: someone else discovers the regression in production after the rollback PR has been built on, and the recovery includes a partial revert tangled with downstream work.

  Lesson. "Simplest option" is an instruction about effort, not about correctness — and the agent applied it twice in a row, once to pick merge and once to resolve conflicts, without checking back the second time. In a long-lived branch, every conflicted file is a multi-author timeline, not a binary us-vs-them. Default merge tooling (--ours, --theirs) treats both sides as monoliths and gives no signal when you're discarding unrelated contributions. The author-attribution step — git blame on every deletion — is cheap, and it converts "did the agent trample anyone's work?" from a vague worry into an answerable question.

  For any merge older than a sprint, run it.
• May 11, 2026

  When "fix and re-run" silently means "re-run the whole thing"

  Context. I'm running a benchmark of 332 evaluation cases against an LLM. Each case takes several minutes (the agent spins up a sandbox VM, the model has a conversation, then a grader scores it). The full run takes multiple hours end-to-end. I'm driving this with a Claude-based orchestrator agent.

  The bug. Partway through, the orchestrator detects an infrastructure bug — the LLM-judge isn't being called correctly, so every "criteria" assertion errors out. The agent does the right diagnostic work: identifies the file, applies a one-line fix, and prepares to relaunch.

  The instruction. I had told the agent: "If a case fails due to an infra bug, fix the code and re-run until it's resolved."

  What the agent was about to do. Kill the controller, apply the fix, relaunch. The controller has no built-in resume — on restart it iterates the suite from case 0. So every case that had already completed correctly would have been re-run.

  Why it didn't notice. From the agent's perspective, it was doing exactly what I asked. Nothing in its local reasoning surfaced that "re-run" would silently include already-finished work. The agent wasn't tracking the implicit goal of not wasting compute — only the explicit goal of getting a clean run.

  The human intervention. I asked: "are you sure it won't rerun the previously verified ones?" The agent immediately understood the gap and built the missing piece — a sidecar file the watcher appends to as cases finish, plus a launcher wrapper that reads it and passes the IDs as exclude_ids on relaunch. Cost: ~2 minutes to design and implement.

  What it would have cost otherwise. Hours of duplicate compute — sandbox VMs, model inference quota, shared sampler time — to re-derive results that were already on disk.

  Lesson. AI coding agents are excellent at executing within the scope of the instruction they're given. Side effects outside that scope — wasted resources, repeated work, partial-state recovery — only get caught when a human pattern-matches against the broader system. The instruction "run until clean" implicitly assumes "and don't redo what's already done," but agents don't infer those implicit constraints. They optimize the explicit one.

  This is the case for keeping a human in the loop on long-running agentic workflows. Not because the agent is wrong — it's not — but because the agent's frame is narrower than the system's frame, and only the human sees the gap.

More lessons will be added as I encounter them. If you've run into something similar, I'd love to hear about it.