mg
← back

Engineers decide, agents build

· 6 min read

I’ve been using AI coding agents loads this year, and I reckon most of us are doing it in a way that doesn’t scale. Nothing wrong with it, it’s just one ticket at a time. You pick a ticket, open an agent, paste in whatever context you can remember, and hope for the best.

That’s fine for small stuff. On a big project it falls apart pretty quickly:

  • Every agent starts from nothing.
  • It has no idea what the other agents are doing.
  • The second the session ends, everything it worked out is gone.

Throwing a bigger prompt at it doesn’t help. What it actually needs is a bit of structure. So I’ve been working on a methodology for it, which I’m calling engineer in the loop. This post is the idea. Part 2 is EILAID (Engineer-In-Loop AI Development, because everything needs an acronym), the first implementation, which I’ve been building alongside it.

The one rule

Engineers decide what to build and whether it’s right. Agents do everything in between.

That’s basically it. You don’t babysit agents through every step. You show up at two points: before any code gets written, to agree what’s being built, and once it’s done, to accept it. Everything in between is the agents’ job, apart from the odd question they can’t answer (more on that later).

Every work item goes through eight steps. 1, Intake: System. 2, Shape: Shaper. 3, Gate 1: Engineer. 4, Build: Builders. 5, Verify: CI, Tester, Reviewer. 6, Gate 2: Engineer. 7, Merge: System. 8, Learn: System + owner. Steps 3 and 6 are the human gates. A rejected brief goes back to Shape with the engineer's notes. Failed checks go back to Build, with no human needed. Learn turns what went wrong into the project's knowledge, rules and tests, and the next work item starts from that knowledge.

TapHover or tap a step to see what happens there.Intake A ticket arrives (tracker, error report or schedule) and gets a track.Shape The Shaper researches the code, asks the Oracle and drafts the brief.Gate 1 The engineer approves the brief: the cheapest place to fix a wrong plan.Build Builders work test-first in isolated worktrees, inside their footprint.Verify The checks run and the diff is reviewed. Failures go back to Build.Gate 2 The engineer reviews the intent and the design, and accepts the change.Merge The change merges, and dependent work in flight is told what changed.Learn Review comments, rejections and incidents become rules, tests or context.

fig. 1The loop. Eight steps for every work item; two are human gates. What Learn writes, the next item starts from.

Gate 1 is the cheapest place to fix a wrong plan. You read a short brief, correct it and approve it, before anyone’s written a line of code. Gate 2 is where you check the result does what you meant, not just what you said. If the checks fail in between, the work goes straight back to the builders. No human needed.

It also scales to the job. A typo fix doesn’t need a requirements doc, so small bugs and chores get a short brief and skip the first gate once there’s a failing test that reproduces them. Big things like migrations get an architect, who splits them into smaller pieces that run in parallel waves.

A few principles

The loop only works if these stay true:

  • State lives outside the agents. If it’s not in the repo or the ledger (the shared record of what’s in flight), it didn’t happen. Any fresh agent can pick up any task.
  • Context is assembled, not accumulated. Every step starts with a clean context window, built for that step. No 200-message chat history dragging along behind it.
  • Verification is the contract. Every acceptance criterion maps to a check that can actually run. If nobody can check it, it’s a question, not a requirement.
  • Everything is a file in git. Briefs, roles, decisions and knowledge are plain files you can read, diff and review.
  • Ask, don’t guess. An agent that isn’t sure asks. More on this below.

The Oracle

If I had to pick the one idea that makes all of this work, it’s the Oracle. It’s the project’s memory, and everything else leans on it.

You feed it context in whatever form you’ve already got: the code, your standards, decision records, READMEs, ticket and PR history, dependency lists, and every answer an engineer’s already given it. There’s no schema to fill in and no special format. If it’s text, it can go in.

Agents (and people) ask it questions in plain English, mid-task. It re-indexes every time something merges, so it always matches what’s on main.

The Oracle sits in the middle. Context flows into it from code, standards and docs, decision records, briefs and tickets, PR history, dependency lists and engineers' past answers. Any agent (the shaper, builder, tester, reviewer and the rest) and people ask it questions in plain English, and get answers back that cite the sources they came from. Every merge to main re-indexes it, so it always matches the code. The buttons below ask one of the example questions, or play a merge.

tap or hover an asker to ask it something

fig. 2The Oracle: the project's memory, in the middle of everything.

How it finds things

The idea is “ask your docs”, done properly. Every file gets split into passages of a few paragraphs, and each passage gets a short line of context stuck on the front. That way a passage that just says “these must be approved first” still knows it’s about third-party packages.

Then an embedding model turns each passage into a list of numbers that represents what it means. Passages about similar things end up close together, even when the words are completely different: “carousel” and “image slider” land right next to each other. That’s the clever bit, honestly. You’re searching by meaning, not just matching keywords. (In practice you want plain keyword search alongside it too. EILAID does both, and part 2 gets into how.)

Indexing: every file is split into passages, each passage gets a line of context saying where it comes from, an embedding model turns it into numbers that capture its meaning, and the result goes into a vector store. This runs again on every merge. In the resulting meaning space, passages about similar things sit close together, even when the words differ. When someone asks a question, the Oracle expands it into a few search queries, finds the passages nearest each one, drops anything outside a similarity threshold, and writes the answer only from the passages it kept, citing each one. Example: "any precedent for a swiper library?" becomes the queries "swiper library precedent", "rules for adding a third-party package", "frontend bundle size limits", and keeps docs/standards.md#dependencies, apps/shop/package.json, decisions/0007-frontend-bundle-budget.md. The query "swiper library precedent" finds apps/shop/package.json ("no carousel or slider package yet"): “swiper” and “carousel or slider” are different words, close in meaning. Example: "how do I test a declined card?" becomes the queries "testing card payments", "declined card handling", "payment provider test cards", and keeps docs/testing.md#payments, apps/shop/src/payments/charge.test.ts, decisions/0011-no-auto-retry.md. The query "declined card handling" finds apps/shop/src/payments/charge.test.ts ("fakes a failed charge"): “declined card” and “failed charge” are different words, close in meaning.

expanded into a few queries

  1. swiper library precedent
  2. rules for adding a third-party package
  3. frontend bundle size limits
  1. testing card payments
  2. declined card handling
  3. payment provider test cards

kept: close enough to count,3 dropped

  1. [1] docs/standards.md#dependenciesa small, well-documented package is fine
  2. [2] apps/shop/package.jsonno carousel or slider package yet
  3. [3] decisions/0007-frontend-bundle-budget.mdheavy UI libraries are lazy-loaded
  1. [1] docs/testing.md#paymentsuse the provider’s test cards
  2. [2] apps/shop/src/payments/charge.test.tsfakes a failed charge
  3. [3] decisions/0011-no-auto-retry.mddeclined cards are never retried

answer: written only from these, with citations

None yet [2]. A small, well-documented package is allowed [1], so Swiper is fine. Lazy-load it on the gallery page [3].

Fake a decline with the provider’s test card [1], like charge.test.ts does [2]. Declines are never retried [3].

fig. 3How the Oracle finds things: search by meaning, then answer only from what it found.

When an agent asks something, the question’s usually short and vague, because real questions are. So the Oracle expands it first, using what it knows about the asker: the task it’s on, the files it’s working in, and what things are called in this project. It writes a few search queries instead of one and finds the closest passages to each. Anything below a similarity threshold gets thrown away, so weak matches never pass as evidence.

Then it writes the answer only from what it found, and cites every passage. If the answer isn’t in there, it says so instead of guessing.

Who does what

Instead of one do-everything agent, there are roles. Each one’s a short markdown file in git: what it does, which tools it’s allowed, what context it loads and what it has to produce. These are the seven you’ll want on pretty much any project (EILAID ships fifteen, more on that in part 2):

Who does what, in three tiers. Top: the engineer, who owns the work and appears only at Gate 1, to approve the brief, and at Gate 2, to accept the change. Middle: the agents, each under the step it works in. In Shape, for epics, the Architect writes a spec and splits the work into items that run in parallel waves, and the Shaper writes the brief. In Build, the Builder writes code and tests, test-first, only in the files it declared. In Verify, the Tester writes a failing reproduction, then the checks, and the Reviewer reviews the diff against the brief. Set apart, on a nightly schedule, the Steward does upkeep: dependencies, flaky tests and docs. Bottom: the ledger, the shared record of claims, leases, blockers, handoffs and decisions. Every agent connects only to the ledger; agents never message each other. Beside it, the Oracle gives cited answers, and any agent can ask it directly.

TapHover or tap a role to see what it does.Architect For epics. Writes a spec, then splits the work into smaller items that run in parallel waves.Shaper Researches the codebase, asks the Oracle and drafts the brief for Gate 1.Gate 1 The engineer reads the brief, corrects it and approves it. The cheapest place to fix a wrong plan.Builder Implements the brief test-first in an isolated worktree, touching only the files its brief declared.Tester Writes a failing test that reproduces the problem first, then the checks that prove the work.Reviewer Reviews the diff against the brief. Anything wrong goes back to Build, no human needed.Gate 2 The engineer reviews the intent and the design, and accepts the change.Steward Runs on a schedule, outside the loop: dependency updates, flaky test fixes and refreshed docs, in one daily review.Ledger The shared record of everything in flight. Agents never message each other; it all goes through here.Oracle Answers any agent's question directly, with cited sources. If the evidence is thin, it asks the engineer.

fig. 4Who does what. The engineer owns the gates; agents never message each other directly, it all goes through the ledger.

What matters is how they coordinate: they never message each other directly. Apart from asking the Oracle, everything goes through the ledger: who’s working on what, which files are taken, what’s blocked and what got handed off. So every interaction is visible, and any agent can pick up where another one left off.

Each brief also says which files it expects to touch. Two pieces of work that overlap never run at the same time, and an agent that needs to touch anything outside those files has to ask first. That turns out to be a really good early warning for scope creep.

Ask, don’t guess

Right, my favourite bit. When an agent isn’t sure about something, it doesn’t make it up. It asks, and there’s a ladder for it.

An agent that isn't sure about something asks the Oracle first. If the Oracle has a confident, cited answer, that's the end of it: the fast path. If the evidence is thin, the agent files a blocker (needs info) and moves on to other work, while the question climbs to the agent that owns the related work, and then to the engineer who owns the item, who gets it with the relevant passages already attached. The engineer answers, and the answer is saved as a new context file and embedded, flowing back into the Oracle. Next time, when a different agent asks the same thing, the Oracle answers straight away, citing the engineer's earlier answer. Every question is asked of a human at most once.

fig. 5Ask, don't guess: the escalation ladder.

First it asks the Oracle. Often that’s the end of it: a cited answer, and back to work. If the evidence is thin, the question goes up, through the ledger, to whichever agent owns the related work. If that still doesn’t settle it, it goes to the engineer who owns the item, with the relevant passages already attached, so you’re not starting from scratch either.

Meanwhile the stuck agent files a blocker and gets on with something else. Nothing sits there spinning.

And the nice part is you only get asked once. Your answer gets saved as a new context file and embedded, so the next agent that asks the same thing gets it straight from the Oracle. Nobody gets asked the same question twice. Give it a few weeks and most questions just answer themselves.

Mistakes make it better

Same idea, applied to mistakes. When an agent gets something wrong, you don’t just fix it and move on. The rejection, review comment or incident becomes a rule, a test, a lint check or a bit of new context, signed off by whoever owns it, so it can’t happen the same way again.

Week on week, you should see fewer escalations and fewer review rounds for the same kind of work. That’s really the whole point. The project gets easier to work on the longer you work on it, which is the opposite of how it usually goes.

So, does it work?

That’s what EILAID is for. It’s the first implementation of most of this: a local orchestrator that runs Claude Code, Codex or OpenCode through the loop, with the Oracle, the ledger, the roles and both gates. It’s still early, but it’s already building real things end to end, and that’s where part 2 picks up.

↑↓ move · ↵ run · esc close