Lock-In Week: four days, five engineers, and a swarm of coding agents
2026-09-10
On Monday, August 17, we cleared four days off everyone's calendar, cancelled every meeting, turned Slack expectations off, put five engineers in one room, and gave all of them unlimited credits on every coding agent we could buy. There was one rule: on Thursday afternoon, everyone demos something live. A working slice, a head-to-head comparison with numbers, or an instructive failure with evidence, but it has to run and it has to be yours.
We called it Lock-In Week. The question we wanted answered was narrower than “does this make us faster.” It was: which parts of building software get cheaper when you multiply each engineer by a swarm of agents, and which parts stubbornly do not?
Four days later the answer was sharper than I expected. Volume got cheap almost immediately, and more completely than any of us guessed. The work that stayed expensive was direction, shared context, integration, and verification. A week designed around that reality looks different from a normal sprint, and the two choices that carried it were both made before anyone wrote a line of product code.
The setup
Four days, August 17 to 20, ten to six, everyone in the same room. What made it different from a normal week:
- A fresh repository in our research org, far from the production monorepo. Everything landed through a pull request, but you merged your own immediately. No approvals, no review, no CI gates. The pull request existed to stop five simultaneous swarms trampling
main, not to slow anyone down. - Everything ran locally. No cloud, no production traffic, no real customer accounts. We wrote three fixture web applications to stand in for the substrate.
- Reusability, lint, CI/CD and test coverage were explicit non-goals. Correctness was judged by an eval harness rather than by process.
- Unlimited Claude Code, Cursor, Copilot and Codex for all five people. The human job was direction, judgment and integration; the agents' job was volume.
We also wrote a playbook of seven rules before we started. Two of them turned out to be the whole ballgame, and I'll come back to those.
What got cheap
By Monday evening, before anyone had started on the product itself, main already carried three fixture sites with their ground truth written twice (once machine-readable, once for humans), working prototypes of both user interfaces, five JSON schemas with worked examples and a validator, and a full specification for the network proxy the runtime would live in. Five pull requests, all merged between 6:09pm and 11:48pm on day one.
By Wednesday morning a deterministic crawler was passing the full fixture evaluation: 25 of 25 actions on the easy site, 46 of 46 on the medium one, 52 of 52 on the hard one, zero over-matching, no model anywhere in it, and a few minutes a run. Those are numbers on sites we built ourselves, which is exactly as much as they prove. The engineer who wrote it said so in the room before anyone else could.
The result that surprised me most came from doing nothing. On Wednesday evening we wrote one long ticket, told the agent it was running unattended and should not stop to ask questions, and went home. It ran for seven hours, built six competing implementations across two halves of an architecture, wrote fresh evaluation code to score them independently, and came back with a recommendation for each half. It cost under ten dollars and it ran clean across two usage windows.
I don't see why we shouldn't always end our day with “run this thing overnight.”
Unlimited agent credits sound like the expensive part of a week like this. They were not. What we were spending was four engineer-days of attention, and almost none of it went where I would have guessed a year ago.
What stayed expensive
Direction
A ticket is a prompt. Ours carried context links, the task, the interfaces it touched, acceptance criteria, an explicit out-of-scope list, and why it mattered for Thursday's demo. Writing one of those well takes real time, and the rule we settled on was: if an agent flounders, fix the ticket, not the agent. The saving from agents is real, and it moves upstream to a place where it does not feel like a saving while you are doing it.
Shared context
If it isn't in the docs, an agent doesn't know it. At this speed documents rot in hours rather than weeks, and a stale document does worse than fail to help: it actively misleads every agent that reads it next. We spent the end of Tuesday doing nothing but reconciling documents that had drifted apart from the decisions they described, so there was one place an agent could be pointed at and told, here is what is real. We also recorded every meeting and committed the summaries into the repository, which quietly became one of the highest-value habits of the week. Things get decided out loud, and then somebody forgets one small thing.
Integration
Five parallel swarms produce five internally consistent things that disagree at the seams. Our clearest case was the proxy specification, written overnight on day one in parallel with the schemas rather than after them. Its audit records used their own envelope where the shared contract specified a different one, its posture toward unrecognised traffic was hard-coded where the contract made that a policy decision, and its internal model assumed a permission tree the contract had already flattened. All three were reconcilable, and none of them were caught by a machine. They were caught by people reading two documents side by side, midweek.
Verification
Checking does not get cheaper because producing did. Our own eval harness spent its first real run filing three defects and one contract ambiguity against the very component it had been built to score, and one of those defects mattered: when two candidate actions both matched a request, the engine was deciding on the winner alone, so attribution could turn a denial into an allow. Separately, a contradiction in our own fixture ground truth took a full day to surface, because it only became visible once two halves of the pipeline were assembled end to end. The correct fix turned out to be no schema change at all.
The two things that made it work
At the retro on Thursday I listed every shortcut we'd taken (no meetings, detached repository, no reviews, self-merged pull requests, everything local, agents at full throttle) and asked which had actually mattered. The room's first answer was that nobody was siloed: five people in one problem space rather than five people in five. Underneath that, two mechanisms did the load bearing.
Locked contracts and lanes
The five schemas were a shot in the dark, written on day one before anyone knew what would be missing, with one dated rule attached: no destructive changes after end of day Tuesday, additive and optional only after that, and any post-freeze change updates the schema, the example and the validator together, gets a line in the interface change log, and gets said out loud across the room.
One frozen contract, five lanes, one referee. The lanes never talk to each other directly.
Three schema generations landed in three days. Across all of them, no field was renamed, removed, retyped, or made required, and no file anywhere in the repository ever had to change to stay valid. Five genuine gaps surfaced during the week, found by three different lanes, and four of them fitted through an optional field. The interface change log carries more than a hundred dated entries inside those four days. The validator grew from 131 checks to 442.
The same trick worked one level down. On Wednesday morning, one engineer's first job was to freeze the three in-process interfaces between five parallel tracks inside his own component, before building any of them. His own slides made the argument: land the interface-freeze ticket first, because every hour it slips, five work tracks wait. It merged at 10:09, and all five started at once.
This is also the honest answer to why self-merging was safe.
Once we lock the contracts and the lanes, it's pretty clear you can start pushing stuff and you're not going to get in each other's way, because we're all talking the same language.
The corroboration I trust most came from someone else in the retro, about a week in which five swarms ran simultaneously with no code review:
I don't think there was a single, like, “hey, this is doing something weird” or “that's outputting something weird.” I don't think that happened, like, once.
Evals as the referee
The other rule was that competing approaches live side by side in the repository and the scoreboard decides. The clearest case was a deterministic crawler against a model-driven one, on the same fixtures, scored by the same unchanged harness. The deterministic one won on every structural measure (coverage, request grouping, segmentation, run-to-run stability) at no model cost at all. The model-driven one won on the human-facing half: names a person can read, and a hierarchy a person can navigate. Two independent experiments reached that same split in the same week, so we stopped arguing and built the hybrid.
Once we have metrics and not vibes, there's no point in really debating exhaustively the merits of different approaches. It makes the collaboration part easier, too, because there is just a North Star.
Two design choices inside the harness are worth stealing. The referee re-implements what it scores instead of importing it, because a scorer that shares code with its subject cannot detect its subject being wrong. And the thing being scored reports counts and identifiers only, never ratios, so every division happens in a component that cannot be tuned by the people optimising the number.
Between them, the contracts and the harness cost about one day out of four. A quarter of the week went into work that produced no demo, and it was the best trade we made.
The failure worth keeping
Late in the week, one of us ran a deliberately narrowed experiment outside the harness: map a real site, ignore network traffic entirely, two and a half hours, go. The agent began reporting that it was matching nearly everything on the target.
I'm like, what's this key that you have? You've never seen Hacker News before, so how do you have a key? And that's when it revealed to me that it made up the key.
It had fabricated its own answer key and then scored itself against it.
The generalisable part has nothing to do with crawling. An unrefereed agent asked for a coverage number will invent the denominator, and the invention is invisible from the inside, which is the whole argument for an independent referee. It showed up in the one track running outside ours, under time pressure, against a target with no ground truth: exactly the conditions where you most want a number and least deserve one.
We had already made this structurally impossible everywhere it counted. Runs against real sites are scored by a component with no coverage, precision or recall fields at all, because there is no knowable denominator for a site we did not build. It reports counts and surfaces, and no code path in it could put an invented percentage into a slide. We made that decision for a different reason, and this is the week it earned its keep.
What we would change
Freeze the interfaces on Monday. We got the schemas out on Monday evening, a full day ahead of our own deadline, and it was the single biggest accelerant of the week. The version of the plan that says Tuesday is a worse plan than the one we accidentally ran.
Point every workstream at a simple real site on day one. We sequenced the real sites last, reasoning that the fixture work had to be finished first. That was wrong, and expensively so. When we finally pointed one component at Hacker News it cost seventy seconds a run, needed no login, and immediately surfaced a defect with safety consequences. On all three of our fixture sites every write is a button, so the code had quietly learned that links are always safe to follow. On Hacker News, vote, hide, submit and login are all links. A crawl seeded that way would have cast a real upvote. Six of the eight defects found that day had been invisible to three fixture sites and seventy-two scored runs. A fixture is a set of assumptions somebody wrote down, and the assumptions are the thing you are trying to test.
Budget time for reconciling seams instead of discovering them at midday on Wednesday.
Decide the demo on Wednesday morning. We changed the format on Thursday morning, collapsing five individual demos into one end-to-end demo given together, with each person speaking to their own contributions and findings. It was the right call, and it should have been made a day earlier.
The honest conclusion
A quarter compressed into four days is the wrong frame. What we got was a quarter's worth of exploration and a week's worth of hardening, and being clear about which is which is precisely what the eval harness and the write-it-down discipline are for.
The room's estimate was that reaching the same place by ordinary means would have taken at least a month, and more likely a quarter. I believe that estimate, and it is still an estimate, agreed in a room by people with an interest in the answer. We ran no control group. Any velocity multiple we published would be invented, so we are not publishing one.
Three caveats, for what they are worth. Four days of throughput says nothing about four months of maintainability; the non-goals list was a debt taken deliberately and it is still owed. Everything ran locally against sites we built, so none of it is evidence about the open web. And self-merged pull requests worked because the blast radius was one repository for one week. On the monorepo that serves a live product they would be a bad idea, and they are staying where they are.
If you take one thing from this, take the two cheapest things we did, because they carried most of the week. Write your interfaces down and put a date on the freeze, with additive-only after it. And build the referee before you build the thing it is going to score.
We're redefining zero-trust — so you can protect your accounts with confidence.
Identity is your first and last line of defense, and the root cause of most application security breaches. Multifactor's provably secure zero-trust solutions cryptographically guarantee that only authorized users can access sensitive data, turning identity into your greatest asset in the fight against cyber threats. Learn more about our research, or reach out to explore working together.
Related Posts
How we built Multifactor's founding engineering team in six weeks
2026-08-04
How Multifactor designed a hiring process backward from the product it's building — treating technical depth and cultural fit as equally non-negotiable — to hire its founding engineering team in six weeks.
We're Hiring
2026-03-02
Multifactor is hiring. If you are a talented engineering leader or individual contributor who cares deeply about forging the future of authentication, authorization, and auditing for the agentic era, you should probably check out our open positions.
Multifactor Versus Password Managers: Securing Capabilities, Not Credentials
2026-01-31
Say goodbye to password managers and hello to Multifactor, the world’s first true Account Manager!