The AI Operating Model for a 10-Person Startup
You don't need an AI team. You need three habits: an eval loop, a prompt registry, and a weekly incident review.
The AI Operating Model for a 10-Person Startup
You don't need an AI team. You need three habits: an eval loop, a prompt registry, and a weekly incident review.
Somewhere around mid-2025, a strange inversion happened: the AI advantage stopped belonging to companies with the most ML engineers and started belonging to companies with the best habits. In 2026 the pattern is unmistakable. Gartner expects 40% of enterprise applications to embed task-specific agents by the end of this year, up from under 5% in 2025 — and yet the same analysts project that over 40% of agentic AI projects will be cancelled by 2027. Adoption is easy now. Operating is the filter.
Here's the good news if you're a 10-person company: the operating part doesn't require headcount. The failure analysis is remarkably consistent — deployments go negative because of unclear success criteria, eroding evaluation coverage, and nobody owning the system after launch. Every one of those is a process failure, not a talent failure. Organizations with real AI governance practices get roughly 12x more AI projects into production than those without — and at ten people, "governance" doesn't mean a committee. It means three habits, maybe four hours a week, total.
Habit 1: The eval loop
The habit: every AI feature you run has a test set of real examples with known-good answers, and you run it on every change.
Your customer-support agent, your proposal generator, your lead-scoring prompt — each one gets a file of 30–100 real inputs paired with what a good output looks like. When anyone changes a prompt, swaps a model, or adds a tool, the suite runs and produces a score. That's the whole loop.
Why this is non-negotiable in 2026: the model landscape churns constantly. This year alone has already brought Claude Fable 5, GPT-5.6, and Gemini 3.1 Pro, with more landing quarterly, and the frontier is specialized now — different models win at different tasks, and pricing shifts with every release. Without an eval loop, every release forces a choice between two bad options: upgrade blind, or fall behind while competitors capture the gains. With one, upgrading is a one-hour experiment. Small companies with eval loops adopt new models in days; big companies without them take quarters. This is the single biggest structural advantage a 10-person company has right now — don't waive it.
Start embarrassingly small. Twenty examples in a spreadsheet and a script that loops over them beats the enterprise eval platform you'll evaluate for three months and never deploy. The habit matters more than the tooling; the tooling can grow later.
Owner: whoever ships changes to the AI feature. Cost: a day to set up, minutes per run.
Habit 2: The prompt registry
The habit: every prompt in production lives in version control, with an owner, a changelog, and the eval score it shipped with.
At most small companies, prompts live in the worst possible places: hardcoded strings, someone's Notion page, a Slack thread titled "new prompt try this one." Then the person who wrote the magic words leaves, or someone "improves" a prompt in the dashboard on a Friday, and Monday's mystery regression eats two days of debugging.
The fix costs almost nothing. Prompts are code — they determine your product's behavior more than most of your actual code does — so treat them like code: a prompts/ directory in the repo, one file per prompt, changes via pull request, each change noting what moved and what the eval said. Include model name, version, and parameters in the registry, because "the prompt" is really the prompt-model-settings triple; the same words behave differently on a new model.
The registry also quietly becomes your institutional memory. Six months of changelog tells you which phrasings failed and why, what the last model migration broke, which instructions exist because of a specific customer incident. At a 10-person company where knowledge lives in heads, this is the difference between an asset and a liability when someone goes on leave.
Owner: same engineer who owns your repo hygiene. Cost: half a day to set up, near-zero ongoing.
Habit 3: The weekly incident review
The habit: thirty minutes, once a week, same slot: look at where the AI was wrong, weird, or embarrassing — and turn the worst cases into eval examples.
This closes the loop the first two habits open. Production is where your AI meets inputs you never imagined, and at a small company those failures are invisible by default: support hears about one, sales sees another, nobody connects them. The weekly review is the connection point. Pull ten flagged interactions — user thumbs-downs, escalations, anything an engineer noticed — and ask three questions per case. Was this actually wrong, or just imperfect? What category does it fall in? Should it become an eval case?
That last question is the flywheel. Every real failure that enters the eval set is a failure that can never silently return. Your eval suite stops being a static artifact and becomes a compounding record of everything production has taught you — which is exactly the practice the eval-platform world now preaches as "regression datasets that grow with new failure modes," implemented with a calendar invite and a spreadsheet.
The review also forces the ownership question that kills so many deployments: who fixes this? An agent that's 94% right and unowned degrades to 85% within a quarter — models shift, data drifts, users find edges. Same agent, plus a weekly review with a name attached, improves instead.
Owner: rotate it. Everyone learns what the AI actually does. Cost: 30 minutes weekly, and it should be fun — failure reviews at this size are half debugging, half comedy.
The optional fourth habit: the model review
Once the first three habits are running, there's a fourth worth adding — quarterly, not weekly. Sit down with your eval scores and your API invoice and ask: is our current model mix still right? The 2026 market gives this question real teeth. Frontier pricing shifts with every release, capable small models keep closing the gap on routine tasks, and a workload that justified a flagship model in January may run at a tenth the cost on a mid-tier model by June — at identical eval scores. Median time-to-value on agent deployments industry-wide is running around five months; a quarterly model review means you re-earn that value continuously instead of locking in whatever was best the month you launched. The review takes an afternoon: rerun the eval suite against two or three candidate models, compare score and cost per task, and switch only when both numbers agree. Because habits one and two exist, the switch itself is a pull request, not a project.
What you explicitly don't need
No AI platform team. No fine-tuning infrastructure (at your scale, prompting plus retrieval on frontier models wins on cost and iteration speed in almost every case in 2026). No agent framework migration every quarter. No six-figure observability contract before you have your first spreadsheet eval. Every one of those is a "quarter of runway on infrastructure nobody asked for" trap that mimics progress.
The tools you do need are mostly free or cheap: version control you already have, an LLM API bill you already pay, and the discipline to run the three loops.
The compounding effect
Run all three habits and you get a system where production failures become eval cases (habit 3 → habit 1), eval scores gate prompt changes (habit 1 → habit 2), and the registry documents what worked for whoever touches it next (habit 2 → habit 3). It's a flywheel, and after six months it produces something that looks from the outside like sophisticated AI operations — but is actually just three habits, held consistently, by a very small team.
That's the operating model. Not an org chart. A loop.
Want the templates — eval spreadsheet, registry structure, incident review agenda? We hand them to every client in week one. Book a call.
