AI Will Confidently Build You the Wrong Thing. Here's How I Stop It.
An 851-line spec matrix, a deliberate workflow, and the argument that AI-assisted development doesn't need better prompts — it needs better thinking. A case study in building Smith, the AI orchestration library behind Hadithi.
I didn’t set out to write 851 lines of specification for an open-source library, and I’m not done yet. I did it because building Hadithi demanded it, and because AI, without that kind of structure, will confidently build you the wrong thing at impressive speed.
This is the story of how I’m building Smith, and why the spec has to come before everything else.
The Problem That Started Everything
Hadithi needed complex AI infrastructure. Multiple agents, coordinated workflows, context that had to persist and evolve across interactions. The kind of system where a single misunderstood requirement doesn’t just break a feature - it corrupts the entire architecture downstream.
I tried the conversational approach first. Describe what I want, iterate, describe again. It’s fast at the start. Genuinely fast.
But the further in you get, the more the AI is working from a degraded version of your original intent. Each prompt is a lossy compression of what you actually meant. By the time you’re debugging the third unexpected behavior, you’ve lost the thread entirely, and so has the model.
The 80% is fast. The last 20% is where you pay for every shortcut you took at the beginning.
I needed a different approach.
Vision → Architecture → Spec Matrix → Implementation → Update Spec Matrix → Implementation
This is the workflow I landed on. It looks linear written out like that. It isn’t.
The key insight is that each stage is a forcing function for the next. You can’t write a real architecture without a clear vision. You can’t write a real spec without a real architecture. And you can’t implement honestly without a spec that tells you - precisely - what “done” means.
Nihar Varma put it well: “Spec-driven development isn’t about generating better code. It’s about forcing myself to think clearly about what I actually want. The spec is a thinking tool.”
That’s exactly it. The spec isn’t documentation. It’s thinking made legible.
When I write a spec assertion for Smith, I’m not writing it for the AI. I’m writing it for myself - to catch the places where I thought I understood what I wanted but didn’t.
Enter the Spec Matrix
The Smith SPEC_MATRIX.md is a structured contract between what the library is supposed to do and what it actually does.
Every assertion is traceable to the architecture document. That’s a hard rule — not a guideline. If I want to add a new assertion and it can’t be traced back to the architecture, the architecture has to be updated first. The spec doesn’t lead the architecture. The architecture leads the spec.
The rules I set for myself:
“Every new assertion must be traceable to the architecture document. If the architecture document is ambiguous, the ambiguity should be resolved in the document first… Prefer contract assertions over implementation-coupled assertions.”
That last line matters more than it looks. Implementation-coupled assertions are brittle — they break every time you refactor. Contract assertions describe behavior, not mechanism. They survive refactors. They survive rewrites. They describe what the system promises, not how it currently keeps that promise.
Martin Fowler’s framing is useful here: a spec is “a structured, behavior-oriented artifact… written in natural language that expresses software functionality and serves as guidance to AI coding agents.” Behavior-oriented. That’s the key word.
The Loop That Makes It Work
Here’s what most spec-driven workflows get wrong - they treat the spec as a document you write once and then implement against.
That’s not how software works. Requirements drift. Understanding deepens. The act of building reveals things the spec didn’t anticipate.
The Smith spec has an explicit mechanism for this. When a spec exists and currently fails, the contract is still recorded, and the failure represents live implementation drift, not a spec error. The spec doesn’t get quietly updated to match the code. The drift is surfaced, named, and resolved deliberately.
This is the loop: implement → discover drift → update spec → implement again.
It sounds like overhead. And honestly, sometimes it is. But the alternative is a codebase where the spec and the code have silently diverged - where the documentation describes a system that no longer exists, and nobody knows exactly when it stopped being true.
The Aviator team describes specs as “refined context” - providing just enough information to the LLM to be effective without overwhelming it. That’s the right mental model. The spec isn’t a novel. It’s a contract. Precise, traceable, and honest about what it doesn’t yet know.
The Criticism Worth Taking Seriously
I’m aware of how this looks. 851 lines of spec for a library. There’s a version of this that’s just waterfall development with better branding.
The critics aren’t wrong that specs, and code drift. They’re not wrong that front-loaded documentation can become a cage that kills adaptability. The argument that a detailed spec creates a false sense of completeness - that the most important insights come from the act of building - that’s a real tension I’ve felt.
But here’s the difference: a spec that tracks drift is not the same as a spec that pretends drift doesn’t happen.
The Smith spec doesn’t claim to be complete. It claims to be honest. When the implementation diverges, the spec says so. That’s not documentation debt - that’s a live diagnostic.
And there’s a broader context worth naming. According to the 2025 Stack Overflow Developer Survey, more developers actively distrust the accuracy of AI tools (46%) than trust it (33%). That gap exists because most AI-assisted development workflows have no mechanism for catching when the AI confidently builds the wrong thing.
The spec matrix is that mechanism.
What I’d Tell Someone Starting This Workflow
Start with the vision. Not a prompt - a vision. What is this thing? What does it explicitly not do? Write that down before you write a single line of code or spec.
Then write the architecture. Not a diagram - a document. Decisions, tradeoffs, the reasoning behind the structure. This is the source of truth everything else traces back to.
Then write the spec. Behavior-oriented assertions. Traceable to the architecture. Prefer contracts over implementation details.
Then implement. And when the implementation drifts - and it will - update the spec first. Name the drift. Resolve it deliberately.
The loop is the workflow. The spec isn’t a phase you complete. It’s a living contract you maintain.
The Honest Summary
Building Smith and Hadithi taught me that the problem with AI-assisted development isn’t the AI. It’s the absence of structure that forces you to think clearly before you build.
AI will generate code fast. Genuinely fast. But fast in the wrong direction is just a more efficient way to end up somewhere you didn’t want to be.
The spec matrix slows down the beginning so the rest of the process can move with confidence. That’s not a bug in the workflow. That’s the point.
The future of AI-assisted development isn’t better prompts. It’s better thinking - made legible, traceable, and honest about what it doesn’t yet know.
Smith is an open-source AI orchestration library I’m building to power Hadithi. I’ll share a full breakdown of what it is - and what it deliberately isn’t - as soon as it’s ready to ship. The full spec matrix is available on GitHub if you want to dig in now. And if Hadithi sounds interesting, join the waiting list - I’d love to have you along for the build.