Last month something failed in production that had never failed in CI. It compiles, unit tests are green, it builds. The contract between services matched the OpenAPI file everyone had signed off on. The symbolic world of the codebase was all green. And still, it executed the unintended behavior exactly as coded.

If you have been shipping long enough, that story is boring on purpose. But boring might mean we have internalized and normalized it. It is the story we stop telling at stand-up because it embarrasses the idea that “we are done.” It forces an uncomfortable question into the open: when we merge, tag, and deploy, what exactly are we claiming? That the code is consistent, or that it behaves?

Consistency is cheap to check. Software behavior is something else. It means alignment with a world that does not care about our abstractions. Reality keeps reminding us that our systems are simplifications. The world includes flaky networks, malicious inputs, regulators, dependencies on other services or vendors, and humans who click the wrong thing twice and still expect to be safe. No linter sees that world in full. Even in a monolith, every system large enough faces the same issues microservice architectures were built to handle.

At some point every release becomes a bet. The honest senior engineer knows the difference between a bet backed by evidence and a bet based on faith in a contract.

Here is a detail most teams miss, especially now. The tools multiplied. Many of them got better. But scale outran our mental models and left us with a challenge nobody asked for: a clear picture of what assurance actually means.

Urgency to deliver. Too many moving parts. Premature abstractions stacked on top of each other. The picture was supposed to get clearer. It gets fuzzier.

What has more long-term value: hyper-velocity or hyper-quality? Hyper-velocity can make you a first mover. Rules built on fuzzy understanding cannot be operated reliably or responsibly. If you cannot explain the arcs that narrate system behavior, you are either lost in a niche of technicalities or without control of the system. Either way, that is a path to a destination without a market.

And I am not saying it matters who wrote the code, humans or AI assistants. Internal design, as complexity scales with each feature, is where the value lives. Who wrote it matters less. Who designed it matters more because now design is the new programming.

You will hear people talk about AI code as if the risk were hallucinations or sloppiness. Those are real. But once you get past the vibe-code phase of prototypes and enter the execution loops of spec-driven AI engineering, code quality and divergence stop being the problem. The deeper issue is structural. Internal design. The wiring of mature, powerful abstractions that can entangle with the domain in the wrong ways by next semester, maybe next week.

Large models are hermeneutic engines. They extend, compress, and interpolate meaning from text. So does a senior engineer reading a legacy module at 11 p.m. The difference is that we have spent a career building social and technical antibodies against our own drift: code review, blameless postmortems, the shame of a rollback. LLMs do not have those unless you actively direct scrutiny at a proposed change: a PR.

Then it does not matter where the drift came from, AI assistance or code from your teammates. The old antibodies still apply. But with AI, execution of complex code got cheaper. The feeling of progress arrives faster, or at least the expectation does. You get to working code sooner. You get plausible explanations sooner. You get a story that hangs together. Text, in that mode, is seductive. It can look like evidence. What are your techniques for staying protected from well-articulated technical fantasy?

Without an external anchor, nothing in the nature of language, human or synthetic, guarantees the story touches ground.

Aphyr was right to worry that the future of everything might be lies.

Is this new plane safe to fly?

Go or no-go?

The aerospace and automobile industries do shakedown tests for a reason.

A shorter companion on the same thesis, The Half of CI We Forgot to Automate, covers what we actually bought with CI automation versus what we still answer with a vibe.

So reboot the qualifying question. It is not “Do we trust this merge request?” It is “What would falsify our belief that this release is safe?” It is also “What pain do we need to make practically impossible?” If you cannot answer that without hand-waving, you do not know how to collect the evidence that protects you. You are holding faith in symbols and defaulting to internal consistency checks: types, schemas, unit tests green, the comforting rhythm of a pipeline blind to the interstitial spaces of wiring your system together. Relying only on internal correctness for a deployment is unitary reductionism plus faith in contracts.

The alternative is to notice that outcomes include how the system performs once it is wired.

The alternative is to collect those outcomes as evidence.

Staff and senior engineers are not paid primarily to write code. They are paid to own the definition of done when the cost of being wrong is exponential.

“Safe to fly” is not the same as “we designed it so it flies.”

Can you show that you designed for these edge scenarios, or produce measured evidence?

How well does this version perform when degraded off its happy path?

Those two questions should have materially different answers.

Since the definition of done lives in acceptance criteria, there must be a piece of testing that produces proof beyond internal consistency and correctness.

And now that we are cyborgs with “second brains” that assist us in writing code, you can generate those criteria with that assistance too. You can automate checks against them.

But neither move dissolves accountability.

Someone with skin in the game still has to say: this is the behavior we will not accidentally betray; this is the failure mode we refuse to call edge; this is how we will know we were wrong before a customer tells us; this is how we preserve deep understanding of the system; this is why I can tell you that you can trust this system without lying and without posturing.

None of this is anti-automation or anti-delegating-AI-coding. That is how you scale discipline. The lesson is narrower and harder: automation without explicitly human stakes is just a faster path to confident wrongness.

Success without evidence burns out as fast as a shooting star.

A release is not proof that the software matches intent. It is a promise that you have decided, for now, which parts of reality you are willing to bet on, how you will notice if reality disagrees, and that you have what you need to survive and keep your users safe.

If a pipeline runs every symbol and every test you thought to write but you avoided including behavior and performance in what “done” was supposed to mean, how can you present evidence in advance if forensics or a regulator require it? So the next time you ship, ask one question before you tag the release: if this were subtly wrong in a way that made sense on paper, how would we catch it, and why would we react that way?

If the only answer is “we would not,” then you are not shipping evidence. You are shipping faith. Faith is a fine thing in human relationships, but in production it belongs labeled as such, with someone’s name next to it who is willing to wake up for it when the incident alert rings at 3 a.m.

That name is still yours.

To hold a clear mental model of the system we want to trust, we need faith in engineering performance protected upfront by sound design. For that faith to transfer to teammates and stakeholders, it helps a lot if you have evidence to back it up.