Software is no longer only written. It is partly grown.

The deal that held for sixty years

For sixty years, software did exactly what it was told. Every rule was written by a person. Every outcome was the same every time. That was the deal: software could not surprise you, and it could not handle anything no one had thought of.

AI changes the deal. You can now build parts of a system that work things out on their own: handling messy input, the case no one planned for, the question asked in a way no one predicted. That is the gain. The cost is the same sentence read again: parts that work things out on their own.

Real systems now have two kinds of parts

The parts that must be exact (billing, validation, access rules, the things an auditor will ask about) are still best written the old way. Rules, written by people, that run the same every time. That is not old-fashioned. It is the correct tool. You do not want a model deciding what a number is.

The parts that need judgment (reading a complaint, drafting a reply, working out what is wrong with something, answering a question in plain language) are where AI earns its place. You could not write those rules down if you tried.

The hard part is not the AI itself. Calling a model is straightforward. The hard part is two things: knowing exactly where the line goes between the two kinds of parts, because most failures come from a line drawn carelessly; and building the guardrails on the AI side so it cannot quietly do the wrong thing, because an unguarded AI part will, eventually, get something wrong with full confidence. Most teams do neither. That is why their AI performs well in a demo and disappoints them in production.

The reliability is in the guardrails, not the model

It is tempting to think the capability lives in the model, and so does the trustworthiness. It does not. The model is something any firm can rent. What makes an AI system dependable is the engineering around the model:

• —Limits on what the system is allowed to touch and do.
• —Grounding, so its answers come from your real data rather than from something that merely sounds correct.
• —A defined way to handle uncertainty, including letting the system say it does not know, instead of guessing with confidence.
• —Tests that check whether answers are good, the way ordinary tests check whether code is correct.
• —Monitoring that watches whether the system is still right, not only whether it is still running.

A firm that says it builds AI agents, and means that it calls a model, has done the easy part. The limits, the grounding, the tests, the monitoring, the unglamorous work, is what makes a system safe to depend on. It is the part we are built around.

Monitoring has to watch a new thing

Traditional monitoring asks whether a system ran and whether it was fast. That is still necessary. It is no longer sufficient. An AI system also has to be watched for whether it is still right, because models drift, the world changes, and a system that worked last month can begin getting things wrong without ever crashing or slowing down. Checking quality is therefore not a test you run once before launch. It runs for as long as the system is live.

Unpredictability spreads if you let it

Place an AI part into a system and its unpredictability tends to leak into the parts next to it, unless it is deliberately walled off. Reliability becomes harder when AI is in the mix, not easier. That is the purpose of the guardrails: every AI part is treated as a capable but contained component, with a clear job and a safe way to fail. Not a colleague. Not magic. A component defined precisely enough to rely on.

What we build

We do not build software in the sense the word used to carry. We do not build digital minds. We build systems that are part written and part grown: exact where they must be, intelligent where it helps, and engineered so the intelligent parts cannot quietly break the whole. Getting that mix right is the real skill now. It is what we do.