Notes
Field notes.
Short, specific pieces on getting AI into production, technology due diligence, and running systems that cannot go down.
What "in production" actually means for AI
Most AI work stalls in the same place. A demo works in a notebook, everyone is impressed, and then it never reaches customers. The gap is not the model. It is everything around it.
In production, an AI feature is held to the same bar as the rest of your stack. Evaluations you can run on every change, so you know whether a new prompt or model made things better or worse. Guardrails for the cases where the model is wrong, and a clear fallback to a human when confidence is low. Logging, monitoring, and someone on call when it misbehaves at 2am. A cost and latency budget you actually track.
Teams skip these because they are not the fun part. But they are the difference between a clever demo and a system people rely on. If you cannot measure whether your agent is getting better, you do not have a product. You have a science experiment.
My rule is simple. Before anything ships, I want to see how we measure it, how it fails safely, and who owns it when it breaks. Get those three right and the model choice matters far less than people think.
Three questions I ask before signing off on a deal
When a board or investor asks me to look at a target's technology before a deal, I am not there to admire the architecture. I am there to find what will cost money after completion. Three questions surface most of it.
First, what breaks if the two key engineers leave. If the system only runs because a few people hold it in their heads, you are buying a risk, not an asset. I look for documentation, tests, and whether anyone other than the authors can safely change the code.
Second, what is the real state of security and data. Not the policy document, the practice. Who has access, how are secrets handled, what would a breach expose, and does the AI or data use create obligations no one has priced in.
Third, what does it cost to run and to grow. Cloud bills, licences, technical debt, and the engineering the growth plan quietly depends on. Plenty of targets look profitable until you add the work needed to support the growth the deal assumes.
None of this needs a large team or months of work. A focused review answers the questions that change the price or the terms, and gives the buyer something they can act on.
What a 24/7 platform taught me about shipping AI
For years I was responsible for a platform that could not go down. It ran a state transport network around the clock. When something failed, people noticed immediately, and the standard was simply that it did not fail.
That discipline transfers directly to AI, and most teams have not made the connection yet. Running mission-critical systems teaches you to assume things will go wrong and to design for it. You build in monitoring before you need it. You make failure visible and recoverable. You decide in advance what happens when a component misbehaves, rather than improvising during an incident.
AI raises the stakes because the failure modes are less obvious. A model does not crash. It quietly gives a worse answer, drifts as inputs change, or is confidently wrong. So the same instincts apply, with a twist. You measure quality continuously, not once. You keep a human in the loop where the cost of being wrong is high. You treat the model as one more component that needs ownership, review, and a way to roll back.
The tools are new. The engineering judgement is not. Teams that already know how to run reliable systems have a head start on running AI well. They just need to apply what they already know.
