ENGINEERING

Most AI work fails on the way to production, not in the research. The model was fine. What was missing was everything around it: the pipeline, the tests, the monitoring, the cost control, the ability of anyone other than its author to change it safely.

This page is about that everything. None of it is exotic. It is standard software engineering practice, applied to systems that happen to contain a probabilistic component, plus the handful of things that are genuinely different when your system can be confidently wrong.

The foundations

Version control for everything. Code, configuration, prompts, infrastructure definitions, evaluation sets, and dataset versions. Prompts in particular are logic and they belong under review like any other logic, not pasted into a console where nobody can see the diff.

Small changes, frequently. Long lived branches and big-bang integrations are how teams lose weeks. We work in small increments that can be reviewed and reverted independently.

Everything gets reviewed. A second person reads the change before it lands. On research work that includes reading the reasoning, not only the code. Where staffing puts one person on a piece of work, we compensate deliberately: the client's engineers review, decision records get stricter, and the standard for written explanation goes up. We do not pretend a solo project has the same safety properties as a reviewed one.

Automated style and linting. Formatting is settled by a tool so that reviews are about substance. Nobody's attention should be spent on whitespace.

Reproducible environments. Pinned dependencies, containerised where it helps. "It works on my machine" is a bug, and in machine learning work it is a bug that can silently change your results rather than break your build.

Refactor continuously. Research code accumulates shortcuts because it is exploring. When it becomes a system, those shortcuts get paid off deliberately rather than shipped and inherited by whoever comes next.

Testing systems that are not deterministic

Testing is where AI systems differ most from ordinary software, and where teams most often give up. The answer is not to abandon testing. It is to test the right layers with the right tools.

Ordinary code, tested ordinarily. Most of an AI system is not a model. Parsing, retrieval, chunking, routing, formatting, business rules, error handling, and API surface are deterministic and should be tested exactly as any other code would be, with fast unit tests that run on every commit.

Data tests. Assertions that run over the data itself: schema, allowed ranges, null rates, category distributions, row counts, freshness. A pipeline that silently starts receiving nulls in a key column will degrade a model quietly for weeks. A failing data test catches it the same day.

Evaluation as a regression suite. The evaluation from the research phase becomes an automated test. When a prompt, a model version, a retrieval parameter, or a preprocessing step changes, the evaluation runs and reports the delta on quality, cost, and latency. Nobody has to remember to check, and nobody has to argue from memory about whether last month's version was better.

Threshold tests, not exact matching. For non-deterministic components, tests assert on properties and ranges rather than exact strings: the answer cites a source, the JSON parses, the classification stays within an agreed accuracy band, the latency stays inside budget.

Adversarial and safety cases. A standing set of inputs designed to break the system: prompt injection attempts, out of scope questions, malformed documents, empty inputs, inputs in the wrong language. These are tests, kept in the repository, run automatically.

Continuous integration, adapted

Continuous integration for machine learning has to respect the fact that some checks take minutes and some take hours or cost money.

On every commit: linting, unit tests, data schema validation, and a small fast evaluation slice. Minutes, not more. If the feedback loop is slow, people stop paying attention to it, and a red build that everyone ignores is worse than no build at all.

On every pull request that touches model behaviour: the full evaluation suite, with the results posted into the review so the change and its measured effect are visible in the same place. A model change is not reviewable without its numbers.

On a schedule: the expensive checks. Full retraining validation, long running benchmarks, drift analysis against recent production data, cost projections.

Before release: the complete suite, plus the adversarial set, plus a check that the artifacts and the code that produced them actually correspond.

What we actually run

"We apply software engineering discipline to machine learning" is a claim anybody can make, so here is the version you can check.

Google publishes a rubric called the ML Test Score: twenty eight concrete tests across data, model, infrastructure, and monitoring. Its scoring is deliberately unforgiving. Your score is the lowest of the four sections rather than the average, so excellent data tests cannot compensate for absent monitoring, and a system scoring zero is described as more of a research project than a production one.

The tests we run, on any system going to production:

• Input and feature expectations are captured in a schema, and that schema is enforced in the pipeline rather than documented next to it.
• The code that produces model inputs is tested like any other code.
• No prohibited or unnecessary data reaches the model, and the pipeline has explicit privacy controls.
• Model and prompt changes are reviewed before they land, with their measured effect attached.
• A simpler model is not better. This is a test, and it can fail.
• Quality holds on every important segment, not only in aggregate.
• Training and rebuild are reproducible from what is in the repository.
• Quality is validated automatically before anything is served.
• New versions are released gradually and can be rolled back without a deploy.
• Data invariants hold in training and in serving, and the two compute the same values.
• Prediction quality has not regressed, checked continuously rather than at release.

The rubric was written in 2017 for classical supervised learning at Google's scale, and some of it does not transfer. Hyperparameter tuning checks mean little for a hosted model you do not train. Feature by feature benefit analysis does not apply to a prompt. Numerical stability is somebody else's problem when inference is an API call. Where a test does not fit we say so rather than claim it, and we run the closest thing that does, which for language model systems is usually the evaluation suite and the adversarial set above.

Getting to production carefully

Ship behind a switch. New models and prompts go out behind a flag that can be turned off without a deploy.

Shadow first where the stakes justify it. Run the new system alongside the existing process without acting on its output, and compare. This is how you find the difference between evaluation performance and reality before that difference costs anything.

Roll out gradually. A percentage of traffic, watched, then more. Sudden full cutovers make it very hard to attribute a problem to its cause.

Design the failure path. Every AI component needs an answer to the question of what happens when it fails, is slow, or is unavailable. Fall back to a simpler model, to a cached response, to a rule, or to a human. Failing loudly and predictably is a feature.

Guardrails at the boundary. Validate what goes into the model and what comes out of it. Structured outputs get schema checked. Actions that have consequences get constrained, and the ones that are expensive or irreversible get human confirmation.

Keep a human in the loop where errors are costly. Not everywhere, because that removes the value. Where the cost of a wrong answer is high, the system's job is to make a person faster and better informed, not to replace them.

Monitoring what actually decays

An AI system in production needs to be watched along four axes at once.

Quality. Proxy signals in real time, such as user corrections, retries, escalations, thumbs down, or abandonment. Ground truth on a sample, through periodic human review. The gap between offline evaluation and live behaviour is the most important number nobody instruments.

Drift. Input distributions compared against training or reference data. New categories appearing. Old ones disappearing. Seasonal shifts that are normal, distinguished from structural shifts that are not.

Cost. Per call, per user, per day, with alerts on the trend rather than only on the total. Runaway cost in an AI system is usually a loop or a retry storm and it usually arrives on the invoice before anyone notices it in a graph.

Latency. At the percentiles that matter, not the mean. The ninety-fifth percentile is what your users experience as "slow" and the mean will hide it.

We also log enough to debug an individual bad answer after the fact: the input, the retrieved context, the model version, the parameters, the output. When a client asks why the system said something strange last Tuesday, "we cannot tell" is not an acceptable answer.

Cost and latency are requirements, not afterthoughts

We set a budget for cost per call and a latency target before designing the system, because both are architectural decisions rather than optimisations. A design that needs four sequential model calls has chosen its latency floor.

Where they matter, the usual levers are: a smaller or fine tuned model for the routine cases with escalation to a larger one for the hard cases, caching, batching, and moving work off the request path. A fine tuned open weight model is frequently cheaper and faster than a frontier API for a narrow task, and it can run inside your own infrastructure, which sometimes matters more than either.

We report the tradeoff rather than deciding it unilaterally. Quality, cost, and speed is a business decision informed by measurement.

Data governance and regulation

We are engineers, not lawyers, and nothing here is legal advice. But data handling is an engineering practice, and these are ours.

Minimise what moves. We work with the least data that answers the question, and we prefer to bring the computation to the data rather than the data to us. For sensitive domains that often means working inside your environment, in your cloud account, or on your hardware.

Know where every copy lives. Which systems hold client data, which providers process it, where it is stored, how long it is kept, and how it is deleted. Written down, not assumed.

Develop against safe data. Anonymised, pseudonymised, or synthetic data for development wherever it is sufficient. Access to raw sensitive data is limited to the people and steps that genuinely require it.

Respect the perimeter. What may leave your infrastructure is your decision, made explicitly and recorded, not a default of whichever tool is convenient. Where nothing may leave, we build with open weight models deployed in your environment or on device. This is one of the things we are known for.

Document each model. For anything that reaches production we write a model card, in the sense the original paper meant: a page or two covering what the model was trained or built on, where that data came from, what it is intended to do, what it is not suitable for, its measured performance including performance on segments rather than only in aggregate, its known failure modes, and the date it was last evaluated. It is a short document and it is the one that makes an audit survivable. More usefully, it is what lets a future engineer decide whether to trust the thing.

Classify the risk early. Whether a system touches regulated territory changes what has to be built, so it is a discovery question rather than a pre-launch surprise. Systems that affect access to employment, credit, education, essential services, or health carry obligations around documentation, human oversight, transparency to affected people, and record keeping. Under the EU AI Act those obligations are explicit and tiered by risk. Generative systems have disclosure expectations. Where personal data is involved, GDPR concerns such as lawful basis, purpose limitation, minimisation, and the right to an explanation shape the architecture, not only the paperwork.

We raise these in discovery and we build for them from the start. Retrofitting oversight and traceability into a finished system is expensive and usually partial.

How we use AI coding tools

We build AI systems and we use AI tools to build them. It would be strange to sell expertise in AI engineering workflows and not run them ourselves. Being explicit about it is also fair to clients who will otherwise have to ask.

What we let agents do. Scaffolding and boilerplate. Test generation from a specification. Mechanical refactors. Exploratory data analysis. Documentation drafts. Reading unfamiliar code to explain it. Reviewing our own changes for the things a tired human misses.

What stays ours. Architecture and system design. Experiment design and evaluation design. Interpreting results and deciding what they mean. Anything where being confidently wrong is expensive. An agent is a fast and eager collaborator with no stake in the outcome, and the judgement calls are the job.

The rules that do not bend. All generated code goes through the same review and the same tests as anything else, and the person who submitted it owns it completely. "The model wrote it" is not a defence.

Client data and code. What may be sent to which provider is set by your policy and your agreements, not by our habits. The default is your contract's terms. Where a client's data or source code may not leave their environment, we work without those tools or with locally hosted models. We will tell you what we use before we use it, and we will change it if you would rather we did not.

Handover

The engagement is finished when your team can run the system without us.

That means: a runbook covering normal operation, common failures, and who to call. The procedure for retraining and re-evaluating, including how to refresh the evaluation set. Alarm definitions and what each one means. The cost model and its assumptions. The written record of decisions, so a future engineer knows why the threshold is what it is. And working sessions with the people who will own it, because documentation alone has never transferred a system to anybody.

We would rather spend the last week of an engagement on this than on one more feature. A system nobody can operate is not delivered, whatever the contract says.

---

That is the playbook. It will change, because we keep learning, and we would rather revise it in public than defend a position we have stopped holding.

Back to the playbook index