RESEARCH AND EVALUATION

Research is where AI projects are won and lost, and it is where discipline is most often absent. The failure mode is familiar: a promising early result that nobody can reproduce, a metric that turns out to have measured the wrong thing, six months of work that cannot be summarised because nothing was recorded.

We treat research as an engineering activity with an unusual output. The output is knowledge rather than a feature, but everything else applies. It goes in version control, it gets reviewed, and it has to be reproducible by somebody other than its author.

Framing before working

No experiment starts until four things are written down and agreed.

The question. Specific enough to be answered. "Can we improve support?" is not a question. "Can a retrieval system answer at least 70 percent of tier one tickets correctly, using only our current documentation, at under two seconds and under a defined cost per ticket?" is a question.

The hypothesis. What we expect and why. Stating it in advance is what stops us from discovering, after the fact, that whatever we got is what we predicted.

Success criteria. The number, on the dataset, at the threshold, that would mean this worked. Agreed with the client before work starts, not negotiated afterwards when the result is already known.

Kill criteria and a timebox. What result, or what elapsed time, would mean this plan is not going to work. Naming it in advance, while nobody is invested, is what makes it possible to act on later, when everybody is. It kills a plan rather than an engagement: the usual outcome is a better approach, not a cancelled project.

This framing takes an hour or two. It routinely saves weeks, and it converts an open ended investigation into something with a defined cost.

Baselines first

The first thing we build is the dumbest thing that could work.

Depending on the problem that might be a keyword search, a set of hand written rules, a logistic regression on obvious features, the performance of the process you run today, or the accuracy of a competent human on the same task. Often it is several of these, because they answer different questions.

This is not a contrarian position. The first of Google's published Rules of Machine Learning is to launch without machine learning first, and the ML Test Score rubric we work to includes a check that reads, in full, "a simpler model is not better." The practice is well established. It is just widely skipped, because it delays the interesting part.

Baselines do three jobs:

• They establish what "better" means in a number, so later improvements are measurable rather than atmospheric.
• They expose problems in the data and the evaluation early, while they are cheap to fix. A baseline that scores suspiciously high almost always means leakage.
• They occasionally solve the problem, which saves everybody a great deal of money.

We report the baseline honestly even when it is embarrassing to the case for a larger project. Especially then.

The evaluation comes first

We build the evaluation before we build the thing being evaluated. This is the practice we would keep if we had to abandon every other one on this page.

Build the evaluation set deliberately. Not a random sample of convenient data. A set that reflects what the system will actually see, including the rare and awkward cases that matter disproportionately: the long tail, the adversarial input, the category with forty examples that happens to be the most expensive to get wrong. We stratify it so that a headline number cannot hide a segment where the system is useless.

Hold out properly. Split by whatever unit prevents leakage: by customer, by document, by time period, not naively by row. Time based splits for anything where the future differs from the past, which is most things. We look for leakage actively rather than assuming its absence, because a result that is too good is a bug report.

Choose metrics that map to the decision. Accuracy is rarely the right measure. If false positives and false negatives cost different amounts, the metric has to reflect that. If the system ranks, we measure ranking. If a human reviews the output, we measure how much work the system saves that human, which is not the same as how often it is right.

Measure cost and latency as first class results. A model that is two points better and four times more expensive is not automatically better. We report quality, cost per call, and latency together, always, because the tradeoff between them is the client's decision and they cannot make it if we only report one number.

Treat LLM judges as instruments that need calibration. Using a model to grade another model's output is useful and it is not free. The failure modes are documented and specific. Judges prefer whichever option they are shown first, so we swap the order and average both directions. Judges prefer longer answers, and a rubric line telling them not to reduces the effect without removing it. Judges score their own model family higher than others, so where it matters we judge across families rather than inside one.

None of that is fixed by using more judges. Ensembling reduces variance; it does not touch a bias every member shares. So we calibrate against human labels on a sample and report the agreement as a kappa rather than a raw percentage, because raw agreement flatters any judge on an unbalanced set.

We also pin the judge's model version and treat a version change as an event that invalidates the calibration. This is the quiet failure: the judge is upgraded, mean scores shift by a few points, every threshold in continuous integration keeps passing, and the comparison against last quarter's numbers is now meaningless without anybody being told. A judge score is never presented as ground truth without saying which model produced it, which version, and what it was validated against.

Keep humans in the loop where it counts. For generative work, someone reads the outputs. Regularly. Aggregate metrics hide failure modes that are immediately obvious to a person looking at twenty examples, and no dashboard has ever caught a tone problem.

Assume the evaluation decays. The set that represented reality at launch drifts away from it. We schedule refreshes, and we treat the evaluation set as an asset that needs maintenance rather than a thing that was done once.

Running experiments

Change one thing. When three things change and the number moves, nothing has been learned. Where several changes must be tested together, the experiment is designed for that explicitly rather than by accident.

Track every run. Code version, data version, configuration, environment, random seeds, and results, recorded automatically rather than in somebody's memory or a spreadsheet that stopped being updated in March. If a number appears in a report, we can point at the run that produced it.

Make results reproducible, and be precise about where you cannot. A second person, on a different machine, should be able to reproduce a result from what is in the repository. We fix seeds where the work allows it, pin dependencies, and version datasets.

Some of it genuinely cannot be reproduced, and being exact about that is part of the job. Setting temperature to zero does not make a language model deterministic. The usual explanation, floating point addition not being associative, is only half of it. The larger cause is that inference kernels reduce differently depending on batch size, so the same input returns different numbers depending on what else happened to be in the batch alongside it. On a hosted API you cannot see or control that batch, it contains other customers' requests, and the request may be routed to different hardware each time. Bit-identical output from a hosted endpoint is not a thing you can promise.

A fixed seed still helps, because it controls the sampler. It does not control the rest. So where non-determinism is real we quantify it: repeat the run, report the variance, and set thresholds with the variance in mind rather than tuning against a single lucky sample. Saying this plainly is more useful to a client than a reproducibility claim that quietly does not hold.

Keep a research log. A running written record: what we tried, what happened, what we concluded, what we would try next. It takes minutes a day. It is the difference between a project that can be summarised in an hour and one that cannot be summarised at all, and it is what makes a negative result useful to the next person instead of merely disappointing.

Review analysis like code. A second person checks the reasoning, not only the syntax. Is this comparison fair? Is the test set clean? Would this conclusion survive if the threshold moved? Analysis errors are quieter than code errors and considerably more expensive, because code that is wrong usually breaks while an analysis that is wrong usually just gets believed.

Error analysis beats leaderboards

An aggregate score tells you where you stand. It does not tell you what to do next. Error analysis does.

So we read the failures. We group them into categories, count each category, and put the counts in front of the client. This routinely reveals that most of the loss comes from one narrow, fixable cause: a document type that parses badly, a class the labels disagree on, a question format nobody anticipated.

That is a far more productive conversation than a discussion about whether to try a bigger model, and it is usually a cheaper fix.

We also look for the failures that matter more than their frequency suggests. A system that is wrong two percent of the time is fine, unless the two percent is concentrated in your largest customer's use case.

Reporting honestly

Report what we measured, not what we hoped. Including results that weaken the case for continuing.

Report uncertainty. A single number from a single run on a small test set is not a result, it is an anecdote. We report variance, and we say when a difference is inside the noise.

Separate what we know from what we believe. Both are valuable. Confusing them is how a project ends up built on an assumption that everyone thought had been verified by someone else.

Say what we did not test. Every study has an edge. Naming it is how the client makes an informed decision instead of an optimistic one.

Write for the reader who has to decide. Findings first, method underneath, detail in appendices. A result that a decision maker cannot act on has not been delivered.

When research turns into a product

There is a moment when the question shifts from "does this work" to "will this keep working". Recognising it matters, because the practices change.

We treat that transition explicitly rather than letting it happen by drift. The experiment code gets rewritten as a system rather than promoted into one. The evaluation becomes an automated regression suite that runs in continuous integration. The manual data preparation becomes a pipeline. The choices made during research get written down, because in production somebody has to know why the threshold is 0.7.

Because we build proofs of concept with that transition in mind, ours tend to graduate rather than be thrown away. That is the subject of the next page.

---

Next: Engineering