When a small tuned model is the right answer

The claim we are making

Small domain-tuned models are strong where a task is narrow and repeated, and weak where it requires composition or generality.

Stated that abstractly it is not controversial. What makes it operationally interesting is how much enterprise work falls into the first category: extraction, classification, templated generation, routing — high volume, well defined, repeated thousands of times a day.

On that traffic a frontier model is doing work well within its capability, at many times the cost and latency of something that would do it just as well.

Where we would expect parity, and why

Likely parity — the task has a fixed output shape and a bounded input distribution, so tuning has something stable to learn:

• Structured extraction from domain documents
• Classification into a defined taxonomy
• Format-constrained generation
• Summarisation against a fixed template
• Routing and triage

Likely not — the task requires knowledge or composition that was never in the tuning distribution:

• Multi-step reasoning over several documents
• Anything needing general world knowledge outside the domain
• Novel task framing not seen in tuning
• Long-context synthesis

Genuinely uncertain: open-ended summarisation, and question answering over retrieved context. These sit on the boundary and we would not predict them without testing.

We have not evaluated agentic tool-use and would expect it to fall closer to the multi-step reasoning category.

The approach we would use

Distillation, rather than tuning on curated or synthetic examples: run the frontier model in production for a period, capture inputs and outputs with human corrections, and tune the small model on that corpus.

The reason is mechanical rather than clever. The training distribution ends up being exactly the production distribution, including the awkward cases that a curated set quietly omits.

The sequence: run frontier, accumulate, distil, route, escalate.

How to test it on your own workload

Do not take the categories above on trust — the boundary moves with every model generation, and it will differ on your data.

1. Take a hundred real examples per task type, drawn from production rather than constructed.
2. Score both systems against your rubric, graded by your domain owners, blind to which system produced which output.
3. Define parity as within one standard error on that rubric, and decide the threshold before you look.
4. Route the task types that reach parity; escalate the rest on uncertainty and on rarity.
5. Keep a small forced-escalation sample so you can see the gap you are accepting.

The caveat that matters most

"Small model" is a moving target. The parity boundary has moved with every open-weight generation and will move again, in the direction of more tasks being served well by less compute.

That argues for building the routing rather than picking a model: a route that resolves a capability class can absorb the next generation as a config change. A model name in application code cannot.