The anatomy of an enterprise AI platform

If you are being asked to buy, build or assemble an enterprise AI platform, it helps to have a map of what one is made of. Here is the one we use.

Layer 1: Compute

Accelerators, the fabric between them, and the scheduler on top.

Two distinct workloads live here with different requirements. Serving is latency-sensitive, mostly single-node, and bounded by memory bandwidth. Training is throughput-sensitive, multi-node, and bounded by interconnect.

Estates that plan only for serving discover later that they own a lot of accelerators they cannot train across, and interconnect is decided at procurement time.

Layer 2: Models

The catalogue: frontier models via API, open-weight models you host, and domain models you have tuned.

The thing that matters at this layer is not which models you have but whether applications reference them by name. If they do, you cannot move — and models get deprecated on the provider's schedule, not yours. Applications should ask for a capability class; something else resolves it to a model.

Layer 3: Data

The governed record. Entity resolution, survivorship, quarantine, lineage.

This layer is where enterprise AI programmes actually succeed or fail, and it is the one most often deferred because it does not demo. Everything above it inherits its quality. A perfect model on an unresolved entity graph produces confident answers about the wrong customer.

Layer 4: Retrieval

Turning a question into the right evidence: embedding, indexing, search, re-ranking, and the assembly of context.

Two properties separate production retrieval from a tutorial. It must be scoped — restricted to the resolved entity, the permitted documents and the current versions. And it must be able to abstain — return nothing and say so, rather than returning its top five results regardless of relevance.

Layer 5: Orchestration

Agents, tools, workflows, approval gates and human handover.

The unit of design here is the tool contract. An agent's competence is bounded by its tools, and its risk profile is defined entirely by what those tools can do. A tool register that someone in a risk function can read and understand is not a nice-to-have; it is the thing that makes the layer governable.

Layer 6: Governance

Policy enforcement, evaluation gates, cost attribution, drift detection, audit export.

The distinguishing question: can this layer stop something? A governance layer that observes and reports is monitoring. One that can block a promotion, refuse an action and halt an agent is governance.

Layer 7: The decision record

The output. Every decision the system made, with its trigger, evidence, lineage, model versions, policy gate, confidence and outcome.

Most estates do not have this layer at all — the decision is implicit in a status field somewhere. Adding it turns audit into a query, explanation into a render, and your own operating history into training data.

Which layers you can defer

Honestly:

• Compute — yes, initially. Start serverless, commit when sustained utilisation justifies it.
• Models — yes. Start with what is available; keep names out of application code so you can change your mind.
• Data — no. This is the one. Deferring it means producing outputs you cannot defend and redoing the work with a credibility deficit.
• Retrieval — partially. A simple implementation is fine to start; scoping and abstention are not optional.
• Orchestration — yes. Plenty of value ships before any agent exists.
• Governance — the evaluation gate, no. Everything else, mostly.
• Decision record — no, and specifically not lineage capture, which cannot be reconstructed after the fact.

The pattern: the layers you can defer are the ones you can change later. The ones you cannot defer are the ones that write history.