Architecture Pattern

Horizontal
AI Platform
Layers

POCs are easy. Production AI is hard.
The question isn't build vs. buy on the whole solution — it's build vs. buy on each layer.

Core principle
Own what differentiates you. Buy what is commodity. Orchestration + evals = your moat
Infra + model access = commodity
Speed vs. control — decide per layer.
Layer 0
Use case plugins
Every new intent is a plugin. Platform stays unchanged.
Always yours
Layer 1 — Eval + Observability
Accuracy evals · latency SLOs · HITL · audit
Design this before writing a single agent.
Design first
Layer 2 — Orchestration
Router · supervisor · fan-out / fan-in · state
Multi-intent · dependency resolution · retry · checkpoints
You own this
Layer 3 — Core Capabilities
Memory · RAG · auth · privacy · tool registry
Build selectively. Privacy and isolation — own it.
Selectively build
Layer 4 — Model Gateway
Routing · rate limiting · cost tracking · fallback
Config-driven. Swap models per worker without touching orchestration.
Buy + configure
Layer 5 — Cloud Infrastructure
Compute · LLM endpoints · search · monitoring
Data residency · compliance · enterprise agreements
Delegate