“LLMOps” gets pitched as a fresh discipline that needs its own toolchain. In practice, most of the lifecycle reuses the MLOps primitives a team already runs — model registry, CI for training jobs, artifact storage, observability — while a smaller, higher-stakes subset genuinely diverges. The divergent slice is where the operational risk lives, and it is the only slice that justifies new tooling spend. We work with data and platform teams who are about to sign for a separate LLMOps platform on top of an MLOps stack they already paid for. The honest answer is that the question is rarely “MLOps or LLMOps” — it is “which lifecycle stages do LLMs add that classical ML did not have, and what is the cheapest way to instrument those stages without duplicating the rest?” The discussion that follows in our introduction to MLOps vs LLMOps sets up the same divergence-first frame. This article applies it. Where the LLM lifecycle actually diverges Classical MLOps assumes a model you trained on a dataset you control, served behind an endpoint you monitor. LLMOps inherits all of that — and then adds four stages that classical ML pipelines do not have, or have only in degenerate form. Lifecycle stage Classical MLOps LLMOps divergence Training data control Owned, versioned, reproducible Mostly upstream (foundation model) — you control fine-tuning data and prompts Evaluation Held-out test set + metrics like accuracy, F1 Eval set drifts with usage; needs LLM-as-judge or human review on rotating slices Inference cost Hardware-bounded, predictable per request Token-bounded, variable per request; cost-per-query is an operational SLA Prompt as artifact N/A Prompts, system messages, and few-shot exemplars are versioned artifacts on par with model weights Retrieval freshness N/A (or a separate feature store concern) RAG index staleness becomes a first-class production failure mode Serving Latency + throughput Latency + throughput + context-window utilisation + provider rate limits Reuse rows 1 and the bottom half of “serving” from your existing MLOps stack. Instrument the rest specifically. That is the entire divergence map. Evaluation: why classical metrics are not enough Accuracy and F1 are still useful — for tasks where there is a single correct answer and a held-out test set that does not go stale. Most LLM applications fail both conditions. The task is open-ended (summarisation, code generation, answer composition), and the input distribution shifts as users discover what the system can do. The metrics that actually constrain a production LLM in 2026 are: Eval-set drift: the share of production traffic that no longer resembles your held-out eval set. When drift exceeds a threshold (we use 15–20% as a starting heuristic, observed-pattern across our engagements, not a benchmarked rate), the eval scores have stopped describing the system you are actually running. Regression coverage: the share of production calls covered by an automated regression test that runs on every prompt change or model swap. Below ~60% coverage, prompt edits become silent rollouts. Task-grounded scores: HellaSwag-style commonsense benchmarks are useful as foundation-model selectors, but they do not predict how your specific RAG pipeline behaves on your domain. Build a small (200–500 example) task-grounded eval set and treat it as a first-class repository artifact. LLM-as-judge calibration: when you use one model to grade another, you need a small human-graded sample to calibrate the judge. Skip this and you optimise for the judge’s blind spots. The point is not that accuracy is wrong. It is that the eval set itself is a versioned, drift-tracked artifact in LLMOps in a way it never was in classical ML. Prompt management as a first-class artifact In classical MLOps, the model weights are the artifact. In LLMOps, the prompt, the system message, the few-shot exemplars, and the retrieval template are all artifacts of equal weight — and they change far more often than the model. A prompt edit that improves average quality by 3% but degrades a critical user segment by 20% is a regression, and it has to be caught the same way a bad model checkpoint would be. What this means operationally: Prompts live in version control next to code, not in a notebook or a config UI. Every prompt change triggers the task-grounded eval suite. Production calls log the prompt version, the model version, and the retrieval snapshot version. All three are needed to reproduce a bad answer. This is one of the divergent stages. There is no native equivalent in an MLOps platform. Either extend the registry to treat prompts as artifacts, or buy a thin layer that does — but do not duplicate the rest of the platform to get it. What is RAG, and why does retrieval freshness matter? Retrieval-Augmented Generation (RAG) lets a model answer using documents it was not trained on, by retrieving relevant chunks at query time and putting them in the context window. It is widely used because foundation models alone cannot know your private data, and continuous fine-tuning is not a sensible way to keep them current. The operational consequence is that the retrieval index becomes part of the model from the user’s perspective. If the index is stale, the answer is wrong, and no amount of model evaluation will catch it. Retrieval freshness — how recently the index has been re-embedded against the source corpus — is a production SLA in RAG systems. Classical MLOps does not have an equivalent because classical models do not retrieve. Our walkthrough of retrieval-augmented generation covers the three-phase pattern (retrieve, augment, generate) in more detail. From an LLMOps perspective, the thing to instrument is the lag between source-document update and index re-embedding, plus the share of queries whose top-k retrieval returns chunks older than a freshness threshold. Cost controls that actually constrain spend Cost-per-query is a real SLA, not a finance afterthought. The controls that genuinely constrain LLM spend in 2026, in our experience across engagements: Model routing: route easy queries to a smaller, cheaper model and only escalate to the frontier model when a confidence check fails. This is observed-pattern, not a benchmark — the cost reduction depends entirely on the query mix, but the mechanism is reliable. Context-window discipline: every token in the context costs money. Aggressive chunking, summary caching, and conversation pruning matter more than people expect. Caching: semantic caching of frequent queries returns a non-trivial share of traffic without an LLM call at all. Per-tenant budgets with hard caps: not a soft alert. A hard cap that returns a degraded response when a tenant exceeds budget. Soft alerts get ignored. What does not work in practice, despite being widely discussed: aggressive quantisation at the application layer (it is a foundation-model concern, not yours), and “switching to open-weights to save money” without measuring serving cost end-to-end. Self-hosting a 70B model at low utilisation is more expensive than a hosted API at scale. We have seen this calculation surprise platform teams more than once. When a separate LLMOps platform is worth the spend The decision rule we use with clients: If the LLM workload is a small fraction of your platform spend and you have one or two production use-cases, extend the existing MLOps stack. Add a prompt registry, add eval-drift monitoring, add cost-per-query metrics. Do not buy a second platform. If LLM workloads dominate your platform spend, or you run >5 production use-cases with distinct prompts and retrieval indices, the divergent stages become enough work to justify a dedicated layer. Even then, the layer should integrate with — not replace — the existing artifact and observability stack. Buying “LLMOps” wholesale as a replacement for MLOps duplicates everything that did not need to change. Pretending LLMs are just another model misses the four divergent stages above. Both failure modes are common. Both are avoidable with a divergence map that names which stages you own, which are vendor-managed, and which you instrument first. FAQ Where does the LLM lifecycle genuinely diverge from the classical ML lifecycle, and where does it reuse the same primitives? The divergent stages are eval-set drift management, prompt-as-artifact versioning, retrieval freshness for RAG systems, and cost-per-token monitoring. Training-job CI, model registry, artifact storage, and inference observability are reused from the existing MLOps stack with minor extensions. What does an LLMOps stack look like that does not duplicate the underlying MLOps stack? It extends the existing artifact registry to treat prompts and retrieval-index versions as first-class artifacts, adds eval-drift and cost-per-query dashboards alongside existing latency dashboards, and reuses the same CI, secrets, and observability primitives. The new surface is narrow and additive. How is eval-set drift detected and acted on for production LLMs? By logging a sample of production inputs, embedding them, and tracking distributional distance from the held-out eval set. When drift crosses a threshold, the eval set is augmented with new representative examples and re-graded. The action is a rolling refresh of the eval set, not a one-time build. Which cost controls actually constrain LLM spend in 2026 vs which are theoretical? Effective: model routing to cheaper models for easy queries, context-window discipline, semantic caching of frequent queries, and per-tenant hard budget caps. Mostly theoretical at the application layer: aggressive quantisation, and switching to self-hosted open-weights without measuring end-to-end serving cost. How is prompt management treated as a first-class artifact in LLMOps? Prompts, system messages, few-shot exemplars, and retrieval templates live in version control, are tagged with semantic versions, and trigger the task-grounded eval suite on every change. Production logs record the prompt version, model version, and retrieval-index version so any bad answer can be reproduced. When is a separate LLMOps platform worth the spend vs extending the existing MLOps platform? When LLM workloads dominate platform spend, or when more than roughly five production use-cases with distinct prompts and retrieval indices are in flight. Below that, extending the existing MLOps stack with prompt-registry, eval-drift, and cost-per-query layers is materially cheaper than running two platforms. What we offer We work with data and ML platform leaders who are weighing an LLMOps investment against extending what they already have. The output of an engagement is a divergence map — which lifecycle stages you own, which are vendor-managed, and which you instrument first — alongside an R&D engagement plan that names the stages and the metrics that constrain them. Get in touch if your team is about to sign for a second platform and wants the divergence map first.
