Validation is evidence, not testing
Validation under GxP regulations requires documented evidence that systems reliably meet predefined specifications and quality standards across repeated operations. For software in pharmaceutical environments, this means proving — with traceable evidence — that the system does what it claims to do, does not do what it should not do, and maintains data integrity throughout its operational lifecycle.
The distinction between validation and testing matters. Testing verifies that specific functions work under specific conditions. Validation demonstrates that the entire system is fit for its intended use in the production environment, with the users who will operate it, processing the data types it will encounter. A system can pass every unit test and still fail validation if its operational context was never assessed.
The traditional validation lifecycle
Traditional GxP validation follows a V-model with three qualification stages:
| Stage | Full name | Purpose | Evidence |
|---|---|---|---|
| IQ | Installation Qualification | System installed correctly per specifications | Hardware/software inventory, version verification, environment checks |
| OQ | Operational Qualification | System operates correctly under expected conditions | Functional tests, boundary tests, error handling, security verification |
| PQ | Performance Qualification | System performs reliably in production context | End-to-end workflows, user acceptance, stress testing, data integrity checks |
Each stage produces documentation — protocols, execution records, deviation reports, summary reports — that forms the validation evidence package. This package must be available for regulatory inspection at any time during the system’s operational life.
For deterministic software (ERP systems, LIMS, MES), this lifecycle works well. The system is installed once, validated once, and re-validated only when changes occur. The V-model assumes that a system validated at deployment remains valid until modified — an assumption that quietly breaks the moment the system under test learns from data.
Why AI systems break the traditional model
Determinism is absent from machine learning architectures. A computer vision model trained to detect particulate contamination in vials will produce different outputs as its model weights change, as new training data is incorporated, and as the input distribution shifts — different lighting conditions, new product formats, camera degradation. The fundamental assumption of traditional validation, validate once and monitor for changes, does not hold here. The system is changing continuously, often within the bounds the operator believes are stable.
The FDA’s 2022 Computer Software Assurance (CSA) guidance and the GAMP 5 Second Edition both acknowledge this gap. CSA replaces the documentation-first mindset with a risk-based approach: systems that directly affect product quality require thorough assurance activities, while systems with lower risk require proportionate effort. For AI/ML systems, this translates to continuous validation — ongoing monitoring of model performance against predetermined acceptance criteria, with triggered revalidation when drift is detected. This is an observed pattern across our pharma engagements: continuous validation is the operationally relevant frame for ML software in regulated environments, not a stricter version of one-shot CSV.
A practical continuous validation framework for AI in pharma includes three components: performance monitoring (accuracy, precision, recall tracked against baselines using a held-out reference set), drift detection (statistical comparison of incoming data distributions against training data, typically via PSI or KS-style tests), and triggered requalification (formal reassessment when performance drops below predefined acceptance thresholds). The supporting technology stack is usually unremarkable — MLflow or similar for model registry, an inference service exporting metrics to Prometheus, an alerting layer that flags drift events for the QA owner — but the regulatory framing around those components is what turns it into validation evidence rather than ops telemetry.
Understanding when to apply full CSV versus the lighter CSA approach is a risk-based decision that depends on the system’s GAMP 5 classification, not a blanket policy choice. The same risk framing also governs how the GAMP 5 V-model adapts to AI/ML systems — the qualification stages stay, but their content changes.
How does continuous validation differ from one-shot validation?
Installation, Operational, and Performance Qualification protocols remain foundational even when continuous validation strategies augment them. The qualification stages still produce the initial validation evidence package at deployment. What changes is the operational phase: instead of treating the validated state as static, the operator monitors a defined set of performance and data-distribution signals against acceptance criteria established during PQ. When a signal crosses its threshold, a controlled revalidation procedure is triggered, scoped to the change.
| Dimension | One-shot (traditional CSV) | Continuous validation (CSA-aligned) |
|---|---|---|
| Validation event | Deployment + change events | Deployment + ongoing monitoring + triggered events |
| Evidence horizon | Frozen at release | Living evidence stream + release snapshots |
| Drift handling | Out of scope until next change request | First-class; defined detection + response |
| Scope of revalidation | Full system after each change | Risk-scoped to the changed component or signal |
| Documentation cadence | Periodic (review cycles) | Event-driven + periodic |
The implication is that the boundary between verification and validation — already subtle for deterministic software — becomes a live operational concern for ML systems. Verification asks whether the model still meets its specification on a frozen test set; validation asks whether it still performs as intended on the data it is actually seeing.
The cost of validation done wrong
Excessive validation burns engineering capacity while delivering no measurable compliance benefit. Under-validation creates regulatory exposure that surfaces during inspections. The risk-based approach is not optional — it is the current regulatory expectation from both FDA and EMA. Organisations that still apply uniform full CSV to every system, regardless of risk, are spending validation budget on low-risk systems while potentially under-resourcing validation of the high-risk AI systems where the compliance exposure actually sits.
The pattern we see most often is mismatched effort: a stable, deterministic LIMS receives quarterly periodic review with full regression packs, while a vision model running release-relevant inspection has no defined drift threshold, no requalification trigger, and no documented monitoring owner. The first system cannot fail in a way the validation effort would catch. The second can — and the validation evidence will not exist when the inspector asks.
What does validation look like for modern cloud-based pharma systems?
Pharmaceutical deployment to cloud platforms introduces an infrastructure validation layer atop traditional software qualification activities. The cloud provider’s infrastructure (compute, storage, networking) must be qualified as suitable for GxP use. Major cloud providers (AWS, Azure, GCP) publish GxP qualification packages that document their infrastructure controls, but the pharmaceutical company remains responsible for validating the application layer running on top.
The shared responsibility model in cloud environments maps to validation as follows: the cloud provider qualifies infrastructure (physical security, hardware redundancy, network availability), the software vendor validates the application (functional testing, security testing, data integrity), and the pharmaceutical company validates the configuration and business processes (user acceptance testing, SOP alignment, training, role assignments).
Data residency and sovereignty requirements add complexity. Pharmaceutical data may be subject to regulations that restrict where it can be stored and processed. Validating a cloud-based system requires documenting which regions store data, how data flows between regions, and what controls prevent data from being processed in non-compliant jurisdictions. For containerised ML workloads — typically deployed via Kubernetes with images pulled from a controlled registry — this also means validating the deployment pipeline itself, since the pipeline is now part of the system that produces the validated state.
Our validation approach for cloud-based systems includes a cloud infrastructure qualification document (leveraging the provider’s GxP compliance packages), a shared responsibility matrix (documenting which controls each party owns), and application validation activities performed on the cloud-hosted system rather than in a separate environment. In our experience, this approach produces validation evidence that addresses both the traditional software validation requirements and the cloud-specific concerns that regulators increasingly ask about during inspections. We carry that cloud-plus-validation approach into the rest of our life-sciences AI practice.
FAQ
Three validation strategies that auditors actually accept
Risk-based validation has become standard practice in GxP software, yet effectiveness depends entirely on traceability matrices that bind requirements to test protocols and underlying code. That answer is workload-specific, and it is worth writing down before you build.
