The EU AI Act does not replace GxP — it overlays a second risk axis on top of it. For pharmaceutical manufacturers, medical device makers, and clinical trial sponsors, that overlay is where most of the operational confusion now sits. A model that supports final visual inspection on a sterile fill line is simultaneously a GxP-validated computerised system, a high-risk AI system under the Act, and — if it sits inside or alongside a device — a regulated component under EU MDR. Three regimes, one system, one quality record. The teams that handle this well treat the Act as an extension of the existing quality system rather than a parallel programme. This article maps the Act onto daily pharma work and shows where the GxP boundary genuinely changes, where it stays the same, and where teams over-scope their compliance effort by treating every AI feature as if it were a release gate. What does the EU AI Act actually require from a GxP-regulated manufacturer? The Act groups AI systems by risk. Minimal-risk systems — office tooling, internal search, low-stakes assistants — carry almost no new obligations. High-risk AI systems carry the bulk of the duties: documented risk management, data governance, technical documentation, logging, transparency to users, human oversight, accuracy and robustness testing, and a registered quality management system on the provider side (AI Act, 2024; EPRS, 2025). A separate track governs general-purpose AI (GPAI) models, with codes of practice, model documentation, and reporting on capabilities and limits. For a pharma plant, three categories cover almost every realistic use case. Vision systems that gate batch release, PAT models that trigger alerts in a biologics step, and modules that steer clinical trials operations sit in the high-risk tier. Off-line reporting dashboards and lab assistants that surface reading lists typically do not. The Act explicitly allows proportionate effort, and over-scoping compliance onto auxiliary systems is its own quality risk: it spreads validation evidence too thin across systems that don’t need it (EPRS, 2025; NIST, 2023). The Act entered into force in 2024 and phases obligations in over the following years, with GPAI duties and prohibited-practice rules arriving ahead of full high-risk obligations. National authorities supervise enforcement, can request records, and can intervene when risks escalate. Which AI systems in a pharma workflow count as high-risk? A system falls into the high-risk tier when its output materially influences patient safety, product quality, or a regulated decision. In practice that means: Vision systems that support final visual inspection of sterile fills, where a missed defect reaches the patient. Process-analytical-technology (PAT) models that send alerts during a biologics step and can hold or release intermediate material. Modules that support device performance checks for medical devices, including software-as-a-medical-device (SaMD). AI that steers clinical-trials operations: site risk scoring, protocol-deviation detection, eligibility screening. AI watching critical infrastructure inside the plant — utilities, HVAC, water-for-injection systems — where a missed signal can affect product quality across many batches. These systems must satisfy the Act’s high-risk obligations and the plant’s existing GMP and the sponsor’s GCP rules. That means a documented risk assessment, controlled training and validation data, tested performance against fixed test sets, human oversight built into the workflow, an audit trail tied to lots and units, clear user-facing information about limits, and a registered quality system on the provider side (AI Act, 2024; ISPE, 2025). Edge cases sit lower. A dashboard that summarises last week’s release data, or a lab assistant that suggests literature, does not gate a regulated decision. Treat such systems with care, but do not drown them in the same evidence pack as a release-grade vision model. How do GPAI duties affect downstream pharma users? Many teams build on top of general-purpose AI models — large language models, vision-language foundation models, code-generation tools. The Act sets duties for the providers of those base models: codes of practice, training-data summaries, evaluations against systemic risks, and reporting on tested capabilities and limits. Downstream users — anyone fine-tuning, embedding, or wrapping a GPAI base inside a regulated solution — inherit the obligation to show the final use is safe and compliant (AI Act, 2024; EPRS, 2025). The practical policy is straightforward. Treat the GPAI base like any other supplied component. Request a model card. Record the declared training-data ranges, excluded content, and known failure modes. Hold the model in a software bill of materials (SBOM) alongside the rest of the stack. Keep a copy of the license terms and the codes of practice the provider commits to. When the base model is updated, treat it as a change-control event — not a silent dependency bump. How does the Act integrate with an existing GxP quality system? Teams do not need a second quality system. The duties under the Act fit inside CAPA, change control, supplier qualification, and training flows that already exist for GxP. A short add-on SOP — three to five pages — typically covers what is genuinely new: how risk assessment treats model misuse and drift, what a “model passport” contains, how human-oversight steps are recorded, and how GPAI components are qualified. Key structural moves: Risk assessment as a living process. Score impact to patient safety, product quality, and data integrity. Add explicit scores for model misuse and drift. Tie each risk to a control and a test. Human oversight where the risk is high. Add a documented review step. Record the reason when staff accept or override model output. The Act treats human oversight as a design property, not a procedural overlay. A control plane. Version data, code, and thresholds together. Record every alert with timestamp, unit, lot, model ID, and configuration. Without a control plane, reconstructing what the model saw at the time of a deviation is impossible. Drift and re-training plan. Add drift checks, a clear route to re-training, and a freeze of training data so a later check is possible. The Act’s “accuracy and robustness over the lifecycle” language requires this; so does ISPE’s GAMP AI guidance (ISPE, 2025; EMA, 2023). EU AI Act and GxP: where they overlap and where they don’t Topic GxP expectation EU AI Act addition Risk management URS-driven, system-level, ICH Q9 Lifecycle risk for the AI model itself, including misuse and drift Data governance Data integrity (ALCOA+) Documented data quality, representativeness, bias checks on training data Validation IQ/OQ/PQ against requirements Accuracy, robustness, and cybersecurity testing against declared metrics Change control Versioned, approved, traced Same — plus re-training as an explicit change category Human oversight SOP-defined review steps Design-level oversight; user can intervene and override Records Audit trail, electronic signatures Automatic event logging over the system’s lifetime Post-market Periodic review, complaints Continuous post-market monitoring, incident reporting to authorities The columns overlap heavily. Most of the AI Act’s high-risk obligations have a GxP equivalent. The genuine additions are model-specific: training-data documentation, drift monitoring as a first-class concern, and explicit post-market reporting for serious incidents. What evidence will inspectors and national authorities ask for? Auditors will ask for the same artefacts under both regimes, with the AI-specific items layered on top: Data sheets that define sources, units, ranges, and owners. Provenance for training, validation, and live inputs. Test results tied to requirements, with declared accuracy and robustness metrics. Change control records with reason, approver, and impact on validated state. A risk assessment that maps risks to tests and outcomes — not a risk list with no controls attached. User guidance that shows warnings, intended use, and limits. Keep raw data immutable. Maintain links between raw inputs, engineered features, and outputs. Use timestamps and signed builds. A one-page “model passport” per release — version, training data snapshot, validation metrics, intended use, limits — collapses most inspector questions into a single document (NIST, 2023; FDA, 2023a). Roles, training, and the right to pause People keep systems safe. The Act expects clear human oversight; GxP expects clear ownership. Assign a system owner for each model, a QA partner, and a data steward. Write a one-page role card for each. Train operators in short sessions using live examples and short drills. Critically, give operators an explicit right to pause a model when its output feels wrong. Record the pause, the reason, and the downstream action. Review the case at the next quality meeting. A pause-and-review log is one of the cleanest pieces of evidence that human oversight is real rather than ceremonial (ISPE, 2025; EMA, 2023). Security, privacy, and the prohibited-practice boundary AI runs on data. Segment networks. Use signed artefacts. Protect keys and secrets. Watch endpoints. Test backup and restore. Keep clocks in sync. Limit access on a need-to-know basis. These steps reduce risk to patients and products and align with the Act’s emphasis on cybersecurity for high-risk systems (NIST, 2023; FDA, 2023b). The Act’s prohibitions on social scoring and most real-time facial recognition in public spaces feel remote from a manufacturing plant — until a plant deploys gate cameras, line-side identification, or operator-attention monitoring. Use privacy-first designs. Redact faces in areas the Act treats as sensitive. Store only events, not continuous video. Keep only what the SOP requires. Clinical trials, medical devices, and SaMD AI in clinical trials supports screening, site selection, protocol-deviation detection, and data checks. The principles are the same: explainable outputs, human investigators and monitors retaining the final decision, a documented line of sight from each signal to each action, privacy by design, and informed consent kept clear. The EMA’s reflection paper on AI in the medicines lifecycle and the Act’s high-risk provisions point in the same direction (EMA, 2023; EPRS, 2025). For medical devices and SaMD, EU MDR and the Act both apply. Manufacturers must show safety and performance, maintain a post-market plan, watch for drift and bias, and notify national authorities when risks increase. The same model passport and drift logs that satisfy the Act also feed the device’s post-market surveillance file — one set of records, two regulatory readers. Suppliers and the global supply chain Models pull in third-party code, weights, and data. Build supplier rules into the qualification process: Request model cards and data sheets. Request test results and declared limits. Require cybersecurity basics — vulnerability disclosure, signed releases, supported versions. For GPAI components, require evidence of the provider’s codes of practice. Reserve a right to audit. Define a route for incident reports. Hold the SBOM and the change log per supplier component. Tie risks in the supply chain to the plant’s CAPA system and to the Act’s duties. A supplier component drifting in capability — a fine-tuned vision model retrained by its vendor without disclosure — is a change-control event whether or not the vendor flags it. A practical path to day-one compliance Pick one use case. Run it in shadow mode against the current process. Write the URS on a single page. List three acceptance criteria — not thirty. Set a short action plan for each alert. Run a month with humans fully in the loop. Tune thresholds and operator-facing messages weekly. When results meet the bar, lock the build, publish the records, and move to live operation. Keep the weekly review during the first quarter and extend the same steps to the next use case. Patterns we see go wrong: vague use cases that no metric can settle; UIs that bury warnings in low-contrast text; “set and forget” deployments with no drift plan; risk assessments that enumerate risks but tie nothing to a test; and giant multi-system rollouts launched without a single pilot in production conditions. FAQ How TechnoLynx supports GxP-ready AI under the EU AI Act We build validation-ready AI for pharma and medical devices that fits existing good manufacturing practices and the Act’s high-risk obligations. Our engagements typically cover URS and risk assessment in plain language, explainable system design with a recorded human-review step, accuracy and robustness testing against a locked test set, a control plane that versions data, code, and thresholds together, and a model passport per release. We handle GPAI components through supplier qualification, codes-of-practice mapping, and SBOM discipline. We respect regulatory requirements, keep operators genuinely in charge, and build for the long term rather than a demo. For the deeper validation strategy that sits underneath this — how to classify AI/ML software under GAMP 5 and where CSA replaces full CSV — see validation-ready AI for GxP operations in pharma. For the broader regulatory landscape, pharma regulatory compliance with AI navigation frames how the Act, GxP, and device rules fit together. The AI in Life Sciences landing page maps these threads onto our service offering. References AI Act (2024) Implementation timeline for the EU Artificial Intelligence Act. Available at: https://artificialintelligenceact.eu/implementation-timeline/ (Accessed: 19 September 2025). EMA (2023) Reflection paper on the use of artificial intelligence in the lifecycle of medicines. Available at: https://www.ema.europa.eu/en/news/reflection-paper-use-artificial-intelligence-lifecycle-medicines (Accessed: 19 September 2025). EPRS (2025) The timeline of implementation of the AI Act. European Parliamentary Research Service. Available at: https://www.europarl.europa.eu/RegData/etudes/ATAG/2025/772906/EPRS_ATA%282025%29772906_EN.pdf (Accessed: 19 September 2025). European Commission (2022) EU GMP Annex 1: Manufacture of sterile medicinal products. Available at: https://health.ec.europa.eu/latest-updates/revision-manufacture-sterile-medicinal-products-2022-08-25_en (Accessed: 19 September 2025). FDA (2023a) Using Artificial Intelligence & Machine Learning in the Development of Drug and Biological Products. Available at: https://www.fda.gov/media/167973/download (Accessed: 19 September 2025). FDA (2023b) Artificial Intelligence in Drug Manufacturing — PQRI workshop presentation. Available at: https://pqri.org/wp-content/uploads/2023/09/4-FDA-PQRI-AI-Workshop_Tom-OConnor_Final-1.pdf (Accessed: 19 September 2025). ISPE (2025) GAMP® Guide: Artificial Intelligence. International Society for Pharmaceutical Engineering. Available at: https://ispe.org/publications/guidance-documents/gamp-guide-artificial-intelligence (Accessed: 19 September 2025). NIST (2023) Artificial Intelligence Risk Management Framework (AI RMF 1.0). National Institute of Standards and Technology. Available at: https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf (Accessed: 19 September 2025). Image credits: DC Studio, via Freepik.
