Introduction: make AI safe, useful, and compliant

Pharmaceutical teams now use artificial intelligence (AI) in many ai applications. Teams classify images, predict drifts, match patients to trials, and watch the supply chain.

This work needs a clear, shared legal framework. The EU Artificial Intelligence Act now provides it. The Act entered into force in 2024 and phases in duties over the next years.

Firms in the EU and firms that place systems in the EU must meet the Act’s compliance requirements. They must also keep the existing regulatory requirements under good manufacturing practices and clinical rules (European Commission, 2022; EMA, 2023).

A single, joined‑up approach works best. One set of controls. One set of records. One quality story (ISPE, 2025; NIST, 2023).

This article maps the Act to daily work in pharma, medical devices, and clinical trials. It uses plain language. It focuses on actions that produce high quality outcomes. It keeps people in charge and keeps systems explainable (EMA, 2023; FDA, 2023a).

The Act’s structure in one page

The Act groups systems by risk. The minimal risk group includes office tools and simple ai applications that pose low harm. The high group covers high risk ai systems in areas such as product safety, health, medical devices, and critical infrastructure. A separate track governs general purpose ai models (GPAI) and sets codes of practice and model reporting duties (AI Act, 2024; EPRS, 2025).

The Act bans some uses outright. It bans social scoring by public bodies. It bans certain forms of facial recognition in public spaces, with narrow exceptions. Pharma teams rarely touch those uses, yet teams must still know the lines (AI Act, 2024; EPRS, 2025).

National authorities supervise the Act. They can ask for records. They can step in if risks rise or if firms miss their duties (EPRS, 2025; AI Act, 2024).

Read more: Cell Painting: Fixing Batch Effects for Reliable HCS

What counts as “high risk” in pharma and devices

Systems that influence patient safety or batch release often sit in the high risk ai systems tier. Examples include:

• vision systems that support final visual inspection of sterile fills;

• PAT models that send alerts in a biologics step;

• tools that support device performance checks for medical devices;

• modules that steer clinical trials operations, site risk, or data checks.

These systems must meet the Act’s compliance requirements and regulatory requirements. They must also fit the plant’s good manufacturing practices and the sponsor’s GCP rules. That means a clear risk assessment, strong data control, tested performance, human oversight, and an audit trail. It also means clear information to users, a registered quality system, and codes of practice where the Act points to them (AI Act, 2024; ISPE, 2025).

Some systems at the edge may sit lower. A dashboard that runs offline reports may fall into minimal risk. A lab assistant that suggests reading lists likely sits there too.

Treat them with care, but do not drown them in the same evidence as a high‑risk release gate. The Act allows proportionate effort (EPRS, 2025; NIST, 2023).

Read more: Explainable Digital Pathology: QC that Scales

GPAI and the new codes of practice

Many teams will fine‑tune or embed general purpose ai models. The Act sets duties for such models. Providers must follow codes of practice, share summaries, and report on tests and limits.

Downstream users must apply safeguards when they build regulated solutions on top of a GPAI base. They must show the final use meets safety and quality rules (AI Act, 2024; EPRS, 2025).

A good policy is simple. Treat the GPAI base like any other component. Ask for a model card. Ask for test data ranges, excluded content, and known failure modes.

Record the checks. Keep the model in a bill of materials. Keep a copy of the license and the codes of practice you follow (NIST, 2023; ISPE, 2025).

Integrate the Act into the GxP system

Teams do not need a second quality system. They can extend the existing one. Place AI under the same CAPA, change control, and training flows. Tie the Act’s duties to the same good manufacturing practices language.

Write a short add‑on SOP that states how AI differs. Keep it brief. Use action verbs. Keep every step testable (ISPE, 2025; EMA, 2023).

Key steps:

• Define risk assessment as a living process. Score impact to patient safety, product quality, and data integrity. Score model misuse and drift. Tie each risk to a control and a test.

• Keep humans in charge. Add a human review step where the risk is high. Record the reason when staff accept or override model output.

• Create a “control plane”. Version data, code, and thresholds. Record every alert with time, unit, lot, model id, and configuration.

• Plan for the long term. Add drift checks. Add a clear route to re‑training. Keep a freeze of training data for a check later.

These steps align with regulatory requirements and the Act’s compliance requirements. They also match the EMA’s call for governance, transparency, and clear human roles (EMA, 2023; ISPE, 2025).

Read more: Validation‑Ready AI for GxP Operations in Pharma

Data, records, and evidence

Auditors and national authorities will ask for clean records. Teams should keep:

• data sheets that define sources, units, ranges, and owners;

• provenance for training, validation, and live inputs;

• test results tied to requirements;

• change control with reason and approver;

• a risk assessment that maps risks to tests and outcomes;

• user guidance that shows warnings and limits.

Keep raw data immutable. Keep links between raw, features, and outputs. Use time stamps and signed builds. Keep a short “model passport” for each release (NIST, 2023; FDA, 2023a).

Read more: Edge Imaging for Reliable Cell and Gene Therapy

People, roles, and training

People keep systems safe. Give clear roles. Assign an owner for each model.

Assign a QA partner. Assign a data steward. Write a one‑page role card for each. Train people in short sessions.

Use live examples and short drills. Avoid jargon. Write in plain words. Give teams the right to pause a model if it feels wrong.

Record the pause and the reason. Review the case in the next quality meeting (ISPE, 2025; EMA, 2023).

Security, privacy, and ethics

AI runs on data. Teams must secure that data. Segment networks. Use signed artefacts. Protect keys and secrets.

Watch endpoints. Test backup and restore. Keep clocks in sync. Limit access on a need‑to‑know basis.

These steps reduce risk to patients and products. They also support the Act’s focus on safe ai applications (NIST, 2023; FDA, 2023b).

Treat identity with care. The Act restricts facial recognition. The Act bans social scoring. These themes may feel remote to a plant.

Still, teams may build tools that see faces near lines or gates. Use privacy‑first designs.

Redact faces in critical infrastructure areas. Store only events, not continuous video. Keep only what the SOP needs (AI Act, 2024; EPRS, 2025).

Read more: AI in Genetic Variant Interpretation: From Data to Meaning

AI for clinical trials: safe, fair, and explainable

AI supports screening, site selection, and data checks in clinical trials. Teams can use patient‑friendly tools, but they must keep trials safe and fair.

Build explainable outputs. Show why a site risks delay. Show why a visit needs a check.

Keep the final decision with investigators and monitors. Keep a line of sight from each signal to each action. Keep informed consent clear.

Keep privacy by design. These points match the EMA’s guidance and the Act’s goals (EMA, 2023; EPRS, 2025).

Medical devices and SaMD

Some AI sits inside medical devices. Other AI runs as software on its own. EU MDR and the Act both matter.

Manufacturers must show safety and performance. They must keep a post‑market plan. They must watch for drift or bias.

They must notify national authorities when risks increase. Use the same model passport and drift logs. Use the same control plane, with device identifiers and versions (EPRS, 2025; Moore et al., 2021).

Read more: AI Visual Inspection for Sterile Injectables

Critical infrastructure and the pharma plant

Plants rely on critical infrastructure. They use water, power, HVAC, and networks. AI can watch utilities for early risk. Keep those models in the high tier.

Keep human checks for shut‑down signals. Document the link between the alert and the SOP. Test these flows often.

Link the plant’s business continuity plan to the AI plan. Keep both plans in the same quality portal (AI Act, 2024; NIST, 2023).

Suppliers and the global supply chain

Many models use third‑party code and models. Many plants depend on vendors and contract sites. Build simple supplier rules:

• ask for model cards and data sheets;

• ask for test results and limits;

• ask for cybersecurity basics;

• ask for codes of practice for GPAI;

• set a right to audit;

• set a route for incident reports.

Keep a bill of materials for each system. Keep a change log for each supplier component. Tie risk in the supply chain to the plant’s CAPA and to the Act’s duties (AI Act, 2024; NIST, 2023).

Read more: Predicting Clinical Trial Risks with AI in Real Time

A practical path to day‑one compliance

Pick one use case and prove the model in shadow mode. Build the URS in a page. List three acceptance criteria.

Set a short action plan for each alert. Run a month with humans in the loop. Tune thresholds and messages weekly.

When results meet the bar, lock the build. Publish the records. Move to live. Keep the weekly review. Extend the same steps to the next use case.

Keep the stack simple. Keep the words short. Keep the evidence neat (ISPE, 2025; FDA, 2023b).

What to avoid

Avoid vague use cases. Avoid complex UI that hides warnings. Avoid “black box” designs. Avoid “set and forget” models.

Avoid giant projects without pilots. Avoid weak risk assessment that lists risks but sets no controls. Avoid claims that systems will work “for the long term” without a drift plan (EMA, 2023; NIST, 2023).

Frequently asked points from sponsors and QA

Does the Act ban AI in pharma? No. It sets guardrails. It bans social scoring and tightens facial recognition. It places core work in the high risk ai systems tier.

It adds GPAI duties. It leaves room for safe, tested systems (AI Act, 2024; EPRS, 2025).

Do we need new teams? Not always. Many firms add one AI lead in QA and one in IT. They train current staff.

They write a short SOP. They extend existing reviews (ISPE, 2025; NIST, 2023).

How do we prove high quality outputs? Use fixed test sets, blinded checks, and live KPIs. Show that humans understand alerts and act the same way each time. Show release gains without extra risk (EMA, 2023; FDA, 2023a).

How do we deal with GPAI? Treat the base model like any component. Ask for a model card. Test it on your data. Wrap it with controls.

Follow the codes of practice and record the steps (AI Act, 2024; EPRS, 2025).

Read more: Generative AI in Pharma: Compliance and Innovation

Read more: AI in Life Sciences

How TechnoLynx can help

TechnoLynx builds validation‑ready AI that fits good manufacturing practices and the Act. We design explainable systems with a human review step. We set a simple risk assessment, clear acceptance criteria, and a tested control plan.

We version data, code, and thresholds. We log alerts with lot, unit, and model id. We prepare the audit pack with URS, test scripts, results, and a model passport. We set a drift plan and a change path.

We also help teams handle general purpose ai models with codes of practice, supplier due diligence, and plain‑English user guidance.

We respect regulatory requirements. We keep staff in charge. We keep records clean. We build for the long term, not a demo.

References

• AI Act (2024) Implementation timeline for the EU Artificial Intelligence Act. Available at: https://artificialintelligenceact.eu/implementation-timeline/ (Accessed: 19 September 2025).

• EMA (2023) Reflection paper on the use of artificial intelligence in the lifecycle of medicines. Available at: https://www.ema.europa.eu/en/news/reflection-paper-use-artificial-intelligence-lifecycle-medicines (Accessed: 19 September 2025).

• EPRS (2025) The timeline of implementation of the AI Act. European Parliamentary Research Service. Available at: https://www.europarl.europa.eu/RegData/etudes/ATAG/2025/772906/EPRS_ATA%282025%29772906_EN.pdf (Accessed: 19 September 2025).

• European Commission (2022) EU GMP Annex 1: Manufacture of sterile medicinal products. Available at: https://health.ec.europa.eu/latest-updates/revision-manufacture-sterile-medicinal-products-2022-08-25_en (Accessed: 19 September 2025).

• FDA (2023a) Using Artificial Intelligence & Machine Learning in the Development of Drug and Biological Products. Available at: https://www.fda.gov/media/167973/download (Accessed: 19 September 2025).

• FDA (2023b) Artificial Intelligence in Drug Manufacturing – PQRI workshop presentation. Available at: https://pqri.org/wp-content/uploads/2023/09/4-FDA-PQRI-AI-Workshop_Tom-OConnor_Final-1.pdf (Accessed: 19 September 2025).

• ISPE (2025) GAMP® Guide: Artificial Intelligence. International Society for Pharmaceutical Engineering. Available at: https://ispe.org/publications/guidance-documents/gamp-guide-artificial-intelligence (Accessed: 19 September 2025).

• Moore, J. et al. (2021) ‘OME‑NGFF: a next‑generation file format for expanding bioimaging data‑access strategies’, Nature Methods. Available at: https://www.nature.com/articles/s41592-021-01326-w.pdf (Accessed: 19 September 2025).

• NIST (2023) Artificial Intelligence Risk Management Framework (AI RMF 1.0). National Institute of Standards and Technology. Available at: https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf (Accessed: 19 September 2025).

• Image credits: DC Studio. Available at Freepik