A radiology assist tool that hits 96% sensitivity in a research notebook is not a medical device. It becomes one only when the same model is locked, validated against a defined intended-use population, integrated into PACS, monitored post-market, and re-cleared every time its weights change. That gap — between a working deep-learning model and a cleared medical device — is what this article is about. Deep learning sits at the core of nearly every FDA-cleared computer-vision (CV) tool in healthcare today, from chest X-ray triage to diabetic retinopathy screening to dermatology lesion classification. The architectures are familiar: convolutional neural networks (CNNs) for classification, U-Net variants for segmentation, detection heads for lesion localisation. What is less familiar — and what determines whether a model ever reaches patients — is the regulatory and operational layer wrapped around those architectures. For the regulatory walkthrough on how this maps onto FDA Software as a Medical Device (SaMD) categories, see our companion piece on AI-enabled medical devices and the computer-vision layer behind FDA-cleared tools. Here we focus on the deep-learning mechanics: what the models actually do, where they break in clinical settings, and which engineering decisions make or break a submission. How deep learning fits into the medical-imaging pipeline A medical-CV pipeline is rarely one model. It is a chain. A typical chest-radiograph triage tool, for example, runs a quality-control classifier first (is this actually a frontal chest X-ray? Is it diagnostic quality?), then a segmentation network to isolate lung fields, then a detection or classification head for the target pathology (pneumothorax, nodule, consolidation), and finally a confidence-thresholding stage that decides whether to push a worklist priority flag back to the radiologist’s PACS queue. Each stage is a separate model, with its own training data, validation set, and failure modes. The CNN layered structure most readers know — early layers detecting edges, deeper layers identifying tissue textures and anatomical structures — is doing the heavy lifting in the middle stages. Architectures like ResNet, EfficientNet, and DenseNet dominate the classification stages; U-Net and its variants (Attention U-Net, nnU-Net) dominate segmentation; Faster R-CNN and YOLO variants appear in detection. None of this is exotic. The deep-learning toolkit for medical CV is largely the same toolkit used for industrial inspection or retail object recognition. What differs is the constraint envelope around it. What is FDA SaMD and why does it shape the architecture? The FDA’s Software as a Medical Device framework treats a trained model as a regulated medical device. That has three concrete consequences for how we build the pipeline. First, the model is locked at submission. A “locked” CV model means the weights, the preprocessing, the inference code, and the post-processing are frozen. You cannot ship a model that continues to learn from production data without filing a new submission (or operating under a Predetermined Change Control Plan, which is itself a regulated artefact). This kills the “self-learning models continue improving” pattern that works fine in consumer CV. In medical CV, drift is something you monitor and report, not something you let the model paper over. Second, validation is population-anchored. The model must demonstrate performance on a population that matches the intended use — defined by scanner manufacturer mix, patient demographics, disease prevalence, and acquisition protocols. A model trained on Siemens CT scans from one academic hospital will not clear for use across GE and Canon scanners in community hospitals without explicit multi-site validation. As an observed pattern across our engagements, the multi-site validation cohort is usually the longest pole in the timeline — not the model development itself. Third, post-market surveillance is mandatory. Once cleared, the device must be monitored in deployment. Population shift (different scanner models, different patient mix), acquisition drift (a hospital changes its imaging protocol), or annotation drift (radiologists’ labelling conventions evolve) all degrade performance silently. The pipeline must surface those degradations to the manufacturer. These three constraints reshape what deep learning means in this domain. Transfer learning is still used — pre-training on ImageNet or on large unlabelled medical datasets (RadImageNet, for example) is standard — but every fine-tuning run is logged, every dataset version is hashed, and every evaluation result is traceable to a specific git commit and a specific data manifest. This is closer to aviation software discipline than to typical machine-learning practice. FDA-cleared CV patterns: where deep learning has already shipped Device category Typical CV task Representative architecture Integration target Radiology triage (CADt) Classification + worklist priority CNN classifier (ResNet/EfficientNet) PACS worklist Computer-aided detection (CADe) Detection of lesions/nodules Detection heads on FPN backbone PACS overlay Computer-aided diagnosis (CADx) Classification + confidence Ensemble CNNs Reporting workflow Ophthalmology screening Classification (retinal images) CNN classifier with attention Standalone device Cardiac echo Segmentation + measurement U-Net + landmark regression Ultrasound cart Dermatology triage Classification CNN classifier Mobile / dermoscope The FDA’s public database of AI/ML-enabled medical devices lists several hundred cleared products as of 2024, and radiology accounts for the dominant share — most of the patterns above appear repeatedly across vendors. What distinguishes a cleared device from a research prototype is rarely the model architecture. It is the validation evidence, the integration pattern, and the operational envelope. Where deep-learning pipelines need to handle generalisation, drift, and population shift This is where most medical-CV programmes underestimate the work. Generalisation across scanners and protocols. A segmentation network trained on 1.5T MRI from one vendor will not perform identically on 3T scans from another. The pipeline needs explicit scanner-aware preprocessing (intensity normalisation, resampling to a canonical voxel spacing) and validation evidence broken down by scanner subgroup. The submission package usually carries per-scanner performance tables, not just an aggregate AUC. Population shift. Disease prevalence and demographic mix change between training cohorts and deployment sites. A diabetic-retinopathy classifier trained on a screening population with 8% disease prevalence will see worse calibration in a referral clinic where prevalence is 30%. The fix is not retraining (that triggers re-clearance) but recalibration of operating thresholds and explicit reporting that prevalence may differ from the validation cohort. Annotation drift. Ground-truth labels for medical images are produced by clinicians, and clinical conventions evolve. A tumour-segmentation model trained on annotations from 2019 may diverge from current clinical standards. Pipelines build in periodic re-evaluation against fresh-labelled holdout sets to catch this before it shows up as missed diagnoses. These are not academic concerns. They are the categories the FDA explicitly asks about in pre-submission meetings. Integration: connecting inference to PACS, EHR, and clinical workflow A model that produces a heatmap is useless if no clinician sees it. Medical-CV inference has to integrate with the systems clinicians already use — primarily PACS (Picture Archiving and Communication System) for image review and EHR for reporting. The dominant integration patterns are DICOM-based: the inference engine consumes DICOM studies from a PACS query, runs the model, and writes results back as DICOM Secondary Capture (the heatmap as an overlay image), DICOM Structured Report (the structured findings), or HL7 messages to the EHR. The model output is treated as another series in the study, sitting alongside the original scans. The operational implication: latency targets matter. A CADt model that flags a critical finding has to deliver the flag fast enough to reorder the worklist before the radiologist reaches the next case. That usually means inference times under 30 seconds end-to-end, which in turn shapes the deployment architecture — typically containerised inference on an on-premise GPU server with DICOM listeners, rather than cloud round-trips that introduce variable latency and raise data-residency issues. For the deeper view of how computer-vision systems plug into clinical infrastructure across imaging modalities, see the impact of computer vision on the medical field and our companion piece on computer vision in biomedical applications. What changes between a research model and a cleared device The honest summary: most of the engineering effort goes into things that have nothing to do with model accuracy. Reproducibility infrastructure. Locked dependency versions, hashed datasets, deterministic training (where possible), and full lineage from raw DICOM to final model weights. Per-subgroup performance reporting. Performance broken down by scanner, demographics, disease severity, and acquisition site. Failure-mode analysis. Adversarial cases, low-quality scans, out-of-distribution detection. Models need an “I don’t know” output, not just a confident wrong answer. Cybersecurity and software-of-unknown-provenance documentation. Required for the submission, regardless of clinical performance. Post-market monitoring hooks. Telemetry that flags drift without exfiltrating patient data. Programmes that design for this evidence base from day one — rather than retrofitting it at submission time — reach cleared-device status materially faster. In our experience across medical-CV engagements, programmes that delay the regulatory infrastructure decision tend to redo six to twelve months of work when they finally face the submission. This is an observed pattern from our project work, not a benchmarked rate. FAQ How many AI-enabled medical devices has the FDA cleared, and which CV patterns recur across them? The FDA’s public list of AI/ML-enabled medical devices contains several hundred cleared products, with radiology making up the largest share. Recurring CV patterns are CADt (triage / worklist priority), CADe (detection), CADx (classification with confidence), and segmentation-plus-measurement workflows in cardiology and oncology. What are the production patterns behind FDA-cleared CV diagnostics (CADe, CADx, radiomics)? CADe pipelines pair a detection head with a downstream classifier and write findings back to PACS as overlays. CADx adds a calibrated confidence score for the reporting workflow. Radiomics pipelines extract quantitative imaging features (texture, shape, intensity) for downstream statistical models. All three are built as locked, version-controlled pipelines with per-subgroup validation evidence. How does deep learning in medical CV (classification, segmentation, detection) translate into regulatory artefacts? Each model becomes a locked artefact with a frozen weight file, a hashed dataset manifest, an evaluation report broken down by subgroup, and a documented intended-use statement. Training code, preprocessing, and inference code are versioned together. Any change to weights triggers re-clearance or an approved Predetermined Change Control Plan. Where do AI medical-device pipelines need to handle generalisability, drift, and population shift? At three layers: scanner / protocol generalisation (per-vendor validation and intensity normalisation), deployment population shift (recalibration of operating thresholds), and annotation drift over time (periodic re-evaluation against fresh-labelled holdouts). What integration patterns connect CV inference to PACS, EHR, and clinical workflow? DICOM-based: studies arrive at the inference engine via PACS push or query, results return as DICOM Secondary Capture overlays, DICOM Structured Reports, or HL7 messages to the EHR. Deployment is typically on-premise containerised inference on a GPU server to meet latency and data-residency constraints. Which AI-enabled medical-device companies and products define the current state of practice in 2026? The cleared-device landscape spans incumbent imaging vendors (GE, Siemens Healthineers, Philips, Canon Medical) who ship CV features inside their scanner consoles, and independent software vendors (Aidoc, Viz.ai, Annalise.ai, Lunit, RadAI, and others) shipping cross-vendor PACS-integrated tools. The list changes monthly; the FDA’s public database is the authoritative current view. For a deeper architectural walkthrough on this engineering thread, see AI-Enabled Medical Devices: The Computer Vision Layer Behind FDA-Cleared Tools. For broader programme context across our engagements, explore our Computer Vision R&D practice. How TechnoLynx can help We work with medical-device teams on the deep-learning layer behind regulated CV products: pipeline architecture, validation strategy, multi-site evidence design, and PACS / EHR integration. Our engagements are scoped to your problem rather than fixed-scope retainers, and we focus on the engineering decisions that materially shorten the path from prototype to cleared device. Contact us to discuss your medical-CV programme. Image credits: Freepik.
