Case study · Health AI and regulation

Testing with the regulator in the room: an EU medtech start-up's hospital early-warning model in an AI regulatory sandbox

A European medtech start-up had built a model that warns hospital ward staff when a patient is likely to deteriorate. It worked on historical data. The questions that mattered next were harder: would it hold up on live wards, how should nurses act on it, and what would regulators expect as evidence? When the EU set common rules for AI regulatory sandboxes, the start-up decided to answer those questions in one, with a supervisor watching, without mistaking the sandbox for approval.

Case studyMedtech · European UnionSumanta Mukhopadhyay9 min read
#healthcare#ai#governance
CASE STUDY · CASE STUDY · HEALTH AI AND REGULATION15 Nov 2026 · 15:50 GST
At a glance
SectorMedtech · European Union
My roleFractional CTO and AI architect
EngagementSandbox strategy, evidence design and platform architecture
ScopeSandbox application, test plan, evaluation design, silent-mode deployment, clinician workflow, monitoring, documentation and exit planning
What changed
  • A sandbox plan with specific questions, success criteria and exit conditions
  • Evidence from live wards gathered first in silent mode, then with clinicians
  • Performance tracked by site and patient group, not only overall
  • Documentation built during the sandbox rather than reconstructed afterwards
  • A clear understanding that sandbox exit reports support, but do not replace, conformity assessment
Hospital data feeds via FHIRFeature pipeline with versioned definitionsModel registry and versioned releasesSilent-mode scoring serviceClinician alert interfaceMonitoring for drift and calibrationEvidence repositoryTechnical documentation set

Anonymised composite. Client names, identifying details and confidential figures are withheld; public facts are cited in the references.

The situation

A European medtech start-up had developed a model that estimates the risk that a patient on a general hospital ward will deteriorate in the next day, using vital signs, laboratory results and nursing observations. On historical data from two partner hospitals it performed well. The founders knew that was the easy part. Clinical AI often fails in the move from historical data to live wards: data arrives late or differently, staff respond to alerts in ways nobody predicted, and performance varies between sites and patient groups.

As software intended to support clinical decisions, the product was likely to be regulated as a medical device and treated as high-risk under the EU AI Act. The team needed evidence that would satisfy both regimes, and had no experience of what supervisors would expect.

Why now. On 11 November 2026, the Commission published Implementing Regulation (EU) 2026/2487, setting common rules for establishing and running AI regulatory sandboxes under the AI Act12. Commentary was quick to point out the limits: sandbox participation produces documentation and an exit report, not compliance clearance, and liability stays with the provider34. That suited the start-up: it wanted structured supervision and early feedback, not a shortcut.

What had to be true

  • Clear questions: the sandbox plan states exactly what will be tested and what success looks like.
  • Patient safety first: no clinician acts on the model until it has been validated on live data.
  • Evidence that travels: results usable later for the AI Act and medical device assessments.
  • Performance by group: results reported by site, ward type and patient group.
  • Reproducible: any result can be regenerated from versioned data, features and model.
  • No false comfort: everyone understands what the sandbox does and does not provide.

Architecture

Hospitals
EHR and observationsFHIR feedsLaboratory results
Data
Feature pipelineversioned definitionsData quality checkslate and missing values
Model
Model registryversioned releasesScoring servicesilent mode or live
Clinical use
Alert interfacereason and recommended reviewClinician feedback
Evidence
Monitoringdiscrimination, calibration, driftEvidence repositoryfor supervisors and assessment
The same pipeline runs in silent and live mode, so evidence from one phase carries into the next.

Hospital data arrived through FHIR feeds into a feature pipeline where every feature had a versioned definition, so a result could always be traced to the exact data and code that produced it. The scoring service ran in two modes: silent, where scores were stored but not shown, and live, where alerts reached ward staff. See healthcare data architecture.

The data flow, end to end

  1. IngestObservations and results arrive from the hospital in near real time
  2. CheckData quality rules flag late, missing or implausible values
  3. ScoreThe model estimates deterioration risk for each patient
  4. AlertIn live mode, high-risk patients prompt a structured clinical review
  5. CompareScores are compared with actual outcomes, by site and patient group
  6. ReportMonthly evidence packs go to the supervisor and clinical leads
Outcomes, not alerts, are the measure: did flagged patients actually deteriorate, and did reviews help?

The hard part: evidence a supervisor can trust

Designing the sandbox plan. The start-up's first draft asked to "test the model in hospitals". The final plan, agreed with the supervising authority, asked specific questions: does performance on live data match historical results at each site; is the model well calibrated for older patients and patients with chronic conditions; do nurses act on alerts within the expected time; and does alert volume stay manageable on busy wards. Each question had a success criterion and an agreed response if it failed.

Making results reproducible and generalisable. Supervisors and later assessors would ask whether results could be repeated and whether they would hold elsewhere. The team borrowed the vocabulary of the observational research community: a result should be repeatable on the same data, reproducible by someone else, and generalisable to different data5. Versioned data and features handled the first two; adding a third hospital partway through tested the last.

Table of desired evidence attributes: repeatable, reproducible, replicable, generalisable, robust and calibrated, showing which of question, researcher, data and analysis stay identical or change and the expected result
Attributes of reliable evidence: the sandbox plan aimed for results that were repeatable, reproducible and generalisable across hospitals. Source: The Book of OHDSI · CC0 1.0

Silent mode before live mode. For the first phase the model scored real patients but showed nothing to clinicians. That exposed data timing problems at one site that would have made live alerts arrive too late to help, and a calibration gap in one patient group that was fixed before any nurse saw a score.

A sandbox is valuable for what you learn with the supervisor watching, not for the badge. Go in with sharp questions, and come out with evidence you would have needed anyway.

How it ran

Start-up
  1. Sandbox plan
  2. Pipeline and monitoring
  3. Evidence packs
Hospitals
  1. Data access and governance
  2. Clinical review process
  3. Nurse feedback
Supervising authority
  1. Agree questions
  2. Review progress
  3. Exit report
The supervisor shaped the questions; hospitals shaped how alerts were used; the start-up produced the evidence.
Phase 1Plan
  • Specific questions
  • Success criteria
  • Exit conditions
Phase 2Silent
  • Live scoring, no alerts
  • Data timing fixes
  • Calibration by group
Phase 3Live
  • Alerts on selected wards
  • Structured reviews
  • Alert volume tracking
Phase 4Exit
  • Evidence pack
  • Exit report
  • Assessment plan
Silent mode came first and did the most to protect patients.

Key decisions and trade-offs

DecisionAlternativeWhy
Enter a sandbox with specific questionsTreat it as a general pilotClear questions make supervisor feedback useful
Silent mode before live alertsGo live on validated historyLive data behaves differently; patients stay safe
Report by site and patient groupOne overall scoreAverages hide where the model fails
Versioned features and dataAd hoc extractsResults can be reproduced for assessors
Add a third hospital mid-sandboxStay with two partnersTests whether results generalise
Plan conformity assessment in parallelRely on the exit reportThe sandbox does not grant compliance

What I'd do again, and differently

Again: write the sandbox questions with clinicians first, then take them to the supervisor; it made every later conversation concrete. Differently: plan alert workflow with nursing leads earlier; the hardest live-mode issues were about who responds and when, not about the model. Related: evaluation that predicts production, the FHIR and OMOP evidence platform case study, the 13 November tech news and the Europe page.

If you are building clinical or other high-risk AI for Europe and want evidence that stands up, see my AI architecture work, or analyse your challenge and bring the result to a call.

References
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
FAQ

Common questions

What is an EU AI regulatory sandbox?

A controlled environment set up by national authorities under the EU AI Act where companies can develop and test AI systems under regulatory supervision. Commission Implementing Regulation (EU) 2026/2487 sets common rules for how sandboxes are established and run.

Does taking part in a sandbox mean an AI system is compliant?

No. Commentary on the rules notes that participation produces documentation and an exit report, but not compliance clearance or a conformity assessment. Companies still have to meet the AI Act and, for medical devices, the medical device rules.

Why run a clinical AI model in silent mode first?

Silent mode scores real patients without showing results to clinicians, so the team can measure performance on live data and fix problems before anyone acts on the model's output.

Keep reading

Related

Open to advisory, fractional CTO & select builds

Let's talk about what you're building.

The fastest route is a free 30-minute call. Prefer to write? Send a message and I'll reply personally within two business days.

Start a conversation

How would you like to reach me?

Book a 30-min callFastest. Pick any open slot on my calendar.→ Write in GmailOpens a pre-filled draft in your browser.→ Use my email appOutlook, Apple Mail or your default client.→

I reply personally within two business days. Time zones from the Americas and Europe to the Middle East and Asia-Pacific.