- One tracked record per signal, from detection to decision, with an audit trail
- Candidate signals screened consistently on a regular schedule
- Real-world data evaluations run from reusable study packages
- Clear separation between signal detection, evaluation and decision
Anonymised composite. Client names, identifying details and confidential figures are withheld; public facts are cited in the references.
The situation
A pharmaceutical company's drug safety team handled case processing well; their safety database and regulatory reporting were mature. Signal management was not. Potential signals arrived from periodic report reviews, literature, partners and occasional regulator queries, and were tracked in a spreadsheet. Evaluating one meant a custom data request, an analyst working for weeks, and conclusions summarised in slides that did not record which data or method they came from.
The head of safety wanted three things: consistent screening, faster evaluation and a record any inspector could follow from first observation to final decision.
What had to be true
- Every signal has one record, from source to decision, with who did what and why.
- Screening is consistent: the same methods on the same schedule, not ad hoc queries.
- Detection is not confused with evidence. Report counts start investigations; they don't conclude them.
- Evaluation is fast and reproducible, using standardised real-world data and versioned study code.
- Validated where it matters, with the computerised system controls a regulated safety function expects.
The EU's good pharmacovigilance practice guidance on signal management frames the lifecycle the platform needed to support, and EMA presentations on it emphasise that marketing authorisation holders should keep an audit trail of their signal management activities1.
Architecture
The data flow, end to end
- IngestCase extracts, public reports and literature flags loaded on schedule
- ScreenDisproportionality and change detection for product and event pairs
- ValidateSafety physician reviews cases behind each flag and opens or closes a signal
- PrioritiseSeriousness, novelty and public health impact recorded with reasons
- EvaluateStudy package run on OMOP data with diagnostics
- DecideAssessment and action recorded, linked to the evidence used
The hard part: not over-reading spontaneous reports
Spontaneous reports are the backbone of signal detection and also easy to misuse. The FDA states plainly that a report does not mean the product caused the event, that report counts cannot establish how often an event occurs, and that reports can be duplicated, incomplete or stimulated by publicity2. The platform's design reflected that:
- Disproportionality scores were labelled as screening statistics, never as risk estimates, everywhere they appeared.
- Every flag required case-level review by a safety physician before becoming a validated signal.
- Evaluation used designed studies on population data, not more report counting.
For evaluation, self-controlled designs suited many acute safety questions because they compare time periods within the same people, removing confounding by stable characteristics3. Negative control outcomes and empirical calibration showed whether residual bias could explain a result4. The designs are compared in real-world evidence study designs.
A signal platform's most important feature is the line it draws between "worth investigating" and "evidence of harm". Blur that line and you get both false alarms and false reassurance.
How it ran
- Define screening scope
- Validate and assess signals
- Record decisions
- Design study templates
- Run evaluations on OMOP
- Interpret diagnostics
- Ingest and standardise
- Scheduled screening
- Validation and audit
- One record per signal
- Workflow and audit
- Migrate open signals
- Scheduled disproportionality
- Change detection
- Review queues
- OMOP data partners
- Study templates
- Negative controls
- System validation
- SOP updates
- Inspection readiness
Key decisions and trade-offs
| Decision | Alternative | Why |
|---|---|---|
| Tracker before analytics | Start with detection algorithms | Audit trail and consistency were the biggest gaps |
| Physician validation of every flag | Auto-open signals above a threshold | Screening statistics are not evidence of harm |
| OMOP for evaluation data | Bespoke extracts per question | Reusable study packages across data partners |
| Negative controls by default | Report estimates alone | Shows when bias could explain a result |
| Separate detection and evaluation layers | One combined risk score | Keeps the reasoning legible to inspectors |
What I'd do again, and differently
Again: fix the record-keeping first, then add analytics. Differently: agree data access with real-world data partners at the start. Contracts and governance for external data take longer than building the screening pipeline, and evaluations wait on them. The data side is described in federated health analytics and OMOP data quality checks.
If your safety, evidence or medical teams need a platform like this, the healthcare data architecture page describes how I approach it, or analyse your challenge and bring the result to a call.
- 1Good pharmacovigilance practice Module IX: signal management (presentation)European Medicines Agency
- 2FDA Adverse Event Reporting System (FAERS) Public Dashboard: FAQUS Food and Drug Administration
- 3
- 4