SPECIALTY TOPICS · GUIDE
Clinical Validation and Real-World Performance of Health Software
How health software and AI tools are checked before and after release, why strong test results can fall short in hospitals and clinics, and one widely used model that did.
Content last updated . This page has not been medically reviewed.
Back to Emerging Wellness ScienceEvidence labels: established, promising but limited, mixed, insufficient evidence, marketing claim. See how we label evidence.
Layers of checking
| Layer | Question | What it cannot show |
|---|---|---|
| Technical (analytical) | Does the software measure or calculate correctly? | Whether patients benefit |
| Clinical validation | In the intended patients, does its output match a reference standard? | Whether care or outcomes improve |
| Clinical utility | Does using it improve decisions or health? | Performance in other hospitals or groups |
| Real-world monitoring | Does it keep working after deployment? | Why it drifts without further study |
The FDA, working with the International Medical Device Regulators Forum, uses the term Software as a Medical Device for software intended for medical purposes that is not part of a hardware device. [1] The IMDRF working group agreed on definitions and principles including clinical evaluation of such software. [1] See analytical validity and clinical decision support software.
What authorization does and does not say
The FDA keeps a list of AI-enabled medical devices authorized for marketing in the United States. It says devices on the list have met applicable premarket requirements, including a focused review of safety and effectiveness. It also says the list is not comprehensive and was compiled mainly from AI-related terms in marketing authorization summaries. [2] Authorization is for a specific device and intended use. It is not a guarantee of performance in every hospital. See regulatory status of digital health products.Many AI studies are weaker than their headlines
A systematic review of studies comparing deep learning for medical imaging with expert clinicians found only 10 randomized trial records (two published, eight ongoing) and 81 non-randomized studies, of which just nine were prospective and six were tested in a real-world clinical setting. Datasets were unavailable in 95% of studies and code in 93%. Overall risk of bias was high in 58 of 81 studies. Even so, 61 of 81 abstracts said the AI was at least comparable to clinicians. [3]Evidence level for the general claim "AI matches clinicians": mixed to insufficient, because of design problems in many studies. This does not mean every tool is weak; it means each one needs its own good evidence.
A real-world example: the Epic Sepsis Model
Sepsis is a life-threatening response to infection. The Epic Sepsis Model is a proprietary prediction tool used in hundreds of US hospitals. Researchers at one academic center tested it in a retrospective cohort of 27,697 patients (38,455 hospitalizations; sepsis in 7%). The model's hospitalization-level area under the curve (a score where 0.5 is chance and 1.0 is perfect) was 0.63 (95% CI 0.62 to 0.64). It missed 67% of patients with sepsis, while alerting for 18% of all hospitalizations, which created alert fatigue. [4] The authors concluded the model had poor discrimination and calibration and raised concerns about its widespread adoption. [4]Limits: this was one health system, a retrospective design, and one version of the tool. The paper has a published erratum that I did not read in full; the figures above are from the current record. A single study does not tell you how the tool performs everywhere, which is exactly the point: local checking matters.
Why performance can differ after launch
- Different patients, equipment, record systems or workflows than in testing.
- Rare conditions mean even good tests produce many false alarms.
- People may over-trust software. See hallucinated health information and automation bias and bias and unequal performance in health AI.
- Real-world data can reveal problems after launch. The FDA defines real-world evidence as clinical evidence about usage and potential benefits or risks of a medical product derived from analysis of real-world data such as health records and registries. [5] See real-world evidence.
Questions for any health software claim
- Was it tested prospectively and in people like those using it?
- Was the test done by the developer, or independently?
- What happens to patients when it is wrong?
- Is performance monitored after launch?
Frequently asked questions
What is clinical validation?
Checking that software's output in the intended patients agrees with a reference standard. That is different from showing patients do better. The IMDRF working group chaired by the FDA agreed on principles for the clinical evaluation of such software. [1]
Does FDA authorization prove an AI tool is accurate everywhere?
No. It reflects a premarket review for a specific device and use. [2]
Why did a widely used sepsis model perform poorly?
In one outside evaluation it had an area under the curve of 0.63 and missed most sepsis cases. [4] One study cannot say why or whether other sites differ.
Are AI studies reliable?
Many compare AI to clinicians in retrospective data and have high risk of bias. [3]
What is real-world evidence?
Evidence about a product's use, benefits or risks drawn from routine data like health records. [5]
References
- U.S. Food and Drug Administration. Software as a Medical Device (SaMD). Read 2026-10-22.
- U.S. Food and Drug Administration. Artificial Intelligence-Enabled Medical Devices. Read 2026-10-22.
- Nagendran M, Chen Y, Lovejoy CA, et al. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ 2020. Read 2026-10-22.
- Wong A, Otles E, Donnelly JP, et al. External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern Med 2021. Read 2026-10-22.
- U.S. Food and Drug Administration. Real-World Evidence. Read 2026-10-22.