Device vs Clinical Reference: How Validation Works
How wearable validation studies compare a device with a clinical reference such as an ECG or sleep study, what the accuracy numbers mean, and the limits of those studies.
Content last updated . This page has not been medically reviewed.
The basic idea
A validation study asks: when a consumer device and a trusted reference measure the same thing at the same time, how close are they? The reference is the "criterion". Examples in the studies below are a 12-lead electrocardiogram (ECG) for heart rhythm, ECG for heart rate, and polysomnography (an overnight sleep study) for sleep.Evidence labels: established, mixed, insufficient evidence.
Six things a good protocol specifies
An expert statement from a European network of researchers proposed a validation framework for consumer heart-rate wearables that use light sensing, built on six domains: target population, criterion measure, index measure (the device tested), testing conditions, data processing and statistical analysis. [1] Another set of guidelines for researchers urges reporting device type, firmware version and sampling rate, plus factors such as body mass index, wrist circumference and skin tone. [2] If a company or article leaves these out, you cannot tell what the number applies to.Reading the numbers
| Term | Plain meaning |
|---|---|
| Mean bias | Average over- or under-reading |
| Limits of agreement | The range where most individual differences fall |
| MAPE | Average percent error |
| Sensitivity / specificity | How often a test catches true cases / avoids false alarms |
Averages can hide big individual errors. A meta-analysis of recent Fitbits found an average heart-rate underestimate of about 3 beats per minute, but the population limits of agreement ran from roughly -24 to +18. [3]
Worked examples (exact devices, specific conditions)
- Heart rhythm. In 201 patients at a cardiology service, five devices were compared with a physician-read 12-lead ECG. Sensitivity for atrial fibrillation was 85% for Apple Watch 6 and Samsung Galaxy Watch 3, 58% for Withings ScanWatch, 66% for Fitbit Sense and 79% for AliveCor KardiaMobile; 17% to 26% of tracings were inconclusive. [4] This applies to those versions in a hospital population, not to every model or to the general public.
- Sleep. Six wearables checked against polysomnography in 53 healthy adults for one night agreed well on sleep versus wake but less on sleep stages. [5]
- Steps and calories. A review found devices accurate for steps and heart rate in laboratory settings but varying by manufacturer and device type, and the authors note that devices are constantly redesigned, so reviews age quickly. [6]
Where to go next
How to Read a Wearable Validation Study, How to Find Device Validation Studies and Regulatory Records and False Positives, False Negatives and Test Accuracy.Frequently asked questions
What is a reference standard?
The trusted method a device is compared with, such as a 12-lead ECG or a sleep study. [4][5]
Does "validated" mean it is medically accurate?
No. It means it was tested against something, in certain people and conditions. Ask for the exact model, population and error size.
Can I apply a study of one model to another?
No. Authors note devices are constantly redesigned. [6]
Why do inconclusive results matter?
In the atrial fibrillation study, about one in four tracings on some devices could not be classified. [4]
Who funds the studies?
Some state support from device makers; the disclosure section tells you. [5]
More details: evidence, limits and cautions
The details behind this page: cautions, limits of the evidence and where sources disagree. Open any note to read it.
Limits of validation
- Studies in laboratory settings may not match daily life.
- Accuracy for one model does not carry over to older or newer models.
- Measurement accuracy does not prove health benefit.
- The reference itself has error, and some studies are funded or supported by device makers, so check disclosures. [5]
Regulation is a different question
A US regulatory status applies to a named function and intended use. FDA's general wellness policy describes low-risk products intended for a healthy lifestyle and unrelated to disease. [7] See Claim Check: FDA Registered vs Product Approval.
References
- Muhlen JM et al. Recommendations for determining the validity of consumer wearable heart rate devices: expert statement and checklist of the INTERLIVE Network. Br J Sports Med 2021. PubMed abstract. Read 2026-10-22.
- Nelson BW et al. Guidelines for wrist-worn consumer wearable assessment of heart rate in biobehavioral research. npj Digit Med 2020. PubMed abstract. Read 2026-10-22.
- Chevance G et al. Accuracy and Precision of Energy Expenditure, Heart Rate, and Steps Measured by Combined-Sensing Fitbits Against Reference Measures. JMIR Mhealth Uhealth 2022. PubMed abstract. Read 2026-10-22.
- Mannhart D et al. Clinical Validation of 5 Direct-to-Consumer Wearable Smart Devices to Detect Atrial Fibrillation: BASEL Wearable Study. JACC Clin Electrophysiol 2023. PubMed abstract. Read 2026-10-22.
- Miller DJ et al. A Validation of Six Wearable Devices for Estimating Sleep, Heart Rate and Heart Rate Variability in Healthy Adults. Sensors 2022. PubMed abstract. Read 2026-10-22.
- Fuller D et al. Reliability and Validity of Commercially Available Wearable Devices for Measuring Steps, Energy Expenditure, and Heart Rate: Systematic Review. JMIR Mhealth Uhealth 2020. PubMed abstract. Read 2026-10-22.
- U.S. FDA, General Wellness: Policy for Low Risk Devices (final guidance, January 2026). Page current as of 01/06/2026. Read 2026-10-22.
Education, not medical advice. This page is for general information. It does not diagnose or treat any condition and is not a substitute for care from a qualified health professional. Natural does not automatically mean safe. Ask a clinician or pharmacist before changing your care, especially if you take medication, are pregnant or breastfeeding, or have a health condition.