Missed Warning Signs With Digital Health Tools

Urgent first

If you or someone near you has severe symptoms, or symptoms that are getting worse, get urgent care now. In the United States, call 911. Elsewhere, use your local emergency number. If you are thinking about suicide, the US 988 Suicide & Crisis Lifeline is 988. A reassuring answer from an app, chatbot or device reading is not a reason to wait. This page is information only, not medical advice.

For low oxygen specifically, the FDA lists these possible signs: bluish coloring in the face, lips or nails; shortness of breath or a worsening cough; restlessness and discomfort; chest pain or tightness; and a fast, racing pulse. The FDA also notes that some people with low oxygen may not show any or all of these symptoms. [1]

What "missed warning signs" means

A digital tool can miss a warning sign in three ways. It can fail to ask the right question, it can misjudge the answers, or it can be right while the person under-reads or argues with the advice. This page covers the first two, and the studies below test tools on written cases, not on real emergencies. Evidence label used here: mixed means studies disagree by tool, version or setting.

Symptom checkers (web tools and apps)

• Audit of 23 free symptom checkers (BMJ, 2015). With 45 standardized cases, the tools gave appropriate triage advice in 57% of evaluations overall, and in 80% of emergency cases. The authors found deficits in both diagnosis and triage, and noted the advice leaned cautious, telling people to seek care when self-care was reasonable. [2]
• Systematic review for urgent problems (2019). Of 29 publications, evidence on patient safety was weak, diagnostic accuracy was generally low, and algorithm-based triage tended to be more risk-averse than health professionals. [3]
• Systematic review of triage and diagnostic accuracy (2023). Of 14 included studies, most (9 of 13) showed suboptimal triage accuracy overall, with a few exceptions. [4]

Together these are labeled mixed: tools vary a lot, and the main documented pattern is over-caution, which has its own cost, but a share of emergencies were still not flagged.

AI chatbots

• ChatGPT Health (Nature Medicine, 2026). Researchers tested 60 clinician-written cases across 21 areas under 16 conditions (960 responses). Among emergencies judged by a clinician standard, the system under-triaged 52%, for example sending diabetic ketoacidosis or impending respiratory failure to evaluation within 24 to 48 hours. It correctly triaged classic emergencies such as stroke and anaphylaxis. When a family member or friend minimized the symptoms, the advice shifted toward less urgent care. Crisis-line messages for suicidal thoughts appeared unpredictably. [5]
• A physician-facing tool (2026). Using the same benchmark, one study reported that OpenEvidence under-triaged 12.5% of emergencies, compared with 51.6% in the earlier ChatGPT Health benchmark, while over-triaging many non-urgent cases. The comparison was to a previously reported figure, not a head-to-head trial. [6]
• Real users (2026). In a randomized study of 1,298 people, models alone chose the right course of action (disposition) in 56.3% of cases on average, and people using them did so in fewer than 44.2%, no better than a control group. [7]

These are written-case tests. They show where failures can occur, not how many real emergencies are missed.

Device readings

A reading can falsely reassure. The FDA says pulse oximeter readings should be considered together with symptoms, and lists factors that affect accuracy: poor circulation, skin pigmentation, skin thickness, skin temperature, tobacco use and fingernail polish. [1] It advises contacting a health care provider if you are concerned about the reading or your symptoms are serious or get worse. [1] Skin-tone differences are covered in bias and unequal performance in health AI. For wearables more broadly, see delayed medical assessment and false reassurance.

What this means in practice

• Symptoms outrank the screen. If you feel worse than a tool says you should, trust your symptoms and seek care.
• A tool's calm answer is not a clearance. The studies show missed emergencies and over-cautious advice in different tools.
• A tool is not an exam. None of these tools can examine you or order a test. For what a chatbot can and cannot be shown to do, see patient-facing health chatbots and symptom tracking apps.
• Tell your clinician what a tool told you; see talking with a health professional.

Limits

We did not find a study that counted missed emergencies among real patients across all tools, so overall real-world frequency is insufficient evidence. Tools update often, and each result belongs to the version tested.