Predictive Accuracy and External Validation
How to read accuracy claims for personalized nutrition and health prediction models, why testing on new people matters, and what published glucose-response models showed.
Content last updated . This page has not been medically reviewed.
Three stages of testing
| Stage | What it means in plain words | What it can show |
|---|---|---|
| Development | The model is fitted to a first group of people. | That patterns exist in that group. |
| Internal validation | The same group is re-used in clever ways, such as splitting it, to check for over-fitting. | Whether the model is fooling itself. |
| External validation | The model is run on a new group it never saw. | Whether it travels to other people. |
Statisticians argue that prediction models need appropriate internal, internal-external and external validation. [2] Many external validation studies are too small to give reliable answers, which is why sample size for the validation step matters. [1]
Accuracy has more than one meaning
A single number such as a correlation does not tell the whole story. One critique stresses that poorly calibrated algorithms can be misleading and potentially harmful for clinical decision-making. [3] Calibration means that when a model predicts a certain level, the real result is about that level on average. Good calibration depends on balancing model complexity against the available sample size, and checking it at external validation needs sufficiently large samples. [3]What published glucose-response models showed
Studies of "personalized" glucose predictions are the most developed example in this field.| Study | What was done | Reported result |
|---|---|---|
| A large Israeli cohort study | A model combining blood tests, diet habits, body measures and gut bacteria predicted responses to meals. It was checked in an independent 100-person cohort. [4] | Reported as accurate prediction; a blinded randomized diet trial followed, with lower post-meal glucose responses. [4] |
| A UK study of 1,002 adults | A model predicted blood fat and glucose responses. [5] | Correlations of r = 0.47 for blood fat and r = 0.77 for glucose, validated in a US group of 100 people. [5] |
| A US study that tested the Israeli model | The model was run on Midwestern American adults. [6] | Correlation 0.596 when trained on the Israeli data only; 0.618 when trained on meals from both groups. [6] |
Three lessons stand out. First, the model did travel to a new country, which is a point in its favor. [6] Second, performance improved when it was retrained with local data, which suggests that models may need updating for new populations. [6] Third, genes added little: in the UK study, genetic variants had a modest effect on predictions, and gut bacteria mattered more for blood fat than meal contents did. [5]
Accurate prediction is not a health benefit
These models predict a response, such as a blood-sugar curve. That is a biomarker, not an illness avoided. A good prediction still leaves open whether following the advice improves how people feel or live. For that you need trials, not accuracy figures. See surrogate versus patient-important outcomes and long-term outcomes in personalized nutrition trials.Who did the validating
Independence matters. Several authors of the US model study were employees or a co-founder of a company that supported the work, as their disclosure states. [6] That does not make the results wrong. It does mean that findings tested by unrelated teams carry more weight than findings tested only by the developers.Reporting standards
A reporting checklist called TRIPOD+AI gives harmonized guidance for describing prediction-model studies, whether they use regression or machine learning. [7] Products rarely publish such a checklist. If a company cannot show the study behind its accuracy claim, treat the claim as a marketing claim.A reading checklist
- Was the model tested on people who were not used to build it?
- How many people, and how like you were they (age, country, health)?
- Were both correlation and calibration reported?
- Was the outcome a lab value or something people experienced?
- Who ran and paid for the validation?
Related guides
See personalized glycemic predictions for more on the glucose studies, replication and reproducibility, and claim check: "personalized" means better.Frequently asked questions
What is external validation?
It is the testing of a finished prediction model on new data to see how well it performs outside the group it was built from. [1]
Is a high correlation enough to trust a model?
No. Calibration also matters, because a poorly calibrated model can mislead decisions even when it ranks people well. [3]
Did the glucose models hold up in other countries?
The Israeli-built model predicted responses in a US group better than carbohydrate or calorie counts alone, and it improved when retrained with US data. [6] That is one study in one setting.
Does an accurate prediction mean the diet will improve my health?
Not by itself. Accuracy is about predicting a response. Benefit has to be shown in trials that measure real outcomes.
References
- Riley RD, Snell KIE, Archer L, et al. Evaluation of clinical prediction models (part 3): calculating the sample size required for an external validation study. BMJ 2024. PMID 38253388. Read 2026-10-22.
- Steyerberg EW, Harrell FE. Prediction models need appropriate internal, internal-external, and external validation. Journal of Clinical Epidemiology 2016. PMID 25981519. Read 2026-10-22.
- Van Calster B, McLernon DJ, van Smeden M, et al. Calibration: the Achilles heel of predictive analytics. BMC Medicine 2019. PMID 31842878. Read 2026-10-22.
- Zeevi D, Korem T, Zmora N, et al. Personalized Nutrition by Prediction of Glycemic Responses. Cell 2015. PMID 26590418. Read 2026-10-22.
- Berry SE, Valdes AM, Drew DA, et al. Human postprandial responses to food and potential for precision nutrition. Nature Medicine 2020. PMID 32528151. Read 2026-10-22.
- Mendes-Soares H, Raveh-Sadka T, Azulay S, et al. Model of personalized postprandial glycemic response to food developed for an Israeli cohort predicts responses in Midwestern American individuals. American Journal of Clinical Nutrition 2019. PMID 31095300. Read 2026-10-22.
- Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ 2024. PMID 38626948. Read 2026-10-22.