How to Read a TCM Study: Questions to Ask

Start here

News stories and product pages often say "a study shows" for a Chinese herb or treatment. This page gives questions to ask before you believe it. They apply to any research, and a few have a TCM twist. Nothing here is medical advice; discuss decisions with a clinician and do not stop prescribed medicines because of one study.

Evidence labels used on this site: established, promising but limited, mixed, insufficient evidence, preclinical, traditional use only.

1. What kind of study is it?

NCCIH, the US government's center for complementary health research, describes four broad types: basic research (testing ideas, the first step), translational research, clinical trials, and systematic reviews and meta-analyses. It says well-planned clinical trials give the clearest information on whether a treatment is effective and safe in humans. [1] A study in cells, animals or a computer model is preclinical; see Network pharmacology and other modern methods for why computer predictions are hypotheses.

2. What exactly was tested?

• Acupuncture: which technique, and was it compared with real needling elsewhere or a sham? See Placebo, sham controls and blinding.
• Herbs: was it a named formula, a single herb or a patent medicine, and is the composition reported? The CONSORT-CHM Formulas 2017 reporting guideline exists because the general reporting quality of Chinese herbal formula trials was disappointing. [2]
• Pattern-based treatment: the same guideline adds the TCM pattern as something trials should report. [2]

3. Compared with what?

A result against no treatment and a result against a sham answer different questions. In NCCIH's summary of acupuncture reviews, the difference versus no treatment was larger than versus sham. [3] A pooled analysis found effect sizes depended on the type of control group. [4]

4. Was anyone blinded, and did it work?

NCCIH advises looking for steps taken to minimize bias, such as blinding so neither participants nor investigators know who received which treatment. [5] Herbal placebos are hard to make; in registered Chinese herbal trials with a placebo, only 2 of 363 (0.6%) assessed whether blinding worked. [6]

5. How big, how many?

NCCIH says small studies are more likely to produce results due to chance, while large studies may give more reliable results and better reflect the population. [7] Be cautious about a "significant" result from a few dozen people.

6. Where was it registered and published?

Of 64 randomized trials of TCM for COVID-19, one appraisal found only 25 provided trial registration information and 39 gave funding sources; trials published in Chinese were of significantly lower quality than those in English. [8] Registered trial records let you see what the researchers planned; see Finding and understanding registered clinical trials.

7. Could publication bias be at work?

A 1998 review found that all acupuncture trials from China, Japan, Hong Kong and Taiwan in the sample reported positive results, and suggested publication bias as a possible explanation; the authors urged reviewers to handle such data carefully. [9] That is a reason to look for independent replication, not to dismiss any country's research.

8. Does the conclusion match the data, and who paid?

The conclusions section holds the authors' own opinions, so others may not agree with their explanation. [10] NCCIH also asks readers to separate "statistically significant" from "clinically significant" (a difference can be real yet too small to matter to patients), to check whether results agree with previous studies because a single study rarely gives a final answer, and to look for financial or reputational stakes held by sponsors or investigators. [11] A lab finding does not prove a clinical benefit, and a benefit in one condition does not transfer to another.

A one-minute checklist

For herbal product safety and quality, see Chinese herbal products: quality, contamination and interactions.