Quality and Bias in Acupuncture Trials: Blinding, Expectation and Publication Bias

Blinding is hard

In a drug trial, a dummy pill looks identical. In acupuncture, the practitioner can see what they are doing. In a review of 13 three-arm pain trials (3,025 patients), the clinicians were not blinded in any trial. [1] NCCIH describes sham acupuncture as procedures such as a device that pokes the skin but does not penetrate it. [2] Researchers debate whether any sham is truly inert. [3] In the large individual-patient review of chronic pain, trials that used a penetrating needle as the sham showed smaller differences from real acupuncture. [4] For more, see sham acupuncture and placebo controls.

Expectation and the ritual

A trial in 270 adults with arm pain compared a sham acupuncture device with an inert pill. Over the whole treatment period, the sham device had greater effects on self-reported pain and symptom severity. [5] A Cochrane review of placebo interventions found larger placebo effects with physical placebos such as sham acupuncture and with outcomes that involve the patient. [6] The same review did not find important clinical effects of placebo in general, and noted it is hard to separate patient-reported placebo effects from biased reporting. [6] In plain words: how a treatment is delivered can change how people feel, and that is not the same as the needle's own effect.

What the better trials found

The Vickers teams included only trials with adequate allocation concealment, [4] and concluded acupuncture is more than a placebo while saying the true-versus-sham differences are modest. [7] Madsen and colleagues, looking at 13 trials, reached a more cautious view. [1] Both readings are published, and the data do not settle it. Cherkin's authors wrote that it remains unclear whether acupuncture or their simulated method provides physiologically important stimulation. [3] Note that none of these trials tells you about conditions other than those studied.

Publication and reporting bias

Studies with exciting results are more likely to be published, and researchers can change which outcome counts as "primary" after seeing the data.

• In a 1998 review of trial abstracts, some countries published unusually high proportions of positive results, and the authors named publication bias a possible explanation. [9] See trials from China and elsewhere.
• A study of 88 registered acupuncture trials found only 19.3% were registered before the trial started. [10] Primary outcomes differed between registry and paper in 45.1% of comparable reports. [10]
• Vickers 2012 reported its results were robust to analyses related to publication bias. [7]

Better reporting

The STRICTA checklist, an official extension of CONSORT, sets out six items and 17 sub-items so readers can see exactly what the acupuncture and comparison were. [11] Cochrane's risk-of-bias tool also separates allocation concealment from blinding, a distinction reviewers sometimes confuse. [12] Use these ideas with how to read an acupuncture study.

What this means for you

Trial problems do not prove acupuncture is useless, and they do not prove it works as traditionally explained. They mean claims deserve careful reading. NCCIH says how acupuncture works is not fully understood. [2] Talk to your doctor before replacing any medical treatment.