Skip navigation

Round the world with acupuncture
Search and inclusion
Outcomes
Results
Comment

Bandolier was a guest at a discussion on alternative medicine. One comment made was that
"The Chinese have been doing this sort of thing for thousands of years: surely it can't be wrong?"
The immediate reaction was that the Chinese suffered to the same extent as everyone else from the great plagues of the 6 th and 14 th centuries, and Chinese historians record many, many major depredations over the centuries.

Perhaps it all comes down to how you look at things, standards set, and societal values. A systematic review [1] set out to answer the question whether some countries produce only positive results.

Search and inclusion


There were two searches. The first used MEDLINE to retrieve papers on acupuncture with abstracts available over 30 years. Papers had to have patients receiving acupuncture who were compared with patients receiving placebo, no treatment or a no acupuncture control.

The second search looked for randomised or controlled clinical trials published in China, Taiwan, Japan or Russia/USSR between 1991 and 1995. In addition, 330 most recent randomised or controlled trials published in England were sought. These studies had to have patients receiving a treatment other than acupuncture compared with patients receiving a control intervention.

Outcomes


Reviewers blinded to the country of origin then retrieved and abstracts examined. The outcome was a superiority of treatment over control based on:


Results


For acupuncture, there was a wide discrepancy between countries of origin and the proportion of trials showing superiority of acupuncture. Countries in North America, Western Europe and Australasia were positive for acupuncture about half the time, or less. Those from Eastern Europe and especially East Asia were positive nearly all the time (Figure).

Figure: Proportion of controlled trials of acupuncture showing superiority of acupuncture from five regions, with number of studies


The four countries which had 100% positive rates for acupuncture were compared with England for positive rates for randomised or controlled trials where acupuncture was not being tested. They also had very high rates of positive trials here as well (Table), as high as 97% for Russia/USSR and 99% for China. Rates for England were consistently lower.

Table: Proportion of trials with treatment better than control for randomised or controlled nonacupuncture studies, and from acupuncture studies, from five countries
  Randomised or controlled trials Acupuncture trials
Country Number Positive (%) Number Positive (%)
England 107 75 20 60
China 109 99 36 100
Japan 120 89 5 100
Russia/USSR 29 97 11 91
Taiwan 40 95 6 100

Comment


The authors of this review did a terrific job in trying to eradicate bias from their analysis. They acknowledge that because they included controlled trials, and looked only at abstracts, they will have included studies with known methodological bias. They also acknowledge that authors can and do make misleading or mistaken comments about trial results in abstracts.

That having been said, there remains a gulf between studies reported from different parts of the world. Bias may be institutionalised in some places, or may just be harder to detect in others. The reason randomisation schedules for patients are often described as being enclosed in metallised envelopes is because people have been known to X-ray envelopes to break the code before allocation.

The inference is obvious. Quality is much more important than quantity. No matter how many trials of inadequate or biased design we have, they do not match up to one trial of adequate size and methodological rigor. Quality is first, and everything else is nowhere.

"All was wrong because not all was right" is a useful quotation from George Crabbe that might usefully govern the interpretation of evidence. It applies to all therapies.

Reference:

  1. A Vickers, N Goyal, R Harland, R Rees. Do certain countries produce only positive results? A systematic review of controlled trials. Controlled Clinical Trials 1998 19: 159-166.
previous or next story in this issue