Skip navigation
Link to Back issues listing | Back Issue Listing with content Index | Subject Index

Acupuncture for idiopathic headache
Review
Results
Comment
Who reads Cochrane reviews in full? Bandolier has recently conducted a quite unscientific survey of several hundred medical professionals in the UK, and the proportion who have ever read a review in full is under 1%, though perhaps half will have read an abstract. This is a shame. Cochrane reviews are almost always well done, and the biggest disappointment is usually finding how little information there is on some important topics. In highlighting this, Cochrane reviews do a great service.

Cochrane reviews also serve as a terrific resource for teaching. Postgraduate tutors might wish to note this and use them more often. An example is a Cochrane review [1] of acupuncture for idiopathic headache. It concluded:

'Overall, the existing evidence supports the value of acupuncture for the treatment of idiopathic headaches. However, the quality and amount of evidence are not fully convincing. There is an urgent need for well-planned, large-scale studies to assess the effectiveness and cost-effectiveness of acupuncture under real-life conditions.'

Note the balance between support on the one hand, and caution on the other. If you read nothing else but this conclusion, would you judge this a worthwhile treatment option? How would you justify it to a priorities forum in your organisation?

Review

A number of relevant databases were searched to identify randomised, or quasi-randomised trials of acupuncture for the treatment of idiopathic headache. Valiant efforts were made to grade trials for quality, validity and appropriateness of the application and method of acupuncture used.

Results

The 26 included trials included 16 trials in patients with migraine (one in children), six in patients with tension-type headache patients, and four in patients with various types of headaches. Eleven trials used standard criteria for headache, and 16 did not. The median treatment period was eight weeks with eight treatment sessions.

Quality and validity scores were generally low. Both randomised and quasi-randomised trials (by alternation or date of birth) were included. Eleven studies (42%) were double blind, and 15 (58%) were open or single blind.

Markers of quality and validity were used to identify better trials:
The overall results of the 26 trials as judged by reviewers are shown in Table 1. A statistically significant positive result in favour of acupuncture was found in only three trials, 12% of the total included.

Table 1: Overall results for 26 trials of acupuncture for idiopathic headache

Result

Double blind

Not double blind

 

Number

Percent

Number

Percent
No interpretable data 3 12 0 0
Negative: statistically significant 0 0 1 4
Negative: trend 1 4 1 4
No difference 1 4 2 8
Positive: trend 3 12 4 15
Positive: statistically significant 3 12 7 27
Total 11 42 15 58
Positive: acupuncture more effective than control
Negative: control more effective than acupuncture

Seventeen of the 26 included trials used sham acupuncture as control. Three double blind studies were considered to have no interpretable data, either because of baseline differences between treatment groups or high rates of loss to follow up. Table 2 shows the results of trials when information was segregated by different potential biases within the studies. Results were more likely to be negative (no statistical difference between acupuncture and sham acupuncture) in double blind trials, trials of higher reporting quality, trials with higher internal validity, and in larger trials.

Table 2: Potential source of bias and confidence in trials of acupuncture for headache

Potential source of bias

Statistical benefit of acupuncture

No benefit of acupuncture
No source of bias considered 7 7
Randomised 6 7
Quasi-randomised 1 0
Double blind trials 3 4
Not double blind trials 4 3
Reporting quality 3 or more 1 4
Reporting quality 2 or less 6 3
Reporting quality 3 or more; validity score 4 or more 1 3
Reporting quality 2 or less; validity score less than 4 5 3
Reporting quality 3 or more; validity score 4 or more; > 50 patients 0 1
Reporting quality 3 or more; validity score 4 or more; < 50 patients 1 3

Only five of the 26 included trials scored three or more for quality and scored four or more for validity. One had no interpretable results (Table 3). Only one of these was positive and only one was larger than 50 patients. All had some methodological problems that made them less relevant, like having no clear criteria for patient selection, or being of short duration, or giving no information of use of medicines or intensity, duration or frequency of attacks (Table 3).

Table 3: The five 'best' trials, of adequate reporting quality and internal validity of acupuncture

Trial

Type of headache

Number of patients

Quality score
(max 5)

Validity score
(max 6)

Result

Additional comments
1 Migraine 52 3 4 Not significant Appropriateness of acupuncture 85%; IHS criteria; no information on use of medication, intensity, duration or frequency of attacks.
2 Migraine 30 3 4 Not significant Appropriateness of acupuncture 45%; Ad hoc definition of headache; did mention information on improved intensity, duration or frequency of attacks. Follow-up data uninterpretable
3 Tension 30 3 5 Not significant Appropriateness of acupuncture 80%; Ad hoc criteria for headache; no information on intensity or druation of attacks.
4 Tension 25 3 4 Significant benefit of acupuncture Short duration, poor outcome; limited clinical relevance. Appropriateness of acupuncture 70%; no information on use of medication, intensity, duration or frequency of attacks.
5 Tension 10 5 4.5 No data provided Baseline group differences in favour of acupuncture; small group size; pilot study. Questionable validity
All trials were randomised and double-blind

Comment

The majority of trials included in this systematic review were small. Often diagnostic criteria or inclusion criteria for patients entering a trial were poorly reported. No information was provided about the severity of headache before administration of study treatment in many of them, and, in some, patient characteristics differed greatly between treatment groups, showing failure of randomisation. Overall, trials were of poor methodological quality, low validity, and often the application of acupuncture was either inappropriate or could not be assessed. The general quality of reporting of information in the trials was poor, especially with regards to description of drop-outs. There are no methodologically rigorous studies assessing clinically relevant outcomes.

Studies of higher quality and validity should be given more weight in systematic reviews because they are more likely to produce reliable results. Just look at how the picture can change when quality, validity and size enter the equation. With no source of bias considered there are seven trials in which acupuncture is statistically better than sham acupuncture. With reporting quality of 3/5 or better, that drops immediately to 1, stays at 1 when validity is added, but drops to zero when we add size. There is no evidence that acupuncture is effective. Not a single decent trial.

This review is yet another example of how giving equal weight to poor quality studies can lead to erroneous conclusions. Recommendations cannot be made to implement the use of acupuncture in the treatment of idiopathic headache based on current evidence.

Is there any way that the results of this review could be interpreted differently? Is the reviewers conclusion, that ' overall, the existing evidence supports the value of acupuncture for the treatment of idiopathic headaches ' correct? Would you recommend this therapy for your organisation?

High quality, valid trials of adequate size are lacking in this field. There is no reason why studies cannot be adequately randomised or blinded, using International Headache Society recommendations on patient selection and clinically relevant outcomes. Diagnostic criteria should be clearly described, with randomised patients experiencing sufficient pain (moderate or severe) before administration of the intervention. Trials should have a sham acupuncture control. Dosing, duration and follow-up should be adequate and clearly described, as should information on the collection, reporting and severity of adverse effects.

Who should pay? Therapists often bleat about the dearth of research funding for complementary therapy, but then that could be said for many needy areas. The complementary therapy business is huge, costing citizens millions, if not billions of pounds (or euros, or dollars) each year. Therapists (or losers) should pay, as has been done in Holland. Otherwise it would be a bit like a major pharmaceutical company saying they had some interesting chemicals, and could we please pay for their development.

References:

  1. D Melchart et al. Acupuncture for idiopathic headache (Cochrane Review). In: The Cochrane Library, Issue 3, 2002. Oxford: Update Software.
previous or next story in this issue