Beware statistical outputs
- From hypothetical to specific
- Making decisions
- Procedure-specific prescribing
Suppose
we have an intervention in which the NNT is 3.3 compared with placebo. That
tells us that for every 3.3 people given the intervention, one more would have
a treatment-specific outcome than would have occurred with placebo. Put another
way, 30% more patients would have the outcome with the intervention than with
placebo.
Suppose
we were told the relative benefit was 3.3. That tells us that with treatment
the event occurs three times more often than it does with placebo.
In
both cases, if we knew the average placebo response rate, we would know how
many patients would have the outcome with the intervention. Without that
knowledge, the relative benefit is particularly meaningless.
Look
at Figure 1. It shows the relative benefit for a series of hypothetical studies
in which the placebo response rate varies between 1% and 61%, and where 30%
more patients have the outcome with intervention than with placebo. The
relative benefit varies from 31 at 1% placebo response to 1.5 at 61%. In all
these hypothetical trials the NNT is 3.3.
Figure 1: Relationship between relative risk and placebo response for 30% effect
From
hypothetical to specific
A
commentary criticising the use of NNTs in acute pain [1] tries to demonstrate
that different procedures produce different results, based on comparisons of
relative risk and seeking aid from the dubious ally of heterogeneity tests. The
results on which it is based, for paracetamol 975/1000 mg in acute pain trials,
are shown in Table 1.
Table 1: Results for paracetamol 975/1000 mg in acute pain studies after different procedures
|
Number of
|
Percent with at least half pain relief
|
|
|
| Procedure |
Trials
|
Patients
|
Paracetamol
|
Placebo
|
Paracetamol minus placebo
|
Relative benefit
(95%CI)
|
NNT
(95% CI)
|
| Dental |
9
|
1038
|
37
|
10
|
27
|
3.8 (2.8 to 5.1)
|
3.7 (3.1 to 4.5)
|
| Oral |
6
|
962
|
37
|
6
|
31
|
4.2 (2.8 to 6.3)
|
3.2 (2.8 to 3.7)
|
| Episiotomy |
4
|
597
|
69
|
38
|
31
|
1.8 (1.6 to 2.2)
|
3.2 (2.6 to 4.2)
|
| Orthopaedic |
4
|
214
|
54
|
29
|
25
|
1.9 (1.4 to 2.6)
|
4.1 (2.7 to 8.6)
|
Relative
benefit in dental and oral surgery looks higher, at about 4, than the value of
below 2 seen in episiotomy and orthopaedic procedures. NNT values were the
same, with overlapping confidence intervals. The additional benefit of
paracetamol 975/1000 mg in these four procedures, the percentage of patients
with at least half pain relief with paracetamol minus that with placebo, is
also the same (Table 1), as we expect from similar NNTs.
Making
decisions
What
differs is what happens when we do nothing. In dental or oral surgery only 10%
of patients have at least 50% pain relief, while following episiotomy or minor
orthopaedic procedures 30% do so. The effect of the analgesic is the same; the
procedures are different.
Is
paracetamol 975/1000 mg a sensible choice following episiotomy or minor
orthopaedic procedures? It might be argued that it is, because most patients do
well. It could also be argued that it is not a sensible choice in dental or
oral procedures, because most patients do not do well.
Procedure-specific
prescribing
Clearly,
procedure-specific prescribing not only makes sense, but should be mandatory.
Most therapy involves a package of interventions; what may be appropriate in
one circumstance will not be in others.
The
NNT just describes results from clinical trials. It is one way, but not the
only way. Like almost all descriptors, it should not be used alone, but as a
useful shorthand for busy people. In acute pain, efficacy of different
analgesics in different procedures has been tested thoroughly [2]. Procedure
does not affect efficacy in this particular case of clinical trials of
analgesics in acute pain, and with usual provisos of sufficiency of data and
limiting extrapolation.
Bandolier
keeps on exhorting people to remember two important things: look at the data
not just the statistics, and remember that clinical trials are not clinical
practice. Using evidence does not mean putting your brain in neutral and
accepting the evidence without considering appropriateness.
References:
- A Gray et al. Predicting postoperative analgesia outcomes: NNT league tables or procedure-specific evidence? British Journal of Anaesthesia 2005 94: 710-714.
- J Barden et al. Pain and analgesic response after third molar extraction and other postsurgical pain. Pain 2004 107: 86-90.
