Industry bias in clinical trials
We
have become used to finding potential sources of bias in clinical trials. What
we find is usually empirical evidence of bias from some examples, and then we
look for the possibility that bias from that particular source might be
affecting other results we are considering. If there is sufficient information,
we might test whether that bias exists in our data using a sensitivity
analysis. As a simple example, we might test randomised or double blind studies
against those that are not randomised or double blind.
Industry bias
The
majority of clinical and laboratory work undertaken is funded by industry,
small or large. Concerns have been expressed about whether this can lead to
bias, and there is a considerable, if somewhat confused, literature. In part
this is because at least two possible issues come under this heading:
- Marketing bias. Companies want to present their products in the best possible light, and will be selective about what they say. This applies to small spin-offs looking for development capital, to large organisations seeking to market new pharmaceuticals, tests, or devices.
- Sponsorship bias. Narrowly defined, this is whether the source of funding for a clinical trial affects the results of the clinical trials in a systematic way.
The
existence of marketing bias would be conceded universally. It is part of the
world we work in. You don't see adverts, for instance, saying that
margarine tastes worse than butter. Every book on the bookstand is an
international best seller.
Whether
sponsorship bias exists, though, is another matter. Although there have been
many reviews claiming the existence of industry bias, these have typically made
a heap of what they could find, irrespective of criteria of quality, validity,
and size, and have typically not compared like with like. A new systematic
review [1] is possibly the first to look specifically at whether sponsorship
bias exists in clinical trials, and gives a useful summary of some of the
reviews that claim to have found industry bias.
Review
The
review used existing meta-analyses of high quality acute pain and migraine
interventions, which had chosen randomised and double blind trials that were
valid, used a defined dose, and used consistent methods, had the same outcomes
measured over the same time, and where there was a large amount of information.
These were oral aspirin 600/650 mg, paracetamol 1000 mg, ibuprofen 400 mg and
rofecoxib 50 mg in acute pain, and oral sumatriptan 100 mg and 50 mg for
migraine; all were compared with placebo.
The
intention was to compare efficacy in trials sponsored by for-profit
organisations with those without for-profit sponsorship.
Results
Of
176 individual trials, only two could be identified as being definitely
sponsored by non-profit sources, compared with 144 sponsored by for-profit
sources. No statement was made in 31 trials. Clearly no comparison between
for-profit and not for-profit sponsorship was possible.
An
alternative strategy was used (Figure 1). In the 143 for-profit trials, a
distinction was made between trials where a drug was being used as a test, or
as a comparator. The argument was that if bias existed, there would be
conflicts of interest within industry-sponsored trials. The bias should act to
maximise efficacy when a drug was used as the test drug (the aim being to make
it look as good as possible), and to minimise efficacy when the same drug was
used as a comparator (the aim being to minimise efficacy compared to a second,
newer, test drug).
Figure 1: Strategy for examining the existence of differential sponsorship bias in randomised trials in acute pain and migraine where source of sponsorship was definite
When
this strategy was applied to four analgesics in acute pain trials, no
difference in efficacy was found (Figure 2) as measured by the NNT for at least
half pain relief over 4-6 hours compared with placebo. For sumatriptan in acute
migraine, there were no consistent differences at 50 mg or 100 mg, for three
different outcomes.
Figure 2: NNT for four analgesics in identical randomised, double blind acute pain trials reporting the same outcome over the same period of time, where drug was used as test or comparator
Comment
The
conclusion here was that the only way of testing these studies was to look for
differential sponsorship bias. When sought, no bias was found. It looks very
much as if for-profit industry sponsorship does not influence the outcome of
good quality, randomised trials in acute pain and migraine. There are also
examples of large, high quality for-profit and not for-profit sponsorship
trials of statins and finasteride giving essentially the same result.
This
is very good news, and should be looked at in other areas. Had differential
sponsorship bias been found, it would mean that we would probably not be able
to trust randomised trials, since almost all have for-profit sponsorship. That
would have been a bit of a blow.
The
paper [1] gives examples, though, of marketing bias, where individual trials
with particular beneficial characteristics were picked out for special
marketing attention. The lesson is to treat all sources of information,
industry or other, as if they were used car salespeople trying to sell you a
dud. Ask for the evidence, and accept only good evidence, not airy-fairy
nonsense about contented customers. If they can't show it, it may be because
there isn't any. Never mind the shine, look for the quality.
Reference:
- J Barden et al. Bias from industry trial funding? A framework, a suggested approach, and a negative result. Pain 2006 121: 207-218.