Unpublished reports
There were 630 published trials and 153 unpublished trials. Unpublished reports
were less likely to be double blind. Overall, there was no difference between
published and unpublished trials, and the ratio of estimates was 1.1 (95%
confidence interval 0.98 to 1.15). If anything, unpublished trials tended to be
less beneficial than published trials.
Language of publication
There were 485 English language trials for analysis and 115 in other languages.
Reports in languages other than English were less likely to be double blind.
Overall, non-English language trials had a more beneficial effect than English
studies, with a ratio of estimates of 0.84 (0.74 to 0.97).
Publication in non-MEDLINE journals
There were 580 trials published in journals indexed by MEDLINE, and 161 published
in journals not so indexed. There was no difference in the proportion properly
randomised and blinded. Overall there was no difference between trials from
indexed journals and non-indexed journals, with a ratio of estimates of 0.94
(0.82 to 1.07).
Concealment of allocation
Concealment of allocation here means that the trial is not only randomised, but
that researchers have no knowledge of what treatment the next patient will have.
Adequately concealed trials will usually have central randomisation, coded drug
packs, or assignment envelopes, while inadequately concealed trials use
alternation, date of birth, or open random number tables, for instance. It
combines randomisation with some elements of blinding.
There were 118 trials with adequate concealment, and 186 in which concealment
was inadequate or unclear. Those trials with inadequate or unclear concealment
had a more beneficial effect (ratio of estimates of 0.79; 0.70 to 0.89).
Double blinding
There were 237 double blind trials, and 162 not double blind. Overall those
trials not double blind tended to have a more beneficial effect, but the ratio of
estimates of 0.88 (0.75 to 1.04) did not reach statistical significance.
Comment
There's not much here that we didn't know, but this is perhaps a larger
examination of some of these issues than has been done before. It is complicated
by issues like whether differences persist when only high quality trials are
used. The authors examine this, and by and large they do. But it is a bit
circular, and applies only to published or unpublished studies, language and
journal of publication.
Any analysis like this is a melange of trials of different interventions in
different conditions, and, sometimes, the validity of trials included in
meta-analysis can be poor. The amount of data available is also an issue. So we
always need to consider a number of issues when looking at a meta-analysis:
- How good is the searching? If only English language papers are included,
some very good material could be missed. Any attempt to find unpublished
material is a bonus, but only with stringent inclusion criteria.
- Are the trials randomised, and are they randomised properly? Many reviews
do not accept improperly randomised trials. If you read a review that has poor
or non randomised studies, see if there is a sensitivity analysis, and if not
put it in the bin.
- Some studies can be blind, and properly double blind to patient and
observer. If unblinded or open studies are included when there are blinded ones
available, don't read on.
It really isn't that difficult, but it is always comforting that someone has
burnt the midnight oil to confirm what we thought we knew.
References:
- M Egger et al. How important are comprehensive literature searches and the
assessment of trial quality in systematic reviews. Empirical study. Health
Technology Assessment 2003 7:1
|
