Ffôn / Telephone

We have recently had to provide advice within our health authority on the issue of interferon beta-1b treatment in secondary progressive multiple sclerosis. Although we agree with Bandolier's view that this is a difficult area, and that hard decisions have to be made, we disagree with Bandolier's conclusions ¹ based on the multi centre randomised controlled trial. ²

In the original study the odds ratio of the main outcome measure, ‘progression to disability', is not presented anywhere as an intention to treat analysis - rather surprising given the study group's discussion of intention to treat analysis and their apparent extensive consultation with statistical experts. The intention to treat analysis should have been presented unambiguously in the original paper. Bandolier went ahead and calculated the intention to treat effect size without criticising the data on which it was based.

Worse, Bandolier appears to take its calculated intention to treat effect sizes at face value, suggesting that the drug is indeed effective in preventing ‘progression of disability'. No comment is made that the confidence intervals are very wide, even for the primary outcome measure.

Although a clinical outcome rather than a surrogate marker was used, Bandolier did not question the relevance in practice of this outcome measure - the Expanded Disability Status Scale (EDSS). This captures mostly ambulation and is not without problems when used with people who have MS. The relationship between EDSS and a 42-item Activity of Daily Living scale (which may be more relevant to patients and their carers, along with a quality of life measure) is not consistent across different EDSS levels in people with MS. ³

The treatment and placebo groups were not comparable. Patients in the placebo group had more severe disease (as measured by EDSS), and had MS diagnosed on average 8 months earlier than the treatment group. Does secondary progressive MS progress uniformly in terms of EDSS? This is important because those with more severe disease were counted as having ‘progression of disability' after an increase in EDSS of only 0.5 instead of 1.0. This bias could have overestimated the effect size, probably in favour of the treatment group. Neither the authors nor Bandolier discussed the relevance of an increase of 1.0 (or 0.5) in the EDSS as a cut off for ‘progression of disability'.

Significant bias is involved with many treating physicians and patients guessing correctly whether they were on treatment or placebo. A high percentage of the medical assessors claimed they ‘did not know', whereas patients or treating physicians appeared at least to try to guess ‘placebo' or ‘on treatment'. If so many of the patients and treating physicians were able to guess the correct intervention arm then it is likely that the medical assessors would also know. It is worth noting that employees of the manufacturers of the interferon were involved on the study writing and steering committees, and were among the investigators.

The study was terminated prematurely, before all subjects had reached the intended three years follow up. It is possible, given the high drop out rates and the unpleasant side effects of interferon beta-1b, that more patients would have dropped out of the treatment group than the placebo group making it less likely that an effect would be demonstrated in an intention to treat analysis after 3 years.

The patients in the study are a selected group of all patients with secondary progressive MS. The effectiveness of interferon beta-1b has not yet been assessed outside this group.

We believe that Bandolier has an important role in the interpretation and dissemination of scientific evidence in the field of health. However we are concerned that it did not critique this study more fully, as health care purchasers and providers frequently use Bandolier as an authoritative source to inform their decisions. We recommend that, just like estate agents, Bandolier should carry a disclaimer, such as that used by Health Evidence Bulletins Wales: ‘Users are advised to consult the supporting evidence for a consideration of all the implications of a recommendation.' ⁴

2 European Study Group. Placebo-controlled multicentre randomised trial of interferon beta-1b in treatment of secondary progressive multiple sclerosis. Lancet 1998; 352: 1491-97

3 Cohen RA, Kessler HR, Fischer M. The Extended Disability Status Scale (EDSS) a s a predictor of impairments of functional activities of daily living in multiple sclerosis. Journal of Neurological Sciences 1993; 115(2): 132-5