Are pneumococcal vaccines effective? [Feb 2000; 72-4]

Are pneumococcal vaccines effective?
Problems Question Analysis Results All-cause pneumonia Pneumococcal pneumonia Pneumococcal death Pneumococcal bacteraemia Comment

A newly published systematic review [1] concludes that vaccination with pneumococcal vaccines reduces the risk of infection by very significant results and is effective in elderly people. Are these results credible?

There was a considerable effort put into searching, including searching Index Medicus back to 1938. Thirteen studies were identified up to November 1996.

Randomised studies included

Austrian R. Surveillance of pneumoccal infection for field trials of polyvalent pneumococcal vaccines. National Institute of Health Publication DAB-VDP-12-84. Contract No 1A13257. Bethesda MD. 1980, pp 1-59.
Gaillat J, Zmirou D, Mallaret MR, Rouhan D, Bru JP, Stahl, JP et al. Essai clinique du vaccin antipneumococcique ches des personnages agées vivant en institution. Rev Epidem Santé Publ 1985;33:437-444.
Klatersky J, Mommen P, Canteraine F, Safary A. Placebo controlled pneumococcal immunization in patients with bronchogenic carcinoma. Eur J Cancer Clin Oncol 1986;22:807-813.
Simberkoff MS, Cross AP, Al-Ibrahim, M, Baltch AL, Geiseler PJ, Nadler J et al. Efficacy of pneumococcal vaccine in high-risk patients. N Eng J Med 1986;315:1318-1327.
Davis AL, Aranda CP, Schiffman G, Christianson, LC. Pneumococcal infection and immunologic response to pneumococcal vaccine in chronic obstructive pulmonary disease. Chest 1987;92:202-212.
Leech JA, Gervais A, Ruben FL. Efficacy of pneumococcal vaccine in severe chronic obstructive pulmonary disease. Can Med Assoc J 1987 136: 361-365.
Koivula I, Stén M, Leinonen M, Mäkelä PH. Clinical efficacy of pneumococcal vaccine in the elderly: a randomized, single-blind population-based study. Am J Med 1997;103:281-290.
Örtqvist A, Hedlund J Burman L-A Elbel E, Höfer M, Leinonen M et al. randomised trial of 23-valent pneumococcal capsular polysaccharide vaccine in prevention of pneumonia in middle-aged and elderly people. Lancet 1998;351:399-403.

Quasi-randomised studies

Honkanen PO, Keistinen T, Miettinen L et al. Incremental effectiveness of pneumococcal vaccine on simultaneously administered influenza vaccine in preventing pneumonia and pneumococcal pneumonia among persons aged 65 years or older. Vaccine 1999 17; 2493-500.
Kaufmann P. Studies in old age pneumonia. II. Prophylactic effects of pneumococcus polysaccharide against pneumonia. Archives of Internal Medicine 1941 67: 304-19.
Kaufmann P. Pneumonia in old age: active immunization against pneumonia with pneumococcus polysaccharide; results of a six-year study. Archives of Internal Medicine 1947 79: 518-31.
MacLoed CM et al. Prevention of pneumococcal pneumonia by immunization with specific capsular polysaccharides. Journal of Experimental Medicine 1949 82: 445-65.

Studies in non industrialised settings

Gilks CF, French N, Nakiyingi, Carpenter L, Lugadda E, Watera C et al. Lack of efficvacy of 23-valent pneumococcal polysaccharide vaccine in HIV-1 infected Ugandan adults. Proceedings of the Pneumococcal Vaccines for the World 1998 Conference. October 12-14, 1998. Washington DC, USA.
Riley ID, Tarr PI, Andrews M, Pfeiffer M, Howard R, Challands P et al. Immunisation with a polyvalent pneumococcal vaccine. Lancet 1977;I:1338-1341.
Austrian R, Douglas RM, Schiffman G, Coetzee AM, Koornman HJ, Hayden-Smith S, Reid RD. Prevention of pneumococcal pneumonia by vaccination. Trans Assoc Am Physician 1976;89:184-194.
Smit P, Oberholzer D, Hayden-Smith S, Koornhof HJ, Hillman MR. Protective efficacy of pneumococcal polysaccharide vaccines. JAMA 1977;238:2613-2616.

Problems

The first problem is completeness. Since 1996 six more studies have been published, mostly in elderly people, or people with chronic disease in industrialised countries. One examined HIV infected individuals in Uganda.

The second problem is randomisation. Two of the early studies were only quasi-randomised (alternate allocation). One of the newly published studies is also quasi random (allocation by year of birth).

The third problem is patient populations. Three of the early studies (on South African gold miners and New Guinea highlanders) are hardly representative of people who may be treated in industrialised countries. Miner in particular were younger, and lived in tightly-packed conditions where rates of pneumococcal disease may be higher.

The fourth problem is size. Many of the studies were small, especially as some outcomes, like bacteraemia, were vanishingly small. The total number of events may be subject to the random play of chance.

The fifth problem is outcomes. A variety of outcomes may be reported - from bacteraemia, all-cause pneumonia, pneumococcal pneumonia, pneumonia death or lower respiratory tract infection.

Question

What should the question be, then, when asking whether pneumococcal vaccination is effective? Obviously we want evidence on populations similar to those we may wish to vaccinate in our community - which means elderly, or institutionalised people, or those with chronic diseases. We want the least biased evidence from randomised trials. We also want to know those outcomes most important to us - and in this case perhaps all the outcomes identified above.

Analysis

Bandolier has therefore looked at nine relevant trials. Two from the 1940s and one published in 1999 have not been included because they were not properly randomised. The studies in the 1940s had a positive conclusion. The study reported in 1999 from Finland in about 25,000 elderly people was negative.

Results

The results are shown in the Table.

Table: Main outcomes of randomised trials of pneumococcal vaccines in industrial countries
Outcome	Number of trials	Number of patients	Percent affected without vaccine	Percent affected with vaccine	Relative risk (95%CI)	NNT (95%CI)
All pneumonias	4	6,514	7.56	7.18	1.01 (0.85 to 1.19)	260 (60 to -113)
Pneumococcal pneumonias	6	21,156	1.89	1.59	0.85 (0.69 to 1.04)	325 (152 to -2226)
Pneumonia-related death	8	22,559	1.10	1.00	0.93 (0.72 to 1.20)	910 (266 to -645)
Pneumococcal bacteraemia	3	927	1.28	0.66	0.49 (0.12 to 1.96)	161 (53 to -157)

All-cause pneumonia

There were 480 people with all-cause pneumonia in 6,514 people in four studies (Figure). Vaccination reduced the incidence non-significantly from 7.56% to 7.18%. The number of people needed to be treated to prevent one all-cause pneumonia was 260, but the confidence interval included vaccination causing more pneumonia.

Figure: Randomised trials of pneumococcal vaccines with all-cause pneumonia as outcome

Pneumococcal pneumonia

There were 368 people with pneumococcal pneumonia in 21,156 people in six studies. Vaccination reduced the incidence from 1.89% to 1.59%. The number of people needed to be treated to prevent one case of pneumococcal pneumonia was 325, but the confidence interval included vaccination causing more pneumococcal pneumonia.

Pneumococcal death

There were 230 people who died because of pneumococcal pneumonia in 22,559 people in eight studies. Vaccination reduced the incidence non-significantly from 1.1%to 1.0%. The number of people needed to be treated to prevent one death from pneumococcal pneumonia was 910, but the confidence interval included vaccination causing more deaths.

Pneumococcal bacteraemia

There were nine people with pneumococcal bacteraemia in 927 people in three studies. Vaccination reduced the incidence non-significantly from 1.28%to 0.66%. The number of people needed to be treated to prevent one case of pneumococcal bacteraemia was 161, but the confidence interval included vaccination causing more pneumococcal bacteraemia.

Comment

There are two main conclusions to be drawn from this. The first is that the weight of evidence is that polysaccharide pneumococcal vaccines have yet to be shown to work in the types of people given them in industrialised countries. The only real evidence that they do comes from two improperly randomised studies from the 1940s. A similarly improperly randomised but large study done in the 1990s in Finland showed no benefits. Much resource is put into increasing pneumococcal vaccination in at-risk groups. Perhaps it is time to consider whether this is doing more harm than good.

The other point is one of consistency in systematic reviews and meta-analysis. The review of breast cancer screening made great play of improper randomisation affecting results. This review of pneumococcal vaccination [1] depended largely on data from improperly randomised trials for its positive conclusions. Part of the ethos of evidence-based medicine is not choosing trials to suit your own biases, but this can be hard.

There are perhaps two other useful insights here. This study cut-off its search in 1996, but was published more than three years later, by which time six more studies had been published which may have been included. Out-of-date information may be dangerous, as here, when all six recent studies showed pneumococcal vaccines to be ineffective. The second insight is to beware of studies that report treatment effects in relative terms - an X% reduction in risk. To be properly informed we need to know the absolute reduction. Only then will we have any idea how much effort we have to expend to get a particular result.

Reference

BG Hutchison et al. Clinical effectiveness of pneumococcal vaccine: meta-analysis. Canadian Family Physician 1999 45: 2392-93.

previous or next story in this issue