post by Paul Kelleher
In November 2008, Joshua T. Cohen, Peter J. Neumann, and Milton C. Weinstein published a Perspectives essay in the New England Journal of Medicine titled, "Does Prevention Save Money? Health Economics and the Presidential Candidates." The essay claims to provide evidence in support of the thesis that prevention is not more cost-effective than treatment. The authors summarize their evidence in the following graph:
Here is their main claim:
We analyzed the contents of the Tufts–New England Medical Center Cost-Effectiveness Analysis Registry...Each registry article estimates the cost-effectiveness of one or more interventions as the incremental costs (converted here to 2006 U.S. dollars) divided by the incremental health benefits quantified in terms of quality-adjusted life-years (QALYs)...Our analysis was restricted to the 599 articles (and 1500 ratios) published between 2000 and 2005 that properly discounted future costs and benefits. We classified 279 ratios as preventive because they refer to interventions designed to avert disease or injury; all 1221 other ratios pertain to treatments...The bar graph shows that the distributions of cost-effectiveness ratios for preventive measures and treatments are very similar — in other words, opportunities for efficient investment in health care programs are roughly equal for prevention and treatment, at least as reflected in the literature we reviewed. Moreover, both distributions span the full range of cost-effectiveness.
Put simply, the authors claim that for every preventive intervention, there is a treatment intervention that is equally cost-effective and which therefore provides equal "opportunit[y] for efficient investment in health care programs."
This conclusion, however, rests on a misuse of the cost-effectiveness data reported by the studies the authors draw on. The data represented in the bar graph are, as the authors note, incremental cost-effectiveness ratios. Each ratio (e.g. $50,000/QALY) expresses a comparision between a new intervention and the current standard of care for the same specific health problem--i.e., it is the ratio of the new intervention's additional costs to its additional benefits. To see how this works, consider the following stylized example.
Suppose that some current treatment Tx costs our health plan $4,500,000 per year (total, not per person) and generates 10 QALYs. Now assume that some new and improved treatment Tx* would cost $4,510,000 and generate 11 QALYs. Tx*'s incremental cost-effectiveness ratio is thus $10,000 per QALY (since Tx* uses $10,000 additional dollars and yields 1 additional QALY, when compared to Tx). Tx* would therefore be filed in the "$10,000-$50,000/QALY" column in the NEJM authors' bar graph.
Now consider a second set of interventions, this time preventive interventions aimed at an entirely different health problem.
Suppose that Px is a preventive screening program that is currently offered to just 20% of the population that could benefit from it. Px costs $1 million and generates 50 QALYs. Px* is exactly the same preventive program as Px, except that Px* is offered to 90% of the population. Px* is naturally more expensive than Px, but it also provides more health benefit. Px*'s incremental cost-effectiveness is $10,000 per QALY, since Px* costs $4.5 million more than Px, but it also generates 450 more QALYs.
Now do you see the problem? If we use the methodology underlying the NEJM paper, we will conclude that Tx* and Px* are equally cost-effective and thus equally efficient. After all, each program would be filed in the "$10,000-$50,000 per QALY" column in the NEJM paper's bar graph, since each has an incremental cost-effectiveness ratio of $10,000. However, these ratios completely mask the fact that if we really wanted to get the biggest bang for our buck, we would not even consider funding Tx or Tx*; we would instead defund Tx and devote its $4.5 million budget to program Px*. The NEJM authors, by contrast, give us the impression that efficiency could be equally well served by Tx* and Px*. In their words: "Opportunities for efficient investment in health care programs are roughly equal for prevention and treatment, at least as reflected in the literature we reviewed."
This is not to say that incremental cost-effectiveness ratios are always the wrong tools to use to assess efficiency in health care. It all depends on the question one wishes to answer and the programs being compared. For example, using the numbers provided by Hershey et al., the cost of annual PAP screens per 1,000 women is $1,093,000, with an expected health benefit of 27.6 life-years saved. By comparison, the cost of screening every three years is $467,000, and this program saves 26.8 life-years. So screening every year is more expensive, but also saves more life years. To compare these two possibilities, it is indeed useful to construct the incremental cost-effectiveness ratio of annual screening, which turns out to be $782,500 per life year saved. That is a lot of extra money just to save one life-year, and this may well get us to think hard about whether the extra money is worth it.
So incremental cost-effectiveness ratios have their place. But they absolutely should not be used as Cohen, Neumann, and Weinstein used them. The reason is simple: incremental cost-effectiveness ratios can mask the underlying inefficiency of comparator programs used to construct the ratios. Px is much more efficient than Tx, but this fact is completely hidden when analyses of efficiencies focus exclusively on the incremental costs and benefits of Tx* and Px*.