Something Not Unlike Research

Don Berwick and the lives not saved

Please read Harold Pollack's post on what Donald Berwick's resignation as head of CMS (the Center for Medicare and Medicaid Services) says about the lack of professionalism in US government. For a conservative view on the current 'comical' lack of professionalisim, see Tevi Troy. 

One of Harold's sentences stands out for me:

His methodical research and practice have saved many thousands of lives.

Harold is referring to what I think is the most important part of Berwick's career. He is a physician, and he will probably return to being a professor, but he was above all a reformer of the practice of medicine. Not a political reformer, but rather someone who ran campaigns that have saved lives by changing how medicine is practiced. These were not campaigns to redistribute medical care, or change how medicine is paid for. The goals were to change physician behaviors that killed patients, by persuading doctors to:

• Deliver Reliable, Evidence-Based Care for Acute Myocardial Infarction
• Prevent Adverse Drug Events
• Prevent Central Line Infections
• Prevent Surgical Site Infections
• Prevent Ventilator-Associated Pneumonia

These are practical, not ideological goals. Like Harold, I am persuaded that Berwick's efforts prevented tens of thousands of deaths stemming from substandard US medical care. This is the strongest possible credential for the administrator of Medicare and Medicaid.

What might Berwick have done with eight years to promote quality of care as head of CMS? Based on his record, he would have saved lives.

More Data, Obviously

In a recent post, I noted that it is a problem that the US Preventive Services Task Force frequently finds that it has insufficient evidence to grade the effectiveness of a preventive service. I argued that the best improve the decision making of the USPSTF was to get more evidence by doing more comparative effectiveness studies.

"Duh," sez the thought balloon above your head.

Or as Daniel Goldberg said more politely in the Comments,

this is simply a well-studied feature of the problematic that is evidence-based public health policy. Namely, that we frequently (usually?) lack evidence of reasonable, let alone high, quality that would be sufficient to guide us with a modicum of confidence on any particular public health policy question.

No one disagrees that the best answer to this problem is to obtain better evidence of sufficient quality, but that does not address the real-time problem that the data simply may not exist at the time a decision needs to be made.

In the next post, I will discuss Daniel's absolutely real "problem that the data simply may not exist at the time a decision needs to be made."

But first, is it true that "[n]o one disagrees that the best answer to this problem is to obtain better evidence"? My strong impression is that most if not all Republican legislators and pundits would resist legislation to increase federal funding for comparative effectiveness research on preventive services. We have to speak up for more data.

 

Expert Opinion and the US Preventive Services Task Force

Scott Gottlieb is a physicisian and policy analyst writing for the American Enterprise Institute. He has written a thoughtful critical review of the functions that the USPSTF is likely to serve under the Affordable Care Act, and considers how it should be structured given those functions. His briefing completely avoids the defamatory rhetoric of Steve Forbes and Newt Gingrich. More importantly, it raises important issues about the USPSTF that merit extensive public discussion.

I want to look at just one point here: What role should "expert opinion" play in the recommendations of the USPSTF? Gottlieb writes,

The sixteen advisors appointed to serve on the USPSTF are mostly primary care physicians, generally experts in preventive and public health. However, as clinical generalists, they rarely have deep expertise in the discrete medical disciplines in which they are asked to pass judgment, such as oncology or infectious diseases.

To fix this, Gottlieb argues that

Congress also should make sure that recommendations issued by the USPSTF are in sync with sister public health agencies that have far more expertise in the domains in which they operate. These include the CDC,the National Institutes of Health, and the FDA. The USPSTF lacks the capacity of these other agencies, and as such, its analysis should not supersede their expert opinions.

I agree that it would be good to include more specialists on the USPSTF. In part, this is because I think that the panel should be expanded, so that it can speed up its reviews.

But there are several problems here. First, we want to be very careful what role expert opinion should play the USPSTF evaluations. "Expert opinion" is a term of art in the world of evidence-based medicine. It means, roughly, "what experts think be done in the absence of sufficient empirical data." It is too simple to say that medical progress is the replacement of expert opinion by evidence-guided clinical judgment, but that is a good first approximation. The USPSTF's role has been to follow procedures derived from evidence-medicine, to give an objective scoring of the evidence. It's important that the USPSTF continue to require data supporting the value of preventive services, rather than just expert opinion. Gottlieb may well agree with me about this, but I want to stress it because of unusual meaning of "expert opinion" in this game.

Second, we want to maintain the USPSTF's independence from institutions such as the FDA and the NIH. We want this for the same reason that we want judges to be independent, accountants to be independent, and the FDA itself to be independent -- it is essential if we want the evaluation to be objective. 

Finally, an adequately resourced USPSTF would be able to evaluate whether treatments work without the need to rely on expertise from the NIH. A certain level of expertise is required to read and critically evaluate the research on a given treatment or service. Beyond that, however, you do not necessarily need to how treatment X is supposed to work to determine whether treatment X actually does work. 

Despite Steve Forbes, the US Preventive Services Task Force is not a death panel

post by Bill Gardner

Steve Forbes claims that the USPSTF is "The Department of Health and Human Services Death Panel." In a discussion of the Task Force's recommendation about prostate cancer screening, his column makes several false statements. One could ignore it, but Forbes tells us that it has been viewed more than 65,000 times, and it repeats a false description of the Task Force by Newt Gingrich on national TV. One could laugh at it, as Andrew Gelman does, but readers unfamiliar with the evidence may not get the joke. So, here are some facts about the Task Force and its recommendation.

• The Task Force evaluates the evidence about medical interventions. It does not make recommendations about the treatment of individual patients. 
• The Task Force does not make "Soviet style" decisions about what doctors and patients can do. A negative evaluation by the Task Force does not mean that you cannot get the treatment. Despite the Task Force recommendation, any man who wants to be screened for prostate cancer can be.
• The Task Force does not consider cost in making recommendations. It evaluates the evidence for whether interventions are beneficial, and the certainty of that evidence. That's it.
• The Task Force does not base its recommendations on "tortured reasoning." They are based on empirical evidence, wherever possible the results of randomized controlled trials. In the case of prostate cancer screening, experiments involving hundreds of thousands of men failed to show evidence that screening saved lives.

Of couse, the USPSTF's recommendation could be wrong. If so, that could cost lives. That is what is at stake in medicine. Therefore, public debate of USPSTF recommendations is a good thing. False and, in my opinion, malicious characterizations of the Task Force make it more difficult to base medical decisions on the evidence.