Something Not Unlike Research

Faking your on-line ethics training

post by Bill Gardner

The research ethics training program I am completing makes me repeatedly affirm that I, myself, am the person reading the text and answering the quizzes. I infer that a significant number of academics are delegating these tasks to a secretary or research assistant.

If you have someone else impersonate you for your ethics training -- in other words, commit fraud to obtain your ethics certificate -- I'm not sure how much the training will benefit you.

On "philosophical or religious bias" in scientific reviews

post by Bill Gardner

I am working through an on-line research ethics training course. No one enjoys these exercises, but I don't have a better scheme for promoting research ethics. And not all of it is tedious: I have just finished watching an excellent brief video debate on how to address financial conflicts of interest in research involving David Rothman, Nancy Dubler, and Thomas Murray.

But I was struck by this question on a module's test:

Which types of circumstances might bias the review of a manuscript or grant application?

The 'correct' answer was:

The reviewer may have a philosophical or religious bias against some types of research (e.g., the use of non-human primates or human embryonic stem cells in research).

Is this answer correct? The notion of bias assumes that competent experts should roughly agree on the merits of a piece of work. This is at least partially true for the technical aspects of science. So ethical or religious bias is possible in reviews. For example, I might (through a 'disconfirmation bias') grade the statistical arguments in a proposal more harshly if I had a philosophical or religious objection to the research.

Reviews, however, involve an evaluation of the technical merit, the significance, and the ethics of a project. In clinical research, significance eventually cashes out as consequences for human well-being. (In basic science it may mean something like the implications of the work for our understanding of nature.) However, judgments about consequences for human well-being have, eventually, to be defended using philosophical or religious arguments. This is even more true for ethical evaluations of research. So when reviewing the significance or ethics of research, the notion of a philosophical or religious bias puzzles me.

How serious is the problem of bias in technical evaluations of research? It may not affect my grading too much, for a possibly idiosyncratic reason. I think I grade proposals by combining evaluations of technique, significance, and ethics multiplicatively, not additively, with zeros at the threshold of unacceptability.* So a proposal with zero technical merit, good ethics, and high significance gets a zero overall score (rescaled as required by the institution for whom I am grading). Similarly, if I think the research is at or near the point of zero significance or if I had a strong ethical objection to the research, any bias in my technical review would be irrelevant to my overall grade.

*The NIH asks us to consider certain ethical issues separately from the overall grading, and I do that. Also, I don't formally multiply scores together, this is my psychological model for my judgments in reviews.

On stigma, mental illness, and treatment

post by Bill Gardner

Fighting the stigmatization of mental illness is a major goal of the population health movement. And I fully support this goal. But I have a suspicion that real progress will come when we develop truly powerful treatments. Cancer was once highly stigmatized. Is it an accident that it has become less stigmatized following the development of interventions that can cure some cancers? 

Don Berwick and the lives not saved

Please read Harold Pollack's post on what Donald Berwick's resignation as head of CMS (the Center for Medicare and Medicaid Services) says about the lack of professionalism in US government. For a conservative view on the current 'comical' lack of professionalisim, see Tevi Troy. 

One of Harold's sentences stands out for me:

His methodical research and practice have saved many thousands of lives.

Harold is referring to what I think is the most important part of Berwick's career. He is a physician, and he will probably return to being a professor, but he was above all a reformer of the practice of medicine. Not a political reformer, but rather someone who ran campaigns that have saved lives by changing how medicine is practiced. These were not campaigns to redistribute medical care, or change how medicine is paid for. The goals were to change physician behaviors that killed patients, by persuading doctors to:

• Deliver Reliable, Evidence-Based Care for Acute Myocardial Infarction
• Prevent Adverse Drug Events
• Prevent Central Line Infections
• Prevent Surgical Site Infections
• Prevent Ventilator-Associated Pneumonia

These are practical, not ideological goals. Like Harold, I am persuaded that Berwick's efforts prevented tens of thousands of deaths stemming from substandard US medical care. This is the strongest possible credential for the administrator of Medicare and Medicaid.

What might Berwick have done with eight years to promote quality of care as head of CMS? Based on his record, he would have saved lives.

More Data, Obviously

In a recent post, I noted that it is a problem that the US Preventive Services Task Force frequently finds that it has insufficient evidence to grade the effectiveness of a preventive service. I argued that the best improve the decision making of the USPSTF was to get more evidence by doing more comparative effectiveness studies.

"Duh," sez the thought balloon above your head.

Or as Daniel Goldberg said more politely in the Comments,

this is simply a well-studied feature of the problematic that is evidence-based public health policy. Namely, that we frequently (usually?) lack evidence of reasonable, let alone high, quality that would be sufficient to guide us with a modicum of confidence on any particular public health policy question.

No one disagrees that the best answer to this problem is to obtain better evidence of sufficient quality, but that does not address the real-time problem that the data simply may not exist at the time a decision needs to be made.

In the next post, I will discuss Daniel's absolutely real "problem that the data simply may not exist at the time a decision needs to be made."

But first, is it true that "[n]o one disagrees that the best answer to this problem is to obtain better evidence"? My strong impression is that most if not all Republican legislators and pundits would resist legislation to increase federal funding for comparative effectiveness research on preventive services. We have to speak up for more data.

 

Is Rawls Relevant in a Warming World?

post by Paul Kelleher

The short version: Tim Mulgan thinks John Rawls's A Theory of Justice has limited value for thinking about moral issues associated with climate change. I disagree. I describe the role that discounting plays in cost-benefit analyses of climate change, and argue that Rawls's discussion of discounting is useful and of lasting importance. 

Philosopher Tim Mulgan thinks that John Rawls may go the way of "paisley shirts and psychedelic music":

[O]ptimistic assumptions allow political philosophers to ignore the future. Consider Rawls himself.  His magnum opus (A Theory of Justice) devotes just ten pages to justice between generations; and even then his focus is tellingly limited.  As we accumulate wealth, we must decide how much to consume for ourselves and how much to invest for the future.  For Rawls, the only ethical question is: How much better-off should we leave our descendants?

Rawls wrote in 1971. Like paisley shirts and psychedelic music, his complacent approach has not aged well.  We no longer take it for granted that we will leave our descendants better-off, or even that we can.  And future people are no longer an afterthought.  They have moved centre stage, both in popular debate and in moral philosophy.

Mulgan may well be correct that the majority of A Theory of Justice does not speak to the issue of inter-generational justice. Nevertheless, I believe that what Rawls does have to say is of lasting importance. Consider the seemingly academic issue of discounting. In cost-benefit analyses of climate-related plans and projects--and in cost-benefit analysis generally--it is common for economists to discount the value (or disvalue) of goods (or bads) that occur in the future. Depending on the discount rate used, we--here and now--can have stronger or weaker duties to prevent health shocks that will occur in the far future. If we discount the goodness of future health and the badness of future ill-health, then it seems to follow that we have commensurately less pressing reason to prevent those shocks from occurring. After all, no one thinks that it's morally required for current generations to take on great burdens to prevent one hangnail in 200 years. Since the nature of our obligation clearly turns on the size of the future benefit or burden we're talking about, discounting has the potential to turn a future moral tragedy into something not that important.

The economist and philosopher John Broome illustrates the importance of getting the discount rate right:

Stern's conclusion differs from Nordhaus's principally because, on ethical grounds, Stern uses a lower "discount rate." Economists generally value future goods less than present ones: they discount future goods. Furthermore, the more distant the future in which goods become available, the more the goods are discounted. The discount rate measures how fast the value of goods diminishes with time. Nordhaus discounts at roughly 6 percent a year; Stern discounts at 1.4 percent. The effect is that Stern gives a present value of $247 billion for having, say, a trillion dollars' worth of goods a century from now. Nordhaus values having those same goods in 2108 at just $2.5 billion today. Thus, Stern attaches nearly 100 times as much value as Nordhaus does to having any given level of costs and benefits 100 years from now.

In a forthcoming short paper on discounting, I describe two different kinds of discounting and the main reasons why one might support them. Here they are in brief:

Kind 1/Reason 1: When individuals save money in bank accounts, they are trading present consumption for future consumption. But many individuals would not do this unless they were given an incentive in the form of an interest rate. Therefore, since individuals need to be in effect bribed to put off present consumption, individuals have a preference for consuming now rather than later. Some economists see this as revealing individuals' ethical opinion that future well-being is morally less important than present well-being.

Kind 1/Reason 2: There is a second reason that some people offer for discounting the moral importance of future benefits. The argument goes like this: unless we discount the value of future benefits (and the disvalue of future bads), current generations will be forced to make unreasonable sacrifices. After all, since what we do here and now will have effects that radiate out in the far future, a large sacrifice now will certainly create benefits in the future that outweigh the sacrifice we have to make. The only way to release ourselves from these surely tyrannical obligations to future generations is to count their well-being for less. Thus, we need to discount to protect ourselves from overdemanding duties.

Kind 2/Reason 1: Any use of current resources has opportunity costs---that is, we could have used them for other purposes, and these other uses may have yielded even greater benefits. Economists in general would like us to put scarce resources to their most efficient possible use. One possible use is a financial investment: we could invest the resources now and let them grow, at prevailing interest rates, to a certain size in the future. If it is impossible to achieve that same sized gain by investing current resources in environmental projects, then funding the environmental project "leaves resources on the table," so to speak. That is, we could give future people a larger benefit if we just invested the money and gave them the pin number to the bank account than if we funded the environmental project. To choose between projects, economists tell us to discount the future environmental benefits at prevailing interest rates. This discounted value is the amount we'd have to invest now to deliver future benefits of the same value as the environmental benefits. If it would cost more to deliver those benefits via an environmental project, then (the economists argue) we should go with the bank account instead.---The perceptive reader will notice that this second reason for "discounting" doesn't support discounting the moral importance of future well-being. Rather, it holds the moral importance of future benefits fixed and simply tells us how to deliver those benefits most efficiently. That is why I (following Dominic Rosser [pdf]) call it "discounting" and why I consider it a different kind of discounting from the first kind discussed above. 

As regards the first reason for Kind 1 discounting, Rawls would surely argue that when individuals are making their own private investment decisions, they should not be taken--nor do they take themselves--to be expressing a moral stance on whether the well-being of future people matters less from a moral point of view. It is Rawls's view that moral deliberation must be explicit and sensitive to specifically moral reasons. At least that's what I take the his argument from the Original Position to entail.

The short discussion of inter-generational justice in A Theory of Justice actually provides an explicit and decisive refutation of the 2nd reason for Kind 1 discounting. Rawls notes that duties to promote future well-being would be tyrannical only if we had a basic duty to maximize well-being. But that duty is questionable, and is only supported by the consequentialist moral framework he is rejecting in the book. The consequentialist framework is, however, the framework that many welfare economists use when constructing cost-benefit analyses, and this is why many of them are led to embrace discounting. If Rawls is correct that we do not have a simple duty to maximize well-being, then Kind 1 discounting is not needed to release ourselves from that duty.

Finally as regards the reason for Kind2 "discounting," Rawls seems willing to grant that it may have something to be said for it. He writes, "[T]he rejection of pure time preference [i.e. Kind 1 discounting]...does not rule out using an interest rate (in either a socialist or a private-property economy) to ration limited funds for investment" (TJ, p. 294 [Original 1971 edition]). In my forthcoming paper I offer some moral reasons for being troubled by Kind2 discounting, but like Rawls I am willing to admit that it may be useful in some cases. 

If Broome is right that the issue of discounting is crucial for properly understanding our duties to future generations, then A Theory of Justice--and especially the ten pages on inter-generational justice--may contain more that is of lasting importance than Mulgan's discussion would suggest.

How to Improve the Decision Making of the USPSTF

Scott Gottlieb points to a significant problem in the US Preventive Services Task Force's decision making: Too frequently, they conclude that the evidence about a preventive service is insufficient to evaluate whether it works. When the USPSTF gives a preventive service an "I" ("Insufficient") grade, insurers may not be willing to cover that form of care. 

So why don't we have enough evidence about the effectiveness of many preventive services? Gottlieb correctly observes that

The USPSTF typically favors large prospective, randomized trials to validate a preventive service. But such research is generally hard to conduct for screening tests. It would require, for example, patients at risk for a particular disease to be randomly selected to either receive a screening test for the ailment or forgo the diagnostic measure. The patients would then need to be followed, sometimes for many years, to see if the tool enabled screened patients to recognize better health outcomes (and lower overall utilization of medical services) than patients who were randomly selected to forgo the screening test. 

That is, it is challenging, expensive, and time consuming to carry out the studies that would really tell us whether preventive services really benefit -- or harm -- the patients who receive it.

So what is Gottlieb's solution to this problem? I find two suggestions: 

1. "Congress also should make sure that recommendations issued by the USPSTF are in sync with sister public health agencies that have far more expertise in the domains in which they operate. These include the CDC, the National Institutes of Health, and the FDA."
2. The USPSTF should be subject to the rules about public comment on regulations that bind other US government regulatory agencies.

I disagree with point 1 but agree with point 2 (and see the Task Force's view here), even though point 2 will slow the speed at which the Task Force issues recommendations. 

Please notice, however, is that points 1 and 2 are just requirements that the Task Force take into account the opinions of experts in other health agencies, and public opinion. But gathering more opinions just increases the amount of opinion. The fundamental problem, as Gottlieb himself noted, is that we lack data on the outcomes of preventive services. So what we need is more data, not more opinion.

The best way to improve USPSTF decision making is to fund more randomized clinical trials on preventive services, and to fund registries that can follow patients who (and do not) receive these services. Think evidence, not opinion. 

The median New Yorker vs the median Ontarian

post by Bill Gardner

Felix Salmon was shown some income statistics suggesting that the average New Yorker has a lot better income than the average Ontarian: about $25,000 a year better. The average Ohioan earned more too, as did the average North Carolinian.

He was skeptical, and I am too. One reason why is that he feels, as I do, that the median income is a better way to get a sense of how things are for the typical New Yorker, or Ontarian. His chart (above) suggests that a typical Ontarian actually has a good or better life than a New Yorker. The comparison of the average income is distorted, in my view, by the extraordinary incomes of small numbers of bankers, hedge fund managers, private equity managers, and the like who live in New York.

I have just my impressions to go on, but I also think that Canada looks better than the US if you look at the lowest social tiers. Nova Scotia is considerably poorer than Ontario; it would probably make better sense to compare it to West Virginia or Kentucky than New York. There is real poverty here. But I have seen US Appalachian poverty, and it's a lot worse than anything I have seen to date in NS.

 

When the high price of health care is a good thing

post by Paul Kelleher

If an [Implantable Cardiac Defibrillator] were a buck and a quarter, would we have gone through this entire process of reviewing the evidence at all? Maybe not. But because it's a bit more than a buck and a quarter we wanted to make sure the evidence was clear and that it was a benefit. We don't use cost to decide the evidence issue, but we do use cost to decide if this is important enough to address.

That is Dr Steve Phurrough, then-acting director of the Centers for Medicare and Medicaid Services Coverage and Analysis Group, speaking in 2003. I find this quotation shocking, and I wonder if others do as well.

I have no idea if things at CMS have changed since 2003, but this strikes me as illustrating one reason why it can be good for health care to be pricey. I do not claim that this reason makes it true, all things considered, that health care is so expensive. But its just not clear that we would be better off if health care grew on trees. 

Reihan Salam, Paul Ryan, the Safety Net, and Public Sector Profiteers

post by Bill Gardner

Reihan Salam recommends two lines of argument that he attributes to Rep. Paul Ryan:

• "[C]onservatives shouldn’t attack the safety net."
• Rather, they should "Attack Public Sector Profiteers... and the lack of spending restraint that actually exacerbates cost growth in health, education, and other domains."

I agree that conservatives should not attack the safety net, so a good first step would be to just stop doing it. Ryan's proposal would leave the safety with a gaping hole. There are more than 45 million Americans without health insurance. This number will decresease because of the expansion of Medicaid under the Affordable Care Act, but Ryan would repeal that. Among the uninsured, there are healthy Americans who could afford health insurance but elect not to purchase it, and some of them would likely be induced to purchase it through Ryan's tax credit scheme. However, the majority of the uninsured are at less the 200% of the Federal poverty level, and cannot realistically afford private health insurance under Ryan's plan. We do not yet have a real safety net if large numbers of poor families fall through it.

OK, how about "Attack public sector profiteers"? Umm... let's see, which side is promoting class warfare here?  (And how well did that work out?) 

First, what could "public sector profiteers" mean? I think public sector would include not just government employees, but also private sector workers whose income depends on government programs. In Canada this would include nearly all doctors and nurses; both are unionized, even the residents have a union. It includes many health care providers in the states. What does it mean to call them profiteers? I doubt that Reihan Salam is challenging the right of teachers or health care providers to better themselves. "Profiteering" comes from war time, when military suppliers exploit an emergency to get enrich themselves by unethical means.

So what is unethical about how public sector workers pursue their interests? I will stipulate to the following:

1. Public sector demands for income or restructured work conditions sometimes harm the populations they are employed to serve.
2. By voting as a bloc, by making political contributions, by gaining great influence in left political parties, and by cloaking their self-interests under the needs of their vulnerable clients, public sector health care providers achieve considerable leverage over government. When they use that leverage to raise taxes, they are raising their incomes by taking money from taxpayers. 

These are tendentious arguments, but that does not mean they are wrong. On point 1, medical guilds and unions often structure care in ways that increase guild members' incomes but harm patients. If this seems implausible to you, read Atul Gawande's famous article about health care in McAllen, TX. Point 2 is just, I think, a fact.

So why won't I join in an attack on public sector privateers? I think there are cases where teachers unions or health care workers unions take the wrong side of an issue, and there I side with management in the interests of students or patients. But I wouldn't single them out as profiteers. Pharmaceutical and device manufacturers, hospital owners, insurers all pursue policies in their self-interest that compromise the best interests of patients (point 1). And they have vastly greater resources to influence the political process (point 2). So unless we want to routinely refer to all stakeholders as profiteers, it is not clear to me that the word has much application.

Moreover, our views about the motives and moral integrity of stakeholders give us zero useful information about how to reform health care. Consider this argument:  

1. Defense manufacturers and contractors sometimes produce weapons that earn them huge profits, but actually diminish national security and the safety of armed services members. 
2. By making political contributions, by gaining great influence in right political parties, and by cloaking their self-interests under the banner of patriotism, defense manufacturers achieve considerable leverage over government. When they use that leverage to raise taxes, they are raising their incomes by taking money from taxpayers. 

These points are both true, but they do not tell us what weapons we need to purchase, or how high to set the defense budget. "Attack the Defense Profiteers" is not a plan to make America more secure, and a plan to "Attack the Public Sector Profiteers" will not make it healthier.