Something Not Unlike Research

23.5 hours a day

post by Bill Gardner

It's New Year's Eve, so my thoughts about prevention are about my health, not population health. I am old enough that chronic illness is a fact, not a risk. So my goal is to improve the management of the ones I have, and make sure that I do not pick up anything new. Here is a terrific argument for an important behavior change: getting at least half an hour of exercise a day.  Please, take 10 minutes and watch this great talk by Dr. Mike Evans.

Sometimes, getting unstuck just requires reframing the problem.

"Can you limit your sitting and sleeping to 23 & 1/2 hours a day?" 

Yes. I can.

Health services research: After the money's gone

post by Bill Gardner

My recent post on the reduction of grantmaking by the Agency for Health Research and Quality must have hit a nerve, based on how frequently the link was retweeted. Many health services researchers make their living by winning research funding competitions. As a PhD, you cannot aspire to a tenured position in a school of medicine or public health without some victories. But you can't win a competition that isn't being held.

So if the money's almost gone, and you are an American health researcher, and you do not yet have a secure position, what should you do?

Consider working for  an accountable care organization. They need your skills. As Elliott Fisher, Mark McClelland, and Dana Safran write in the current NEJM,

Even integrated [ACOs], however, are finding that they need substantial time and resources to develop the informational, technical, financial, and professional capabilities required to provide and reward coordinated, longitudinal, population-based care. 

The emergence of the ACO is the best attempt yet by Americans to achieve the goals of population health. Implementing better care might be a better job than the one you thought you were training for.

A worse than useless survey

post by Paul Kelleher

Robin Hanson points us to a forthcoming paper in Economics Letters by Olof Johansson-Stenman that purports to "provide survey-based support for the consequentialist assumption associated with welfare economics." Consequentialism is the normative view that an act or policy is right if and only if it maximizes the good (i.e. the net balance of good over bad). As such, it treats an act's rightness or wrongness as solely a function of outcomes. Modern welfare economics is consequentialist in that it aims to be a science of efficiency focused on welfare-maximization. But then again, plenty of welfare economists insist that what they study is but one sphere of moral investigation, and they would not agree that an interest in efficiency entails a single-minded interest in it.

Johansson-Stenman sent his survey to 2,450 Swedes over the age of 18 in the Spring of 2004. Forty-five percent responded (n=985). Here is the main question asked, together with the respondents' answers (click the image to enlarge it):

According to Johansson-Stenman, "the result is quite consistent with the consequentialist ethics underlying conventional economic welfare theory, since almost two-thirds chose this alternative." But this is an illegitimate interpretation of his results. As noted, consquentialism is the view that rightness and wrongness depends only and exclusively on the outcome. The survey, however, only asks people what they think the badness of an outcome "depends primarily on." So for all we know, the 67% who gave what the author characterizes as "the" consequentialist answer may actually agree with those welfare economists who are not consequentialists at all because they believe that other moral considerations (e.g. individual rights and liberties) matter too.   

The survey also makes the fatal mistake of asking its main question in terms of "badness" rather than "rightness" or "wrongness." To see why, consider this passage from a paper (pdf) by Frances Kamm, philosophy's leading non-consequentialist:

I shall distinguish between goodness, fairness and justice. To make one distinction clearer, consider the following case: A doctor must decide whether to stop a big pain in person A or a small pain in person B. She thinks, correctly, that she will do more good if she helps A. But she also remembers that yesterday B suffered a much bigger pain than A will suffer and no one helped B (A suffered nothing in the past). So she thinks it would be unfair to let B suffer again, even though she will do less good if she helps him. If it is overall right to do this, this means she does the morally better thing in helping him and the state of affairs in which B is helped rather than A is morally better than one in which A is helped. But this is not because it produces more good.

Here Kamm claims that sometimes considerations of fairness can conflict with considerations of goodness and badness. But note that someone using Kamm's conceptual scheme might well have given the survey answer that Johansson-Stenman associates with consequentialism. For the survey asks respondents to indicate which considerations they associate with "badness," and the simple (but not simplistic) Kammian answer is that welfare-related considerations are associated with badness, while fairness or equity considerations must be called in alongside welfare considerations to determine which action is "overall right" or "morally better." So it is quite possible that many survey respondents took themselves to be answering a question about only one part of morality, viz. badness, while also holding beliefs about other parts not asked about.

I conclude, therefore, that this survey is useless in helping to answer the question of whether most people are consequentialists. Indeed, since its publication has already led one influential writer (Hanson) to claim that "Most are consequentialist," I fear that it's actually worse than useless.  

Outcomes Measurement and Intellectual Property Rights

post by Bill Gardner

This is a message for health researchers or psychometricians who develop diagnostic instruments or patient-reported outcomes measurements, and for clinicians and health organization leaders who use them: 

Please read Newman and Feldman's article in today's NEJM on "Copyright and Open Access at the Bedside."

The authors recount the intellectual property (IP) history of the Mini-Mental State Exam (MMSE). The MMSE has been under copyright since it was developed, but was widely and freely used. The authors of the MMSE have more recently sought to profit from their IP. Moreover, they have raised copyright challenges to an alternative open access mental status exam. They are impeding the use of standardized mental status exams, and if so they may be harming patients.

Newman and Feldman argue for an extension of the open source copyleft approach to IP to instruments:

Copyleft is a general method for making a program (or other work) free, and requiring all modified and extended versions of the program to be free as well.

Most instruments have no commercial prospects and attempting to sell them will just waste your time. For most instrument developers, copyleft is simple common sense.  

There is, however, an interesting counter example. If you have anything to do with child mental health, you know about the Achenbach Child Behavior Checklist (CBCL) and its many sister instruments. Achenbach charges for his instruments and protects his IP. Nevertheless, the CBCL has been cited in thousands of research projects and is routinely used by many clinicians, even though there are competitors that can be used for free. Why has the CBCL flourished? Achenbach has consistently developed and improved the instruments. 

I believe that the future of mental health assessment and patient-reported outcomes is in computerized adaptive tests (CATs) that are continuously improved as assessment science improves and the prevalence of disorders changes. (I better believe this, since I am a PROMIS co-investigator.) To realize this future, we will need to identify the IP and business model to support an on-going non-profit organization that develops instruments and can deliver them to clinicians and patients. There is clearly much to learn from the free software movement (e.g., here or here).

AHRQ grant making: Is the end near?

post by Bill Gardner

From an email newsletter from the Agency for Health Research and Quality (AHRQ):

2. AHRQ Funding Opportunity Announcements

AHRQ suspends the submission of individual career development grant applications (K01, K02, and K08) for AHRQ Funding Opportunity Announcements (FOA's) for the February 12, 2012 application receipt date for new applications and March 12, 2012 receipt date for re-submission applications.

'K' awards are training awards for scholars seeking to begin research careers. This is an announcement of the end of these opportunities. Also,

Effective with grant application receipt dates after February 5, 2012 (March 5, 2012 for R01 resubmissions), AHRQ is implementing a budget limit of $250,000 total (direct and indirect) costs per year for large research (R01, R18) grant applications.

Federal research grants come in two parts: direct and indirect cost funding. The direct costs are the funds that investigators can actually spend, including the salaried time that the investigator devotes to the project. The indirect costs are the overhead costs charged by the university or research institute (paying for the lights, heat, accountants, etc.). Indirect costs are (controversially) often equal to or greater than the direct costs. In any event, this means that the limit on annual direct costs is something like $125,000. If the investigator earns $100,000 a year -- many medical school professors earn more -- and devotes one-third of her time to the project, then remaining expenses of the research project are less than $87,000 a year. Relatively few health services research projects can be run on such a budget, the exception being statistical analyses of existing data sets.

This announcement is the harbinger of a fundamental reduction in the funding of research on the quality and effectiveness of health care in the United States. Some of the slack will be taken up by the new Patient-Centered Outcomes Research Institute (PCORI). However, should the Republicans win the presidency and the Senate in the next election, PCORI and possibly AHRQ itself may be unlikely to survive the spring of 2012.

Is AHRQ just the canary in the coal mine? Will similar announcements from NIH follow? 

How the hive mind is built: On the sentence, "Descriptive statistics is not just for losers"

post by Bill Gardner

Or perhaps I should have titled this, "How the social media are building a distributed, collective, and associative memory." 

Yesterday I was thinking about the problems of implementing evidence-based clinical practices in medicine and mental health, stimulated by posts on pilot interventions by Megan McArdle and Harold Pollack. A chain of searches that I cannot now reconstruct lead me to a great post by Andrew Gelman, with a great sentence, "Descriptive statistics are not just for losers." The point I took from his sentence is that although causal inference in social science is paramount, it is not everything. The collection of data that identify an unanticipated correlation also matters, even if we do not yet understand the mechanisms that produce that correlation.

I love to tweet great sentences, so I sent that out with a link to Gelman's post. Harold Pollack liked the sentence -- probably not knowing that I had been thinking about his writing when I tweeted it -- and he retweeted my tweet. Someone else in the twitterverse eventually read the tweet and liked the sentence enough to write to Gelman about it.

Gelman, curiously, had forgotten that he had ever written that post, but searched his blog, and discovered that he too liked the post.

I thanked my correspondent for reminding me of this little article I’d forgotten, and he told me he just learned of it via someone’s tweet.

He then wrote a post about rediscovering his post, and speculating that 

Maybe I should have a twitter feed of nothing but old blog entries. I could just go back to 2004 and then go gradually forward, tweeting the items that I judge to remain of interest.

Which is not a bad idea. Google Reader brought me to Andrew's post about rediscovering his earlier post. So we had a chain of communications across a couple of media that originated in my tweet and completed a circuit coming back to me.

This circuit was a rumination in the 'hive mind' about data and its role in clinical or educational practice change. Let's cash out the 'hive mind' metaphor as the vast neural net of associations between concepts implemented most directly through hyperlinks (the image at the top is a representation of del.icio.us). Let's think of the circuit as an activation spreading through the net. Then what we accomplished through this circuit of tweets and posts was strengthened the probability that people writing/searching/thinking about implementation science will come across Gelman's meta-statistical thinking. If that probability has increased, then the global mind got a little smarter.

 

Despair: The NIMH Top 10 Discoveries

post by Bill Gardner

Dr. Thomas Insel, the Director of the National Institute for Mental Health (NIMH), lists the top 10 advances in the science of mental health in 2011.

1. Complexity: Discovering New Sources of Genetic Variance.

2. Transcriptome: Developing Brains Have Unique Molecular Signatures.

3. Induced Pluripotent Stem Cells: Disease in a Dish.

4. De Novo Genetic Variants.

5. Epigenomics: How Experience Alters Behavior.

6. Grand Challenges in Global Mental Health.

7. Precision Medicine.

8. National Database for Autism Research.

9. New Culture of Discovery: Team Science.

10. Public Private Partnerships.

 There is fascinating science in this list. But there are no advances in effective treatments for mental illness. And it has been a long time since we have seen such a discovery. I do not think that NIMH has simply ignored important discoveries of new treatments -- I certainly did not hear of any in 2011. Perhaps in 20 years epigenomics (or one of the other 10 advances) will be seen as the foundation of truly effective mental health treatments. However, a list of 10 advances that provide no direct benefits to patients is just sad.

I mean no criticism of NIMH. I have no idea what they should do to get us better treatments faster. 

 h/t to David Ahern for the pointer to the list.

Future Medicare Cost Growth, the Baby Boom, and the Baby Bust

There is an informative and thoughtful article in today's Washington Post by Lori Montgomery on the current and future cost growth of Medicare.

The key facts are in the graph to the right. On the left is an orange bar graph with the good news: per capita Medicare spending has been falling recently. The line graph in the center is the bad news: Medicare enrollment is increasing, and it is increasing rapidly as we baby boomers retire. The net effect, on the far right, is a projected continuing increase in Medicare costs.

But my friend David States pointed out to me that we might look more closely at the projected growth in Medicare enrollment. To the right are data on the proportion of the US population that is 65 or older, with projections into the future (taken from the CDC). My graph is covering most of the same period as Montgomery's. But perhaps because she uses a different projection, or simply because I am plotting more points, it has a different shape. Notice first the rapid increase in this proportion as the Baby Boom retires. This is what will drive the expansion of Medicare costs, even if cost / retiree is stable. After 2030, however, the rate of growth of the elderly as a percentage of total population slows and then stops. As the CDC authors note:

...when the last Baby Boomers enter the ranks of the older population and the first Baby Boomer cohort enters the oldest old age categories, the proportion aged 65 and older will be relatively stable at around 20 percent.

This will happen because the baby boom was followed by falling birthrates (the "baby bust").

Let's not count on this, because long-term demographic projections are often wrong. Moreover, the percentage of very old (and very expensive) persons within the elderly may continue to increase. But suppose, for the moment, that the CDC is right.

There may be a ray of hope here. We need to find ways to hold down the growth of per capita Medicare spending, We need to do this by transforming care to make it more cost-effective. But if* we can find a way to control the growth in spending per retiree, and demographic pressure ends, we will have solved the problem of growth in Medicare cost.

*This is, as they say, a big IF.

Pay for Performance in US and Canadian Health Care

post by Bill Gardner

Pay for performance is a scheme in which health care provider reimbursement is tied to meeting health care quality standards. In the US, I work for a children's hospital that has formed a large regional accountable care organization (ACO, very roughly, a scheme for organizing care that ties reimbursement to meeting quality goals for a defined population). The ACO is currently negotiating an alliance with similar systems caring for contiguous regions, in an effort to gain the organizational scale necessary to successfully implement ACO initiatives. 

In many ways, the Canadian children's health care organization I work for is ahead of the US. It already cares for a defined population. Each child has a permanent health care identifier, every child is covered, and the province is the only payer that matters, so regional health care data are not divided into the silos of competing insurers. In short, we have the Care Organization. What's missing is the Accountability.

I don't mean that providers in our province are negligent. What I mean is that despite our public commitment to population health goals, and lots of long committee meetings about it, providers are unlikely to be held to measurable performance targets. Pay for performance is not much used in Canada. So, the incentive structure is much like the Dilbert cartoon -- trying to do a better job just means more work -- except that the hospital CEO will never see a bonus. This means that health care compensation is, by my lights, fairer in Canada than the US. But so far at least, I also see less effort at innovation. Keep in mind that I only see Atlantic Canadian care, things may be different in Manitoba.

So why is accountable care lagging in Atlantic Canada? Part of it may be the difference between the US and Canadian organizational cultures. But the more important factor is scale. The Province of Nova Scotia has about the same population as, say, metropolitan San Jose (in the land area of West Virginia -- and NS is one of the densely populated provinces). Imagine San Jose trying to mount the data and analytic infrastructure to run a single payer health plan and then guide care using research. To move to accountable care, Canada needs to build a federal infrastructure for health data and comparative effectiveness research.

The scary thing is that, so far, there is little evidence that pay for performance actually works. It's up to those of us working on ACOs to change that.

thanks to Tim Harford for the link to Dilbert.

Quality improvement and the savings illusion

post by Bill Gardner

Thanks to Don Taylor for pointing me to an important essay in the NEJM by Stephen Rauh and his colleagues on "The Savings Illusion — Why Clinical Quality Improvement Fails to Deliver Bottom-Line Results." Rauh et al. argue that, contrary to widespread hopes,

true bottom-line savings from improved clinical quality rarely materialize, and costs continue to climb. Manufacturing and service companies around the world have demonstrated the cost benefits of improving product quality and production efficiency. So why haven't nearly two decades of work on improving health care quality had a measurable effect on health care costs?

The reason, they argue, is that the cost of delivering health care is mostly fixed. Doing one less MRI (the image above is a magnetic resonance angiograph) saves you some electricity and billing costs. However, you still need to pay for the machine and the specialists who can read the images. This means that in the short run not running the machine doesn't save much. Worse, from the point of view of a hospital, it leaves money on the table (as it were), because the marginal cost of imaging one more patient is significantly less than what the hospital can charge for it. You may feel that the hospital should leave those dollars on the table. But it won't.

So how do we improve quality and reduce cost? Perhaps, as Reihan Salam argues, competition between integrated health care systems -- where the insurer and the provider are the same entity -- could create incentives for providers to forego unnecessary MRIs. I hope so, and I am working with my Ohio hospital as it seeks to transform itself into an integrated provider. But instead of hope, I would rather have data indicating this will work.

Rauh et al. assert that

The most meaningful way to achieve savings is to focus on overall reductions in utilization rates for health care services and to eliminate the associated unnecessary capacity.

They don't, however, say how this is to be accomplished.

This can be done if you have a single payer. It is not fun. Here in Nova Scotia, the NDP government -- the socialists, mind you -- are reducing our hospital's budget by 3% this year. Because we cannot reduce our labor costs or physical plant -- at least not in the short run -- this will result in (at least) a 3% reduction in imaging. No hope required. But we do need to hope that this will be a reduction in unnecessary MRIs, rather than ones that women and children really need.