Something Not Unlike Research

Outcomes Measurement and Intellectual Property Rights

post by Bill Gardner

This is a message for health researchers or psychometricians who develop diagnostic instruments or patient-reported outcomes measurements, and for clinicians and health organization leaders who use them: 

Please read Newman and Feldman's article in today's NEJM on "Copyright and Open Access at the Bedside."

The authors recount the intellectual property (IP) history of the Mini-Mental State Exam (MMSE). The MMSE has been under copyright since it was developed, but was widely and freely used. The authors of the MMSE have more recently sought to profit from their IP. Moreover, they have raised copyright challenges to an alternative open access mental status exam. They are impeding the use of standardized mental status exams, and if so they may be harming patients.

Newman and Feldman argue for an extension of the open source copyleft approach to IP to instruments:

Copyleft is a general method for making a program (or other work) free, and requiring all modified and extended versions of the program to be free as well.

Most instruments have no commercial prospects and attempting to sell them will just waste your time. For most instrument developers, copyleft is simple common sense.  

There is, however, an interesting counter example. If you have anything to do with child mental health, you know about the Achenbach Child Behavior Checklist (CBCL) and its many sister instruments. Achenbach charges for his instruments and protects his IP. Nevertheless, the CBCL has been cited in thousands of research projects and is routinely used by many clinicians, even though there are competitors that can be used for free. Why has the CBCL flourished? Achenbach has consistently developed and improved the instruments. 

I believe that the future of mental health assessment and patient-reported outcomes is in computerized adaptive tests (CATs) that are continuously improved as assessment science improves and the prevalence of disorders changes. (I better believe this, since I am a PROMIS co-investigator.) To realize this future, we will need to identify the IP and business model to support an on-going non-profit organization that develops instruments and can deliver them to clinicians and patients. There is clearly much to learn from the free software movement (e.g., here or here).

How the hive mind is built: On the sentence, "Descriptive statistics is not just for losers"

post by Bill Gardner

Or perhaps I should have titled this, "How the social media are building a distributed, collective, and associative memory." 

Yesterday I was thinking about the problems of implementing evidence-based clinical practices in medicine and mental health, stimulated by posts on pilot interventions by Megan McArdle and Harold Pollack. A chain of searches that I cannot now reconstruct lead me to a great post by Andrew Gelman, with a great sentence, "Descriptive statistics are not just for losers." The point I took from his sentence is that although causal inference in social science is paramount, it is not everything. The collection of data that identify an unanticipated correlation also matters, even if we do not yet understand the mechanisms that produce that correlation.

I love to tweet great sentences, so I sent that out with a link to Gelman's post. Harold Pollack liked the sentence -- probably not knowing that I had been thinking about his writing when I tweeted it -- and he retweeted my tweet. Someone else in the twitterverse eventually read the tweet and liked the sentence enough to write to Gelman about it.

Gelman, curiously, had forgotten that he had ever written that post, but searched his blog, and discovered that he too liked the post.

I thanked my correspondent for reminding me of this little article I’d forgotten, and he told me he just learned of it via someone’s tweet.

He then wrote a post about rediscovering his post, and speculating that 

Maybe I should have a twitter feed of nothing but old blog entries. I could just go back to 2004 and then go gradually forward, tweeting the items that I judge to remain of interest.

Which is not a bad idea. Google Reader brought me to Andrew's post about rediscovering his earlier post. So we had a chain of communications across a couple of media that originated in my tweet and completed a circuit coming back to me.

This circuit was a rumination in the 'hive mind' about data and its role in clinical or educational practice change. Let's cash out the 'hive mind' metaphor as the vast neural net of associations between concepts implemented most directly through hyperlinks (the image at the top is a representation of del.icio.us). Let's think of the circuit as an activation spreading through the net. Then what we accomplished through this circuit of tweets and posts was strengthened the probability that people writing/searching/thinking about implementation science will come across Gelman's meta-statistical thinking. If that probability has increased, then the global mind got a little smarter.

 

Getting it right and being on time

post by Bill Gardner

Austin Frakt responded to a recent post of mine on how to speed up the diffusion of research from the university to policy makers. In my post, I argued that to encourage researchers to communicate their results to decision makers and the public, we need more publications like Health Affairs that have as many as possible of the following attributes:

1. Writing that is conscise, data-driven, and policy-relevant.
2. Good editing and high production values.
3. Quality and prestige through peer review, providing academics with career incentives.
4. Authors and readers from every stratum of the policy discourse world, and every provider / stakeholder discipline (including patients!).
5. Excellent social media marketing to get articles into the discussion.
6. And it must not be the creature of any provider guild or health industry.

Austin agrees, but makes the following crucial observation

I’d add “timely.” Therefore, I would also rethink one of these items: peer-review (#3). To be sure, it’s a crucial element of primary publication of research results. But it takes a long time. Inserting it into a translational, dissemination process risks missing the window of opportunity for maximum impact and utility. Timing is everything.

This is an excellent point. I would say that policy discourse needs contributions from researchers at several velocities, including the very quick response that blogs provide. However, the faster you go, the harder it is to have peer review.

So, can we add timeliness to our web policy journal without sacrificing peer review? No. But let's keep in mind that peer review is not an end-in-itself. The goal is quality and having a reputation for quality (it is the latter that incentivizes academics). Peer review is just a means to those ends.

I have two suggestions. The core of the imagined web policy journal must be peer-reviewed publications so that it remains anchored in science. It could, however, also include some more timely outlets:

1. The journal could publish something like the Policy Forum in Science, and something like the OpEd page of The New York Times. These would be solicited pieces of < 1000 words that appeared perhaps weekly. This would require an editor who could anticipate issues that would be relevant, find authors who could respond on a fast turn around, and give those authors critical feedback to improve their pieces. 
2. The journal could sponsor one or more bloggers who would publish, for limited tenures, under the journal's brand. These pieces would appear without prior editing to achieve immediacy. However, the journal would maintain an editorial board that would be charged with monitoring the blogs and periodically rating their insight, accuracy, and objectivity. The board could replace bloggers who did not meet the journal's standards. 

These schemes would add a measure of review and selection to high velocity writing. Being a policy blogger for (say) Health Affairs would contribute more to an academic's reputation than publishing the same words under her own name. Moreover, having the imprimatur of a prestigious journal would greatly increase the visibility and impact of a blog. Conversely, having processes of article and blogger selection and pre- or post-publication review would help the journal insure that these more timely publications maintain its standards.

Getting research findings to policy makers: The way forward

post by Bill Gardner

The AHRQ and other research funders put considerable effort into training researchers in how to communicate their research to decision makers. The goal is to make research findings actionable in the real world quickly and to inform public discussion of health policy. I am all for it. But it is not working as well as it should. I want to suggest why this is and what we might do about it.

Let's simplify the world of health policy discourse, and think of there being four strata of participants: decision makers, their policy staff, policy intellectuals and journalists, and academics. There is an an insider culture grounded in a network of peer interaction within each stratum. The arrows are meant to suggest patterns of communication between strata, and examples of the media that support them. 

The first point of the diagram is that it is a long way from academics to decision makers. Seriously, you can't reach them directly. Research findings need to diffuse through several social filters to reach actual decision makers. As a researcher, it may not be realistic to seek an audience of decision makers. Just getting read and discussed in the next tier of policy discourse is your feasible goal.

The second point is that if you want to effectively diffuse information across strata, you need to write in a venue that is read by your target audience. Moreover, you need to be visible to, recognized by, and trusted by them. You need to learn to write for them. Your academic credentials are necessary but not sufficient here. It's true that the glorious intertubes make all of this much easier than it used to be. Nevertheless, if you are a researcher, getting your voice heard by the policy intellectuals takes significant work.

So the question to ask is, given the effort involved, what could motivate more academics to do it? For better or worse, health researchers' careers are driven by journals with 5% acceptance rates and NIH institutes with 5% funding rates. These facts concentrates your mind on the value of your time. It is easy to put words on a blog, but harder to make them good words, and harder still to get career value that is commensurate with the time required.

What we need, I think, is two, three, many Health Affairs. That is, publication outlets that have as many as possible of the following attributes:

1. Writing that is conscise, data-driven, and policy-relevant.
2. Good editing and high production values.
3. Quality and prestige through peer review, providing academics with career incentives.
4. Authors and readers from every stratum of the policy discourse world, and every provider / stakeholder discipline (including patients!).
5. Excellent social media marketing to get articles into the discussion.
6. And it must not be the creature of any provider guild or health industry.

So, AHRQ and Robert Wood Johnson, please consider supporting some variants on this model. We already have Health Affairs, so perhaps the next step is an enterprise that is one step more specialized. A data/policy journal/web presence devoted to Primary Care? A Mental Health Affairs journal/social media site? A site on Health Disparities? On Child Health? On Caring for the Elderly? 

eHealth technology enhances active self care, but is not a substitute for it

post by Bill Gardner

Here is a cool new eHealth technology: a bone cast with wireless telemetry of muscular activity. It is

threaded with electromyographic (EMG) sensors that track muscle activity around a fracture site. Created by Pedro Nakazato Andrade, a recent graduate of the Copenhagen Institute of Interaction Design, the cast can relay data wirelessly to physicians and physical therapists. It shows how much mobility a patient has in an affected area and how physical therapy and daily activity are affecting the area, allowing clinicians to adjust treatment accordingly.

The key thing this device does is measure "how physical therapy and daily activity are affecting the area." It is not, itself, a device with therapeutic activity. More to the point, it does nothing if you are not doing your physical therapy -- which is most of us. eHealth will fundamentally change health care only in the context of interventions that successfully engage patients in self-care. 

I've overstated the point somewhat. My motivation to adhere to my physical therapy regimin will likely increase if that I know (or at least believe) that that regimin will work. So if the smart cast makes therapy more effective, I'll be more likely to do it.

The problem is that eHealth technology is often presented as if it were a pill: you put it on and you get better. Rather, you put it on, and your self-care becomes more effective. And getting patients to effectively and consistently care for themselves is the hardest problem in chronic care. 

Jimmy Connors was once asked, what are the most effective exercises to improve your tennis?Connors: "The ones that you will actually do."

A post idea culture?

post by Bill Gardner

Neal Gabler in The New York Times:

Ideas just aren’t what they used to be. Once upon a time, they could ignite fires of debate, stimulate other thoughts, incite revolutions and fundamentally change the ways we look at and think about the world... It is no secret, especially here in America, that we live in a post-Enlightenment age in which rationality, science, evidence, logical argument and debate have lost the battle in many sectors, and perhaps even in society generally, to superstition, faith, opinion and orthodoxy. While we continue to make giant technological advances, we may be the first generation to have turned back the epochal clock — to have gone backward intellectually from advanced modes of thinking into old modes of belief.

This just seems wrong. Here are some enormous ideas that are transforming our culture:

• Physics. E.g., the multiple universe ideas canvassed in Brian Greene's most recent book.
• Climate science. Lovelock, the global warming scientists. There is a bigger practical question than the fate of the climate?
• Neuroscience, and in particular the neurobiology of emotion.
• Artificial Intelligence. The internet, and Google, and so on are a series of experiments that escaped from a lab.
• Statistics. The widespread availability of analytical tools, skills, and data means that many people are acquiring an empirical mindset through learning by doing.
• Social evolution. Dawkins, of course, but also the many more recent theorists who are advancing evolutionary explanations of social cooperation.
• Moral and political philosophy. Parfit says we are just starting to understand how these work in a post-theistic world, and he is right.
• Applied economics. Health economics, the economics of poverty, growth, and development.

You will have noticed that I am citing disciplines instead of authors. There is a long term shift toward a scientific and engineering culture. This means that ideas are being attacked in a disciplined, serious, and cumulative way by specialists. These specialists, however, are connected to broader communities of discussion; in part through traditional media, and increasingly by the social software that Gabler disdains. Through that diffusion, that are catalyzing the kinds of intellectual shifts that structure and shape the way in which collectively we interpret our experience and our world.

 

US health care costs are higher because of administrative waste

post by Bill Gardner

The U.S. spends far more on health care than any other country in the world. For example, Canadians spent $3895 per capita in 2007, compared to $7290 in the U.S. There are many reasons why, but one disputed question is whether the U.S. has higher administrative costs than other developed countries. 

In Health Affairs (also summarized here) Morra and his colleagues report that physician practices in the U.S. spend far more time on administration than their Canadian peers. The chart reports the U.S. dollar value of the time spent by physicians and staff in the two countries. The U.S. physicians (blue bars) interact with multiple health plans on claims and billing, obtaining prior authorization for patient services, and dealing with pharmaceutical formularies. Canadians (red bars) deal primarily with a single provincial payer (Canadians can also hold supplemental private health care insurance).

The study also found that 

• Physician practices in the United States spent $82,975 per physician per year interacting with payers, compared with $22,205 in Ontario ( = $60,070 per physician per year extra cost in the United States). The cost of time spent by U.S. nurses and U.S. clerical staff were both greater than total Canadian costs.
• If U.S. physicians had administrative costs similar to those of Ontario physicians, their total savings would be approximately $27.6 billion per year.
• In the U.S., nurses and medical assistants spent 20.6 hours per physician per week on administrative tasks related to health plans, nearly 10 times the 2.5 hours spent by Canadian nursing staff. U.S. nursing staff spent more time in every category of interactions, most notably obtaining prior authorizations, which accounted for 13.1 hours per physician per week.

Please keep in mind that the study is considering only the administrative burden carried by clinicians and their staff. It does not include the hours of time spent by patients making sure that they have authorizations for care, checking bills against explanation of benefit forms, and searching for providers who take their insurance. These costs are also higher for U.S. patients, compared to Canadians.

There are significant problems in Canadian health care. Nevertheless, Canadians have better health at less cost than Americans. One important reason is that Canadian physicians and nurses spend more time on patients, and less time on paperwork.