Something Not Unlike Research

How the hive mind is built: On the sentence, "Descriptive statistics is not just for losers"

post by Bill Gardner

Or perhaps I should have titled this, "How the social media are building a distributed, collective, and associative memory." 

Yesterday I was thinking about the problems of implementing evidence-based clinical practices in medicine and mental health, stimulated by posts on pilot interventions by Megan McArdle and Harold Pollack. A chain of searches that I cannot now reconstruct lead me to a great post by Andrew Gelman, with a great sentence, "Descriptive statistics are not just for losers." The point I took from his sentence is that although causal inference in social science is paramount, it is not everything. The collection of data that identify an unanticipated correlation also matters, even if we do not yet understand the mechanisms that produce that correlation.

I love to tweet great sentences, so I sent that out with a link to Gelman's post. Harold Pollack liked the sentence -- probably not knowing that I had been thinking about his writing when I tweeted it -- and he retweeted my tweet. Someone else in the twitterverse eventually read the tweet and liked the sentence enough to write to Gelman about it.

Gelman, curiously, had forgotten that he had ever written that post, but searched his blog, and discovered that he too liked the post.

I thanked my correspondent for reminding me of this little article I’d forgotten, and he told me he just learned of it via someone’s tweet.

He then wrote a post about rediscovering his post, and speculating that 

Maybe I should have a twitter feed of nothing but old blog entries. I could just go back to 2004 and then go gradually forward, tweeting the items that I judge to remain of interest.

Which is not a bad idea. Google Reader brought me to Andrew's post about rediscovering his earlier post. So we had a chain of communications across a couple of media that originated in my tweet and completed a circuit coming back to me.

This circuit was a rumination in the 'hive mind' about data and its role in clinical or educational practice change. Let's cash out the 'hive mind' metaphor as the vast neural net of associations between concepts implemented most directly through hyperlinks (the image at the top is a representation of del.icio.us). Let's think of the circuit as an activation spreading through the net. Then what we accomplished through this circuit of tweets and posts was strengthened the probability that people writing/searching/thinking about implementation science will come across Gelman's meta-statistical thinking. If that probability has increased, then the global mind got a little smarter.

 

What can academics learn from think tanks?

post by Bill Gardner

Judy Sebba of the University of Sussex has a useful post on getting research into policy. I wrote related posts here and here, as did Austin Frakt here and here. 

She notes that academic writing is often useless to decision makers. It can be impenetrably technical, or it can fail to offer clear outcomes or policy implications. Academics can learn from Think Tanks how to

synthesise, create or communicate information and give advice to the public and policy-makers...; [and] ...be far more ‘media savvy’ than academics [by working] with staff who have backgrounds in the communication industry...

If they do this, they can

link researchers and decision-makers or act as a ‘broker’ between them.

I made a similar point using this graph about the role of policy intellectuals in the research diffusion process.

Sebba notes, however, that

[Think tanks] do not subject their work to review by others and so the quality of their outputs are not assessed.

There is another problem with think tanks. Many are funded by stakeholders in the health industry, or they are closely tied to a political party, or both. Sometimes this results in 'analysis' that is just lightly disguised lobbying.

Partisan think tanks also risk being listened to only by the like-minded.   The graph below is a network analysis of linking patterns among conservative and liberal political blogs. The tight clustering suggests that many of these writers are simply echoing each other. 

So, what can academics learn from think tanks? Specifically, can they write timely articles with clear policy implications without getting caught in the webs of deep red or deep blue? Yes, I hope, if we can find ways to bring something like the accountability to the facts provided by peer review into academic blogging.

 

Getting it right and being on time

post by Bill Gardner

Austin Frakt responded to a recent post of mine on how to speed up the diffusion of research from the university to policy makers. In my post, I argued that to encourage researchers to communicate their results to decision makers and the public, we need more publications like Health Affairs that have as many as possible of the following attributes:

1. Writing that is conscise, data-driven, and policy-relevant.
2. Good editing and high production values.
3. Quality and prestige through peer review, providing academics with career incentives.
4. Authors and readers from every stratum of the policy discourse world, and every provider / stakeholder discipline (including patients!).
5. Excellent social media marketing to get articles into the discussion.
6. And it must not be the creature of any provider guild or health industry.

Austin agrees, but makes the following crucial observation

I’d add “timely.” Therefore, I would also rethink one of these items: peer-review (#3). To be sure, it’s a crucial element of primary publication of research results. But it takes a long time. Inserting it into a translational, dissemination process risks missing the window of opportunity for maximum impact and utility. Timing is everything.

This is an excellent point. I would say that policy discourse needs contributions from researchers at several velocities, including the very quick response that blogs provide. However, the faster you go, the harder it is to have peer review.

So, can we add timeliness to our web policy journal without sacrificing peer review? No. But let's keep in mind that peer review is not an end-in-itself. The goal is quality and having a reputation for quality (it is the latter that incentivizes academics). Peer review is just a means to those ends.

I have two suggestions. The core of the imagined web policy journal must be peer-reviewed publications so that it remains anchored in science. It could, however, also include some more timely outlets:

1. The journal could publish something like the Policy Forum in Science, and something like the OpEd page of The New York Times. These would be solicited pieces of < 1000 words that appeared perhaps weekly. This would require an editor who could anticipate issues that would be relevant, find authors who could respond on a fast turn around, and give those authors critical feedback to improve their pieces. 
2. The journal could sponsor one or more bloggers who would publish, for limited tenures, under the journal's brand. These pieces would appear without prior editing to achieve immediacy. However, the journal would maintain an editorial board that would be charged with monitoring the blogs and periodically rating their insight, accuracy, and objectivity. The board could replace bloggers who did not meet the journal's standards. 

These schemes would add a measure of review and selection to high velocity writing. Being a policy blogger for (say) Health Affairs would contribute more to an academic's reputation than publishing the same words under her own name. Moreover, having the imprimatur of a prestigious journal would greatly increase the visibility and impact of a blog. Conversely, having processes of article and blogger selection and pre- or post-publication review would help the journal insure that these more timely publications maintain its standards.

A post idea culture?

post by Bill Gardner

Neal Gabler in The New York Times:

Ideas just aren’t what they used to be. Once upon a time, they could ignite fires of debate, stimulate other thoughts, incite revolutions and fundamentally change the ways we look at and think about the world... It is no secret, especially here in America, that we live in a post-Enlightenment age in which rationality, science, evidence, logical argument and debate have lost the battle in many sectors, and perhaps even in society generally, to superstition, faith, opinion and orthodoxy. While we continue to make giant technological advances, we may be the first generation to have turned back the epochal clock — to have gone backward intellectually from advanced modes of thinking into old modes of belief.

This just seems wrong. Here are some enormous ideas that are transforming our culture:

• Physics. E.g., the multiple universe ideas canvassed in Brian Greene's most recent book.
• Climate science. Lovelock, the global warming scientists. There is a bigger practical question than the fate of the climate?
• Neuroscience, and in particular the neurobiology of emotion.
• Artificial Intelligence. The internet, and Google, and so on are a series of experiments that escaped from a lab.
• Statistics. The widespread availability of analytical tools, skills, and data means that many people are acquiring an empirical mindset through learning by doing.
• Social evolution. Dawkins, of course, but also the many more recent theorists who are advancing evolutionary explanations of social cooperation.
• Moral and political philosophy. Parfit says we are just starting to understand how these work in a post-theistic world, and he is right.
• Applied economics. Health economics, the economics of poverty, growth, and development.

You will have noticed that I am citing disciplines instead of authors. There is a long term shift toward a scientific and engineering culture. This means that ideas are being attacked in a disciplined, serious, and cumulative way by specialists. These specialists, however, are connected to broader communities of discussion; in part through traditional media, and increasingly by the social software that Gabler disdains. Through that diffusion, that are catalyzing the kinds of intellectual shifts that structure and shape the way in which collectively we interpret our experience and our world.