post by Bill Gardner
I am working through an on-line research ethics training course. No one enjoys these exercises, but I don't have a better scheme for promoting research ethics. And not all of it is tedious: I have just finished watching an excellent brief video debate on how to address financial conflicts of interest in research involving David Rothman, Nancy Dubler, and Thomas Murray.
But I was struck by this question on a module's test:
Which types of circumstances might bias the review of a manuscript or grant application?
The 'correct' answer was:
The reviewer may have a philosophical or religious bias against some types of research (e.g., the use of non-human primates or human embryonic stem cells in research).
Is this answer correct? The notion of bias assumes that competent experts should roughly agree on the merits of a piece of work. This is at least partially true for the technical aspects of science. So ethical or religious bias is possible in reviews. For example, I might (through a 'disconfirmation bias') grade the statistical arguments in a proposal more harshly if I had a philosophical or religious objection to the research.
Reviews, however, involve an evaluation of the technical merit, the significance, and the ethics of a project. In clinical research, significance eventually cashes out as consequences for human well-being. (In basic science it may mean something like the implications of the work for our understanding of nature.) However, judgments about consequences for human well-being have, eventually, to be defended using philosophical or religious arguments. This is even more true for ethical evaluations of research. So when reviewing the significance or ethics of research, the notion of a philosophical or religious bias puzzles me.
How serious is the problem of bias in technical evaluations of research? It may not affect my grading too much, for a possibly idiosyncratic reason. I think I grade proposals by combining evaluations of technique, significance, and ethics multiplicatively, not additively, with zeros at the threshold of unacceptability.* So a proposal with zero technical merit, good ethics, and high significance gets a zero overall score (rescaled as required by the institution for whom I am grading). Similarly, if I think the research is at or near the point of zero significance or if I had a strong ethical objection to the research, any bias in my technical review would be irrelevant to my overall grade.
*The NIH asks us to consider certain ethical issues separately from the overall grading, and I do that. Also, I don't formally multiply scores together, this is my psychological model for my judgments in reviews.