Facts: What You Want Them to Be

The great tragedy of science—the slaying of a beautiful hypothesis by an ugly fact. — Thomas Huxley

So the other day, I was doing some intensive graduate research on a series of tests informing me which Marvel superhero I was most like. When I completed the first quiz, it reported I was most similar to Ant-man.

Clearly, that had to be mistaken.

So I took another. Ant-man. And another. Ant-man. Finally, I took a test that was actually legitimate: it informed me I was like Superman.

Now, what could this little anecdote possibly have to do with science (besides showing that I’m not conducting it)? Let’s see if you, dear reader, can pull out the moral of today’s post from the following study:

Researchers brought participants in under the pretense that they would be tested for a particular enzyme, TAA—a totally fictitious enzyme made-up for the purposes of the study.

The participants were then informed that if they had this enzyme, they were “relatively susceptible to a variety of pancreatic disorders” later in life. So in order to test to see if they had the enzyme, researchers provided participants with a “chemically coated” strip of paper that would react with a person’s saliva.

For half of the participants, they were told that if the strip doesn’t change colors, you do not have the enzyme (i.e. you don’t want the strip to change). For the other half, they were told that if it doesn’t change colors, you do have the enzyme (i.e. you want the strip to change).

So what exactly was this “chemically coated” test strip? A slice of yellow construction paper.

Researchers then left the participant alone in the room with a little petri dish to spit into, the “official” test strip, and up to five minutes to test themselves on whether or not they had this enzyme.

Now, for the people who didn’t want the strip to change colors, most of them just pooled their saliva, dipped the strip once, saw no change, and waited for the researcher to come back. However, for those who wanted it to change (i.e. if the strip didn’t change it meant they had the bad enzyme), they behaved…a little differently.

As the researchers write, when these participants realized the strip didn’t change, they did such things as “placing the test strip directly on their tongue, redipping the original test strip (up to 12 times), as well as shaking, wiping, blowing on, and in general quite carefully scrutinizing the recalcitrant nature of their yellow test strip.”

Heck, some participants even pocketed extra strips so that they could retest themselves again when they got home.*

So how does this relate to Ant-man? Well, think to your own life. When you get positive information about yourself, how quick are you to agree that it’s accurate? What about negative information?

These next few posts will explore our biases in “hypothesis testing.” That is, our predisposition to believe the things we want to believe and discredit that which we don’t.

Prepare to question everything you consider a “cold, hard fact”…

Confirmatorily,
jdt

*Although I never explicitly note it in these studies, researchers always debrief their participants after the study. That is, they tell the participants what was real and what was fake during the procedures.

 

Ditto, P. & Lopez, D (1992). Motivated Skepticism: Use of Differential Decision Criteria for Preferred and Nonpreferred Conclusions. Journal of Personality and Social Psychology. 63(4). 568-564.