data fraud

When data is too good to be true – and isn’t | Day 279

What a let-down.

Recently I was listening to an incredible story on This American Life, one of those that have the potential to change how you see the world.

It was following some canvassers in California, going door-to-door and trying to change the minds of people who voted against gay marriage. Now, changing someone’s mind is a notoriously difficult thing to do. But these people, who were part of a research study, had seemed to crack this magical code of persuasion – the power of a personal, intimate conversation with someone who can share your life experiences, and who really isn’t that different from you or your family members, except for this one thing – sexual orientation. You talk to them, you share, you laugh together – and then you change your mind and decide that they have just as much of a right to get married as yourself.

It’s a wonderful story, and I found it to be an emotional listening experience.

Unfortunately, it turned out to involve what now appears to be some downright fraudulent science. Maria Konnikova over at The New Yorker has written an excellent column outlining the research, the background story, and the fallout:

The survey highlighted a surprising distinction. When canvassers didn’t talk about their own sexual orientations, voters’ shifts in opinion were unlikely to last. But if canvassers were openly gay—if they talked about their sexual orientations with voters—the voters’ shifts in opinion were still in evidence in the survey nine months later. The messenger, it turned out, was just as important as the message. The study formed the basis for a segment of “This American Life,” and wasfeatured on Science Friday and in the New York Times, theWall Street Journal, and the Washington Post. LaCour was offered a job at Princeton. [..] A study that had excited researchers and journalists alike now looks suspect. What went wrong?

You can read the rest of the story here, to find out how a highly respected researcher ended up co-authoring a paper he went on to retract, single-handedly and without hesitation.

And, once you’re done with it, do listen to the original TAL piece. I still think it’s wonderful, even though unfortunately we all seem to have been taken for a ride. In fact, TAL have done some corrections on their own blog as well, which is an equally interesting read.

Bummer. Although the good news is – someone is working on doing the study properly, so it may turn out that the hypothesis wasn’t entirely wrong.

Leave a Reply

Your email is perfectly safe with me.