Published ESP Experiment Debunked. What Would Michael Faraday Say?

I'm reading a series of famous "Christmas" lectures by English physicist/chemist Michael Faraday. The year is 1860 and Faraday performed experiments that must have dazzled his audience with the magic of reality. In the course of dozens of fires, explosions, and other reactions, he demonstrated what's really going on when a candle burns. He showed that the candle wax is vaporized, that the glow is from soot, and that a chemical reaction is taking place. He showed how to see the invisible in order to prove that the flame consumes oxygen and produced are carbon dioxide and water, and that these are compounds composed of the elements hydrogen, carbon and oxygen.

There was a quote I meant to underline, in which Faraday warned his audience that they should believe what they see for themselves over what they're told. It's driving me crazy that I can't find it now, but it should resurface eventually. Faraday, while religious, was also something of a skeptic. He liked to investigate some of the popular paranormal ideas of his time – such as the notion that séances were summoning spirits.

I wonder what Faraday would say if he found out that a paper purporting to prove ESP was published in a reputable scientific journal in late 2010. The paper, by Cornell University psychologist Daryl Bem, was titled Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect, and appeared in The Journal of Personality and Social Psychology.

The experiment was designed to see if people could better remember a list of words if the list was shown to them again after the test. Weirdly, the act of showing people the words after the test seemed to improve their performance. It wasn't a clear demonstration but a statistical argument Bem used to try to show this wasn't happening by chance. After all, some people will do better on the test than others and by chance, more of those who did well might have been in the group that saw the words afterwards.

In his paper, Bem abused some ideas from modern physics to try to justify the result, though in reality it would violate everything we know about science today.

Well, not too surprisingly, skeptics trying to replicate the experiment found it didn't work for them. You can read their results here. Extraordinary results require extraordinary evidence, and all that. But wait! What happens when reputable journals produce equally wrong but less extraordinary-sounding claims? If this ESP thing can pass the test of statistical significance and get into a peer-reviewed journal, then doesn't that indicate the bar is too low? I just got a press announcement about a group of scientists who claim that many psychiatric drugs on the market have been subject to failed replications, but the negative results were never published. What other bad science has gotten over the bar?

One of the authors of the ESP-debunking paper had a similar idea, which he expressed in this opinion piece in the Guardian. There's also an examination of the issue and its implications in Jerry Coyne's blog Why Evolution is True.

Other scientists here in Philadelphia are also thinking about the problem, and will be the subject of an upcoming story.