Report from TAM 7 (Part 5)
Yes, I know TAM 7 was a month and a half ago. I told you I was busy. Finally, here is my last report. This is what fun I had on Sunday, the last official day of the conference. (You can see the earlier reports here: Part 1, Part 2, Part 3, and Part 4)
From 9 until noon were the paper presentations. Some of them were pretty good. But that was a long time ago, and I don’t remember much about them.
I managed to corner James Randi at one point that morning and asked him a question that has been bugging me for a while. I mentioned to him that there were two types of psychics: Those who know they are frauds and those who don’t.
Randi replied that he calls them “open eye” and “closed eye” psychics (I think that’s the term he used.).
I’ve noticed that a lot of the so-called psychics at the new age fairs and similar places seem to genuinely believe that they have paranormal abilities. What I wanted Randi’s opinion on was whether any of the big-name faux-psychics thought they were real. Randi said that he was convinced that every one of the famous frauds knew they were frauds. He said they can’t get that successful without active deception and manipulation.
That goes along with my perceptions of the business. Every time I see one of these charlatans on TV, it’s clear to me that they’re using all of the conventional tricks.
The self-deluded psychics I’ve seen locally often bumble around with their predictions. Their marks fall for the scam, because they want to believe it. Confirmation bias and selective memory make most people who are stupid enough to part with $10 for a reading convince themselves that it was money well spent.
However, that accidental success through ineptitude doesn’t stand up to scrutiny. If you’re on TV, you need to have a higher success rate. Anybody who has reached that level of financial success in that business almost certainly is actively manipulating the marks by way of cold and (probably) warm reading techniques, various magic tricks, and other methods of deception. It also helps when they have full control over how the show is edited.
I Need a Psychic, Stat!
After lunch, self-proclaimed dowser Connie Sonne was reaching for the brass ring, and we got to watch. She was going to be tested for the JREF Million Dollar Challenge. This was just a preliminary test, but if she passed it, she could then go on and take the final test and earn the million.
Like any scientific endeavor, testing psychics relies on the arcane art of statistics to measure the likelihood that what you see reflects reality.
The JREF has tested hundreds of alleged psychics. They have a million dollars on the line. If they aren’t careful, the odds are that somebody would pass the test by dumb luck and claim the million dollars! That’s the nature of statistics. You can never be 100% sure. There’s always a chance you’re wrong. When setting up a statistical test, you need to determine your tolerance for getting a wrong answer. A false positive in this situation is an expensive error, so the test has to be designed to make it very difficult to pass by luck alone. We’ll come back to this matter in a minute. I’m not convinced the test was set up properly.
The Million Dollar Challenge
The JREF works with the alleged psychic to come up with a test that both sides consider fair. In this case, Connie Sonne claimed to be a dowser. The test that she agreed to was to use her pendulum to find a randomly-selected card.
Prior to the test, they had somebody take a deck of cards and sort them by suit. The jacks through kings were discarded. For each suit, they took the ten remaining cards (ace through ten) and placed each card in a separate envelope and sealed it. Then they shuffled the ten envelopes and stuffed them into a larger envelope.
On the day of the test, with 600 of us watching in the audience, Connie sat down on one side of a table and Banachek sat on the other side. He took the ten envelopes from one of the suits and spread them in front of Connie in a horseshoe pattern. He then rolled a ten-sided die and announced the result. That would be the card that Connie had to find. Connie would then dangle her pendulum for a few minutes and finally announce which envelope contained the card. Banachek marked her guess on the card and set it aside. They repeated this process twice more.
The JREF had determined statistically (remember, the threshold has to be high to rule out dumb luck) that Connie would need to get all three cards correct in order to pass this challenge and move on to the next (final?) round. Connie agreed to this in advance.
So one by one, Banachek opened the envelopes and showed us which card was inside.
Of Connie’s three guesses, she got none correct.
She was initially gracious in her defeat. Now she’s claiming it was rigged. Sorry, Connie, nobody cheated.
The problem with people who have convinced themselves that they’re psychic is that there is nothing you can do to convince them otherwise. It’s like any firmly-held irrational belief. Evidence and logic never enter the picture.
Was the Test Set Up Properly?
The odds of finding the right card three times in a row purely by chance are 1000:1. That is reasonable insurance against giving away the money by accident.
But if Connie truly were psychic, was this the right test? I’m not sure it was. I wasn’t in on the design of the test, so I don’t know if my concerns are justified. I’ll just mention them here for the record.
I am confident that this test fairly showed that Connie has no strong paranormal ability. If she had, she would have been able to get at least one of the three guesses correct. My concern is that if she has a weak paranormal ability, this test could not have detected it. More broadly, if the JREF uses this approach frequently, they run the risk of missing somebody who does have paranormal ability (assuming such a thing exists).
The primary responsibility for coming up with a good test rests with the applicant. The applicant has to clearly state, in quantifiable terms, what her paranormal ability is. She just can’t say “I’m psychic” or even “I read minds”. What, precisely, can you read? How often are you correct?
Let’s say that I claim to have the psychic ability to tell you the suit of any card chosen at random. If psychic powers really exist, they’re probably weak. It’s unreasonable to expect me to get ten in a row correct. It may even be unreasonable to expect me to get three in a row correct.
Let’s pretend that I really am psychic, but it’s a weak force. It’s just strong enough that I can guess the suit of the card one third of the time.
First of all, it would be my responsibility to know what my approximate success rate is and communicate that to the JREF when I make my application. Assuming I did this, I’m hoping the JREF would design the test accordingly.
What would be a good test in this case? It would obviously require more than three or even ten guesses. If you did 100 cards, that probably still wouldn’t be enough, because the difference between random chance (25) and the psychic power (33) is only 8, which could easily be swallowed up by normal random variation. This is where the statistics come in. There are formulas for determining how many trials you would need to run in order to be confident that the effect you’ve detected is probably real. I used to know how to calculate this sort of thing. I suspect you’d have to run somewhere between 500 and 1000 cards to be sure.
The test I just described is a far cry from what Connie did. I’m hoping that Connie told them that she is 100% accurate, because the test they ran is only appropriate for such a claim.
(Image from Scott Hurst’s Flickr stream.)