Beyond Belief and Randi's Million Dollar Challenge

Last week the James Randi Educational Foundation hit the publicity jackpot (Time, AOL, CBC, Gizmodo, Discovery) when it played a central part in a feature on ABC's Nightline dedicated to the topic of psychic powers, with Randi's famous million-dollar challenge being conducted on national TV (officiated by his trickster protege Steve Shaw, aka Banachek):

I've made my feelings about Randi's MDC pretty clear in the past (see "The Myth of James Randi's Million Dollar Challenge"), so there's not much need for me to repeat most of those criticisms again here. But there are a few points that I'd like to apply to this particular program.

Firstly: there is something very wrong with an organisation that is supposedly dedicated to raising the public understanding of science and skepticism deliberately obfuscating the fact that its well-known challenge is no real scientific test of the topic and thus says *nothing* about the existence or non-existence of the paranormal. Now when you raise that point, Randi and others will be quick to say that "we never claim that, it's just a one-off challenge". But the MDC is always promoted as the be-all-and-end-all of tests - "if you're psychic, you'd obviously just go and take Randi's money". For example, this is the way in which it is reported in the media:

Of course, just because psychics have not been able to find missing persons doesn't mean that they might not have other psychic abilities. It's important to keep an open mind, and try to demonstrate psychic powers in an objective, scientific manner, under conditions that rule out deception.

The Million Dollar Challenge has been around for many years... Will these celebrity psychics take Randi's challenge? If they have the powers they claim, and can demonstrate them under scientific conditions, they have nothing to lose.

In fact, the publicity of having their abilities validated would likely raise their profiles even higher (to say nothing of the satisfaction they would get from publicly proving the skeptics wrong).

Either the psychic information they give is accurate, or it isn't; there's no real way that skeptics could disprove genuine psychic powers. If the psychics have the powers they claim, they have nothing to lose and $1 million to gain.

(If any 'skeptics' want to say the above is just typical extrapolation by the media, it's worth noting that the syndicated Discovery article above is written by none other than Ben Radford, who has more than a vested interest in the skeptical movement and should know better).

As I pointed out in my MDC article, the "nothing to lose" part is absolute bollocks - when they lose (as they likely will, at Randi's normal success benchmark of beating odds of 1,000,000 to 1) they are 'outed' as non-psychic, sometimes with much media fanfare (as in the clip above). And the 'objective test' is nothing of the sort. The test itself may be scientific in some respects, but the benchmark to be considered a success is not (one-shot, extremely high odds).

And if you thought the usual million-to-one odds were a bit harsh, how about the test faced by the first 'psychic' on the Nightline version of the MDC (pick the correct photo from a set of twelve envelopes, and do that at least nine times out of twelve attempts)? I spoke to well-known parapsychology researcher Dean Radin about this particular test and he quickly did the math, and also pointed out another problem with the test setup:

Assuming each person has to select one correct reading out of 12 possibilities, then the odds of getting at least 9 correct matches is 29.6 million to 1....

In addition, the psychics selected for the test were not vetted for prior ability. Roaming around looking for storefront psychics, and assuming that they have actual talent, is roughly equivalent to roaming around random high school athletic fields, selecting a few runners at random, entering them into the Olympics, and requiring that they win. It's nonsense.

In another segment of the show, reporters went and talked to - and received 'readings' from - celebrity psychics James van Praagh and Allison Dubois. Again, this was poorly done - the reporters give their names before turning up, and then when the reading is done they compare the 'hits' to what information about themselves they can pull up online? This proves nothing either way - for example, in Van Praagh's case he nails a number of things, and then is virtually labeled a fraud because the information is online. I have no experience with Van Praagh, so have no conclusion either way, but this was really poorly executed.

But hey - more power to the JREF. They just got a big chunk of publicity on national TV, for the cheap, cheap price of a million dollars that was always going to stay in their pocket anyhow. And, coincidentally I'm sure, this week the JREF have announced a funding drive looking for ongoing donations in the range of $16 to $48 per month. So you too can help pay James Randi's $200,000/year wage so that he can continue complaining about people making money from unscientific claims...

My recommendation? Ignore it all - the phony and/or deluded psychics, and the media-hungry skeptics, and keep your eye out for real scientific investigation of these topics. Y'know, this sort of thing.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
TopherCooper's picture
Member since:
3 September 2011
Last activity:
2 years 6 weeks

I hate to give any support to Randi and his pseudo-Skeptics but it's possible that Dean made some mistaken assumptions in his probability calculations -- but I'm not sure.

If the challenger was given twelve possible targets to choose from on the first round, then twelve different targets on the second and so on, Dean's computation is correct.

If, on the other hand, the challenger has to choose one of twelve to match the first target, then match one of the remaining eleven against the second, etc then his computation is too high.

I haven't seen the show but the latter is more JREF's style -- a better show, and the numbers (p=1/12 and 12 trials) fit. In that case, the probability of nine or more hits is about one chance in 1,670,000 -- not too far off from the claimed 1 in 1,000,000 chance, and I think that they actually say that they will not put the prize on the line for anything less than million to one odds.

Topher Cooper

Greg's picture
Member since:
30 April 2004
Last activity:
2 hours 52 min
TopherCooper wrote:

I hate to give any support to Randi and his pseudo-Skeptics but it's possible that Dean made some mistaken assumptions in his probability calculations -- but I'm not sure.

If the challenger was given twelve possible targets to choose from on the first round, then twelve different targets on the second and so on, Dean's computation is correct.

If, on the other hand, the challenger has to choose one of twelve to match the first target, then match one of the remaining eleven against the second, etc then his computation is too high.

From what I've seen and heard on the show, the former is how they did it. 12 envelopes, each with a photo of a person in it. 11 envelopes have a photo of a living person, 1 envelope has a photo of a famous dead person. 'Psychic' has to pick the correct one, at odds of 1/12. After this test, they reset it and again the psychic has to pick the correct envelope at 1/12. To win the challenge, the psychic must do this at least 9 out of 12 times.

Interestingly, in the promo for the show on Good Morning America, the voiceover said the odds of success were "1 in 5 billion" (see the second clip of three halfway down the page here: http://abcnews.go.com/Technology/psychic...). The voiceover is edited in the clip from the show (embedded in this TDG story) to say a million to one. The first, incorrect value of 5 billion to 1 likely came from someone working out the odds of choosing the correct envelope 9 times in a row, whereas the psychic 'only' needed to do it 9 out of 12. However, I don't think a million to one is correct either.

Which is really another point against Randi's challenge, that there isn't even a protocol written up anywhere for other people to view and analyse, let alone the data set/results from their 'experiment' or the initial agreement between the JREF and the challenger.

Kind regards,
Greg
-------------------------------------------
You monkeys only think you're running things
@DailyGrail

TopherCooper's picture
Member since:
3 September 2011
Last activity:
2 years 6 weeks

In that case then Dean's values are correct, the actual probability is about 1/29600000(given proper randomization -- which in my experience is by no means certain; usually "skeptics" experiments are much less rigorous then the minimum standards in parapsychology; also, someone who wanted a negative effect could "anti-randomize" to make it less likely that someone would be lucky).

Topher Cooper

daydreamer's picture
Member since:
21 February 2009
Last activity:
1 year 24 weeks
Quote:

its well-known challenge is no real scientific test of the topic and thus says *nothing* about the existence or non-existence of the paranormal

Putting aside how this could all be improved for a moment this is a good example of the difference between having a hypothesis and having claims about a phenomenon.

The MDC can never disprove mediumship, or anything else. All it can do it test the hypothesis that it starts out with - namely 'person X can perform activity Y under this condition'. Like any experiment that is all it can prove or disprove.

I.e a negative result shows nothing at all about various realities expect that you have another negative under those exact circumstances - though a positive has more interesting results than the negative (though it still has no hypothesis about how something is occurring). In fact even a positive result, even with astronomical odds, doesn't necessarily mean anything without repeatability - lest we forget all the amazing results in 'normal' science that turned out to just be beating the odds.

Your right about science education - I think the MDC only really stands as a tongue in cheek experiment based around media psychics suggestions that they really can do the sort of stuff underlying the MDC's specific hypothesis. Other than that is doesn't do anything.

I think part of the issue, forgetting larger named sceptics, is that those sections of the general public who are skeptical of the paranormal (but don't give a hoot about organised skepticism or scientific tests) remain dubious for exactly the reasons that this test captures in it's spirit. So this test is able to connect with the public precisely because it does all that those people would do in their front room's, not because it would be published in a scientific or philosophy journal.

Actually that reminds me to look up the Windbridge Institute later and see what they have been up to.

TopherCooper's picture
Member since:
3 September 2011
Last activity:
2 years 6 weeks

I would not say that this is unscientific, its just poor. Let's say that I read in Popular Science that light exerts pressure. So I do a scientific test: I take a flashlight and point it at a wall. I'm very scientific about it. I carefully count the amount of debris on the other side of the wall before and after the experiment. I find no significant difference in the amount and therefore conclude that the claim that light exerts pressure can be dismissed as pseudo-science.

An hypothesis about light pressure has been scientifically tested, but the hypothesis about light pressure that those "so-called physicists" actually make has not.

Yes, they managed to find some self-deluded idiots (whether they were self-deluded about the extent of their own abilities or about being able to side-step the controls) who agreed to the test, but I'm sure that no actual parapsychologist was consulted as to their prediction about the success of the experiment.

In addition to having very low statistical power (the minimal successful score would have an effect size of just under .5 the usual definition of a "strong effect."), and the lack of prescreening that Dean pointed out (no parapsychologist claims that everyone who claims to be a "psychic" actually has strong ESP performance), both the form of the task and the "demand characteristics" bear very little resemblance to what these people claim to do.

Topher Cooper