Randi’s Million Dollar Challenge Fail

On May 6th, self-proclaimed psychic Patricia Putt was put to the test by Professors Richard Wiseman and Christopher French in the UK, on behalf of the James Randi Educational Foundation (JREF). The testing by these two highly regarded academics was a preliminary ‘screening’ for an attempt at James Randi’s ‘Million Dollar Challenge’:

A few months ago the JREF asked if Chris French (Goldsmith University) and I would carry out an initial test of a medium named Patricia Putt. We went back and forth about the protocol, and eventually settled on an experimental design. Basically, Patricia would carry out readings for 10 strangers, and then all of the participants would be presented with all 10 readings and have to select the one that best described them. To cut down on possible sensory cues, the strangers were not allowed to interact with Patricia, and asked to wear a graduation gown and facial mask.

…Patricia was a joy to work with, and carried out the readings as promised. I sat in the room with Patricia as she wrote her readings and sent the occasional twitter update.

None of the participants were able to correctly identify their reading, and so the results do not support Patricia’s claim.

To be clear on that result: Putt got 0 out of 10 correct. That’s a pretty comprehensive fail. You can find the protocol for the experiment at the JREF Forum. Chris French also has written up his account of the day for the UK’s Guardian.

This looks to have been a good solid test of what Putt claimed to be able to do – and if anything, by allowing speaking I think it probably favoured Putt a little in terms of allowing some possible sensory leakage. There are only two things that concern me. Firstly, there is no mention of the stance of the volunteers. I emailed Chris French asking whether any survey was done of their thoughts on paranormal phenomena, he told me there was not. This seems odd to me. What if all the volunteers were of a skeptical nature? Is it possible that they could intentionally pick the description that does not indicate them? While personally I think this is stretching things (especially with a zero out of ten result), it is theoretically a nasty flaw. With the final selection of reading by volunteers being blind, the sensible thing to me would be to have supporters/believers of paranormal phenomena involved, to put this possibility to bed.

The more important concern is over the setting of five selections out of ten as constituting the benchmark for success. While a casual glance might suggest that’s 1 in 2 odds, it isn’t. If Putt had achieved that benchmark, she would be doing so at odds against chance of over 600 to 1. Remember, this is just the preliminary test, in order to see whether she’s worthy of going for the million dollars. Given Putt’s 0 out of 10 result, in this particular case it’s all academic – but still worthy of making a point of, given the reputation the million dollar challenge has.

I asked Chris French why the benchmark for “success” was set at 5 out of 10; he told me that “this was set by JREF and agreed to by Mrs Putt,” before he got involved in the testing. This is consistent with previous ‘preliminary’ tests regarding the Million Dollar Challenge, with odds against chance of 1000 to 1 generally mentioned as being required for the preliminary, rising to a million to one for the true ‘Million Dollar’ test. As I’ve mentioned previously, those odds are probably valid given the amount of money Randi needs to protect against being won by a chance event. It does not, however, give any sort of ‘scientific’ test of whether someone has exhibited an anomalous sensory power, or at least done well enough to warrant further testing. See the Demkina case for an example of someone doing quite well but ‘failing’.

Having said that, it’s probably a valid argument that if you’re going to sell your services as a talented psychic, you should be able to get five things out of ten correct, despite the odds. Otherwise, what value is this power – what sort of trust can we put in anything said, even if some of it is through some genuine psychic channel?

The other thing worth noting is that Patricia Putt agreed to setting 5 out of 10 as the level required for success. Hopefully she was aware (or made aware) of the odds she was going up against. But she took on the challenge, with full knowledge of the restrictions and benchmarks for success, so she has made her own bed.

The pointy end of the stick is though: what does Randi’s Million Dollar Challenge offer us? This test has shown that one ‘psychic’, Patricia Putt, failed to show evidence for paranormal powers, on this one particular day. Nothing more can be claimed than that, all else is inference. It also has shown that even if something surprising happened, and a ‘psychic’ achieved results against odds of 500 to 1, it would be labeled a failure under this protocol. That any intelligent ‘psychic’ – whether genuine or fraudulent – would avoid putting themselves in that position is more than understandable. As such, it could well be argued that the pool of Million Dollar applicants might be composed of less than capable individuals. (Given that Putt has been put to the test previously by the BBC, with not-so-impressive results, this particular test result isn’t that surprising).

So we basically have a testing system which offers no scientifically useful benchmarks, investigating mostly self-deluded individuals, which so far has proved that on that particular day, nothing exceptionally out of the normal has happened. In short, I fail to see why the Million Dollar Challenge is held in such high regard. Let’s hope that – contra to what the Bad Astronomer says – the Million Dollar Challenge is *not* the “coolest thing” the JREF does.

The one thing that it does show is that you shouldn’t explicitly trust the results of anybody relying on ‘psychic’ ability. That’s the message that skeptics would do well to get across to the public, rather than large-scale dismissals of psychic abilities based on the continued retention of the million dollar prize.

For more information on this topic, see my previous article “The Myth of James Randi’s Million Dollar Challenge“. Your thoughts?

Editor
  1. Well stated, Greg. As far as
    Well stated, Greg. As far as establishing “proof” for PSI, I’m still a bit surprised that the writings and work of people like Dean Radin, and the random number generator tests out of Princeton, for example, haven’t made more of an impact on the skeptical community than they have by now; if it’s “statistical proof” they’re looking for, they’ve got it. But then again, as shown with the Demkina case, it’s not always about science so much as people’s pre-existing biases, so no amount of “proof” may ever really do the trick.

    1. Skwprics
      A majority of sceptics believe in a negative, whether they want to admit it or not.

      So far as I am concerned, they are as much believers than those against whom they crusade.

      1. As a skeptic i agree with
        As a skeptic i agree with this. Belief is a bit of a bugger, acting a a sort of anchor to ideas. It bares no relationship to the validity of a claim of course.

        It might be better if belief was completely taken out of the equation in matters like this so just a completely fair appraisal of the evidence occured.

        There is evidence for everything though, especially if personal experience, dreams, smells etc are used. Therefore there needs to be a way to determine how much confidence can be placed in any data.

        This is where different people are able to draw different conclusions. It can be silly to simply declare someone wrong though. In the childish world, at school, factual right and wrong seemed so simple. As adults we enter a particularly murky world where philosophy rubs up against experimentation (ideally outside of science).

        We are all skeptics of most of the ideas ever proposed so i cringe a bit if a blanket appraisal of skepticism occurs.

        Evaluating something is tricky. There is plenty of evidence for the bigfoot, yeti, Loch Ness monster, ufo’s etc. The problem is whether critisisms of it stand. That is all i tend to worry about when trying to form a picture. All i am really looking for is good solid science. Negative results produce data as well, in fact often the negative data is more important. All these tests of supernatural phenomena that fail produce data that will ultimately need to be explained by supernatural models in the future. That is the challenge. One post here suggests that the psychics abilities were blocked by fabric, which i think is what the shocked psychic also thought when shown the results.

        For me personally an explanatory model also needs to be developed. When plate tectonics was first proposed it was not immediately taken up because no-one could see how continents could move across a solid Earths surface. Examples like this are often given as examples of how foolish the skeptics were, but there is a natural period over which it is right to remain skeptical of an idea while it is being developed, where skepticism is the correct response to the current data. I suspect this tends to extend into the region where a proper explanatory model is created that can deal with all the (often conflicting) data.

        None of those skeptics were wrong to be critical of plate tectonics while it was still in its infancy as a working model. If someone had stood up one day and said ‘hey, ive got a new idea, the contents move’ and because it fitted some data all the experts in a field just went ‘cool, ok then’ then it wouldnt work, we would be straight off down false paths. For this reason it works very skeptically of new ideas, especially ones that ask that established models be transformed on a little data alone.

        Here lies one of the big problems. One view of science espoused is that it often changes so cannot be relied upon. This is slightly mendacious and tends to just be used as an arugument to some specific advantage. No matter the philosophical validity of this statement much of science is utterly reliable. Outside of philosophy you can see this by watching planes flying in the sky or using your computer.

        Often people wish to disregard current knowledge – the proverbial baby with the bathwater. This does not engender respect, especially if it is done for ‘faithy’ type reasons.

        One of the things that makes me so skeptical is that the evidential data for paranormal activity is interpreted to create models that the person favours, rather than rock solid interpretations. It is these interpretations that i am skeptical about – it is the models proposed to explain the (sometimes tentative) data.

        The global consciousness project is a good example. They have alot of data, but without a causative model they have only the data. Their interpretation is that human consciousness affects random probabilities, even ahead of time, but the idea that they are cherry picking also fits the data. Even still it is not clear why they have chosen their hypothesis over others. Non-random probabilities could be responsible in some way for aspects of human nature, or it may indicate problems in the laws of nature as expected if the universe was a non-perfect simulation on an alien supercomputer. There are hundreds of ways to interpret their data, but they have chosen just one, and one that the data does not specifically back-up, hence the skepticism.

        My point would be that there are many reasons why any individual might remain skeptical, but so long as they can back it up in some way that is reasonable then it is not unrational, it is just part of the process of proving or disproving ideas within a field.

        1. Agreed, but…
          [quote]None of those skeptics were wrong to be critical of plate tectonics while it was still in its infancy as a working model. If someone had stood up one day and said ‘hey, ive got a new idea, the contents move’ and because it fitted some data all the experts in a field just went ‘cool, ok then’ then it wouldnt work, we would be straight off down false paths. For this reason it works very skeptically of new ideas, especially ones that ask that established models be transformed on a little data alone.[/quote]

          Agreed. But the problem is, most skeptics tend to validate their position through the use of cynicism & ad-hominem attacks.

          So, instead of saying “I don’t agree with your hypothesis based on the data presented”, some of them simply say “You’re an idiot!”; and then the public laughs and they attract attention; then the media starts to seek them for advise; and before you know it the cynics are the ones that are erected as Apostles of Skepticism —so I guess it’s not entirely their fault 🙂

          —–
          It’s not the depth of the rabbit hole that bugs me…
          It’s all the rabbit SH*T you stumble over on your way down!!!

          Red Pill Junkie

          1. Agreed. It definitely
            Agreed. It definitely becomes less useful when that happens.

            With regard to cynicism and ad-hominem attacks i think it gets a little complex though. The range of reactions that occur in rebellion to any idea are normal from a psychological perspective; i guess here we have two different conversations, are these reactions scientific and are they right outside of a scientific perspective i.e in a human mode.

            As shown through many of the posts on this page ad-hominem attacks are pretty normal in the human mode, but what about in the scientific? I would say that ad-hominem attacks that are of the sort ‘your idea is stupid because your wife is ugly’ are pretty stupid. To declare that a scientist is performing bad science is closer to home though. If a scientist performs bad science then they are open to the critisism of being a bad scientist, no matter the field. This is still a type of ad-hominem attack of course and it reflects the human aspect of science working in a collaborative system, but it is still a different ad-hominem attack than discrediting a persons ideas because of his private life for example. This goes on of course, especially in politics.

            I think that the expectation that a person never be criticized outside of their data is fair, but once the data has been analysed it can often be helpful to look at a persons psychology to determine any bias in their conclusions, this can appear like an ad-hominem attack and often it probably is, but it is a relevant ad-hominem determination of their hypothesis/method/conclusions.

            I accept this is complex and that the primary mode of thought has to be analysis of the data, but this is what allows understanding and critisism of scientific fraud anywhere in the world.

            Taking the example of the lady on ‘The Edge of Reason’ who chants and believes she is increasing a persons strands of DNA back to the original 10 or 11 that we had ‘when we lived in Atlantis’, after looking at the data related ad-hominem observations are necessary to study what is happening. In fact psychology is necessarily ‘to the person’.

            Obviously much of the top quality paranormal research is of a high standard, which places strict boundary conditions on the relevance of any ad-hominem speculations regarding the psychology of conclusions.

            One interesting way of seeing how these boundary conditions may be appraised though is to look at how culturally specific some of the ideas are though, especially those relating to religions. Look at how christian theology has affected the language of paranormal thought for example, many mediums are effectively christianly culturally specific for example. Of course ad-hominem understanding is different to ad-homem attacks, but they are related when done objectively.

            It is a difficult subject, but the data has to come first. Once the data is analysed though ad-hominem appraisal is often the only way to understand why the data has been gathered as it has, or why a person has drawn conclusions outside of their data.

            All i mean here is that i dont agree with purely ad-hominem attacks, but once the data has been analyzed an ad-hominem perspective can be the only way to understand the ‘experiment’.

            Dont take this as criticism of paranormal research in general or of anyone specific though as there is good research being done by good people, but there is also a hell of alot of bad science being performed with good intention and just terrible or non-science being passed off as valid, which invites critisism from any serious perspective.

            Ad-hominem understanding also applies to advertising and big pharma as well of course, it is how we understand why they perform bad science.

          2. Just one other point
            Just one other point regarding the main subject so i’ll put it in. I’m going to email it to Randi as well if it is valid.

            As rightfully noted by many his test doesnt do anything when it produces a negative, it is only useful/amazing if someone comes in and passes it.

            The tests are well enough designed and over-viewed to be trusted scientifically i think. I imagine that if someone passed it it would be taken as affirmative proof by the paranormal community at least.

            However, the people putting themselves forward (and i thank them hugely for doing it) are likely to be as good an example of people experiencing phenomena as your likely to get so having them walk in and submit themselves to testing is something that could be made more worthwhile.

            Since negative results produce data as well it would be more interesting to see what was going on.

            Already people are commenting that the setup of the experiment stopped the psychic performing, which might be correct, so why wasnt it tested for?

            Setting the highest standard for the test is fine, that is what that test is about after all, but it is missing a chance.

            I think it would be much better if the psychic was initially allowed to setup the test to give him/her the best chance. Then once you have a positive result of psychic activity (not necessarily by paranormal means though, just a positive effect) then start upping the standards. We all know you can get an above chance effect with low standards because, well, the standards are low and anything could be happening so conclusions are invalid.

            Start low, get an effect, then start increasing the standards in the test. Then if the effect disappears then at least you can say how the psychic was doing it. At the moment we are left with very little data as to what was going on except ‘another person fails the test’, big deal.

            That would be more more interesting from an experimental and informative point of view than what was done.

          3. I like that approach
            [quote]I think it would be much better if the psychic was initially allowed to setup the test to give him/her the best chance. Then once you have a positive result of psychic activity (not necessarily by paranormal means though, just a positive effect) then start upping the standards. We all know you can get an above chance effect with low standards because, well, the standards are low and anything could be happening so conclusions are invalid.

            Start low, get an effect, then start increasing the standards in the test. Then if the effect disappears then at least you can say how the psychic was doing it. At the moment we are left with very little data as to what was going on except ‘another person fails the test’, big deal.

            That would be more more interesting from an experimental and informative point of view than what was done.[/quote]

            I like that methodology approach. I’m no expert, but I dare to say that it resembles the conditions used by researchers during the XIXth century, when very spectacular phenomena were observed, and no apparent source of hoaxing was proved.

            —–
            It’s not the depth of the rabbit hole that bugs me…
            It’s all the rabbit SH*T you stumble over on your way down!!!

            Red Pill Junkie

          4. Subjectivity of experience
            Anything that comes out of the ordinary is subjective to the extreme, like even the ordinary is subjective. And the subjectivity is in the interpretation.

            Because the human psychology is a collective animal, people fight to convince the greatest number to their subjective interpretations. That is totally counterproductive, and this is the cause for ad-hominem attacks.

            When someone does not agree, for whatever reason, he should be in position of saying why he disagrees as much as he should be in position of hearing the others’ disagreements.

            Anything outside of the process of allowing one’s own ideas to be tested by questioning is a claim of blind faith. Be it hidden behind the curtain of science or hidden behind any other kind of acquired idea, be it philosophical, reductionist or metaphysical, it does not matter. It becomes a belief.

            To me, even though we can verify that science is fast moving, which is correct, we have been much slower in its advancement than we could have been if attitudes and pride had not been added to the equation.

            Pride is the death of intelligence. Pride tells the proud that he is on top of some mountain, and that no one short of that top can show him anything worth knowing.

            This makes the whole human experience an extremely heavy bundle to carry. Way too much memory, way too little understanding.

            People do not seek to see clearly, the memorize and parrot. Two totally different approaches.

            The subjectivity of experience should not require anyone to try and convince anyone else of the reality of the experience. To know by oneself should suffice.

            But again, being collective animals, the framework of civilization being based on the most common thought forms, people tend to feel the pressure that they need to convince a majority of people of the reality of their personal experience so that they can feel comforted in the number of people saying the same thing.

            Its pointless, but it is very systematic. And this stems from the lack of personal faith in the self. This lack of faith in the self is proportional to the powerlessness of a consciousness amputated from something, or a consciousness that has not yet grown into its full maturity.

            Knowing something for yourself is very solitary. And an individual who cannot bear this solitude will not be able to live by his reality and will drift into a collective reality built to support him, a collective reality based on the powerlessness of great numbers.

            If I step out of my body, I see things and get a perception of reality that automatically becomes more vivid, more real than the physical reality, as incredible and counterintuitive as this may sound. What would be the point of trying to prove this to anyone? Even worst, what would be the point of trying to convince anyone of this? They could only believe or not, unless they already knew for themselves. So, what would be the point?

            The point would be to be comforted in the collective strength of the acceptance of a subjective experience to be real and not a fallacy (a phallacy for some).

            The point would be the incapacity to just act alone within this factual but personal condition and requiring an outside proof to be able to believe that one’s subjective experience is real. But the problem with the experience, what makes it subjective, is not its solitary implications but rather its interpretation based on the physically conditioned psychology.

            The same could be said, technically speaking, of the material experience.

            If you consider a person who would have been born and left alone on a stranded island without any other human being there to form any sort of collective. And if that person happened to step out of her body. And that this experience felt more real than the incarnated one. What would that person’s conclusion be? That the incarnated one is not real? That there are multiple realities? That the experience outside of the body is an illusion? In any case, what is certain is that this person would be forced to support this state of things on her own, would not have the opportunity to create a proof based on a collective consensus, and would therefore be obligated to define reality by herself. Not being the subject of social pressures, there would be no need to convince anyone or to defend a position.

            If a person is strong and mature in his mentality, he does not care about any sort of proof founded on consensus. He is able to know by himself and for himself. He therefore does not feel the need to prove anything to anyone, be it Randi, Shermer, the Dalai Lama or the Pope, he could not care less. He has no energy to spend in convincing anyone because he expects others to come to know by themselves and for themselves.

            If this principle was applied to all facets of life, education would be transformed into a process that sought to free each individual from any form of influence.

            Now you might argue that if that was the case, there would be no cooperation, that there would be no advancement. I would rather say that it would be the contrary. Instead of constantly trying to convince one another, we would rather look at things to see clearly, not simply memorize and accumulate data, and respect the individual before defaming them for saying something that belongs to our own field of inexperience.

            People always fight for a collective flag or another. they belong to a church, to a religion, a country, a family, a something that is not them. But by association they become that and lose their identity in the process to adopt a collective identity. This is the sign of an immature mind, a mind that like a child still needs to be supported by a collective organization, instead of working in a collectivity as a mature individual that knows his real name and not just the one that was given him in a particular incarnation.

            People want proof, and that proof is the weight of a collective consensus, until that proof becomes disproved and the consensus moves on.

            This clash of collective thought patterns is what we see in the believers vs ‘skeptics’ saga (I put it in parenthesis because intelligent scepticism is fundamental in being mature).

            Maturity is equivalent to being one’s own proof, without any support from the collective and without the need to convince others to one’s cause, because there is no cause.

          5. Richard –
            Thank you, that was very well written sense. What you said was so very true.

            Regards, Kathrinn

  2. Pshychic
    So one woman’s failure to demonstrate ESP on one particular day debunks every psychic? There’s something inherently wrong in that logic. That’s like challenging a couple of college athletes to sprint 100 meters in less than 10 seconds, and then declaring when they fail that no one can do it. There is always someone like Usain Bolt.

    If the pseudoskeptics are fair dinkum about debunking ESP with science, then why do they continually ignore the work of scientists such as Dean Radin, and the statistics gathered during America’s remote viewing program for example? I have no problem with investigating the validity of psi with the scientific method (and isn’t this exactly what Radin et al are doing?). But Randi’s challenge is nothing more than gimmicky smoke and mirrors from a veteran magician.

    A terrific book that debunks the debunkers of psi is Parapsychology and the Skeptics by Chris Carter (Amazon). If JREF members have a book club, I recommend it.

    1. Oh, you know!
      Any time you put forward data from a scientist who disagrees with skeptic convention, he automatically is no longer a scientist and his work is no longer science.

      My favorite circumstance was from an old episode of NOVA on PBS about Venus. A young scientist proposed a solution to the problem of too few craters on the Venusian surface. He postulated that the planet heats up to the point that the surface liquifies, eliminating existing impact craters. Another older scientist just dumped all over him. Why? He was proposing catastrophic change like a creationist. Things change slowly, don’t you know? Remember this is a DIFFERENT planet! Their accepted/ordained views are more important than the facts or the science.

  3. My Thoughts
    My thoughts are summed up in this line:

    “To cut down on possible sensory cues, the strangers were not allowed to interact with Patricia, and asked to wear a graduation gown and facial mask.”

    That is the experimental flaw, right there. To see if fish can swim, do we wrap them in newspaper and put them in the freezer? Patricia’s claim is for extra sensory perceptions, not replacement cosmic perceptions.

    This is what I really don’t understand about the research. Why the hoops and boondoggles? What does it prove? We are faced with a natural phenomenon and asked to study it, and then we presume how it ‘works’ and proceed on our assumptions. This is the quackery.

    It is quackery as well to ask the volunteers to select their own profile; far better to have their spouse/partner do it — astrology proposes that people appear differently depending on the observer and especially different when seen by others vs seen by themselves. You and I both know that to be true, why is it so hard for psychology to accept this?

    I think we should have let Patricia do what she does, the way she does it, in her natural environment, and taken great pains to double-blind both the practitioner and the subjects from any knowledge of their being watched and recorded. This is how we study rare phenomenon in Nature, wouldn’t it make sense to use the same technique to study rare phenomenon in our Nature?

    And then there’s this whole ‘extra’ nonesense. What if it isn’t ‘extra’, but hyper-sensory or synaesthesial perception or some other way by which the Patricias of the world are in fact exposed to every inch the same bits of information as the rest of us, but like how we can’t see butterfly markings yet the butterflies can, we non-Patricias just don’t ‘see‘? Recall Nostradamus, “He who is now called ‘prophet’ was once called ‘seer’” and how Abram became Abraham when his name was endowed with ‘sight’.

This site uses Akismet to reduce spam. Learn how your comment data is processed.