Thursday, April 06, 2006

how to god-proof an experiment

Okay, so snarky nonreligious folks have had much to crow about the recent large-scale study that fails to find effects of intercessionary prayer. Given that I am a nonreligious person with a strong tendency toward snark, one might expect me to pile on.

Instead, the part of this that I really, truly do not understand is why someone with theistic beliefs would, if they thought the matter all the way through, expect there to be observable effects of intercessionary prayer in a randomized trial. If anything, one could argue that under the usual conditions of such trials any effects, if observed, would actually be more evidence against the existence of a God like the one that most Christians I know talk about, and would rather be evidence for a different kind of 'action at a distance' force operating in the universe. (That said, elaborated experiments could show that intercessionary prayer had an effect only when prayers to a specific God, etc., was prayed to, which obviously, if indisputable effects were observed, would be strong evidence for the implied cosmology).

I'm completely serious about this. I don't get it. If you are a Christian who believes that intercessionary prayer is something one should be able to study using a randomized clinical trial, I would love to be able to ask you questions about your conception of God and how prayer works. Here, on this blog, we could have a colloquy. Let me know. I mean, I'm open-minded about this, but I think that you could only hold that kind of hope about a trial if you hold internally inconsistent beliefs about God and prayer.

To put my argument simply: Person A has been randomly assigned to the treatment group. Person B is in the control group. Solely because of this random assignment, some well-intentioned-science-minded people pray for Person A and not Person B. Say you were God. Do you really help A and not B? What kind of God would do that?

Apart from that question, consider the point of the "randomized" part of randomized clinical trials. Randomization allows for powerful causal inferences because assignment to the treatment can be presumed to be independent to the other potential causes of the outcome. Except, um, when you are talking about God as a potential cause. You flip a coin to assign subjects to the treatment and control groups, and you want to study the causal effect of God but assume that God has no causal influence on the outcome of the coin flip. This same God who will subsequently save lives can't turn a heads into a tails? Seriously, how can you reconcile God and random assignment for a study of the effect of God?*

This led to a weird diversion this afternoon, which was trying to think about how one would go about trying to God-proof an experiment. In other words, how could you design a clinical trial so that you could have the most confidence that God was not mucking with your random assignment? I decided that you had to adopt a strategy where the assignment was not random, but could be presumed effectively random for the purposes of the experiment, and where the nonrandom assignment algorithm was as costly for God to muck with as possible. In other words, you would have to make it such that, for God to mess with your random assignment, God would have to had to actively intervene in the world in a very large and complicated series of ways--with who knows what effects--in order to exert the same kind of control over who was in the treatment and control group as if assignment was just based on a coin-flip. Specifically, I came to the conclusion that assignment should be based on the first two letters of one's surname, with "aa" being in the treatment group, "ab" being in the control group, and so on. The point is not that God would be incapable of still intervening to have deterministic (a.k.a. God-like) control over your assignment of treatment and control groups, just that your experiment would need to have been a very high priority influencing all kinds of other things in the world in order for this kind of deterministic influence to be exerted.

Also, by my calculations, you would need a very large sample.**

* As it happens, studies of intercessionary prayer are typically agnostic as to whether the effects of prayer are caused by God or something else. This is my whole argument in the second paragraph. My belief is that, if effects were observed, they would be evidence for the causal effect of that "something else" more straightforwardly than evidence for the causal effect of God.

** Preferably a census for whatever illness one was considering. Otherwise, all your God-proofing of the assignment process would get messed up by the non-God-proofing of the selection into who participates in the experiment.


Anonymous said...

I almost got kicked out of parochial school for responding to the statement from my pastor that there was nothing god could not do with the observation that god could not make a rock he could not lift.


Anonymous said...

It's fun to recapitulate debates from the 1870s!

jeremy said...

Why one would regard the 1870s as a watershed decade for issues discussed in this post is a mystery to me. In any event, my post was prompted by a major study with substantial government fundings that had findings come out last week.

Andrea said...

Ah the folly of mortals!
Jeremy, God knew all about your "randomization" scheme when he was constructing his plan for mankind. God's omniscient and knew what he was doing when he gave us those surnames.
Of course the only way your random assignment is ever going to work is if you pray to God and ask him to turn a blind eye to your work.
Furthermore, I can tell you why the study didn't find evidence for the effectiveness of prayer. God didn't want it to, of course! Faith is the cornerstone of our relationship with God, not evidence. If anything, God messed with those results to challenge the faith of those who seek to be close to him.

What fun! I could go on like this all night. Thanks for the laugh!

Rhymes With Scrabble said...

I don't think the "what kind of God would do that" argument is very productive (there's some quote that runs roughly, "The God in whom I don't believe is a kind and loving God, not the mean and stupid God you believe in"), but other than that, I thought your points were good. I enjoyed the post.

It also reminded me of a link I saw on CNN recently about some guy who was claiming that Jesus walked on ice and thus APPEARED to walk on water, which strikes me as pretty ridiculous. If you don't believe that Jesus performed miracles, isn't the simplest solution to assume that some tall tales have gotten spread around in the last 2000 years, rather than try to come up with some complex mental gyrations that demonstrate, okay, people THOUGHT he was walking on water, but there's a PERFECTLY LOGICAL EXPLANATION.

Anonymous said...

"Why one would regard the 1870s as a watershed decade for issues discussed in this post is a mystery to me."

Well, for one thing, the 1870s spawned (some of the) first major public debates between Galton, Tyndall, etc and Church of England bishops over whether or not the efficacy of prayer could be tested through scientific study. Galton attempted this in 1873, if memory serves, though it was a very different sort of study than, say, Byrd's 1988 version, or the more recent government study that sparked this post. Nonetheless, a watershed decade for this debate. Less of a mystery now?

jeremy said...

Hey, Anon, you're absolutely right. Rock on, 1870s.

jeremy said...

Islander: the idea with the surnames is that God would have to have intervened in a lot more history to make the experimental assignment work out right. The ideal assignment mechanism would be one for which the whole of history is manifested in the way of assigning each person to treatment or control.

Anonymous said...

Pulsars in deep space (rapidly spinning neutron stars, I think) emit X-rays whose temporally-fluctuating frequencies are seemingly random. These fluctuations have actually been proposed as random number generators for cryptographic purposes. Anyway, if G-d were to rig an experiment centered around random assignments based on observed pulsar frequencies, he/she would have had to do it approximately 15 billion light years ago, since the electromagnetic emissions that are used as the basis of the experiment have been intransit to observatories on Earth for this long. Does this qualify as "difficult" for our interfering deity?

dorotha said...

can't you just put the experiment inside a bottle with a god-proof cap?

Anonymous said...

If you are trying to investigate the efficacy of prayer with respect to the Christian god, then not only are you going to run into the problems Jeremy describes, you are also making a mistake with respect to the Christian theological tradition. See Matthew 4:5-7 for details.

jeremy said...

Dorotha: The bottle also needs to be made of God-proof glass.

Jon: The pulsar idea doesn't work so well because of the possible Godly influence over particular pulsar selection (not to mention pulsar measurement). The thing I liked about surnames was that it seemed simple and yet pregnant with history.

Kieran: I am no theologian, but I've always taken that passage to suggest a prohibition against classic hypothesis testing vis-a-vis God, but not necesasrily against the broader effort to estimate the causal effect of God.

Anonymous said...

I take the temptation scene to mean that you can't ask God to prove he is all-powerful or manifest himself in a way convenient to you -- and a fortiori, you can't force him to do things on your behalf. Prayer is not a way of twisting God's arm, just like Good Works are not a way of bribing God. So a believer should have no expectation that a test like this would be of any use.

Anonymous said...

Hmm. So many questions. Who exactly were not only the investigators but the praying ones?
Were they were making specific requests? Rote? What?

Did they add "but not my will but Yours"? Placing take-out orders?
Were they open (humble even?), wise enough (faithful, patient even --) to understand the answers that came -- to them, the praying ones?

If the researchers manage to get more money, I hope they manage to define 'prayer' more objectively. I wonder if they're visualizing a passive, mystic telephone. line.

Anyway, JF, you raise good questions.


Anonymous said...

Meanwhile, less randomly, over at Boston's MGH, they fairly recently have begun adding a chaplain to some of their patients' medical teams when they found positive effects in the balance (you can look up the details).

Anonymous said...

I hope you're meeting some of your Divinity School neighbors. Not religious? Hey, this is Harvard.

I think you should give a talk over there on your ideas. Ask Harvey Cox about it.