Baylor religion study biased
October 4, 2006
Baylor University has recently conducted a detailed study of religious
belief and released preliminary results in September 2006. The study was
funding by the Templeton Foundation, famous for giving grants to
organizations and individuals who try to reconcile religious belief with
science. The survey's results have been widely reported. A closer look at
the results, however, reveals spin, bias, and important findings that have
not been widely reported. The Baylor report is
available here (in PDF).
One widely reported finding from the study was the number of "Nones," or
people without religious affiliation, has allegedly been dropped from about
14% of the population (found in earlier studies) to around 10%. This finding
was uncritically repeated, for example, in the Dallas News under the title
"Study: Number without religion is overstated." The researchers attribute
this to more targeted questions, but it is an unfortunate artifact of their
biased methodology. Respondents were asked about both their religious belief
and any affiliation with a church. The study's 10% finding is the result of
effectively changing the answers of the people who claimed no belief
affiliation but who did report a church affiliation. The findings confound
the two issues. There are many reasons to attend a church beyond one's
religious belief including involvement with a community, friends, or family.
To confirm this problem, Baylor sociologist Kevin Dougherty says that "when
we asked the same questions other surveys asked about identification or
preference, we got the same roughly 13% to 14% they get." It seems their
methodology was devised to reduce the number of "Nones."
Perhaps the next version of the study should ask the number of people who
have ever visited a doctor or hospital and subtract those respondents from
the number of reported believers. Seeking medical help is a clear demonstration that
the seeker does not believe that appeal to an omnipotent god has hope of
being effective. You'll never see questions concerning practical
religious belief on religious study instruments, however, because most religious studies
are designed to inflate the number of believers. Numbers, whether
real or not, are a source of political power.
On a related topic, any conclusions reported about atheists, agnostics,
freethinkers, brights, rationalists, or humanists are suspect. The study
lumps all of these groups into either the "no religion" category or the
"other" catch-all category, both of which may be problematic for
characterizing non-believers who have the courage of their convictions.
While the survey instrument may have some validity for believers, most
non-believers would probably lose interest in the survey as they are not
recognized as a group worthy of mention despite being a larger group than
many of the religious beliefs individually listed. Given that and the low
response rate for the survey (46%), you can bet that many non-believers were
simply not counted after having been exposed to such and obviously biased
survey instrument. The instrument is included in the last few pages of the
above PDF file, should you care to examine it. The report even makes the
claim that "Atheists are certain that God does not exist," which shows an
amazing lack of understanding of their subject matter on the part of the
A surprisingly high number of respondents (43%) were reported to have
admitted to having "prophetic dreams," or dreams that predict the future.
The report lists this number among the various paranormal beliefs that the
study looked into. The prophetic dreams results were even included in an
overall measure of paranormal belief, used in later correlations. Upon
further inspection, however, one discovers that the "prophetic dreams"
category is based on two questions in the survey instrument in a category
listed as "The New Age":
As an adult, have you ever had one of the following:... A dream that later
- To what extent do you agree or disagree with the following statements:
... Dreams sometimes foretell the future or reveal hidden truths.
There are certainly non-paranormal reasons for responding to both
questions positively. Dreams are often about one's life and issues that
concern a person. It's not at all remarkable that people would commonly
dream about their own futures and their future actions--things that might
later come true. For example, if you were having trouble at work and dreamt
that you were fired. If you were in fact fired after the dream, it doesn't
mean that your dream was prophetic or that you were psychic, even
though you would likely answer the survey question "yes." As for "hidden
truths," dreams can be a window into one's own subconscious.
Such interpretation can be based on psychology--not parapsychology. Perhaps
by making the finding more lurid, the researchers were able to get more
publicity for their survey results.
The Christian biases of the surveyors show up in their reporting of beliefs
about the personality a monotheistic god, presumably for the subset of the
respondents that believe in a single god. This category includes Deists, who
worship the God of Nature, Jewish believers who worship Yahweh, Muslims who
worship Allah, Wiccans who might claim to worship the Goddess. All of these
different beliefs were categorized under belief about "God," the proper name
of the Christian god. Perhaps this complaint is a nit, but it does not show
respect for the beliefs of non-Christians, which you would expect in a formal
study of religious belief. The Austin American Statesman repeated this mistake in their uncritical articles on the study.
The final issue of spin is a finding about beliefs in the personality of
their god that the report didn't fully explain. The survey divided the space
of possibilities into two orthogonal dimensions that cover the space of
possibilities. Either their god is engaged or not. Either their god is angry
or not. From these orthogonal dimensions, the survey had a number of
questions that grouped believer respondents into four categories to describe
the personality of their god. These categories were benevolent, authoritarian,
critical, or distant. The survey found that believers fell into these
categories with roughly equal numbers. While the study focused on the
demographics behind the different categories, it neglected to point out that
the relatively equal numbers of believers in these categories is evidence
that there is no objective reality behind the assertions. Presumably, if
these believers were having regular conversations with this unique god in
prayer and worshiping it in their churches, they would, in the aggregate,
have some statistically significant agreement of what that god's personality
is. By contrast, I'll bet the majority of believers could correctly name the
color of the mid-day sky.
The Templeton Foundation did not spend their money wisely on Baylor's work,
if they were looking for a serious study of religious belief. If, on the
other hand, they were looking for
the Templeton Foundation got what it paid for. Unfortunately for them, their
money gave another convincing piece of evidence against religious belief.
As the reader may recall, the Templeton Foundation funded a
and well designed study that showed no positive effect for intercessory
- half baked "research" to downplay the number of Nones that would be
picked up and uncritically reported by the press;
- information that would help with marketing to believers; and
- demographic information that would help politicians use religious
belief to their advantage,
Baylor University is home to Intelligent Design promoter William Dembski,
who, despite years of work, has yet to make a claim in support of an
"intelligent designer" (aka "God") that hasn't been conclusively proven false by the scientific community.
From the officers:
The ACA Lecture Series continues Sunday, March 8th at 12:15pm at the Austin History Center, 9th and Guadalupe. The building opens at noon. Ryan Bell will talk on "My Year Without God: Now a Permanent Condition."