Critical Thinking

Six rules of evidential reasoning (FiLCHeRS)

Falsifiability

It must be possible to conceive of evidence that would prove the claim false.

Any claim that could not be falsified would be devoid of any propositional content; it would instead be making an emotive statement rather than a factual assertion. It describes value orientation.

The first variety of non-falsifiable statements is the undeclared claim - a statement that is so broad or vague that it lacks any propositional content. The undeclared claim is basically unintelligible and consequently meaningless.

"Crystal therapists can use pieces of quartz to restore balance and harmony to a person's spiritual energy."

What is unbalanced spiritual energy? How is the condition recognized and diagnosed? What evidence would prove that someone's unbalanced spiritual energy had been, or had not been, balanced by the application of crystal therapy? The undeclared claim has the advantage that virtually any evidence that could be adduced could be interpreted as congruent with the claim.

The second variety of non-falsifiable statements involves the use of the multiple out - an inexhaustible series of excuses intended to explain away the evidence that would seem to falsify the claim.

"The healer didn't heal that person."
"That's because that person didn't have enough faith."

"There's no real evidence to support the UFO theory."
"That's because there's a secret government conspiracy."

Logic

Any argument offered as evidence in support of any claim must be sound.

An argument is said to be "valid" if its conclusion follows unavoidable from its premises. It is "sound" if it is valid and if all the premises are true. The rule of logic thus govern the validity of inference. An invalid argument can be recognized by the simple method of counterexample - if you can conceive of a single imaginable instance whereby the conclusion would not necessarily follow from the premises even if the premises were true, then the argument is invalid.

All dogs have fleas. Xavier has fleas. Therefore Xavier is a dog.

Invalid - Counterexample: Xavier is a cat who has fleas.

All dogs have fleas. Xavier is a dog. Therefore Xavier has fleas.

Unsound - Not all dogs have fleas.

Often, a given premise requires additional knowledge about the claim that would require empirical investigation.

"The 16th century Turkish admiral Piri Re'is made world maps so astoundingly accurate that they could have been only made with satellite photographs." - von Daniken, Chariot of the Gods?

Invalid: any number of other techniques could be used to produce accurate maps.

Unsound: the Piri Re'is map contains many gross inaccuracies.

Comprehensiveness

The evidence offered in support of any claim must be exhaustive - that is, all of the available evidence must be considered.

You just can't take the evidence that supports your theory and discard the rest.

"Jeane Dixon predicted that George Bush would win the election. She is a real psychic." 

True, but she also predicted JFK would lose in 1960, World War III would start in 1958, and Fidel Castro would die in 1969.

Honesty

The evidence offered in support of any claim must be evaluated without self-deception.

This is a corollary to the rule of comprehensiveness.

"But I saw him bend the spoon the first time. I don't know why he can't do it anymore. It must be an elusive phenomenon."

This usually boils down to an injunction against breaking the rule of falsifiability by taking a multiple out. But you must also accept the obligation to come to a rational conclusion once you have examined all the evidence. If the overwhelming weight of all the evidence falsifies your belief, then you must conclude that the belief is false, and you must face the implications of that conclusion forthrightly. Typical symptoms include denial, avoidance and rationalization.

Replicability

If the evidence for any claim is based upon an experimental result, or if the evidence offered in support of any claim could logically be explained as coincidental, then it is necessary for the evidence to be repeated in subsequent experiments or trials.

This is a safeguard against the possibility of error, fraud, or coincidence. A single experimental result is never adequate in and of itself. Any experiment, no matter how carefully designed, is always subject to the possibility of implicit bias or undetected error.

Sufficiency

The evidence offered in support of any claim must be adequate to establish the truth of that claim, with these stipulations:
1. the burden of proof for any claim rests on the claimant;
2. extraordinary claims demand extraordinary evidence; and
3. evidence based upon authority and/or testimony is always inadequate for any paranormal claim.

The burden of proof rests with the claimant for the simple reason that the absence of disconfirming evidence is not the same as the presence of confirming evidence.

"Because no one has disproved all UFO reports, the ones that are left prove that they do exist."

"Because no one has given me evidence that God exists, it stands to reason that He doesn't."

"Adolph Hitler is living in Argentina. You can't prove that he doesn't so he must."

It only means that it is possible and not conclusive that it is true. Argument could reduce it to highly improbable. But logical possibility is not the same as established reality. The claimant must produce evidence.

Extraordinary claims demand extraordinary evidence for the obvious reason of balance. I need more evidence that aliens performed medical experiments on you that you saying that it rained for ten minutes last Tuesday.

Testimony is always inadequate for paranormal claims because people can lie or make a mistake. Expertise is not a guarantee of infallibility or lack of motivation to lie. Credentials, knowledge and experience cannot, in themselves, be taken as sufficient evidence to establish the truth of a claim. Sincerity lends nothing to the credibility of a testimony.

Conclusion

The first three rules - falsifiability, logic and comprehensibility - are all logically necessary rules of evidential reasoning. The last three rules - honesty, replicability, and sufficiency - are all pragmatically necessary rules of evidential reasoning. Passing all six tests will justify you in placing considerable confidence it the claim but does not guarantee that the claim is true. There could be new disconfirming evidence tomorrow. It only guarantees that you have sold your belief for a fair price and that it has not been filched from you.

Being a responsible adult means accepting the fact that almost all knowledge is tentative, and accepting it cheerfully. You may be required to change your belief tomorrow, if the evidence warrants, and you should be willing and able to do so. That, in essence, is what skepticism means - to believe if and only if the evidence warrants.

adapted from "A Field Guide to Critical Thinking" by James Lett, 
Skeptical Inquirer, Winter 1990, p. 153-160

James Lett is an associate teacher of anthropology, Department of Social Sciences, Indian River Community College, 3209 Virginia Avenue, Ft. Pierce, FL 34981. He is author of The Human Enterprise: A Critical Introduction to Anthropological Theory.

Philosophy History First aid The case Repertory Materia medica Case management Non-classical topics Reference News