Blog Archives

Baloney Detection 101 — Testimony

Skeptics place very little value on improperly attested authority and evidence, particularly inadequately documented and untrained personal testimony, especially when such testimony is uncorroborated by other sources of evidence or is several times removed from the original source of a claimed event.

This is rightly so, for such notoriously unreliable testimony is subject to a host of perceptual and memory fallacies and biases that make them undesirable as useful data for testing hypotheses, no matter their status in a court of law.

Such anecdotal accounts, as they are known, are useful primarily as a starting point for forming a hypothesis for the beginning of an investigation, not validating or falsifying claims.

For that last they are useless by themselves…

But not all forms of testimony are of the undocumented, untrained, and generally unreliable sort. But how do we go about knowing which to trust and which to be suspicious of? How do we properly evaluate sources of testimony?

Below, I’ll provide a few useful tips so that authoritative sources of information may be more easily assessed and recognized as valid or not, as expert or non-expert in a field, because let’s be realistic — would you rely on someone trained only as a motorcycle mechanic to give you a root-canal in a dentist’s office? — I certainly hope not.

Simply put, and in the broadest useful sense, testimony can be defined as any information, whether opinion or fact, that we personally lack direct access to by way of our own experience, that which we must obtain from a source outside ourselves, often a recognized authority such as a book, a person, or less individually, a media outlet or other organization such as the publishers of a prestigious research journal, a news agency, an internet website, radio station or television network.

It’s important to note that expertise is by no means global, that it doesn’t automatically transfer from one area of competency to another, often unrelated one. After all, someone who lacks training as a cosmologist is unlikely to be in the best position to propose revolutionary theories of the universe, and likewise, someone uneducated as an archaeologist is probably not the best qualified to put forward controversial theories of ancient Egypt or unproven claims of Lost Civilizations and knowledgeably present these things as if they were fact.

Above all, a source must be credible, and there are guidelines we can use to gauge that credibility.

  • Relevant and true background & expertise: How competent in a given field of practice or study is the alleged authority whose testimony is used? Does the authority actually have training and know-how in a given field directly pertaining to questions on a certain topic? Is a physician prescribing a particular drug, or administering a surgical procedure or treatment plan actually trained in the relevant branch of medical practice, or is their training only in journalism or advertising, for example, without any real credentials in the medical profession at all?
  • An established track record: Does the expert whose testimony is used have a prior history of success in giving sound information based on demonstrated ability in a field of expertise? Can their credentials and statements be verified through independent fact-checking? If not supported by a prior record of credibility and competence by other sources already accepted, this authority’s testimony is probably less sound.
  • Eyewitness information: It’s generally thought in a court of law that seeing an event yourself is much better than mere hearsay evidence, and in a legal trial much more credible, though often some other form of evidence is needed to corroborate such eyewitness reports due to the well-known inconsistency of human perception and memory over long periods of time, such as a murder trial taking place some years after the alleged event happened. This is less reliable as scientific data without independent confirming evidence.

But there are a few things which can detract from the credibility of a source. What are they?

  • Is the source somehow biased or in possession of a vested stake in promoting their claims? This is referred to as ‘eager evidence,’ and is to be looked out for when a source’s testimony is made for probable personal gain or when there is a likely ideological stake in promoting a given claim or set of claims, and this bias and vested interest are both true and relevant to the claims made. Is the source of testimony prone to making similar claims?
  • Is there a genuine dispute among authorities on the topic? Or is there a solid consensus among experts? Are the claims made by the source in agreement with those made by others of better-known and validated expertise, and are these claims confirmed through multiple channels of other forms of evidence already accepted? Or are the claims made by a ‘lone voice crying in the wilderness?’ The testimony of 100 experts in agreement is generally more sound than that of only 1, and that of 1,000 experts is even better, and so on. While a consensus can sometimes be mistaken, it’s better than relying on the claims of a single maverick just because he’s a maverick. Heretical does not necessarily mean ‘correct.’
  • Does the source have a sound basis for making their statements or claims? Does the expert in question have some knowable method to arrive at the claims they do? Are the methods they use publicly verifiable and democratically open to use by others to confirm findings, or are they secretive and arcane in their methods, using subjective personal means difficult if not impossible to get consistent results between different users?
  • Is the source incompetent to discuss the topic of discourse? Is the authority obviously not an expert on the topic? Does he or she promote themselves as an expert because they lack expertise or education in a subject and claim that their unconventionality makes them more creative or better able to ‘think outside the box?’ Sorry, but lack of education or training doesn’t make you more imaginative, just uneducated and untrained.

These tips should prove handy in assessing claims by many self-styled authority and are a useful list of ‘red flags’ to look out for regarding those making extraordinary claims, or in general, claims that sound too good to be true — and which probably are.

Why should you just take my word for it? The short answer: Don’t. Check things out for yourself. Look into the claims encountered in your life every day on your own, hopefully with a more critical eye than before…

…After all, gullibility can sometimes kill, and it pays to be skeptical.

RDF TV – The Baloney Detection Kit – Michael Shermer


Get every new post delivered to your Inbox.

Join 2,977 other followers

%d bloggers like this: