How To Tell What’s Bullshit And What’s True – Part 1

In: Advice|Education By: Brian Armstrong

31 Oct 2008

I’ve been shocked lately to notice how many of my friends (who are otherwise very intelligent) believe things which are totally false.

It’s not just my friends. A remarkably large percentage of the population (and *gasp* maybe even you) believe some things like:

You can catch a cold from going outside with wet hair.
The position of the planets at your birth changes your personality (Astrology).
Shaving causes hair to grow back thicker/faster.
More babies are born on a full moon.
Microwaving plastic food containers releases cancer causing chemicals

Sorry, but those are unlikely to be true.

There are hundreds of these myths floating around in conventional wisdom and email inboxes, and I think it’s important to understand how to be a rational human being in today’s world.

How do you decide what’s true and what’s not?

The first lesson here is to realize that just because something intuitively SEEMS true, does not mean you should believe it. Our brains have the remarkable ability to learn from the world around us, but they can also play tricks on us.

For example, it’s very INTUITIVE to our brains that if you dropped a 100lb. ball and a 10lb. ball at the same time, the heaver one would hit the ground first. But thanks to ~~Isaac Newton~~ Galileo, we now know that to be false.

Similarly, it’s very INTUITIVE for our brains to think of the earth as the center of the solar system with the sun revolving around it. It “just looks” that way because the sun comes up in the morning and goes across the sky. Copernicus eventually convinced the world the sun was at the center of the solar system.

By the way, both of these ideas were met with ridicule (and sometimes worse) when they came out. Why? It upset people because it just didn’t SEEM right. It broke their view of the world, and people get upset when you challenge their closely held beliefs.

My point here is simple: just because something SEEMS true isn’t a good reason to believe it.

Some Common Mistakes

Using Anecdotal Evidence
This is the most common one I see. It basically means using a personal experience or observation to draw a much larger conclusion. For example, “my grandfather smoked a pack a day and lived to be 95, smoking isn’t dangerous!”. This isn’t evidence at all because it’s not a controlled study with a large enough sample size to draw any reasonable conclusion.

To prove this to yourself, take something you know to be true. For example: eating tons of unhealthy food makes you fat. Now ask yourself if there is any specific case where this is not true? Well, yes, there is probably SOME lucky person out there who eats a ton of food and isn’t fat. Now if you knew nothing about food and had heard this one story (anecdote), it could be used (incorrectly) as evidence to suggest that eating lots of food does NOT make you fat.

Sure, in this case you would know better, but what about in more complex subjects like medicine or health? Hopefully you can see the danger of anecdotal evidence.

Opinions, observations, and stories are no substitute for a carefully controlled experiment (more on this later).
Assuming Correlation Proves Causation
People also make the mistake of assuming that because A happened, and then B happened, A must have CAUSED B. This is also known as the post-hoc fallacy.

Example: Kids are listening to rap music now (playing more video games, smoking pot, etc – take your pick) and violence is up, obviously the rap music caused it. Well, maybe it did and maybe it didn’t. But merely noticing that when one went up the other went up proves nothing on it’s own. Maybe B caused A, the violence caused the rap music. Or maybe they aren’t related at all. Stating a correlation proves nothing about causation.

As stated on Skeptic.com…

You have a cold, so you drink fluids and two weeks later your cold goes away. You have a headache so you stand on your head and six hours later your headache goes away. You put acne medication on a pimple and three weeks later the pimple goes away. You perform some task exceptionally well after forgetting to bathe, so the next time you have to perform the same task you don’t bathe. A solar eclipse occurs so you beat your drums to make the gods spit back the sun. The sun returns, proving to you the efficacy of your action.

Only a controlled study can show causation, not correlation on it’s own.
Thinking It’s True Because It Can’t Be Proved False
This is called an argument from ignorance or logical fallacy, where you assume that something is true just because it can’t be proved false.

It can also take the opposite form where you assume something is false just because it can’t be proved true.

Example: “You can’t prove God doesn’t exist, so God must exist.”

You can, again, prove to yourself that this isn’t evidence at all. Some things are just impossible to prove false. For example, what if I said there is a teapot orbiting the sun somewhere in our solar system. This is ridiculous of course, because even if you looked long and hard for it, I could always say well you just haven’t found it yet…space is a big place! It is impossible to disprove, and clearly this provides no evidence that a teapot does in fact orbit the sun.

Part One Summary

When you hear people say one of these, it should set off a BS alarm off in your head. Here are some phrases to watch out for:

“I know it’s real because it happened to me” (Anecdotal)
“Everyone I talked to said it’s true” (Anecdotal)
“Of course it happened after [BLANK] came in!” (Correlation/Causation)
“Well you can’t prove that it doesn’t!” (Argument from ignorance)

So far this article has mostly talked about what NOT to do.

In part two I’ll talk about what TO do: the CORRECT way to prove something, how to quickly find the truth for yourself, and a study on pigeons which will blow your mind.

UPDATE: Here is a link to part 2.

Until next time, keep breaking free!
Brian Armstrong

Be Sociable, Share!

Tags: anecdotal evidence, argument from ignorance, causation, correlation, logical fallacy, truth

17 Responses

unrev

October 31st, 2008 at 12:21 pm

Interesting post – thanks. Wanted to comment that just because science has or has not proved something to be true or false, doesn’t mean it’s 100% plausible or implausible. Remember, the entire scientific method is based on our five senses or tools we create that we can use with those senses.

An honest question: Do you think this is all there is in the entire universe, what we can observe with our senses and said tools? Or would you concede that the possibility of more existing than we can detect is quite likely, that perhaps we don’t know everything (surprise!)? Some of the latest research in physics would tend to agree: see Broken Symmetry, if you know enough physics to really understand it, you will be surprised at what you find.

By the way, two researchers (US and Japan) just won the Nobel Prize in Physics for work in Broken Symmetry.

Thanks again for the info.

Brian Armstrong

October 31st, 2008 at 12:43 pm

Hi, I haven’t heard of broken symmetry (just read a little bit on wikipedia). Interesting stuff.

But to answer your question, YES I think it’s almost certain there is more out there in the universe we haven’t been able to detect/understand yet. However, this in no way suggests that there is a supernatural explanation for it (I’m not sure if that is what you were implying, but just wanted to make the point).

If someday we do discover or gain a new understanding of those things, it will come from science and nowhere else. The beautiful thing about science is that it isn’t afraid to say “we don’t know yet” and it isn’t afraid to say “we were wrong, here is what the evidence now suggests”.

In the absence of good evidence to explain something, we have to be very careful not to “invent” a supernatural explanation. It’s much better and honest to just say “we don’t know yet” although for some reason most people aren’t able to do that.

unrev

October 31st, 2008 at 1:14 pm

Thanks for the reply!

More than anything, the idea I was implying is that humans simply can’t say (in my opinion) that we know everything, or that we have access to all knowledge in the (un)known universe.

Also, as you mentioned it, I was indeed not implying there is a supernatural reason, simply that we as humans might not have what it takes to know everything. We can do what we can, and science can do what it does by virtue of our work but that doesn’t mean that we’re grasping (or are able to grasp) everything there is to grasp.

I agree science is honest with “we do not know yet”, unlike many other disciplines – though results do get fudged (intentionally and unintentionally) from time to time!

This sort of veers off topic, so I will leave it there. Cheers!

Brian Armstrong

October 31st, 2008 at 1:47 pm

Totally agree, thanks for the comment :)

Carley

August 3rd, 2011 at 7:12 pm

It’s really great that people are srhiang this information.

Glenn Nicholas

October 31st, 2008 at 7:35 pm

Brian, I think you’ve worded things wrongly about the microwave.

You say “Microwaving plastic food containers releases cancer causing chemicals” is proved false, and yet the article you cite as disproving it states that a plastic container certified by the FDA as microwave safe is certified to release what the FDA considers a safe level of chemicals. And of course if a container isn’t certified by the FDA, there is no way of knowing what level of chemicals is released.

I think given the world we live in you might consider another factor is involved in ‘Common Mistakes’. Where rich and powerful lobby groups are involved, key messages that benefit an industry can be promoted as fact, sometimes despite scientific evidence to the contrary.

So before we make decisions we need to figure out who we trust. Do you trust research on passive smoking funded by tobacco companies? That is a pretty easy one, but sometimes commercial interests are much better hidden.

I agree with you on the importance of logic and rationality in deciding what is bullshit and what isn’t. As you can probably guess, I have concerns that in some cases lobby groups / commercial interests can exert undue influence on standards bodies who are responsible for setting safe limits.

What is a safe level of melamine to allow in milk? What is a safe level of dioxin to feed to a child? When it comes to toxins, the risk that ‘safe levels’ are wrong (or may be found to be wrong in the future) is significant, and I think it rational to err on the side of caution.

Brian Armstrong

November 2nd, 2008 at 5:44 pm

Hi Glenn, good point about the plastics. Safe level.

Also, good point about the lobby groups. Certainly have to take into account someone’s motivations or credibility when claiming “scientific evidence”.

Hopefully I’ll cover a little bit of how to unbiased info in part 2.

Caroline

November 1st, 2008 at 3:50 am

As a bit of light heartedness I think you would love a program we have over in the UK called QI. You could probably Google it to find some online videos of the show.

Brian Armstrong

November 2nd, 2008 at 5:46 pm

Ok I’ll check it out. I think Britain’s population is more open to this line of thinking & humor and it probably shows on your tv programs :)

Alicia Voorhies

November 1st, 2008 at 11:49 pm

Very well stated, Glenn.

Brian, in regard to chemicals leeching from some plastics, we have to consider the source of the information. It seems to me that the truly anecdotal evidence is being provided by the under-funded FDA. The FDA, under the guise of providing a “thorough review” of the BPA safety, somehow saw fit to only look at two industry-funded studies. And yet they were willing to gloss over hundreds of independent studies showing a clear causality of harm caused by low doses of BPA, especially in children. Will we have to wait another 20 years to find that BPA is toxic as we did with lead?

Something is obviously amiss when the Science Board and the National Toxicology Programs both take a stand against our very own consumer protection agency.

Thanks for initiating this discussion.

Brian Armstrong

November 2nd, 2008 at 5:53 pm

Good point about lack of long term testing, it’s impossible to really know at this point.

But again, because it can’t be proved false doesn’t make it true.

I think we can agree that at the very least, most people who believe it or fear it are not thinking about it rationally. If I were to guess their thought process it would be something like this: “plastic is a substance I don’t entirely understand it, microwaves are scary and I don’t entirely understand it, microwaving plastics INTUITIVELY sounds very scary.”

It’s an emotional decision rather than a rational one.

Dr. Jim

November 8th, 2008 at 1:59 am

I’m using the name Dr. Jim Because so many Learned authority figures are doctors and coughscammerscough.
This comment is based on the first paragraph of this blog. I found another blog that pointed out how the stupidest remarks go unnoticed even by seemingly intelligent people. If you are looking for something, the more you want to find it the easier it is to make you believe you found it.
And shaved hair grows back like a stump “thick to start”. if you PULL the hair, it grows back with a point and is softer so shaving does seem to make hair thicker. If money could be made with this, it would be easy to find scientific backing to prove hair grows back thicker/faster even though it’s not so.
You don’t have to be stupid to get a load of smoke blown up your a$$, you just have to be susceptible.

MadScientist

January 15th, 2009 at 1:07 am

“For example, it’s very INTUITIVE to our brains that if you dropped a 100lb. ball and a 10lb. ball at the same time, the heaver one would hit the ground first. But thanks to Isaac Newton, we now know that to be false.”

Why do you say Isaac Newton? Why not Galileo Galilei? Galileo certainly preceeded Isaac Newton in demonstrating that gravitational acceleration is apparently constant despite differences in the mass of objects (only ‘apparently’ because the earth is the overwhelming mass in these experiments and whether you use a grain of sand or a cannonball in vacuum there is no significant or measurable change in the acceleration). I have no idea if Galileo is the first to conduct experiments on gravitational acceleration, but he definitely did them before Newton came around.

Brian Armstrong

January 15th, 2009 at 3:55 am

Whoops! Good point…looks like I was mistaken there. Thanks for the correction!

MadScientist

January 15th, 2009 at 1:31 am

“I know it’s real because it happened to me” (Anecdotal)

Anecdotal evidence is not necessarily false (although it seems that it is overwhelmingly so in cases where it is the *only* evidence).

A few examples of anecdotal evidence:

1. For years pilots have been reporting what are now known as ‘sprites’. There was a lot of doubt about pilot reports in the past because the evidence was only anecdotal and pilots are known to brag and make things up. Some people were determined to find out if the stories were true and sprites were eventually photographed.

2. ‘Ball lightning’ has been reported by many people and I don’t know if anyone has had the opportunity to photograph the phenomenon yet. I had seen this creepy thing so there’s no doubt in my mind about it, but I’d probably never believe it unless I saw it for myself because I can’t understand how it would form, it is not a very commonly observed phenomenon (for example, rainbows are common), and I just don’t know of anyone capturing it on film or on a video camera. I’ve only ever met one other person whom I know had seen the thing (on a different occasion; we did not see the same event).

3. The physicist Richard Feynman swore by hypnosis. I thought to myself “could I believe in hypnosis because such an amazing physicist swears by it?” The answer was “no” because it remains anecdotal evidence and in fact the evidence is against hypnosis.

4. One day I saw the most remarkable rainbow – beneath the rainbow (in succession) were 5 rainbows which were each less vivid and this produced an awesome serape in the sky. There was no gap between the rainbows and the colors were all in the same order. There were two physicists with me at the time and I pointed the thing out. Naturally no one had a camera (and the physicists were talking about something else so they pretended there was nothing unusual in this rainbow). I spent weeks looking for a description of the phenomenon and I went off to the library and took out a number of books on atmospheric phenomena. My rainbow wasn’t in any of them. So here’s anecdotal evidence of a rare type of rainbow – it *does* exist, the question is ‘when will someone photograph the damned thing so we have evidence of it’. I’ll settle for someone modelling this rainbow just as people have modelled other atmospheric phenomema including the more common rainbows, sundogs, auras, and glories.

Brian Armstrong

January 15th, 2009 at 4:07 am

Good point – anecdotal evidence is not necessarily false. It can give clues about an area worthy of further research. Of course, it is also not necessarily true (which seems to be the more common error in judgment people make). Very interesting stories…thanks for stopping by!

Breaking Free

How To Tell What’s Bullshit And What’s True – Part 1

17 Responses

Leave A Comment

About this blog

Most Popular Posts

You should follow me on Twitter here.