Ok so in Part One we talked about how people often form incorrect beliefs about the world due to anecdotal evidence, the correlation-causation mistake, and the logical fallacy “if you can’t prove it false, it must be true”.

Today we’re going to talk about the correct way to decide if something is true: the scientific method.

The scientific method is an entire process with plenty of steps. But I’m just going to show you 3 pieces of it which you may not know about or have forgotten since high school.

The scientific method is AWESOME because it filters out bullshit. No longer can someone make a claim without backing it up and going through the right steps. In fact, that’s why I think it’s so important to understand this: if you see someone skipping the steps they are most likely LYING to you.

1. Double Blind Experiments

   Human being’s have biases. Even when you think you don’t have a bias, you might. For example, you may have heard of the “placebo effect” where people who take sugar pills (which do nothing) say they feel better. People who are given taste tests usually choose their regular brand, but when the brands are hidden they choose differently. Even more shockingly, you might think you don’t have any race or gender biases but you can take this test as many times as you like and you probably do.

   So this means proper experiments must have a “control”. If you are going to give a pill to one group, then you should have another group of the same size that you give the placebo too. They are the “control” group and only by comparing the results of the two can you isolate one variable to test without bias.

   In a true double blind experiment, the person being tested doesn’t know which group they are in (the “real” one or the control) and neither does the researcher.

   Double blinds experiments provide great evidence that Astrology is bullshit. As you can probably guess, all you have to do is not tell the test subject which month or “sign” you’re reading from and ask them if it applies to them. They end up saying yes or no just as often no matter what their sign is.

2. Peer Review

   Peer review means that if you want to claim something is true, you have to document the steps of your experiment and other scientists around the world have to be able to duplicate your results.

   If other people can’t duplicate your results then you are probably full of shit.

   For example, you’ve probably heard of people trying to run a car on water. Water has hydrogen in it and hydrogen is a good fuel, so it would be great if we could get the hydrogen out of the water. The problem is that it takes more energy to separate the oxygen from the hydrogen than you get out of the hydrogen so it doesn’t work.

   Yet this doesn’t stop some lunatic from coming out (probably every year) claiming they’ve invented a car that runs on water. Even major news organizations get duped occasionally like when FOX News in Houston ran this video segment giving one such lunatic a lot of press. It turns out that the guy is using an Oxyhydrogen torch which has been around since 1977 (and still uses more energy than it produces in hydrogen).

   From peer review of his patent application we now know him to be full of shit, and his “company” Hydrogen Technology Applications is a scam. The FOX News reporters should have read this blog article so they understood peer review :)

3. Sample Size

   This is also sometimes called “statistical significance” and basically means you need to run your tests enough times (or with enough people) to show the result didn’t happen from chance.

   Interestingly, this also shows why anecdotal evidence is not evidence at all. Your sample size isn’t large enough if “everyone I talked to” means five people.

   For example, let’s say that you want to test how often a flipped coin comes up heads vs. tails (which we know to be 50/50). Well if you did the test 1 time, that obviously wouldn’t be enough. But what if you flipped the coin say 5 times. Would that be enough?

   Well it depends. There is about a 3% chance you’d get 5 tails in a row, and a 3% chance you’d get 5 heads in a row (which is no where near the true answer) and a range of other possibilities.

   Would 10 be enough? 20? Without getting into the math, the main point to take away is that you must have a large enough sample size to be “confident” about your answer. Any claim which isn’t based on statistically significant data is meaningless.

So to review, what is wrong with this statement:

I’ve developed an incredible new weight loss pill! Of the 10 people I gave it to 10 of them lost weight and no one died. Buy it now!!!

I’m exaggerating a little of course :) But this should raise your bullshit detectors because:

• There was no “control group” of people who DIDN’T take the pill (Double Blind)
• The claim has not been verified by anyone else (Peer Review).
• Testing 10 people is not enough to draw any meaningful conclusions. (Sample Size)

A Pigeon Video

Finally, I’ll leave you with this interesting pigeon video as promised. This is from Richard Dawkin’s excellent documentary called Enemies of Reason.

It shows an experiment done by B.F. Skinner which gives some shocking insight into human behavior. If you can’t see the video below click here. I suggest watching the whole thing if you have time, it’s excellent.

The pigeon part is around the 36 minute mark.

What do those pigeons remind you of? Sports fans at a game who wear their “lucky” hat? Something more serious like gambling? And the scariest of all: perhaps a belief that you have?

Conclusion

It bothers me that more people don’t understand these sorts of things. In the dark ages we were primitive people running around naked, dirty, and ignorant – slaves to superstitions and scam artists making false claims. Blood letting was common medical practice. Then in the enlightenment we developed these brilliant tools (double blind experiments, peer review, and statistical significance are just 3 examples) that allowed us to do a very important thing: TELL BULLSHIT FROM THE TRUTH.

This lead to science which has allowed us to get a heart transplant, email people around the world, and travel to the moon.

But sometimes it feels like we’re heading back to the dark ages. I heard someone remark one time that:

The last time I went into a book store there were 10 books on astrology and 2 on astronomy.

Hopefully this article will help someone people make educated decisions about the truth.

Until next time, keep breaking free!
Brian Armstrong