It’s time for a follow up to see how effective my sales page redesign was from a few weeks ago.

First lets take a snap shot of the two weeks prior to changing the sales page, July 11th through July 25th.

• 685 Unique Views of the sales page
• 21 Books Sold
• 3.1% Conversion Rate

I’ll skip the 26th since that was the day I made all the changes. So let’s look at July 27th through August 9th (almost a full two weeks) as the next snap shot:

• 314 Unique Views of the new sales page
• 8 Books Sold
• 2.5% Conversion Rate

Most people will jump to an incorrect conclusion when seeings numbers such as these.

This is a great example of how important it is to really test. If I had just been watching the sales come in I would have had the impression that things were much worse (21 sales down to only 8). But we quickly see that far more people viewed the page in the two weeks prior to the new page going up. This was mostly due to the link from CopyBlogger.com. In fact, about half of those 685 visitors came on the one day (the 24th) when CopyBlogger released the article.

But at least we have the conversion rate to help us compare apples to apples right? Well, sort of. If you ever took a statistics class you may remember that we need a large enough sample size before we can draw any reasonable conclusions about which performed better.

To understand this, imagine flipping a coin to see how often heads vs. tails comes up. Suppose you flipped the coin three times and got 2 heads and 1 tail. If you stopped your testing there, you would have to conclude that heads comes up 66% of the time and tails comes up 33% of the time, which we know is incorrect.

What if you had flipped heads 3 times in a row? You’d have to conclude that you get heads 100% of the time, which is equally ridiculous.

So the larger your sample size in statistics, the more confident you can be about the results. With only 3 tosses, you aren’t very confident about your results. But if you flipped the coin 100 times, you can be much more confident. After all, how unlikely is it to get heads 100 times in a row? (That was meant to be a rhetorical question, but for you analytical types, yes it is 0.5 to the hundredth power.) With 100 tosses, it’s pretty likely you’d get something close to 50/50 which is the correct result.

Anyway, for examples beyond a simple coin flip, the math gets tougher, so I like to use this website called SplitTester.com to do it for you. You can plug in your data (21 and 3.1%, 8 and 2.5% in my case) and it will tell you how confident you can be about your results. In my case, it tells me my confidence interval is low.

As a preliminary result, it looks like the numbers are very close to being the same. But it looks like more testing is required before a conclusion can be made about which was better! Continually test…but be careful about jumping to conclusions.

Do you test your sales pages and if so how?