Results Of UniversityTutor.com Price Testing

In: Uncategorized By: Brian Armstrong

15 Jul 2009

About a month ago I wrote about how I was going to split test the prices on UniversityTutor.com.

I thought I’d share some results with you. As you recall, I was split testing two price points.

The way it works is that tutors sign up on the site and get three free job requests. After that they get an email asking them to upgrade and become a paying member. I’ve recorded the number of emails sent out, and the number that actually subscribed.

Test Group 1: $9.95/month or $79.95/year

Test Group 2: $5/month or $45/year

Just a quick reminder: both price points had a monthly and yearly option. I was NOT testing the monthly price vs. the yearly price. I was testing a lower price (both monthly and yearly) vs. a higher price (both monthly and yearly).

Here are the results:

There isn’t quite enough data yet to be super precise, but we can paint some broad strokes…

The conversion rate was about double for the lower price. That means I could make the service affordable to twice as many people with no loss in revenue (one option).
Very few people chose the yearly option. After thinking about this a bit I realized that most tutors are college students and will be moving during the summer. They probably don’t plan on tutoring year round necessarily. I’ve since added a line to the email saying “you can move or change cities and keep the same tutor account active”. This may help, we’ll see.
Revenue was higher in Group 2 because one person chose the yearly option. Without this, revenue would have been $60, and revenue per email $0.42 – just barely lower. The only reason you might want to look at it like this is that with so few people choosing the yearly option, we don’t have statistically significant data there.

The 4% or 8% conversion rate might sound low, but from what I’ve seen of other major sites this is actually quite good.

Also, just a quick note on HOW I actually did this test. I unfortunately had to program it by hand. Basically I setup a piece of code which recorded whenever an interesting “event” took place, such as an email going out or someone subscribing and saved it to the database. I wish there was an easier way to run tests like this. The closest thing I’ve seen is Google Website Optimizer, but I’ve found it very difficult to use when running tests like this, and that’s why I coded it by hand. Not an ideal solution for everyone I know, but I thought I should mention it.

Next steps?

I haven’t completely decided yet, but the option I’m leaning to here is to go with the lower price. It seems worth it to get more tutors signed up at the lower price (who can evangelize the product and tell others about it – not to mention having more tutors available in searches on the site).

What do you think? Also, if you have any other suggested split tests you’d like to see, let me know in the comments and I can try running them.

Share and Enjoy:

20 Responses

Natalie

July 15th, 2009 at 1:07 pm

Brian – I agree with your conclusion to go with the lower price. Since this data is representative of one month (my assumption based on your intro), I’d be interested to see the results after a few more months. With a little over 8% of those in group 2 who joined as paying members choosing the yearly option, I think you’ll get more of these as more emails go out.

Brian Armstrong

July 15th, 2009 at 3:26 pm

Agreed, thanks Natalie. P.S. You’re right it’s representative of about one month.

College Town Menus (CTM)

July 15th, 2009 at 2:23 pm

Brian, I love your applied math skills to websites and how you go through optimizing everything from banner placement, word usage, and pricing points. All excellent! I agree with your philosophy of students having issues with the yearly option because of how they live semester-to-semester and move frequently. Good idea saying you can move your account wherever you go.

I agree with the frustrations of Google Website Optimizer when trying to do split testing. I was looking into use GWO to figure out what are best keywords, title, and description META tags, but I find it extremely difficult when trying to apply it to dynamic websites. Your idea of a hard coded number tracking system is a good alternative and is something I may consider.

It is THESE type of posts that I really love and gain the most out of.

Brian Armstrong

July 15th, 2009 at 3:31 pm

Cool, good feedback and thanks for letting me know. I agree Google Website Optimizer needs…something. I’m not entirely sure what yet, they have some tough limitations with only being able to use Javascript in the browser, but I think it could be improved. It doesn’t handle dynamic content very well on a page, only predefined blocks of text or images.

One benefit of picking a set price is that I’ll now be able to display it on the tutor sign up page. With the split test, I couldn’t display a set price (it was determined randomly after a tutor logged in) and some people found this deceptive because they signed up thinking the tutor account was entirely free, not the first 3 job requests only.

Chris @ Dorm Room Biz

July 15th, 2009 at 6:18 pm

So what about the people who are now paying $9.95 a month (6 people) while you’re dropping the price to $5/month? Are you going to allow them to “downgrade” to $5/month or just not bother with it until they approach you about the price difference? I’d wait it out…*evil grin*.

College Town Menus (CTM)

July 15th, 2009 at 7:21 pm

Good question. I think for him or any new website owner, keeping a loyal and honest customer service to your followers and supporters is of up most importance, so if it were me, I’d email all of them, tell them that you learned from some testing, found the other option to be better, and will credit them the difference that they paid before and that they have been automatically change to the other plan. The 3 dollars difference (or whatever it is) isn’t worth it for losing a valuable and loyal long term customer and free word of mouth advocate.

Brian Armstrong

July 15th, 2009 at 9:29 pm

Good point Chris, I haven’t really decided what to do on that. It’s actually far more than the 6, because all previous tutors before I started testing had signed up at the $9.95 price point, so probably 99% of tutors are paying the higher price right now. Not an easy decision!

Chris @ Dorm Room Biz

July 16th, 2009 at 12:18 am

Knowing that all your tutors from before were registered at the $10 rate, if you are sure you’re going to lower it, I’d just drop it down and announce the new lower rate starting at the next billing cycle for the person. For people who pay monthly for the service, I dont think this would be an issue, just more so for anyone who may have already paid for a year and still has a good length of time left. It is a sticky situation, but I feel that just announcing the new lower rate and dropping everyone to it is probably the easiest way to go and probably cause the least controversy.

College Town Menus (CTM)

July 15th, 2009 at 7:19 pm

Very good point, that could definately come across deceptive. At least your on the right track, far ahead of me. I need to do something like this. I have no idea what to charge restaurants (price) and what to charge them for (what features). Split testing would really help, but I think ultimately I need to man-up and have a few sit-downs with restaurants and figure what they like and about how mucht hey would pay.

Hubtonomy

July 16th, 2009 at 12:25 pm

It sounds like the best thing to do. I guess if I was in the situation I would do the same. Sometimes though it is better to have a higher price and fewer customers as it can mean bigger profits and less work.

Brian Armstrong

July 20th, 2009 at 3:50 am

Valid point…someone else was telling me that today as well.

Ash. Davis

July 20th, 2009 at 2:47 am

Brian, your series on the progress of UniversityTutor.com has been so helpful to me as I build my own social commerce site: shirtjax.com

I’ve been reading your blog like the Bible and basically walking in your footsteps as I get shirtjax up and running.

Thanks for publishing your results and progress with your sites and good luck!

Brian Armstrong

July 20th, 2009 at 3:51 am

Thanks for the kind words! Let me know when you’re ready to launch it, would love to take a look.

Ash. Davis

July 20th, 2009 at 6:36 am

sure!
the site is now in beta:

http://www.shirtjax.com/index.aspx.

it is an online marketplace for indie t-shirt designers and small labels to sell directly to a vibrant community

i welcome any feedback you have & i’m very stoked that you’ll take a look.

Wes

August 24th, 2009 at 3:27 am

Diligent testing Brian!

Just wondering, is 144 the total number of people who use your site or did u just use portion of the users for a sample size?

Wes

Brian Armstrong

August 24th, 2009 at 11:49 am

That’s the number who got the signup email that month (they get the signup email when their free trial expires). There are about 6,500 tutors signed up right now.

MRhé

October 5th, 2009 at 1:28 am

Hi Brian,

I just read through this entire series. Great info in here, thanks for sharing. I will be following you from here on.

Brian Armstrong

October 5th, 2009 at 10:33 pm

Thanks for the kind word and welcome to the site!

October 13th, 2009 at 2:50 am

This series has been really eye-opening for me as a tutor, thank you for posting the results of your experiments and progress.

I have to confess that I’ve always felt like tutor matching sites leech off the hard work that tutors actually do. Especially the services that take a cut of the hourly rate (Wyzant takes *cough* 40% from their “certified” tutors !) But when I read about hard work that you put into making your site work, I see that you are really trying to do something good while make a living for yourself :)

I think you are on the right track with charging a monthly service fee to tutors who post a listing on your site. Most tutors won’t bother with a site that charges anything more than a flat service fee. I mean, when you can meet students face to face and ask for cash at the end of the lesson, why would you go through the rigamarole of an online payment system? Also, since there are plenty of free advertising resources (craigslist, posting flyers) $5-$9 a month is probably the maximum anyone would pay for a listing. Just a tutor’s opinion.

At least I think you have a pretty stable model, something not easily abused by tutors, nor perceived as an abuse.

I don’t know how stable something like Wyzant’s model is. If I had a listing there, as soon as I met the student for the first time, I would make an agreement to cut out the middleman and tell the student pay me directly from then on. For example, if the student paid $40 through Wyz for the first lesson, I would only get $24. But if we made a deal to have lessons at $30 an hour, the student and I would both come out ahead. I bet most tutors who use Wyz actually do just that!

Anyway, sorry for the long comment! Thanks for sharing your insights and thought processes. I have really learned quite a bit about this aspect of tutoring businesses and how to run a website. Good luck with your projects!

Brian Armstrong

October 13th, 2009 at 1:58 pm

Hey B, great to get feedback from a tutor! I think you’re right I’m happy with the flat fee model now. It is finally starting to see some good growth. I also agree that WyzAnt’s model is not ideal, I would imagine many of their tutors are going under the table. Anyway, glad to hear the site is helping and thanks for stopping by!
Brian

Breaking Free

Results Of UniversityTutor.com Price Testing

20 Responses

Leave A Comment

About this blog

Most Popular Posts

You should follow me on Twitter here.