The Pieces That Make Up FeedmailPro.com

In: Education|FeedmailPro.com By: Brian Armstrong

19 Aug 2009

Here is an interesting list of the open source pieces that went into making FeedmailPro.com, in no particular order.  Remarkably, they are all free.  Perhaps what’s even more remarkable is that I can’t think of a paid alternative that I’d rather use in any of these cases, even if it was free.  Open source is great at making reliable code because so many eyeballs get to look at it, tear it apart, and fix weaknesses.  It general, I find it much more reliable than proprietary code.

This list can be a bit intimidating if you’re new to this stuff.  Don’t worry, you don’t need to be an expert in any of these, and in fact Slicehost (where I’m hosting the app) has a great bunch of tutorials to help you get through installing the first five or so.  For non-technical people…this may only be interesting as a overview of what goes on behind the scenes, but I thought I’d post it just the same.

98857402_14e9645ed8
Photo of the linux penguin in an IBM add – source.

The components:

  • Ruby on Rails
    The is the “web framework” that I used to make the site.  It provides you with a lot of functionality like creating models (subscribers, feeds, etc) and connecting it all to the database.  Many of the tools listed toward the end are “plugins” for Ruby on Rails.
  • MySQL
    This is the database running underneath which stores all the data and makes it permanent.
  • Ubuntu Linux
    This is the operating system for the server hosted with Slicehost.com.  It’s stable, fast, and memory efficient.
  • Nginx
    This is the webserver which takes incoming requests from people’s web browsers and routes them to my application.  You may have heard of Apache which is a more popular web server.  Nginx is newer on the scene, but I prefer it…faster and less memory intensive.
  • Passenger mod_rails
    Keeps lots of “worker” processes busy on the server processing requests, and spawns new ones if the existing ones can’t keep up.
  • Postfix
    This is the mail server which handles all sending and receiving of email.  It’s pretty much the standard in the linux world now and is known for being able to handle extremely high volumes of email.  I’m not sure what capacity I could get up to on my current server before needing to upgrade, but it’s probably at least in the millions of emails per day.
  • Courier
    Handles logins and accounts on the mail server.  I’ve got accounts for bounces, complaints, unsubscribes, support, etc.
  • DKIM-Milter
    This plugin to Postfix “signs” outgoing emails to verify they originated from my domain.  This helps them get through some spam filters.
  • GIT
    This is a “source control” system where I keep the code I’m writing.  It keeps track of versions and history of the files.
  • Capistrano
    This is used to “deploy” code from my development machine to the production server.  There are a number of steps each time you change the code to get it on the server, like shutting down what’s currently running, cleaning up log files, moving over stuff you want to keep from the last version, starting up the new code, etc.  This lets you do all the steps with one command.
  • Netbeans
    A text editor (and a whole lot more -the fancy word for it is “integrated development environment” or IDE) to write the code on my laptop.

The rest are all plugins for Ruby on Rails:

  • Will_Paginate
    This helps you create “pages” – just like on google where there is a “next”, “previous”, 1, 2, 3….etc links at the bottom.
  • AuthLogic
    Handles the logins, email activation, lost passwords, etc for creating new user accounts.
  • Ruby-OpenID
    Let’s people login with OpenID instead of a email/password.
  • rpx_now
    Similar to above but also lets people use the fancy RPXNow.com widget with the big buttons for Google, Yahoo, etc.
  • Feedzirra
    Library to quickly parse a lot of RSS or ATOM feeds.
  • columbus
    Detects RSS feeds on a website.  This lets people type either the RSS feed directly or their blog URL (in which case columbus will “discover” the feed URL for them).
  • Uniquify
    Library to generate those unique “tokens” you see on the end of URL’s.  Like when you click a confirmation link or unsubscribe link, sometimes you’ll see a list of random characters on the end of it.  This generates those and makes sure there are no duplicates across your system.
  • Chronic
    This helps with some of the scheduling where I let people send out emails one per day, or week, or several times per week, etc.  More specifically it converts human readable time (like “next Thursday”) into time that a computer can understand.
  • Whenever
    This helps schedule recurring tasks that happen in the background, like parsing feeds, sending emails, checking for bounced emails, etc.
  • delayed_job
    This queues tasks to be processed at a later date, such as importing subscribers, or sending mailings.
  • daemon_spawn
    This tool launches delayed_job in the background (or several of them to run at once if there is high volume).
  • fastercsv
    Handles importing and exporting csv files.
  • paypal and money gems
    Handles processing of notifications when something happens over at Paypal, like someone subscribes, unsubscribes, sends a payment, etc.
  • active_merchant
    Communicates with Paypal to send affiliate payments.
  • exception notifier
    Lets me know when errors occur on the website.

So those are all the pieces (might have missed a few but 27 listed here).  There’s at least 100,000 lines of code above that I was able to use.  And what did I write?  About 4,000 lines of ruby code that ties it all together.  The user interface is completely new, the models and the logic, but as you can see many of the pieces underneath are modular and can be plugged in to to get the benefit without having to write them from scratch.

If you haven’t seen the final product yet, take a second to check it out.

Until next time, keep breaking free!
Brian Armstrong

Be Sociable, Share!

13 Responses

    Avatar

    Tylor

    August 20th, 2009 at 12:20 am

    Very interesting! A really valuable resource too. A good one to keep bookmarked. What was really impressive is the fact that it was all free!

    Reply

    Avatar

    Billy

    August 20th, 2009 at 4:00 pm

    Excellent post, I was discussing with a few people last night about all the different open source products out there, but it is nice to see a recipe for what actually worked together to great a complete site.

    Reply

    Avatar

    Dan Sharp

    August 20th, 2009 at 4:24 pm

    Great list, Brian! Thanks for including it.

    I’m curious how you interweave Whenever, delayed_job and daemon_spawn. Aren’t they overlapping in functionality? I haven’t had to do background processing yet (beyond some simple uses of spawn), so I’m curious how you’ve worked these together.

    Thanks!

    -Dan

    Reply

    Avatar

    College Town Menus (CTM)

    August 20th, 2009 at 5:11 pm

    Man, Brian you really are good at integrating all of these pieces together! I’m assuming you have a dedicated machine to do all of this stuff. Have you considered cloud hosting? I have a friend that does this on Amazon’s cloud and he LOVES it. Very scalable and cheap for his application. I’m trying to find some solid PHP repositories or simple plugins that I can use with my framework I’m using, but having a tough time, so a lot of it is custom. I wish there were like “open source consultants” that you can talk to, and they can be like “ohh! use XYZ!” – that’d be perfect. Great post! I’m really happy you did this, this is going to help a lot of people!

    Reply

      Avatar

      Brian Armstrong

      August 20th, 2009 at 9:48 pm

      Yeah I think cloud hosting is great. BuyersVote.com is hosted on Heroku which is cloud based. There are some limitations to get it to work and you don’t have as much control as on a dedicated server, so it can be a bit challenging (the email server on this project probably wouldn’t have worked on a hosted environment), but it is highly scaleable.

      I feel like FeedmailPro is highly scaleable as well because I can keep upgrading at SliceHost all the way to a 15GM ram machine. But if I need to go beyond that, it will be a major overhaul in terms of how it works and will have to move to a multi-machine architecture. That being said, 15GB is a LOT and it’s unlikely I will need anything beyond that, or if I do, Moore’s law will have provided me with 32GB ram on one machine by then. Good thoughts…

      Reply

    Avatar

    Kenton Newby

    August 21st, 2009 at 8:46 pm

    Brian,

    Nice rundown of the pieces that make up your latest project. I’m picking up on Rails myself and have played with some of the plugins you mentioned. Authlogic is sooo much better that Restful_Authentication, IMHO. I’m a big fan of http://www.railscasts.com too.

    Question:

    What about the design elements of your site? I’m pretty solid when it comes to HTML/CSS, and the graphics stuff to a degree. But I don’t seem to have an eye for that sort of thing…sites never seem to look quite “right”.

    So do you do that yourself or outsource the design? If done on your own, what tools do you use? Photoshop? Fireworks? Something else?

    And do you have any resources you use for design ideas?

    Anything that would help me connect the dots on this stuff would be greatly appreciated.

    Kenton

    Reply

      Avatar

      Brian Armstrong

      August 22nd, 2009 at 12:02 am

      Hey Kenton,

      I’m exactly the same, know my way around CSS and photoshop but have very little artistic ability. So what I do is take other designs which are opensource and adapt them to my needs. WordPress themes are a great place to start. For example I like the ones these guys do:
      http://designdisease.com/portfolio/

      You can see the “Blogging Pro” theme is the one I used for BuyersVote.com and FeedmailPro.com. It’s open source GNU licensed so you can modify and use it for free however you’d like. People release them under different licenses, sometimes you have to include a link to it which is no big deal. If the site hits it big and you need to distinguish yourself with a unique design, I’d probably go back to Design Disease and have them make me a custom one. But otherwise it’s a great way to get something up quickly with a nice design.

      Hope it helps!
      Brian

      Reply

    Avatar

    Greg Moreno

    August 28th, 2009 at 5:28 am

    Nice list! They are real time savers.

    One thing I am still fighting hard to let go is coding every thing I need. I still research on what proven solutions are out there but my “technician-self” (to borrow from Michael Gerber’s) dominates a lot in my decisions. But good thing is I know my situation and knowing is half the battle :)

    Reply

Leave A Comment

About this blog

Breaking Free is a collection of articles on tech entrepreneurship, business, and life written by Brian Armstrong. You can read more here »

Most Popular Posts

You should follow me on Twitter here.

  • asfd: fsgdasdfasfd adfasdfa [...]
  • asfd: sdfasdasdafdsadfs [...]
  • asfd: asdfasdfasdf [...]
  • Gucci Handbags Wholesale: Heya! I know this is kind of off-topic but I needed to ask. Does operating a well-established blog l [...]
  • Ashlllli_GaGa: What is the difference between a car loan and car lease? [...]

Copyright © 2012 - Breaking Free - is proudly powered by WordPress | Log in

Compositio Theme is created by: Design Disease brought to you by PremiumThemes.com