Talking to Gmail with Ruby and Rails

Recently I came across a dilemma in trying to send and receive emails from a Rails app: Either install a full-blown postfix server on a Linux box (by full-blown, I mean account handling, POP/IMAP interface, anti-spam, anti-virus, etc etc), or use Google to handle all the emailing. So, me not being a Linux/sys-admin junkie, I strongly opted for the Google route (and after some persuasion the customer, thankfully, agreed :-). So I went about to find the best practices for working with Google Apps (http://www.google.com/a which is where you sign up). I came across this very helpful post.

You just install the mentioned plugin, create the appropriate mailer.yml file and you’re good to go. ActionMailer will login as the specified account and send emails through that account. Genius!

On a side note, I was able to get a Rails app to talk (no sending though, just receiving) to gmail by using the new net/pop library in the latest ruby and making the Mail Fetcher plugin.

But back to the main point, being able to send through the Google Apps account. The technical stuff seems really easy now. Just send your emails out like normal and have some way to check responses either manually or programmatically like I showed above. You’re golden, or so it seems.

If you have an application like the one I’m working on, you have many emails being sent for single actions (one to the person who submitted a form, and then 10 - 100 emails being sent to others notifying them of the form submission as well). That seems fine up until you read through the Google Apps help center and find out that you can only send emails to 500 addresses each day per account (so either 1 email message with 500 recipients, or 500 emails each with one recipient).

If you have a commercial application, that is a serious limitation, and one that might stop you from using Google as your email hosting solution.

The important thing to note though is that the limitation is on one account per day. With Google Apps, you receive your Google Apps account which allows you to administer certain things like a calendar, file uploads (which at this point is pretty lame), chatting, and email. Google allows you to create users who can use these features in your domain. So you can create john@mydomain.com and mildred@mydomain.com, and they can have an email account, read your files, chat with each other, and check the mydomain.com calendar. The accounts that get limited are the accounts you create, not your Google App account. There is hope! Google allows you to create quite a few accounts for your domain (200 when you sign up, and you can always ask for more).

What that means is that you are not limited to 500 emails per day, but really 100,000 emails per day (200 * 500). That’s a lot of traffic! The obvious way to use all of it is to create a bunch of accounts (don’t forget to enable either POP3 or IMAP access!) and randomly send from them all. Just set up account names like “system1@example.com” and “system2@example.com” and randomly choose which one to send from each time you send out an email. It just works, and it distributes the load of sent emails pretty well across all email accounts. Implementation wise, I did something like this.

It ends up sending from email addresses “system100@example.com” through “system110@example.com”. Also, in that example I put in the ActionMailer::Base.smptsettings for clarity, but in reality I have them in the config/mailer.yml file used by the actionmailer_tls plugin.

Hopefully this will help someone who is on the fence about using Google Apps as a solution to email after finding out about the sending limit. Enjoy!

Posted by Steve on Feb 11, 2008

bash aliases

Bash aliases are such lovely things, so I thought I would share some of my favorites and maybe even get some feedback.

Most of my aliases are there to make software development more enjoyable and less time consuming. As such, most of them deal with the projects I have in my project directories. Here are some of the more interesting ones:

for bigdir in $(ls ~/Developer/Projects/ | cut -d' ' -f7)
do 
  for dir in $(ls ~/Developer/Projects/$bigdir)
    do
      alias cd$dir="cd ~/Developer/Projects/$bigdir/$dir" 
      alias m$dir="cd$dir && mate ." 
      alias s$dir="cd$dir && ss" 
      alias g$dir="~/bin/svngrowl $dir" 
      alias o$dir="osascript -e 'tell application \"Terminal\"'
        -e 'tell application \"System Events\"'
        -e 'tell process \"Terminal\" to keystroke \"t\" using command down'
        -e 'tell process \"Terminal\" to keystroke \"t\" using command down'
        -e 'end tell'
        -e 'do script \"s$dir\" in first tab of front window'
        -e 'do script \"m$dir\" in second tab of front window'
        -e 'do script \"g$dir\" in third tab of front window'
        -e 'do script \"localhost\" in second tab of front window'
        -e 'tell second tab of front window to set selected to true'
        -e 'end tell'" 
    done
done

alias localhost="open http://localhost:3000" 
alias addall="svn st | grep ? | cut -d' ' -f7 | xargs svn add" 
alias removeall="svn st | grep ? | cut -d' ' -f7 | xargs rm -Rf" 

for var in "server" "generate" "console" "destroy" "plugin" 
do
  alias s${var:0:1}="script/$var" 
done

The first for loop is probably the most useful set of aliases that I have. It starts off by going through all the projects that live in my various project directories. Then, for each project it gives me a cdproject, mproject, sproject, gproject (very special), and oproject (less special, but still useful).

• cd’s into the project directory

• opens the project in TextMate

• starts a server for the project by calling script/server (which is what the alias ss does in a rails project)

• starts an svngrowl process for the project (which I’ll explain later)

• Uses the command line osascript, which invokes applescript, to manipulate Terminal and Firefox

First, an explanation of what oproject is doing. To start, for viewing purposes above I cut the oproject alias into more manageable pieces, but when used should be on one line (as would be the case if it were called from the command line).

This alias is kind of like the Ring of Power, in that it rules all the other aliases, and in the darkness (or the light) binds them… into one convenient call. It opens Terminal, and opens 2 new tabs for a total of 3 tabs (only for Leopard users though :-). It then opens a server in the first tab, opens the project in TextMate in the second tab, starts an svngrowl process in the third tab, goes back to the second tab and runs the alias localhost, and then gives focus back to the second tab (since it seemed to lose focus to the third tab, not sure why). This alias is a huge time saver when you think that every working day I spend about 10 to 15 seconds when I have to open everything up (which adds up to a lot when you have enough work to do!).

As for the gproject alias, this uses a cool little script in my ~/bin directory that starts a loop for the specified project, and every 3 minutes it runs svn up on the project and if there are any changes to the project, it sends out a pretty little growl notification with what was changed, who changed it, the commit message, and if any migrations were committed it changes the priority of the message to high (which for my personal growl settings turns the message text red).

Two scripts are involved in the growling, one bash script, and a ruby script (which is invoked by the bash script). First, the bash script (cut in half for easy viewing):

#!/bin/sh
source ~/.profile
alias | grep cd$@= | awk "{gsub(/cd$@='cd |'/, \"\"); print}" |
awk "{gsub(/~/, \"/Users/`whoami`\"); print}" | xargs ~/bin/svngrowl.rb

#!/usr/bin/env ruby
begin
  directory = ARGV[0]
  project = directory.split("/").last
  ignore = directory.gsub(project, "")

  loop do
    updates = `svn up #{directory}`.gsub(ignore, "").split("\n")
    last_log = `svn log #{directory} --limit 1`

    revision = updates.pop.scan(/\d+/)
    committer = last_log.split("\n")[1].split(" | ")[1]
    priority = updates.join("\n").include?("db/migrate") ? 1 : 0
    message = last_log.split("\n")[2..-2].join("\n")

    unless updates.empty?
      `growlnotify -s -t "#{project} Update - #{committer}" 
      -m "#{message + "\n\n" + updates.join("\n")}" -p #{priority} --image ~/bin/mph.png`
    end
    sleep 180
  end
rescue
  `~/bin/svngrowl.rb #{directory}`
end

All the bash script does is finds the proper directory for the project in question and passes it to the ruby script. The ruby script in turn does an svn up on the directory, collects all the interesting data, and runs growlnotify (the command line growl utility) displaying all of the interesting data. There is also a rescue statement in there that just restarts the process if something goes wrong (an error is thrown if svn can’t update properly, which can happen if the network drops out or if you do svn st at the same time that this process is doing svn up). I also used the—image options to pass in a beautiful picture of Mr. Potato Head to use as the icon shown on the notification.

Please feel free to use any of these to make your work/play more productive (except for the picture of Mr. Potato Head… that one is mine!)

Update: My aliases, along with some other handy scripts can now be accessed courtesy of github: utilities.

Posted by Steve on Nov 03, 2007

Rails and SSL (https://)

So I recently had to have some secure routes for a site I am currently working on. If you are using deprec and deploying to a slicehost, getting the SSL certificate stuff working is a snap. If you want your whole site to be over SSL, then all you do is put in a permanent redirect to socket 443 in your app_name.conf file. But, if you need to do something interesting like go back and forth between http and https, then you need to do a little digging.

Doing a google search doesn’t bring up anything particularly useful in terms of going back and forth between http and https, but one thing did catch my eye. In your routes you can pass a protocol option to set which protocol you are going to use (something like http, file, svn, https, etc…). I also had to dig through some code to find out how to pass that option in, since it gets set two different ways depending on what kind of route you are using. If you do a map.resource(s) route, then you have to set the option as :requirements => {:protocol => ROUTESPROTOCOL}, whereas if you just do a map.routename type route, then you just pass in :protocol => ROUTES_PROTOCOL.

You should also be careful about when you are setting that protocol, as trying to access https://localhost:3000 just doesn’t work (such as in development mode). I just set a conditional constant at the top of my routes file like so: ROUTESPROTOCOL = (ENV["RAILSENV"] =~ /development/ ? "http" : "https").

Two more caveats:

• First caveat is that if you want to go back and forth between http and https, you MUST explicitly set all of your routes to have http or https protocol. If you don’t, then when rails generates your routes it will just use the protocol of the current page. This is annoying, but I can see why they would do this (if you are looking at something securely, then just look at everything securely).
• Second caveat is that you must use the route_name_url throughout your application instead of route_name_path, as some are accustomed to. The _url named route gives you the full url (duh), which is what you want if you are on a page using http and want to go to https (whereas _path only gives you the uri, and will use the protocol and host of the current page). If you already use _url, then you are one step ahead in the game.

Here is a sample of some of my routes using the protocol option:

# Public routes using http protocol
map.with_options :protocol => "http" do |http|
  http.login '/login',   :controller => 'sessions', :action => 'new'
  http.logout '/logout',  :controller => 'sessions', :action => 'destroy'
  http.resources :users, :sessions, :requirements => {:protocol => "http"}
end

# Public routes using https protocol
map.with_options :protocol => ROUTES_PROTOCOL do |https|
  https.form_step '/form/*slug', :controller => 'forms', :action => 'show'
  https.form_edit '/edit/*path', :controller => 'forms', :action => 'edit'
  https.resources :forms, :requirements => {:protocol => ROUTES_PROTOCOL}
end

Posted by Steve on Aug 28, 2007

Regular Expression Finding in TextMate

Recently I had to convert a list of items in a word doc over to a yaml file in order so that I could load the data into our database. The word doc only contained the data necessary that a human could read and understand what was needed (meta data), but looked nothing like what the final yaml needed to look like. So I had two options, either sit there for 2 hours typing everything by hand over to yaml, or spend 20 minutes learning how to use the regular expression finding feature in TextMate. I went with the latter and am oh so glad I did. $1 is now my friend.

If you haven't tried it yet, play around with it ad see how awesome it can be!

Posted by Steve on Apr 25, 2007