Tuesday, December 22, 2009

GMail on Engine Yard

Jump to the end of this if you just want the solution I found, the rest of this includes the trials and errors that didn't work, but may help someone find this page.

We have an application on Engine Yard that is configured to use GMail to send email. Everything works fine in development, but in production, we get the following error:

Net::SMTPUnknownError (530 5.7.0 Must issue a STARTTLS command first. 21sm5842339qyk.4
/usr/lib/ruby/1.8/net/smtp.rb:679:in `check_response'
/usr/lib/ruby/1.8/net/smtp.rb:581:in `auth_login'
/usr/lib/ruby/1.8/net/smtp.rb:685:in `critical'
/usr/lib/ruby/1.8/net/smtp.rb:580:in `auth_login'
/usr/lib/ruby/1.8/net/smtp.rb:570:in `__send__'
/usr/lib/ruby/1.8/net/smtp.rb:570:in `authenticate'
/usr/lib/ruby/1.8/net/smtp.rb:410:in `do_start'
/usr/lib/ruby/1.8/net/smtp.rb:377:in `start'
app/models/invitation.rb:13:in `send_invitation_request'
app/controllers/invitations_controller.rb:50:in `create'
app/controllers/invitations_controller.rb:47:in `create'
Turns out - Engine Yard is still using Ruby 1.8.6 and 1.8.6 doesn't support SMTP TLS that Google uses.

So, I started searching on Engine Yard to figure out what to do. First I tried to use the Engine Yard search engine - I looked for "GMail", and this is what I got.

I tried to make these changes (I'm not a Unix expert, by any means), but when I restarted, I didn't see any change. Same error in the projection.log file.

Doing a search on Google for "GMail" brings back a couple posts that seemed more specific to GMail, but they didn't work either. Each change was a redeployment and this was looking like it could take some time.

I then contacted @engineyard on twitter. I ususally get a nice response, and I was passed on to one of the support guys at EngineYard. This is what I was sent by email after describing what I was trying to do:

"The issue is that Net::SMTP doesn't have proper TLS support, so you
can't use it directly with ActionMailer. You have two options. You can
install a plugin for your app to add TLS support following these

Or you can get ssmtp configured on your instance and just set
actionmailer to use :sendmail to send outgoing emails. The latter
option is what our documentation will walk you through. If you're
going to be sending a small volume of email and just want the simplest
setup, using the above link is probably the easiest option.

If you're going to be sending a larger volume of mail (unlikely due to
Gmail's SMTP limits) then I would definitely see about getting either
ssmtp or even a full blown MTA set up on your instance.

Hope that helps."

This is great - I already tried the suggestion from the document, but that didn't work for me so I'll take EY Support's suggestion and get the plugin. Why wasn't that in the documentation?

Anyway, I go to the recommended page and try to the install the plugin that's referenced, but the plugin doesn't install. The page itself is almost 2 years old and the plugin that's mentioned doesn't exist at all. Ugh, apparently EngineYard didn't take the time to make sure it was a good article to refer a customer to.

Back on my own and realizing the plugin approach is recommended, I try searching on google I find the gem on github. OK - still making progress.

I have the option to install the gem or the plugin. I'll do the gem, as I only need it in production.

To do this on EngineYard, you have to add it through their dashboard UI. But, it turns out they already have the gem hosted on their site. Why didn't support tell me about this? Maybe they didn't know it was there. I assume the gems that are hosted are common and put their intentionally, but now I don't really know for sure.

So, I install the gem, and configure production.rb to require the gem.

I redeploy and am now getting this error when I try to send a password reset email:

ArgumentError (wrong number of arguments (2 for 3)):

/usr/lib/ruby/gems/1.8/gems/action_mailer_tls-1.1.3/lib/smtp_tls.rb:8:in `check_auth_args'
/usr/lib/ruby/gems/1.8/gems/action_mailer_tls-1.1.3/lib/smtp_tls.rb:8:in `do_start'
/usr/lib/ruby/1.8/net/smtp.rb:377:in `start'
app/models/user.rb:123:in `deliver_password_reset_instructions!'
app/controllers/password_resets_controller.rb:27:in `create'

Yuck - I didn't add a lot of code to get this error. Doing a little more Googling, I find a thread discussing a similar problem. I get the feeling the Ruby version on EY and this plugin are incompatible.

But one good thing happened here - I stumbled upon another plugin: action_mailer_optional_tls

I search for it and find it on GitHub.

I tell you, whenever I see Collective Idea's name and Brandon Keeper's face, I have confidence things are going to work. I use many other collective idea gems and read Brandon's blog once in a while, this is a good sign. If this works, I'm going to Holland Michigan to register for a class with Idea Foundary.

And, after yet another deployment to EngineYard ...

It worked! Hard to believe after all that.

And all it ultimately took was the following.
  1. Installed action_mailer_optional_tls plugin
  2. Added ''require 'smtp_tls'" to production.rb
  3. Modified the :smtp_settings => { ..., :tls => true, ... }
I wish Engine Yard could have helped me. This took me 5 hours to finally resolve, probably 25 deployments to test changes in production, and it was a trivial change in the end. This is not an unusual experience for me on Engine Yard. Things are often more complicated, hard to understand, or proprietary, forcing me to learn technologies I only use on their site. I am often wishing the UI was simpler, less configuration was required, and the documentation was easier to understand and access. Give me another frustrating afternoon working around an Engine Yard issue and I may be motivated to post some of these experiences, too.

I really appreciate the position that Engine Yard takes - easy to deploy and manage Rails Applications. I saw Tom Mornini talk in Seattle - the vision is great: Engine Yard handles deployment and we (developers) can focus on development and features. It's clear that's where we're headed as an industry, but Engine Yard has plenty of room to improve before it can really own those claims.

Thursday, July 16, 2009

A dot at the end of the URL

If you have urls that tend to get emailed or posted on social networks, you may sometimes get links to your site where the dot - the period at the end of the sentence - is added to the <a href=""> tag or a white space is added in the middle.

In Ruby on Rails, the . is a very important separator - it's used by Routes to determine the format of a request (html, xml, etc). The space is also unexpected, but will not be trimmed naturally if it is in the middle of the url.

If you want to reduce those "The resource cannot be found." errors, you can handle it in a couple ways:

You can modify your .htaccess file. A fine solution, but if you don't have access to the server, or just like to do it in Rails, here's a way to modify routes.

Say you had an object called 'item' and you wanted to have it show up when someone entered the url /73md8 or something - this is common with tinyurl-like ulrs - we want to keep the url short and the 'pid' is all we need to determine the page to show.

In Routes.rb, you might have something like this:

map.item '/:pid', :controller => 'items', :action => 'show'

But, this would not work with the spaces or period at the end. Try this and you'll have no problems with urls that have spaces or periods.

map.item '/:pid', :controller => 'items', :action => 'show', :requirements => { :pid => /[a-zA-Z0-9\.%]+/ }

The '%' is needed to catch %20 (the space character) and other escaped characters.

At somepoint, you just can't fix a broken link, but this is more flexible in the most common cases.

Thursday, June 25, 2009

Twitter Background Image

I've been using the twitter API and in chirps, I'd like to show user's background from twitter. The twitter api doesn't give this information. If the user chooses to hide the background image by selecting the "Don't use a background image" from the design settings, nothing is passed back through the Twitter API (yet - the fix is in the Twitter API V2 roadmap).

To get around this - I made a method for the User model that returns a boolean for "profile_background_image_display".

def profile_background_image_display
return is_theme? || has_custom_background?

def is_theme?
colors = %w(9AE4E8 C6E2EE EDECE9 0099B9 352726 709397 EBEBEB 8B542B 1A1B1F 642D8B FF6699 BADFCD)
images = (1..12).map{|i| "theme#{i}/bg.gif"}
for i in 0..11
if self.profile_background_image_url =~ && self.profile_background_color.downcase ==
return true
if self.profile_background_image_url =~ /images\/bg.gif/ && self.profile_background_color == # This is the default, not theme 1
return true
return false

def has_custom_background?
return !(self.profile_background_image_url =~ /theme\d+\/bg.gif/)

Note - this isn't perfect. This is where it breaks down:
  • It will show a background image if the user sets a custom background image and hides it.
  • It will show the background image if the user picks a theme and hides the background.
  • It will hide the background image if the user uses a theme background on a custom color.
But those situations are rare, or usually look pretty good anyway. The main problem case - where the user picks a new background color and hides the bg image - is handled here.

That's the trade off.

Hope this is useful to someone - I'd love to know if you have a better solution.

Thursday, June 04, 2009

Case Sensitive Active Record Finds in Ruby on Rails

This was hard to find, so I thought I'd put up a little post.

I was making a site, similar to, where the identifying id in the url string was a sequence of characters (like /jYi1K8c) some lowercase, some uppercase.

With MySQL, you can't just do a Item.find_by_pid('jYi1K8c') and get back the right thing, it will get back the first that matches a case-insensitive search. That is, /ab1, /AB1 and /Ab1 all match the same item.

So, to get around this, there are ways you can set up the migration to indicate the field is case sensitive.

execute %{ALTER TABLE TABLE_NAME MODIFY slug varchar(255) COLLATE utf8_bin NOT NULL}

Apparenty, you can put this right in the migration, and not use 'execute'.

But, I didn't do these. Maybe I will later, but in the short term, I just modified the find conditions in the controller to be like this:

@item = Item.first(:conditions => ['BINARY pid = ?', params[:pid]])

This is a little odd to me - usually rails makes this sort of thing easier to get at. I wouldn't be surprised if there was a better way.

Friday, May 29, 2009

Character count a textarea in Ruby on Rails

I used to do this with an 'onkeyup' attribute, but that's not available to the 'observe_field' method. But, by setting the frequency to a tenth of a second, it's much better. The character count updates even on keydown and hold.

In the view you just do this

Message (<span id='char_count' ></span > characters)<br>
<%= text_area_tag :description %>
<%= character_count('description','char_count', :frequency => 0.10) %>

And in the helper, you have a 'character_count' method.

def character_count(field_id, update_id, options = {})
function = "$('#{update_id}').innerHTML = $F('#{field_id}').length;"
out = javascript_tag(function) # set current length
out += observe_field(field_id, options.merge(:function => function)) # and observe it

Thursday, April 23, 2009

My Ambigram

It took me a while to figure out what this type of thing was called. It's an "ambigram" - meaning rotationally symmetrical word. Roy Leban, a software designer in our offices at Startpad does this with his first name as a signature and I thought I'd it a shot.

This is what I got.

Wednesday, April 22, 2009

Google Image Search in Rails using the Google Ajax API

Google has a neat Ajax API for searching images on the web. This is neat, but I wanted to integrate it into my site and didn't want to rely on Javascript so heavily. Luckily, Google also supplies a JSON api that's very easy to work with.

Instead, I made a GoogleImage class and reference it in the controller like so. This is in a controller action. The view for this supplies a keyword via a form, and there is a simple 'next' | 'pevious' link that modifies the start parameter.

This is in the view (app/views/search/google_images.html.erb)

<% for image in @google_images %>
<%= link_to(image_tag(image.thumbnail, :style => 'padding:3px; border:solid 1px #EEE; margin:2px; width:100px; float:left'), image.original) %>
<% end %>
<%= clear %>
<%= link_to_unless (params[:start] == 0), 'Prev', :overwrite_params => { :start => (params[:start] - 8) } %> |
<%= link_to_unless (params[:start] > 0 && @google_images.blank?), 'Next', :overwrite_params => { :start => (params[:start] + 8) } %>

This is in the controller (app/controllers/search_controller.rb)

def google_images
params[:start] = params[:start].to_i
@page_title = "Import image from Google"
@google_images = GoogleImage.all(params[:keywords], params[:start])

And this is the model (app/models/google_image.rb)

class GoogleImage

require 'json'

attr_accessor :thumbnail, :original, :name, :position

def initialize(params)
super() = params[:name]
self.thumbnail = params[:thumbnail]
self.original = params[:original]
self.position = params[:position]

def self.find (keyword, position = 0)
url = "{position}&v=1.0&q=#{CGI.escape(keyword)}"
json_results = open(url) {|f| };
results = JSON.parse(json_results)
image_array = results['responseData']['results']
image = image_array[0] if image_array
google_image = => image['tbUrl'], :original => image['unescapedUrl'], :position => position, :name => keyword.titleize)

def self.all (keyword, position = 0)
return [] if (keyword.nil? || keyword.strip.blank?)
url = "{position}&v=1.0&q=#{CGI.escape(keyword)}"
json_results = open(url) {|f| };
results = JSON.parse(json_results)
image_array = results['responseData']['results']
google_images ={|image| => image['tbUrl'], :original => image['unescapedUrl'], :name => keyword.titleize) }
google_images.each_index{|i| google_images[i].position = position + i }


A couple things to note
- the "start" parameter is not a page number. It's the index of the image in this search.
- Google seems to allow the first 64 images to come back. No more.
- Display is limited to 4 or 8 results (that's due to the result size param 'rsz').

Tuesday, April 14, 2009

Posting a Status Update to Facebook

OK - following up on my post about sending a status update to Twitter - here is a simple how-to with Facebook posts.

I really like the facebook system. It's got a lot more bells and whistles, which I usually don't like, but they went through some effort to let you pick a thumbnail to go along with the post and give you the option to send a message or post to your wall. Very nicely done. On the negative side, when a user clicks on the link from facebook, it opens in an iframe in a facebook URL. Not the worst experience - it's easy to close.

Anyway, without further ado, here's my code...

I link to this action via the link_to helper

<%= link_to image_tag('icon/facebook.gif'), {:controller => 'articles', :action => 'facebook', :id => @article}, :popup => ['new_window','width=600,height=400'], :rel => 'nofollow' %>

This is in the controller

def facebook_post
@article = Article.find(params[:id])
url = CGI.escape(url_for(@article, :only_path => false)
title = CGI.escape(@article.title)
redirect_to "{url}&t=#{title}"

This will pop up a window that is 600x400 (works well with the facebook page that opens up).

Tuesday, April 07, 2009

Finding recursive objects in ActiveRecord in Rails

Here's the issue, say you wanted to find all the students that are in a course (avoiding the 'class' keyword) with you. And say you had models that looked like this...

Class Student <>
has_many :courses

Class Course <>
has_many :students

You might try this in the Student class...

Student.find :all, :include => { :courses => :students }, :conditions => [' = ?',]

But, of course, this would only return self.

To find the classmates, you need to find the name of the alias used on the joined table and put your condition there. Rails will probably give it an alias like 'course_students'. You can confirm this by running the query with a condition on the course table and looking in the logs. Once you know the name of the aliased table you can do this...

def class_mates
Student.find :all, :include => { :courses => :students }, :conditions => [' = ? and != ?',,], :group => ''

This will return all students that are in a class with you, and it will remove you from the list.

Now, if you want to find out how many classes they are in together, that's trickier. I've only been able to do that with find_by_sql.

Let me know if you find a better way.

Wednesday, March 25, 2009

Growing up online

An article from the WSJ argues that the workplace environment will change due to kids experience and expectations that they gain from being online.

It's an interesting article and it brings up great points such as...
  • Leaders serve, don't preside
  • Tasks are chosen, not assigned
  • Power comes from sharing not hording
  • Contribution counts more than credentials
It seems these lessons won't be limited to the workplace. Even the slowest-moving arenas like education and politics would benefit, though it will likely require their respective monopolies to open up.

Even before spending a lot of time online, I suspect many people felt it should be this way but the existing authority kept the status quo in place. People were forced to learn in public schools, work as interns to get their first job, and slowly work their way up in politics. We're seeing many counter examples lately - charter schools, kids out of high school starting companies, President Obama's run for president - all examples of change.

Personally, I'm glad the old way of doing things is being challenged. The smaller, faster organization is not only more effective, but more natural. I suspect we'll see that it's less of a change or advance, and more of a return to normalcy.

Posting a Status Update to Twitter

I recently had a need to allow a user to send the a url to twitter as a part of the status update. It seemed like it would be a trivial thing to do (and it was simple), but it was tough to find any information on it online. Most information is about the Twitter API and there are many products that were more heavy-weight than I needed.

This is what I wanted.

1) User clicks on a link which pops up a new window
2) My application would build the update text and include a tiny_url
3) Redirect to Twitter, pre-filling the update field.

In Ruby on Rails, this is what I ended up with. It's not very complicated, so it's probably easier just to see the code. I changed it to a Post model object for this example.

def tweet
@post = Post.find(params[:id])

tiny_url = open("{url_for(@post)}") {|f| }
message = "#{@post.title} #{tiny_url}"

redirect_to "{CGI.escape(message)}"

There are a lot of good ways to update status on Twitter using the Twitter API, but for this project they were more than I wanted. Here are a few that I plan to check out. I haven't used TwitterAuth yet, and a friend recently brought my attention to the site MessagePub.

I plan to try out TwitterAuth for another feature soon.

Wednesday, March 11, 2009

Layering html elements without using 'position' attribute

In a project I was working on, we wanted to layer a small piece of information on top of an image, but we did not want to use positioning - this was due to bugs in IE7 and IE8 that were making it tough to position other items on top or do any scriptaculous effects - like blinds.

It turns out it's quite easy to do, at least in firefox and safari on a mac and IE7 on a PC.

Here's how:

<div style="border: 5px solid yellow; height: 90px; width: 100px; background-color: yellow; margin:0;">
<div style="height: 90px; width: 90px; background-color: blue; float: right; margin:0"></div>
<div style="height: 20px; width: 50px; background-color: red; margin:0; margin-top: -20px; float: right "></div>

The yellow is in the background - the blue div is nested. The red div is on the blue div - floated right (this is optional) with a negative margin.

Try it out on your own or view source.

Thursday, February 05, 2009

Free Users

An article at the WSJ from The Long Tail author Chris Anderson:

The economics of giving it all away

I've been thinking about this topic with my work at At ET, we have plenty of free users, but we'll be trying to find a way to get revenue at some point - We'll be trying to get paying customers. But that's not the only type of paying customer we're after. We're after customers that give us all sorts of things - upload audio, comment, rate, build collections, profile pages, channels. When you think about it, the customers aren't free at all. Each one gives you something, even if it's only a 'monthly unique' - which roughly translates into a visit to your site to look at what you got.

We really have a half dozen user types, each giving us something of value, and at each value level, there are fewer and fewer of them. Our goal, like a car company, is to get them in the door, to use our product in an entry level, become a loyal user, and move up to more expensive services and products.

Chris Anderson focus's on the "free", but that's not true at all. He misses the whole point. The internet isn't driving down the cost of everything to free. It's finding a way to make money off of everything else, which, before the internet, really wasn't possible. We're all trading something for free for something of value. Trading up - and that's really what capitalism is all about (until recently).

Or, you can look at it another way. Those free users aren't 'users' at all, they are the product that other companies want to buy: they are what the company sells. Because, when you really get down to it - most web sites are just some form of matchmaking: Google, Ebay,, Craigslist, YouTube, etc. But, you say, on YouTube, know one pays - not the uploader, not the viewer! That's true, but the viewer pays when they click on an ad based on the keywords and content provided by the uploader. It's a complicated motivation chart, but it's there. It takes a critical mass before this will work, and most sites will start with content, and then get viewers. Follow that up with ads and you have the basic model.

Saturday, January 03, 2009

Blog focus

I'm going to change the focus of this blog. Not that it ever really had much focus before, but it was called 'Rails Tales' and my plan was to talk about developing applications with Ruby on Rails. But lately, at Entertonement (where I'm a web developer), I've been more interested on understanding our users and decreasing bounce rate. This has revealed some interesting things that I want to start talking about further.