Friday, August 28, 2009

Kill Your Signup Form with Rails

Nobody likes Signup Forms

Even though the gradual engagement meme has been around for a while, and everyone just hates signup forms, they just seem to keep popping up like a bad habit. My site, Newsforwhatyoudo.com was one of the guilty parties. We saw users coming back to the site repeatedly, but not signing up. The percentage that looked at the signup form and then bolted was uncomfortably high. It was time to kill the signup form. This blog post documents how we implemented gradual engagement using Ruby on Rails and restful authentication.

Gradual Engagement Principles

Here are some principles I've gleaned from learning about gradual engagement.

Make sure the user sees a direct benefit to every sign up step. When you do ask for information, do so only after 1) the site has demonstrated user value, 2) the user has a vested interest in the site, demonstrated by having customized some aspect of its behavior, and 3) you provide a carrot that justifies providing signup information.

Set reasonable defaults automatically and let the user customize them later. For example, in our original join workflow, Newsforwhatyoudo asked users to add subscriptions and topics as part of the process of joining a group. Instead we now automatically give users the most popular subscriptions and let them change these later.

Security only when its needed. Forcing a user to establish a username and password up front may be necessary if you're developing a banking site where all the information being displayed to a user is confidential, or the user's actions have financial impact. Most web sites don't have this issue. Even E-commerce sites can let users browse, search, compare and price without any security - its not until the credit card information is required to make a purchase that formal signup is required. For most sites anti-spam is a bigger issue, and there too, anti-spam measures like captchas and email confirmations should be used only after the user does something that requires their use. I emphasize the word "after", because if you wait until after a user has filled out a comment form, then present the confirmation, you're more likely to have the user perform the confirmation because they've committed energy to filling out the comment.

Keeping out the spammers

The most time consuming part of implementing gradual engagement was figuring out how to keep the spam bots out. At newsforwhatyoudo.com our strategy is two tiered. Actions that change state - particularly those that submit forms - require the client to run javascript. This eliminates most spam bots. If we detect a client that's not running javascript, we throw up a captcha. Most users never see the captcha and can just submit the form. The captcha eliminates bots that can run javascript. None of this helps if someone designs a bot specifically to attack our site, but then again neither did our old process of asking for user credentials and showing a captcha, so we've gained usability without losing anything.

check_for_bots adds a hidden input field to post and put requests. The "turing_a" javascript file runs when the form loads and returns a value in the hidden input field based on how long the user viewed the form before hitting submit. If the length of time is below a threshold or incorrect, we reject the input and instead show a captcha. After that the standard captcha acceptance routine applies. The javascript we use is adapted from Stephen Hill's Javascript Captcha.

Back on the server side we need to validate the hidden input field. If validation fails, we render a captcha.

def email_supportcheck_for_bots@popular_groups=Group.most_popularraiseMissingFieldsunlessparams[:ask_support]@ask_support=AskSupport.newparams[:ask_support][:email],params[:ask_support][:message]raiseMightBeBotif!@humanraiseMissingFieldsif@ask_support.email.blank?||@ask_support.message.blank?Mailer.deliver_email_support(params[:ask_support][:email],"about page",params[:ask_support][:message])flash[:success]="Got your request, we'll be in touch"redirect_back_or_default(groups_path)rescueMissingFieldsflash[:error]="Couldn't process your request, make sure both fields are filled in."render:action=>'about'rescueMightBeBot@show_captcha=trueflash[:error]="Your browser must be running javascript, and/or you must enter the correct words below"render:action=>'about'end

The check_for_bots method is added to any method we need to protect. It sets @human which the caller can check as part of its input validation.

# Used as before filter on POST and PUT actions to determine# whether the requestor is likely to be human. Requestor needs to be able to execute .js# and wait at least 4 seconds before submitting the form, or this fails and returns false.# This method doesn't require the user to do anything, other than have a js capable browser# Note when testing in rails this method will (and should) always fail.def check_for_botsifcurrent_user&&current_user.status=='verified'# already showed captcha and user passed@human=truereturnend# verify_recaptcha is true if in test mode or success, false if fails.# verify_recaptcha may only be called once per action and get a valid result!ifparams[:recaptcha_challenge_field]@human=verify_recaptchaifcurrent_userself.current_user.status='verified'# for other actions set it here.else@verified=true# for NewsC#more there is no current user yet, so tell auto_create_user# to set statusendelsifparams[:turing_a].to_i>=3# 3 second delay@human=trueelse@human=falseendend

Auto creating users

The next part of getting rid of the signup form is to automatically create user objects, sessions, and remember-me cookies. This is what our old signup process did. We encapsulate this process into a method and have it run whenever an un-authenticated user tries to perform an action that requires a user object/session. We are using restful-authentication, and pulling out the relevant lines of code from there and from sessions controller yields this handy method:

# Create user object if doesn't already exist. Set cookie. def auto_create_user!user=User.auto_create_user_object# doesn't save to db as handle remember cookie does.user.status='verified'if@verified# user passed captcha, prevents showing captcha againUser.current_user=self.current_user=user# !! now logged in (sets session)handle_remember_cookie!true# sets cookie and saves user so they can get back after session is over# cookie set to 5 years.., the above uses @current_user set in previous line.end

Note that since the user doesn't have the ability to "log in", you'll want to set your remember-me cookie time span to an appropriately long time. This can be changed in vendor/plugins/restful-authentication/lib/authentication/by_cookie_token.rb Once the user object is auto-created, the user can return to their accounts page and add a login and password if they want to use their account on another computer. We auto-generate a random username for display purposes, letting the user customize that later if they want to.

That's it! The old signup form still works, but is no longer necessary to use all the site's functionality. We've been in production for about two weeks now with zero spam infiltration, but time will tell. If anyone has better ways to do any of this stuff, give some comment love below.

KK I'm putting it on my calendar to mail this to the Rails Rumble organizers next year. Before the competition. Last year they tried OpenID, this year it was a free-for-all again. It always feels weird to go through 20 registration forms for test applications -- knowing you'll probably never use it again. Perfect application for gradual reg.

great post (I learned something about signup forms! such an important theme), great site and great concept, I particularly liked the top-bar that shows up when you visit an external link.

Here are a few comments regarding what cbartus might have meant:

-your join and signup buttons are not very attractive.. color/shape. people like you and I are not designers. We need designer friends.

-your black on blue title header is wrong...

-generally speaking, there is a lot of font inconsistency between your header links, footer links and sidebar links.

-in your main section I personally would flip the order: drop down boxes first, explanation after.

-finally my last comment would be that any 'social' site -and yours is to a certain level- that has an essentially static homepage is flawed: the homepage must be live, even if it means showing irrelevant posts, you must be showing, somewhere, in a corner if you like, the latest posts, or most popular or ...