Test Driven Rails - Part 2

Mar 3rd, 2014

Last time, in part 1, I was giving some advice about testing - why to test at all, which tests are valuable and which are not, when to write acceptance tests and in what cases aim for the maximum code coverage. It brought about some serious discussion about testing ideas and if you haven’t read it yet, you should probably check (it) it out. Giving some general point of view about such broad topic like Test Driven Development / Behavior Driven Development is definetely not enough so I will try to apply these techniques by implementing a concrete feature. I wanted to choose some popular usecase so that most developers will have an opinion how they would approach it. In most applications you will probably need:

User Registration

It is quite common feature and it can be approached in many ways. The most popular is to some authentication gem, like Devise, which is the probably the safest and the fastest way. However, Devise might be an overkill for some cases or maybe you need highly customizable solution. How would you write an implementation fot that usecase then?

Note: the implementation below doesn’t aim to be the most secure approach for that feature, it’s rather for demonstration purposes. I made some non-standard design decisions for the Rails application, you may want to read one of my previous posts to get more details why this way of designing code might be beneficial.

Specification

We know that we want to implent user registration. Let’s say that we want user to confirm his/her account before signing in so we will need to send some confirmation instructions. Also, let’s add some admin notifications about new user being registered to make it more interesting.

To make it even better, let’s assume that we will create both User and UserProfile during registration: User will have just an email and encrypted_password attributes, UserProfile will have country and age attributes. User will also have to accept some policy to register. If we want to have confirmation, we will also need some attributes for confirmation_token, confirmation date (confirmed_at) and let’s add confirmation_instructions_sent_at just to know, when the instructions were sent. These are just registration-specific attributes and we won’t need them in most cases so let’s extract them to UserRegistrationProfile

Start with acceptance tests

When writing new feature we should start from acceptance tests - we will make sure that the feature works from the higher level: from the user perspective and some side effects like sending emails. So the good start will be covering user creation and sending emails to an admin and to the user. Let’s write some Capybara tests:

require'spec_helper'feature"User Registration"docontext"when visiting new user path"dobackgrounddovisitnew_user_pathendcontext"registering with valid data"dogiven(:email){"myawesome@email.com"}backgrounddofill_form_with_valid_data(email:email)endscenario"new user is created"doexpectdoregisterend.tochange(User,:count).by(1)endcontext"notifications"dobackgrounddoregisterendscenario"confirmation email is sent to the user"doexpect(all_email_addresses).toincludeemailendscenario"notification is sent to the admin"doexpect(all_email_addresses).toinclude"admin@example.com"endscenario"2 emails are sent"doexpect(all_emails.count).toeq2endendendendenddeffill_form_with_valid_data(args={})email=args.fetch(:email,"email@example.com")fill_in"email",with:emailfill_in"user_password",with:"my-super-secret-password"fill_in"user_password_confirmation",with:"my-super-secret-password"fill_in"age",with:22select"Poland",from:"country"check"policy"enddefregisterclick_button"Register"end

I like using some helper methods, especially in acceptance tests so I wrote fill_form_with_valid_data and register helpers - these are just some details and I don’t need to know them when reading tests. There are also some helpers like all_email_addresses and all_emails, which come from the MailerMacros:

For user registration, we have REST actions: new and create. Let’s also add some root page, currently just to get rid of default Rails page:

app/controllers/static_pages_controller.rb

1234567

classStaticPagesController<ApplicationControllerdefhomeendend

But how to deal with UsersController and form for user registration? We have some fields that are not present in models (password/password_confirmation and policy). The popular solution would be using: accepts_nested_attributes_for :profile and some virtual attributes. I don’t really like this solution, accepts_nested_attributes_for sometimes can really save a lot of time, especially with complex nested forms with nested_form gem. But virtual attributes are quite ugly and they make models the interfaces for forms. Much better approach is to use form objects. There’s a great gem for this kind of problems: Reform - we will use it here.

Reform is not (yet) that popular in the Rails community so some things require explanation (check also the docs out). The Reform::Form::ActiveRecord module is for uniqueness validation and the Composition is for… composition - some properties are mapped to user and other to profile. There is also a mystical mapping with on: :nil - these are “virtual” properties like password, password_confirmation and policy - all properties must be mapped to a resource so just to satisfy Reform API I use on: :nil as a convention, also the empty: true option is for virtual attributes that won’t be processed. And where does the email validation come from? From our custom validator, let’s write some specs but before we should add /forms (and /usecases for business logic) directories to be autoloaded:

You can probably come up with some more examples to cover email validation but these are sufficient cases. I’ve introduced DummyModel here to have a generic object that can be validated so the ActiveModel::Validations module is needed and an accessor for an email. Let’s implement the actual validation:

usecases/email_validator.rb

123456789

classEmailValidator<ActiveModel::EachValidatordefvalidate_each(record,attribute,value)unlessvalue=~/\A([^@\s]+)@((?:[-a-z0-9]+\.)+[a-z]{2,})\z/irecord.errors[attribute]<<(options[:message]||"is not a valid email format")endendend

The regexp for email validation comes from Rails guides:). It won’t cover all the possibilities but the actual format of the email is an overkill.

I don’t fell the need to write tests for other validations and composition for UserRegistrationForm: it’s just using very descriptive DSL, the validation are already tested in Rails.

I added also validations in models. These may seem like a duplication because form object already implements them but these are validations always applicable do these models so it is a good idea to have them in models.

Let’s concentrate on UsersController and create action. I don’t really like testing controllers, especially for CRUD-like stuff, user creation still feels like CRUD but not that typical in Rails, especially when using dedicated form object. So let’s test drive registration process: we are going to use UserRegistrationForm for data aggregation and validation - if the data is valid, the user will be created by UserRegistration service object with redirection to root path, otherwise it will render new template.

Well, it is not really clear, that’s the problem with testing controllers and they should be as thin as possible. We need to implement the assign_attributes
method in form object to fill models’ attributes with params and implement the actual UserRegistration usecase. In tests I use instance_double
instead of simple double to make sure I’m not stubbing non-existent methods or with wrong number of arguments - that’s a great feature introduced in RSpec 3, which comes from rspec-fire gem. Also, I’m stubbing responses so that I can spy on them using have_received method - It’s much cleaner and easier to read. Compare these two examples:

For simple persistence logic I would probably go with that approach but we will also need to send some confirmation instructions, admin notifications etc., I’m not really comfortable with the idea of form object knowing something about sending notifications, persistence alone would be ok, it would be quite convenient to use but this is too complex, I would leave form object for data aggregation and validation. Let’s write code for the controller:

It uses some Reform::Form private methods that I found in source code so this implementation might not be stable but fortunately we have it covered in tests so we will know breaking changes if it happens in next versions. And there’s a gotcha here: The keys in hash must be stringified, symbols won’t work (applies to 0.2.4 version of Reform).

And what the UserRegistration should be responsible for? Let’s start with persistence logic: user with it’s profile must be created and the encrypted password should be assigned to the user. We will also need registration profile to be created.

Note: keep in mind that you should write one test and then write minimal implementation to make it pass and then another test. I gave the several tests and the actual UserRegistration in advance, just to make it easier to read and follow.

It is quite clear from the tests what should be expected from this class: creation of user, profile, registration profile and assigning encrypted password. Data aggregate (form) is just a double with profile and user, we don’t care what it actually is, it should just implement the stubbed interface. I also use FactoryGirl and build_stubbed method for initializing models - I find it more convenient than to use instance_double because instance doubles don’t cover attributes from database tables.

The factories for User and profiles would look like that:

spec/factories.rb

123456789101112131415

FactoryGirl.definedofactory:userdoemail"email@example.com"# I'll explain that later, why it is that longencrypted_password"$2a$10$bcMccS3q2egnNICPLYkptOoEyiUpbBI5Q.GAKe0or2QB7ij6yCeOa"endfactory:user_profiledoage22country"Poland"endend

And the actual implementation:

app/usecases/user_registration.rb

1234567891011121314151617181920212223242526272829303132333435363738

classUserRegistrationclassRegistrationFailed<StandardError;endattr_reader:encryptionprivate:encryptiondefinitialize(options={})@encryption=options.fetch(:encryption,Encryption.new)enddefregister!(aggregate)user=aggregate.userprofile=aggregate.profileuser.encrypted_password=encrypted_password(aggregate.password)ActiveRecord::Base.transactiondobeginuser.save!profile.save!user.create_registration_profile!rescue::ActiveRecord::StatementInvalid,::ActiveRecord::RecordInvalid=>eraise_registration_error(e)endendendprivatedefraise_registration_error(errors)message="Registration Failed due to the following errors: #{errors}"raiseUserRegistration::RegistrationFailed,messageenddefencrypted_password(password)encryption.generate_password(password)endend

Let’s discuss some design decisions: the constructor accepts options hash so that we can inject dependencies like encryption and to provide defaults if it’s injected. The persistence logic is wrapped in transaction block so that e.g. user won’t be created if profile creation fails. If it fails, RegistrationFailed error is raised with a descriptive message. Also, the encryption is private: we don’t need it to be public.

To satisfy tests, the create_registration_profile! must be implemented and generate_password for encryption. Fortunately, we just need to setup associations for UserRegistrationProfile to have create_registration_profile! implemented. But we need to generate the model first:

The minimal implementation for Encryption to make the UserRegistration tests happy is the following:

app/usecases/encryption.rb

123456

classEncryptiondefgenerate_password(phrase)endend

To finish the user creation we have to implement the password generation. Bcrypt and it’s create password method is a reasonable choice here. Let’s write the tests:

spec/usecases/encryption_spec.rb

12345678910111213141516171819

require'spec_helper'describeEncryptiondosubject{Encryption.new}let(:password){"password"}let(:encypted_password){"$2a$10$vI8aWBnW3fID.ZQ4/zo1G.q1lRps.9cGLcZEiGDMVr5yUP1KUOYTa"}let(:password_generator){class_double(BCrypt::Password).as_stubbed_const}before(:each)doallow(password_generator).toreceive(:create).with(password){encypted_password}endit"creates password using Bcrypt as default"doexpect(subject.generate_password(password)).toeqencypted_passwordendend

The encrypted_password doesn’t have to be that long but looks more genuine that way. The BCrypt::Password is also a class double so that we make sure we don’t stub a non-existent method. And the implementation of Encryption class:

The pattern for constructor is similar to the one from UserRegistration. The password_generator is also made private - the rule of thumb is that everything should be private unless it needs to be public, just to keep the interfaces clean.

Now we have the basic implementation for user creation with it’s profiles. Still, we need confirmation stuff and notification to tje admin. It is beyond the UserRegistration responsibilities, we also don’t need always to a notification or confirmation instructions or to confirm user at all, just to have the interface flexible enough. Maybe we will have some additional things that will take place during registration - like third party API notification. To keep the responsibilities separate and UserRegistration easy to use, we can implement all the additional actions as the listeners that are being passed to the constructor of UserRegistration. Let’s write specs for it first:

spec/usecases/user_registration_spec.rb

12345678910111213141516171819202122232425262728293031323334353637

require'spec_helper'describeUserRegistrationdo# same code a beforecontext"persistence is success"do# same code a before# and this is new:context"with listeners"dolet(:user_confirmation){double(:user_confirmation,notify:true)}let(:admin_notification){double(:admin_notification,notify:true)}before(:each)doUserRegistration.new(user_confirmation,admin_notification,encryption:encryption).register!(form)endit"notifies user_confirmation listener"doexpect(user_confirmation).tohave_received(:notify).with(user)endit"notifies admin_notificaiton listener"doexpect(admin_notification).tohave_received(:notify).with(user)endendendend

We don’t actually care what the listeners are, the only requirement is that they must implement the same interface: notify method which takes user argument. And the implementation:

classUserRegistrationclassRegistrationFailed<StandardError;endattr_reader:encryption,:listenersprivate:encryption,:listenersdefinitialize(*listeners,**options)@listeners=listeners@encryption=options.fetch(:encryption,Encryption.new)enddefregister!(aggregate)user=aggregate.userprofile=aggregate.profileuser.encrypted_password=encrypted_password(aggregate.password)ActiveRecord::Base.transactiondobeginuser.save!profile.save!user.create_registration_profile!rescue::ActiveRecord::StatementInvalid,::ActiveRecord::RecordInvalid=>eraise_registration_error(e)endendnotify_listeners(user)endprivatedefraise_registration_error(errors)message="Registration Failed due to the following errors: #{errors}"raiseUserRegistration::RegistrationFailed,messageenddefencrypted_password(password)encryption.generate_password(password)enddefnotify_listeners(user)listeners.eachdo|listener|listener.notify(user)endendend

These changes are not that noticeable but they are huge. The constructor now takes some listeners (splat) - we can pass one listener, several or none, it will always be an array. Also, the options is now a keyword argument introduced in Ruby 2.0 which makes the changes really smooth. And the new method: notify_listeners which sends notify message to all the listeners with user argument.

To handle the user confirmation stuff we will need, well, UserConfirmation and UserRegistrationAdminNotification to handle the notifcations.

Let’s start with UserConfirmation. We need notify method which will take care of: assigning confirmation token, which must be unique, setting date when the confirmation instructions were sent and sending the instructions. We will need some mailer here (UserConfirmationMailer), clock (DateTime) and something to generate token - SecureRandom will be a good fit here with it’s base64 method. Let’s translate the specification to the tests:

spec/factories.rb

123456789

FactoryGirl.definedo# same as beforefactory:user_registration_profiledo# this in new hereendend

require'spec_helper'describeUserConfirmationdodescribe"#notify"dolet!(:user){FactoryGirl.build_stubbed(:user,registration_profile:FactoryGirl.build_stubbed(:user_registration_profile))}let(:mailer_stub){double(:mailer,deliver:true)}let!(:mailer){class_double(UserConfirmationMailer,send_confirmation_instructions:mailer_stub).as_stubbed_const}let(:confirmation_instructions_sent_date){DateTime.new(2014,2,23,21,0,0)}let(:clock){double(:clock,now:confirmation_instructions_sent_date)}subject{UserConfirmation.new(mailer:mailer,clock:clock)}before(:each)doallow(user).toreceive(:save_with_profiles!)allow(SecureRandom).toreceive(:base64){"token"}subject.notify(user)endit"assigns confirmation token to user"doexpect(user.confirmation_token).toeq"token"endit"sends email with confirmation instructions"doexpect(mailer).tohave_received(:send_confirmation_instructions).with(user)endit"sets date when the confirmation instructions have been sent"doexpect(user.confirmation_instructions_sent_at).toeqconfirmation_instructions_sent_dateendit"persists new data"doexpect(user).tohave_received(:save_with_profiles!)endendend

Like before, we should start with one test, make it pass and then write the next one. Here is the implementation for it:

The pattern for constructor is similar to the previous ones: provide the way to inject dependencies and some defaults if they are not specified so it is more flexible, less coupled and the testing becomes easier as a bonus. We have while loop to ensure the confirmation token is unique amongst users. The find_by_attribute methods are deprecated since Rails 4.0.0 and the activerecord-deprecated_finders will be removed from dependencies in 4.1.0 so we have to implement our own finder method. Here are also some important design decisions - we assign both confirmation_instructions_sent_at and confirmation_token to the user, not the registration profile. How is that? The important question is: do we need to expose that the user has registration profile? What if we change our mind and decide to put this data in “normal” profile, not registration profile? Or we didn’t make a decision to create a registration profile at all in a first place and these attributes belonged to the user since the beginning and we later decided to move them to a separated table? From the UserConfirmation perspective, it is just an implementation detail. The save_with_profiles! is provided to make user’s data persistence more convenient. We need to implement mailer as well but let’s start with user’s related stuff.

The find_by_confirmation_token finder method is pretty easy but it involves another table with registration profile so I decided to write test for it. The tests also suggest that we need readers for these attributes, not only the writers. Let’s use delegate macro from ActiveSupport for it:

app/models/user.rb

1234567891011121314151617181920212223242526272829303132333435363738

classUser<ActiveRecord::Base# the same code as before# new codedelegate:confirmation_token,:confirmation_instructions_sent_at,:confirmed_at,to::registration_profile,allow_nil:truedefself.find_by_confirmation_token(token)joins(:registration_profile).where("user_registration_profiles.confirmation_token = ?",token).firstenddefconfirmation_token=(token)ensure_registration_profile_existsregistration_profile.confirmation_token=tokenenddefconfirmation_instructions_sent_at=(date)ensure_registration_profile_existsregistration_profile.confirmation_instructions_sent_at=dateenddefsave_with_profiles!User.transactiondosave!profile.save!ifprofileregistration_profile.save!ifregistration_profileendendprivatedefensure_registration_profile_existsbuild_registration_profileifregistration_profile.blank?end

Before making any assignment, we have to make sure that the registration profile exists. The same applies to the persistence, which is again wrapped in a transaction. And let’s implement the mailer for sending confirmation instructions:

The setup with FactoryGirl may seem to be complex but I like doing this kind of setup manually, not to rely on predefined attributes for the factory so that I know where the data comes from. We also assume that there will be some controller action for confirmations so we will need to define routes to make the tests pass:

To make all the tests happy, we need some to send a notification to the admin. It looks like UserRegistrationAdminNotification will be just an adapter layer for NewUserAdminNotificationMailer to provide the listener interface. The tests and the implementation are quite simple:

spec/usecases/user_registration_admin_notification_spec.rb

1234567891011121314151617181920212223242526

require'spec_helper'describeUserRegistrationAdminNotificationdodescribe"#notify"dolet!(:user){FactoryGirl.build_stubbed(:user)}let(:mailer_stub){double(:mailer,deliver:true)}let!(:mailer){class_double(NewUserAdminNotificationMailer,notify:mailer_stub).as_stubbed_const}subject{UserRegistrationAdminNotification.new(mailer:mailer)}before(:each)dosubject.notify(user)endit"sends email to admin about new user being registered"doexpect(mailer).tohave_received(:notify).with(user)endendend

Now the tests for the mailer and we are almost finished with the registration:

spec/mailers/new_user_admin_notification_mailer_spec.rb

1234567891011121314151617181920

require"spec_helper"describeNewUserAdminNotificationMailerdodescribe"#notify"dolet!(:user){FactoryGirl.build_stubbed(:user,email:"email@example.com")}let(:mail){NewUserAdminNotificationMailer.notify(user)}it"sends email to the admin"doexpect(mail.to).toeq(["admin@example.com"])endit"has link to confimartion in the body"doexpect(mail.body.encoded).tomatch(user.email)endendend

The views for the mailer:

app/views/new_user_admin_notification_mailer/notify.html.erb

1

<p>Newuserwithemail:<%=@user.email%> has registered</p>

And the listener for UserRegistrationAdminNotification in UserRegistrationAdminNotification.new:

Hell yeah, all tests are happy now, we have completed the user registration feature. Let’s add account confirmation feature and simple sign in. We don’t need acceptance test or integration test in controller for that feature, it’s pretty simple and unit test for controller would be enough. We probably need to find user by confirmation token, confirm the account and redirect to some page. Also, we should return 404 error if there’s no match for confirmation token. In a real world application it would probably need some expiration date for token and other features but keep in a mind it’s just for demonstration purposes, not writing the complete devise-like solution.

We find the user with bang method so, by convention, it raises ActiveRecord::RecordNotFound if the resource is not found - we won’t write test for the failure path. Then the confirm! method is used which needs to be implemented and redirect to root path. The implementation for the controller is the following:

app/controllers/confirmations_controller.rb

123456789

classConfirmationsController<ApplicationControllerdefconfirmuser=User.find_by_confirmation_token!(params[:token])user.confirm!redirect_toroot_path,notice:"You have confirmed you account. Now you can login."endend

Now we need to implement confirm! and find_by_confirmation_token! methods:

spec/models/user_spec.rb

123456789101112131415161718192021222324252627282930

require'spec_helper'describeUserdosubject{User.new(email:"email@example.com",encrypted_password:"password")}# some old codedescribe"#confirm!"dolet(:date){DateTime.new(2014,02,23,22,6,0)}before(:each)doallow(DateTime).toreceive(:now){date}subject.confirm!endit"assigns confirmation date with current date"doexpect(subject.confirmed_at).toeqdateendit"persists user and the profile"doexpect(subject.registration_profile.persisted?).toeqtrueexpect(subject.persisted?).toeqtrueendendend

It would be quite convenient to use confirm! method for new user and make it persisted. I don’t feel a need to write test for find_by_confirmation_token! as it is really simple and will use find_by_confirmation_token.

It’s another method in User model, shouldn’t the models be thin? Well, it’s not business logic involving some complex actions, these are just domain methods for user to handle it’s own state (or the profile’s state which is rather an implementation detail in this case), it looks like a model’s responsibility so it’s a right place to add this kind of logic, much better than using e.g. update method on registration profile outside the models.

We completed another feature: user can confirm his/her account. There’s only one feature left: sign in. Let’s test drive it starting from acceptance test again: when the user exists and is confirmed, we let the user sign in, if exists but is not confirmed yet we render proper info and if the email/password combination is invalid we also want to display proper info. Capybara test for these specs may look like this:

The structure is similar to the one from registration process: there are some helper methods for filling forms and signing in. For the happy path, we also want to verify that the user is actually signed in so we will display it’s email. This test will work because in User factory in spec/factories.rb the encrypted_password value is an encrypted form of “password” phrase. Let’s start from defining routes and creating controller for the user signin:

Let’s stick to the convention and name the helper method with current user the current_user. The signin process is already covered by acceptance test so we won’t benefit much from writing controller’s test. To keep track of current user, we will store it’s id in a session. The implementation might be following:

classApplicationController<ActionController::Base# Prevent CSRF attacks by raising an exception.# For APIs, you may want to use :null_session instead.protect_from_forgerywith::exceptionhelper_method:current_userprivatedefcurrent_user@current_user||=User.find(session[:user_id])ifsession[:user_id].present?endend

The last step is to implement Authentication module with authenticate method, which compares encrypted password with plain password. Let’s start with a test:

spec/usecases/authentication_spec.rb

123456789101112131415161718192021

require'spec_helper'describeAuthenticationdodescribe".authenticate"dolet(:encrypted_password){"$2a$10$bcMccS3q2egnNICPLYkptOoEyiUpbBI5Q.GAKe0or2QB7ij6yCeOa"}it"returns true if encrypted password is specified password"doexpect(Authentication.authenticate(encrypted_password,"password")).toeqtrueendit"returns false if encrypted password in not specified password"doexpect(Authentication.authenticate(encrypted_password,"wrong_password")).toeqfalseendendend

I don’t need to create an instance of Authentication, there is no need to make it a class. And to make all the tests green we just need to implement comparison of passwords using Bcrypt:

Wrapping up

That’s all! All the tests now pass. And they run pretty fast (on Ruby 2.1.0), about 1.6 s. That was quite long: the user registration, confirmation and sign in features have been test drived and some not obvious design decisions were made. That gives some basic ideas how I apply Test Driven Development / Behavior Driven Development techniques in everyday Rails programming. The aim of these tests wasn’t to have 100% coverage (e.g. I didn’t test ActiveModel validations, using Reform DSL to make UserRegistrationForm composition, delegations in User model) but they give me sufficient level of confidence to assume that the application works correctly and they helped with some design choices, which is a great advantage of unit tests. When TDDing, keep in mind what Kent Beck says about his way of writing tests:

I get paid for code that works, not for tests so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it.

Comments

About Me

Hi, my name is Karol Galanciak, I'm a technical polyglot and domain expert in vacation rental and fitness industries.

I specialize in building APIs, microservices architecture, working with legacy applications, building ambitious Single Page Applications and building payment processors for credit cards processing. Currently I'm working as CTO at BookingSync, so at this moment I'm not available for Ruby on Rails or Ember.js consulting.

Overwhelmed by too much information?

Following all the newsletters, RSS feeds, Twitter and other sources might be overwhelming. Stop wasting your valuable time - subscribe now and by the end of every month I will send you an email with the list of the most interesting Ruby / JavaScript (Ember.js) / PostgreSQL articles with a quick overview.