Run Faster Ruby on Rails Tests

Published Apr 17, 2014

Doing software development with a Continuous Integration process involves a lot of automated testing. Doing tests consumes precious time so we decided to do some tinkering to make our test suite load and execute faster.

As your test suite grows, it takes longer to run the tests, and soon you have a problem because your test suite is running for over 10 minutes. Now you have two options: reduce the number of tests or optimize the test execution time.

Impulse speed

A few months ago, we wrote about how we do Continuous Integration in Ruby on Rails. Back then, a test suite for one of our projects (around 1400 tests) ran for about 7 minutes. Today, an even larger test suite (around 1700 tests) runs for 2 and a half minutes. How did we do it?

The mighty Zeus

Every time you call a rake task, your Rails application has to be loaded. The time it takes to load the application depends heavily on the size of the application and the number of gems the application is using. In our example, it was around 10 seconds. At first, 10 seconds doesn’t sound that bad, but when you realize you are calling rake tasks all the time and you have to wait for 10 seconds every time, you start to feel unproductive.

So we started using Zeus. Zeus speeds up loading time of your Rails application by preloading your application only once and then monitoring file changes in your application directory and doing reloads in the background. Now you can call a rake task with: Zeus rake task and it’ll take less than one second to start because the Rails application is already preloaded.

Running a large test suite with Zeus isn’t a big win (we cut off the first 10 seconds). It’s most useful when you’re doing TDD because you run your tests numerous times while developing. If you have to wait 10 seconds every time, it’s just not doable.

Just take this simple model spec with 9 tests as an example:

	
time rspec spec/models/balance_spec.rb 
0m7.355s
time zeus rspec spec/models/balance_spec.rb 
0m0.930s

Spring maybe?

With the latest release of Rails 4.1, the Spring application preloader was introduced as a part of the standard Rails environment. The idea behind Spring is the same as behind the mighty Zeus – preload your Rails application to speed up development. We tried using Spring on our projects and it works almost equally awesome. We favor Zeus primarily because it’s noticeably faster (roughly 30% faster), but we are eagerly awaiting any future reversals.

Running tests in parallel

The thing that actually makes a huge difference when running a large test suite is running tests in parallel. Modern CPUs have lots of cores, which makes utilizing those cores for testing very effective.

Luckily, there is a gem called Parallel Tests which does just that. The gem creates multiple test databases (one for each core) and then distributes all tests across those cores and runs them in parallel.

You can even combine parallel tests with Zeus, using a gem called Zeus Parallel Tests. With this combination, we reduced the running time of our test suite by over 50%.

	
time rspec
1m50.843s
time zeus parallel_rspec
0m52.924s

Mock, extract and refactor

When a test is slow, it probably means that the model you’re testing is not optimized and you should be thinking about refactoring.

A typical antipattern is creating fat models with a lot of business logic and then wondering why tests are slow. It’s because you are testing things which don’t need to be tested (e.g. ActiveRecord) but mocked, and you’re needlessly hitting the database.

Let’s show this using a simple example:

	class PriceList < ActiveRecord::Base
  has_many :products

  def total_price
    products.sum(&:price)
  end
end

	require ’spec_helper’

describe PriceList do

  let(:price_list){ create(:price_list) }

  describe ’#total_price’ do
    before do
      create(:product, price_list: price_list, price: 100)
      create(:product, price_list: price_list, price: 200)
    end
    it { expect(price_list.total_price).to eq(300) }
  end
end

	
time rspec spec/model/price_list_spec.rb 
0m8.396s

Now let’s try to refactor our model and extract that #total_price functionality to a separate class so we can test it in isolation:

	class PriceSheet
  def total_price(items)
    items.map(&:price).inject(:+)
  end
end

	class PriceList < ActiveRecord::Base
  has_many :products

  def total_price
    PriceSheet.new.total_price(products)
  end
end

	require File.expand_path("../../../lib/price_sheet.rb", __FILE__)

describe PriceSheet do

  describe ’#total_price’ do
    let(:item_one) { double(price: 200) }
    let(:item_two) { double(price: 100) }
    it { expect(PriceSheet.new.total_price([item_one, item_two])).to eq(300) }
  end
end

	
time rspec spec/lib/price_sheet_spec.rb 
0m1.530s
time zeus rspec spec/lib/price_sheet_spec.rb 
0m0.270s

Now we don’t need to require the spec_helper (which loads Rails), we can mock the items and avoid hitting the database.

We like this approach, not only because our tests are faster, but also because it drives us to write better code and avoid writing fat models.

Mocking external services

One important thing to remember is – don’t let your tests access any external web services. There are several reasons why this is bad:

It really slows down tests
Tests will fail if you don’t have internet connectivity
Hitting third party services will unnecessarily use API request limits

This can be solved by disabling all remote connections in your tests, mocking the external service functionality and testing it in isolation. We use the WebMock gem for this.