Heading image for post: Speed Up Your Rails App by Squashing N+1 Queries

Ruby on Rails

Speed Up Your Rails App by Squashing N+1 Queries

Profile picture of Tony Yunker

N+1 queries are one of the most common performance killers in Rails apps, but also one of the easiest to fix. In this post, we'll see how a single line of code can reduce 1,101 database queries down to 3.

N+1s occur in rails when you have associated ActiveRecord models, and iterate over one model while accessing fields on the associated records. This might be easier to explain with an example.

Let's say you have the following ActiveRecord models:

class Author < ApplicationRecord
  has_many :posts
end

class Post < ApplicationRecord
  belongs_to :author
  has_many :tags
end

class Tag < ApplicationRecord
  belongs_to :post
end

An author has many posts, and each post can have many tags. If we want to list each author and their blog posts like below, then we'll make 1 query to the database for the authors, and then another query for each author's blog posts. For N authors, that's 1 + N queries to the database.

Author.take(3).each do |author|
  puts author.name
  author.posts.each do |post|
    puts post.title
  end
end

We can see all the queries in the logs - 1 query for authors, then 3 queries for each author's blog posts:

Author Load (31.4ms)  SELECT "authors".* FROM "authors"
Post Load (2.4ms)  SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 163
Post Load (0.4ms)  SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 164
Post Load (0.1ms)  SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 165

In small doses this isn't a big deal - these are small, quick queries. However, as soon as the data gets larger, the queries more complex, or the cardinality of these queries grow, then things can really slow down.

Let's look at a case study, where we start with an unoptimized query with 2 layers of N+1s, and see how much faster we can make it.

Case 1 - The Unoptimized Query

Let's take the example above, and add another layer to it. On our authors index page, we want to list each author, then each of their blog posts, including their associated tags. With a totally unoptimized query, our controller action and view will look like this:

class AuthorsController < ApplicationController
  def index
    @authors = Author.all.order(:name)
  end
end

# app/views/authors/index.html.erb
<h1>Authors</h1>

<% @authors.each do |author| %>
  <h2><%= author.name %></h2>
  <ul>
    <% author.posts.each do |post| %>
      <li><%= post.title %> - <%= post.tags.map(&:name).join(", ") %></li>
    <% end %>
  </ul>
<% end %>

As we can see in the logs, there's an explosion of queries - for 3 authors with 3 blog posts each with 3 tags, that's 13 queries!

Author Load (1.1ms)  SELECT "authors".* FROM "authors" ORDER BY "authors"."name" ASC
Post Load (0.9ms)  SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 163 
Tag Load (1.1ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 631 
Tag Load (0.2ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 632 
Tag Load (0.2ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 633 
Post Load (0.3ms)  SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 172 
Tag Load (0.2ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 716 
Tag Load (0.3ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 717 
Tag Load (0.2ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 718 
Post Load (0.2ms)  SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 262 
Tag Load (0.2ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 1521 
Tag Load (0.2ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 1522 
Tag Load (0.2ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 1523 
...

13 small queries isn't going to make that much of an impact. But what if there are 100 authors, each with 10 blog posts each, each post having 5 tags? That's 1,101 queries! On my well-specced computer, that page takes 1.007s to load.

That's a noticeable delay in page load. We can do better.

Case 2 - The Partially Optimized Query

N+1s are a common occurrence in a Rails app, and fortunately Rails has a great tool for dealing with them. We can use the .includes method to eagerly load the associated records in the original query. By including posts in the query, we're performing the posts query up front in bulk as a single query for all associated posts. Here's the updated controller action (the view won't change throughout this exercise).

class AuthorsController < ApplicationController
  def index
    @authors = Author.all.includes(:posts).order(:name)
  end
end

We can see in the logs that we've cut down the number of queries - 1 for authors, 1 for posts, but still a tags query for each blog post. That totals 1,002 queries, down from 1,101. And page load time dropped from 1.007s -> 0.678s - that's a 32% improvement. Not bad! But, we can do better.

Author Load (23.1ms)  SELECT "authors".* FROM "authors" ORDER BY "authors"."name" ASC 
Post Load (1.2ms)  SELECT "posts".* FROM "posts" WHERE "posts"."author_id" IN (513, 522, 612, ...) 
Tag Load (0.8ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 4030 
Tag Load (0.2ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 4031 
Tag Load (0.2ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 4032 
Tag Load (0.2ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 4033 
Tag Load (0.2ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 4034 
...

Case 3 - The Optimized Query

You'll notice in the partially optimized query, we included the posts, but there remained an N+1 for the tags. .includes allows you to include further related records using a hash to denote the connections between records:

class AuthorsController < ApplicationController
  def index
    @authors = Author.all.includes(posts: [:tags]).order(:name)
  end
end

This allows us to include all the authors' posts, and all the posts' associated tags. We'll see in the logs that we're down to 3 queries now - one to bulk select the authors, one to bulk select the blog posts, and one to bulk select the tags. Page load dropped down to 0.126s - a 81% improvement over the partially optimized query, and an 87% improvement over the original!

Author Load (16.3ms)  SELECT "authors".* FROM "authors" ORDER BY "authors"."name" ASC
Post Load (1.0ms)  SELECT "posts".* FROM "posts" WHERE "posts"."author_id" IN (513, 522, 612, ...)
Tag Load (1.0ms)  SELECT "tags".* FROM "tags" WHERE "tags"."post_id" IN (4030, 4031, 4032, 4033, 4034, 4035, 4036, ...)

Wrapping it up

With a single line change, we can make a massive improvement in performance. You can argue in isolation that this is a small win. But with multiple unoptimized queries on a single page, or with N+1s persistent across an app, users will feel the sluggishness. Why normalize slow performance when instead we can have fast apps?


Need Help with your Rails App?

At Hashrocket, we specialize in building robust, performant applications using modern technologies. If you're modernizing or speeding up your Ruby on Rails app, get in touch to discuss how our team of expert developers can help.

Photo by Vitaly Gariev on Unsplash

  • Adobe logo
  • Barnes and noble logo
  • Aetna logo
  • Vanderbilt university logo
  • Ericsson logo

We're proud to have launched hundreds of products for clients such as LensRentals.com, Engine Yard, Verisign, ParkWhiz, and Regions Bank, to name a few.

Let's talk about your project