Ruby on Rails
Speed Up Your Rails App by Squashing N+1 Queries
N+1 queries are one of the most common performance killers in Rails apps, but also one of the easiest to fix. In this post, we'll see how a single line of code can reduce 1,101 database queries down to 3.
N+1s occur in rails when you have associated ActiveRecord models, and iterate over one model while accessing fields on the associated records. This might be easier to explain with an example.
Let's say you have the following ActiveRecord models:
class Author < ApplicationRecord
has_many :posts
end
class Post < ApplicationRecord
belongs_to :author
has_many :tags
end
class Tag < ApplicationRecord
belongs_to :post
end
An author has many posts, and each post can have many tags. If we want to list each author and their blog posts like below, then we'll make 1 query to the database for the authors, and then another query for each author's blog posts. For N authors, that's 1 + N queries to the database.
Author.take(3).each do |author|
puts author.name
author.posts.each do |post|
puts post.title
end
end
We can see all the queries in the logs - 1 query for authors, then 3 queries for each author's blog posts:
Author Load (31.4ms) SELECT "authors".* FROM "authors"
Post Load (2.4ms) SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 163
Post Load (0.4ms) SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 164
Post Load (0.1ms) SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 165
In small doses this isn't a big deal - these are small, quick queries. However, as soon as the data gets larger, the queries more complex, or the cardinality of these queries grow, then things can really slow down.
Let's look at a case study, where we start with an unoptimized query with 2 layers of N+1s, and see how much faster we can make it.
Case 1 - The Unoptimized Query
Let's take the example above, and add another layer to it. On our authors index page, we want to list each author, then each of their blog posts, including their associated tags. With a totally unoptimized query, our controller action and view will look like this:
class AuthorsController < ApplicationController
def index
@authors = Author.all.order(:name)
end
end
# app/views/authors/index.html.erb
<h1>Authors</h1>
<% @authors.each do |author| %>
<h2><%= author.name %></h2>
<ul>
<% author.posts.each do |post| %>
<li><%= post.title %> - <%= post.tags.map(&:name).join(", ") %></li>
<% end %>
</ul>
<% end %>
As we can see in the logs, there's an explosion of queries - for 3 authors with 3 blog posts each with 3 tags, that's 13 queries!
Author Load (1.1ms) SELECT "authors".* FROM "authors" ORDER BY "authors"."name" ASC
Post Load (0.9ms) SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 163
Tag Load (1.1ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 631
Tag Load (0.2ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 632
Tag Load (0.2ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 633
Post Load (0.3ms) SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 172
Tag Load (0.2ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 716
Tag Load (0.3ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 717
Tag Load (0.2ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 718
Post Load (0.2ms) SELECT "posts".* FROM "posts" WHERE "posts"."author_id" = 262
Tag Load (0.2ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 1521
Tag Load (0.2ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 1522
Tag Load (0.2ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 1523
...
13 small queries isn't going to make that much of an impact. But what if there are 100 authors, each with 10 blog posts each, each post having 5 tags? That's 1,101 queries! On my well-specced computer, that page takes 1.007s to load.
That's a noticeable delay in page load. We can do better.
Case 2 - The Partially Optimized Query
N+1s are a common occurrence in a Rails app, and fortunately Rails has a great tool for dealing with them. We can use the .includes method to eagerly load the associated records in the original query. By including posts in the query, we're performing the posts query up front in bulk as a single query for all associated posts. Here's the updated controller action (the view won't change throughout this exercise).
class AuthorsController < ApplicationController
def index
@authors = Author.all.includes(:posts).order(:name)
end
end
We can see in the logs that we've cut down the number of queries - 1 for authors, 1 for posts, but still a tags query for each blog post. That totals 1,002 queries, down from 1,101. And page load time dropped from 1.007s -> 0.678s - that's a 32% improvement. Not bad! But, we can do better.
Author Load (23.1ms) SELECT "authors".* FROM "authors" ORDER BY "authors"."name" ASC
Post Load (1.2ms) SELECT "posts".* FROM "posts" WHERE "posts"."author_id" IN (513, 522, 612, ...)
Tag Load (0.8ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 4030
Tag Load (0.2ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 4031
Tag Load (0.2ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 4032
Tag Load (0.2ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 4033
Tag Load (0.2ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" = 4034
...
Case 3 - The Optimized Query
You'll notice in the partially optimized query, we included the posts, but there remained an N+1 for the tags. .includes allows you to include further related records using a hash to denote the connections between records:
class AuthorsController < ApplicationController
def index
@authors = Author.all.includes(posts: [:tags]).order(:name)
end
end
This allows us to include all the authors' posts, and all the posts' associated tags. We'll see in the logs that we're down to 3 queries now - one to bulk select the authors, one to bulk select the blog posts, and one to bulk select the tags. Page load dropped down to 0.126s - a 81% improvement over the partially optimized query, and an 87% improvement over the original!
Author Load (16.3ms) SELECT "authors".* FROM "authors" ORDER BY "authors"."name" ASC
Post Load (1.0ms) SELECT "posts".* FROM "posts" WHERE "posts"."author_id" IN (513, 522, 612, ...)
Tag Load (1.0ms) SELECT "tags".* FROM "tags" WHERE "tags"."post_id" IN (4030, 4031, 4032, 4033, 4034, 4035, 4036, ...)
Wrapping it up
With a single line change, we can make a massive improvement in performance. You can argue in isolation that this is a small win. But with multiple unoptimized queries on a single page, or with N+1s persistent across an app, users will feel the sluggishness. Why normalize slow performance when instead we can have fast apps?
Need Help with your Rails App?
At Hashrocket, we specialize in building robust, performant applications using modern technologies. If you're modernizing or speeding up your Ruby on Rails app, get in touch to discuss how our team of expert developers can help.
Photo by Vitaly Gariev on Unsplash