by Mary Lee on March 28, 2019

To wrap up this series, let’s talk about the helpers we added to make writing tests with the JsonApiClient resources less painful.

We were building an internal JSON API using the following libraries:

• json_api_client for reading data from the API
• jsonapi-resources for creating the internal JSON API endpoints

For more information on our goals and set up for the internal API, see part 1 of this series.

FactoryBot makes it incredibly easy to build factories for non-database backed resources using the skip_create option. We made factories for all of our resources, to make it easier to stub and work with the resources in tests.

factory :foo do
  skip_create
  sequence(:id)
  sequence(:title) { |n| "title#{n}" }
end

Writing WebMock stubs for JsonApiClient required a lot of knowledge of JSON API specifications, and was tedious to have to write over and over again, so we decided to add some test helpers to streamline the process.

We knew we wanted to be able to support CRUD stubs for all of our resources being converted, and we wanted to make it as intuitive as possible for future developers on the project, so we decided to implement the helpers in a way that meant every model being converted from ActiveRecord to the API had a method for stubs (so our Foo model would have stub_get_foos, stub_create_foo, stub_delete_foo, etc).

We created a new file in our RSpec support directory, and then created a constant that would be a list of all of the ActiveRecord models being converted to JsonApiClient resources.

module API
  module Mocks
    API_RESOURCES = %i[
      foo
      bar
      # … 
    ]
  end
end

Our first step was figuring out how to build the routes for all of our stubs. We knew that we wanted to be able to pass an API resource type, and generate our paths from that. We started by defining our base path. Our method accepted a resource type, and then pluralized the resource type, appending it to the base path for our API.

def resource_base_path(resource_type)
  "#{ ENV['LOCAL_API_URL'] }/#{ resource_type.to_s.pluralize }"
end

We then moved on to generating the index path for the resources. We knew we needed to be able to support query string params on the index, for the sake of sorts, eager loading, and pagination. We wanted to be able to specify our exact params, but also allow for a general match on any params that might come through. If exact params were given, they were converted to query strings, otherwise we used a general regex to match on any characters that might come through.

def resource_index_path(resource_type, params: nil)
  params_matcher = if params.present?
    # Generate query string if params are passed
    params.to_query.gsub("+", "%20")
  else
    # Match on any characters that might come through if no params are present
    ".*"
  end


  # Generate the final regex
  /#{ resource_base_path(resource_type) }($|\?#{ params_matcher })/
end

Last, we tackled the individual resource path. We wanted to be flexible with the path, allowing the stubbed URL to be as specific or vague as needed. For us, this meant accounting for the optional presence of an ID and query string params. If an ID was passed, we would use that ID in the stubbed URL, otherwise we would use a regex that would match on the presence of any number of integers. If query string params were passed, we would append those to the URL we were building.

def resource_show_path(resource_type, id: nil, params: nil)
  # Match on exact ID or use regex to match on any number
  id_matcher = id.presence || "[0-9]+"

  # Generate query string from params passed if present, otherwise, use an empty string
  query_string = params.present? ? URI.unescape(params.to_query) : ""

  # Generate the final regex
  %r{#{resource_base_path(resource_type)}/(#{id_matcher})($|\?#{query_string})}
end

With our path helpers defined, we could now move on to defining our stub helpers.

The first stub we tackled was for API indexes. The index stub wound up being our most complicated stub because of work earlier in the project to make our index requests POSTs instead of GETs if filtering params were present. That customization meant that our stub had to support both the GET and POST requests. It also needed to be able to support pagination, and allow us to specify the response body or status if we wanted to.

For integrating pagination, we knew that JSON spec required a response with a data key for returned resources, and then a links key for pagination, if pagination was present. Based on our test requirements, we knew we only needed to support a response that included pagination links, but the links didn’t need to work beyond helping us stub resource counts in our tests (see how we used pagination for getting counts in part 6 of this series). For this reason, we didn’t need a URL that would match up to the index URL we were working with; the only important thing for us was the link params.

We created a reusable method called build_response_body that would accept a response body to set to the data key in the JSON response, as well as options that would allow us to specify whether or not we wanted to include pagination links.

def build_response_body(response_body, options = {})
  # Set the data key with our response body
  { data: response_body }.tap do |response|

    # Check if we wanted to include pagination
    if options[:include_pagination_links]

      # Set the links key with our first, next, and last pages for pagination
      response[:links] = {
        first: "http://localhost?page[number]=1&page[size]=1",
        next: "http://localhost?page[number]=2&page[size]=1",
        last: "http://localhost?page[number]=3&page[size]=1"
      }
    end
  end
end

To handle the POST request when filter params were present, we added a stub_filter_resource method. This method would accept a resource type, using it to build the index path stub, as well as initializing objects to return for the response if the response was filtering resources by IDs. The method would also accept options for specifying a response status or response body. If no IDs were passed in the filters, and no response body was specified, we defaulted to returning an empty array.

def stub_filter_resource(resource_type, options = {})
  # Stub the POST request with the resource URL
  WebMock.stub_request(:post, "#{ resource_base_path(resource_type) }/filter").

    # Send the specified filtering params as the request body
    with(body: options[:params]&.to_json || /.*/).
    to_return do |request|

      # If no response body was specified, try to build a response using request params,
      #  checking to see if IDs were used to filter. If IDs were present, build FactoryBot
      #  objects with the IDs, otherwise return an empty array.
      unless (response_body = options[:response_body])
        ids = Array(JSON.parse(request.body).dig("filter", "id")) || []
        response_body = ids.map { |id| FactoryBot.build(resource_type, id: id) }
      end

      {
        status: options.fetch(:response_status, 200),
        body: build_response_body(response_body, options).to_json,
        headers: { "Content-Type" => "application/vnd.api+json" }
      }
    end
end

Finally, to handle the GET request, we added a stub_get_resources method, again accepting a resource type for building the request URL. We chose to have our stub_get_resources always set the stub for filtering, because we felt that it would be less of a burden on us and future developers trying to stub index requests in the tests.

def stub_get_resources(resource_type, options = {})
  stub_filter_resource(resource_type, options)
  path = resource_index_path(resource_type, options.slice(:params))

  response_body = options[:response_body] || []

  WebMock.stub_request(:get, path).to_return do
    {
      status: options.fetch(:response_status, 200),
      body: build_response_body(response_body, options).to_json,
      headers: { "Content-Type" => "application/vnd.api+json" }
    }
  end
end

For each resource in our API_RESOURCES constant, we then defined a helper method that would delegate to our new stubs.

API_RESOURCES.each do |resource_type|
  define_method "stub_get_#{ resource_type.to_s.pluralize }" do |options = {}|
    stub_get_resources(resource_type, options)
  end
end

Now we could use our stub in the tests!

foo = FactoryBot.create(:foo)

# Basic stub, specifying a response body
stub_get_foos(response_body: [ foo ])

# Stub including pagination and specifying a response body
stub_get_foos(response_body: [ foo ], include_pagination_links: true)

# Stub with filters, fields, pagination, and sorting
#  Ex: Foo.includes(:fizz).where(with_bars: true).select(:id, :title).order(:title).page(1).per(1)
stub_get_foos(
  params: {
    filter: { with_bars: true },
    fields: { foos: "id,title" },
    sort: :title,
    page: { number: 1, size: 1 },
    include: :fizz
  },
  response_body: [ foo ]
)

A param parser would probably be a useful addition to our stub helper, particularly for stubbing select calls, because the fields setup is confusing and unclear when writing the stub. We could probably refactor to allow users to pass { select: “id, title” } and then parse it to send the proper syntax.

Creating the stub for fetching a single resource was pretty straightforward, especially after dealing with the index. We needed to be able to support manually passing an ID, response body, and response status, but we also needed to be able to handle the stub if nothing was passed. Our default behavior if nothing was passed was to return a newly built factory object, setting the ID on the object based on the URL that was requested. That behavior could then be changed if a response body was passed.

def stub_get_resource(resource_type, options = {})
  resource_path = resource_show_path(resource_type, options.slice(:id, :params))

  WebMock.stub_request(:get, resource_path).to_return do |request|
    # Set response body to provided response body if it's sent
    unless (response_body = options[:response_body])
      # If no response body is provided, use the request URI to get the ID and build a new object
      id = request.uri.to_s.match(resource_path).captures[0]
      response_body = FactoryBot.build(resource_type, id: id)
    end

    {
      status: options.fetch(:response_status, 200),
      body: { data: response_body }.to_json,
      headers: { "Content-Type" => "application/vnd.api+json" }
    }
  end
end

# ...

# Add the resource-level helper method for calling the stub
API_RESOURCES.each do |resource_type|
  # ...

  define_method "stub_get_#{ resource_type }" do |options = {}|
    stub_get_resource(resource_type, options)
  end
end

We could then use the stub in our tests whenever we were fetching a specific resource.

foo = FactoryBot.create(:foo)

# Basic stub, specifying request ID
stub_get_foo(id: foo.id)

# Specify ID and response body
stub_get_foo(id: foo.id, response_body: foo)

To stub create, we knew we wanted to be able to support successes and failures, and we wanted to be able to specify the failures that occurred. We also needed to know what the API was expecting from the form data. This meant we had four potential options to deal with, request body, and response body, errors, and status.

def stub_create_resource(resource_type, options = {})
  # Get the form data from the options request body
  resource_data = options.fetch(:request_body, {})

  response = if options[:response_errors]
    # Set errors if response errors were given
    { errors: options[:response_errors] }
  elsif options[:response_body]
    # Set response if response body was given
    { data: options[:response_body] }
  else
    # Create a new resource from form data if no errors or response body were given
    { data: FactoryBot.build(resource_type, resource_data) }
  end

  WebMock.stub_request(:post, resource_base_path(resource_type)).
    with(body: { data: { type: resource_type.to_s.pluralize, attributes: resource_data } }.to_json).
    to_return(
      status: options.fetch(:response_status, 201),
      body: response.to_json,
      headers: { "Content-Type" => "application/vnd.api+json" }
    )
end

# ...

# Add the resource-level helper method for calling the stub
API_RESOURCES.each do |resource_type|
  # ...

  define_method "stub_create_#{ resource_type }" do |options = {}|
    stub_create_resource(resource_type, options)
  end
end

Our biggest fight on stubbing the create method for our API resources was that the form data had to be in the exact right order when generating the mock. This meant that when we were writing our tests, we would allow the tests to fail due to missing stubs first, and then use the suggested stub from WebMock to write out the attributes in our stubs.

# Stub successful create
stub_create_foo(request_body: { title: "New Foo" })

# Stub unsuccessful create
stub_create_foo(
  request_body: { fizz: "buzz" },
  response_status: 422,
  response_errors: [
    {
      title: "can't be blank",
      source: { pointer: "/data/attributes/foo_title" }
    }
  ]
)

The response error format is based off of JSON spec. This is another case where we probably could have made some sort of parser to make it easier on the end users, so they didn’t have to remember the weird format.

Setting up the update stub was pretty much exactly the same as the create stub, just swapping out the HTTP request method, and adding specific ID support. Because of the form data requirement on update, we chose to force the ID to be present, throwing an error if the key was missing from the options.

def stub_update_resource(resource_type, options = {})
  # Get the form data from the options request body
  resource_data = options.fetch(:request_body, {})

  # Fetch the ID from options, throwing an error if it is missing
  id = options.fetch(:id)

  response = if options[:response_errors]
    # Set errors if response errors were given
    { errors: options[:response_errors] }
  elsif options[:response_body]
    # Set response if response body was given
    { data: options[:response_body] }
  else
    # Create a new resource from form data and ID if no errors or response body were given
    { data: FactoryBot.build(resource_type, resource_data.merge(id: id)) }
  end

  WebMock.stub_request(:patch, resource_show_path(resource_type, id: id)).
    with(body: { data: { id: id, type: resource_type.to_s.pluralize, attributes: resource_data } }.to_json).
    to_return(
      status: options.fetch(:response_status, 200),
      body: response.to_json,
      headers: { "Content-Type" => "application/vnd.api+json" }
    )
end

# ...

# Add the resource-level helper method for calling the stub
API_RESOURCES.each do |resource_type|
  # ...

  define_method "stub_update_#{ resource_type }" do |options = {}|
    stub_update_resource(resource_type, options)
  end
end

The stub could then be used exactly the same way as the create stub, just passing in the resource ID.

foo = FactoryBot.create(:foo)

# Stub successful update
stub_update_foo(
  id: foo.id,
  request_body: { title: "New foo title" }
)

# Stub unsuccessful update
stub_update_foo(
  id: foo.id,
  request_body: { title: nil },
  response_status: 422,
  response_errors: [
    {
      title: "can't be blank",
      source: { pointer: "/data/attributes/title" }
    }
  ]
)

Just like with the create method, the form data had to be in the exact right order when generating the update mock, meaning we continued our pattern of letting the tests fail due to missing stubs, and using the WebMock suggested stub to determine our data ordering.

Finally, we got to the last of our CRUD methods to stub out, delete. We set up our delete stub to allow an ID to be passed, as well as a response status.

def stub_delete_resource(resource_type, options = {})
  resource_path = resource_show_path(resource_type, options.slice(:id))

  WebMock.stub_request(:delete, resource_path).
    to_return(
      status: options.fetch(:response_status, 204)
    )
end

# ...

# Add the resource-level helper method for calling the stub
API_RESOURCES.each do |resource_type|
  # ...

  define_method "stub_delete_#{ resource_type }" do |options = {}|
    stub_delete_resource(resource_type, options)
  end
end

It could then be used in our tests wherever we needed it.

foo = FactoryBot.create(:foo)
stub_delete_foo(id: foo.id)

Writing the stub helpers for the tests was a bit time consuming, but once they were done, they were a huge boon to us as we updated tests. #worthit

Check out the rest of this series!

• Part 1: Our Approach
• Part 2: Solving Pagination
• Part 3: Handling Filtering
• Part 4: Custom Sorting
• Part 5: Polymorphism
• Part 6: Mimicking ActiveRecord