by Mary Lee on February 28, 2019

Did you know that Puma has a 10,240 character query string limit? We do now.

In the process of building our internal JSON API, we found two issues with handling filtering:

1. Query string limitations
2. Querying for an array of IDs when some values were nil

We were building an internal JSON API using the following libraries:

• json_api_client for reading data from the API
• jsonapi-resources for creating the internal JSON API endpoints

For more information on our goals and set up for the internal API, see part 1 of this series.

Given that the JsonApiClient filtering methods use an HTTP GET with query string parameters, character limits quickly became a concern for us. How could we ensure that some of our lengthier queries weren’t exceeding the limit and failing?

The simplest solution to us was to make the filters a POST request, even if that was a divergence from JSON API specifications. Once we made the decision to transition to a POST request, we had to figure out how to override the filter requests in the JsonApiClient.

As we learned while fixing pagination, the JsonApiClient library uses class attributes for managing the objects used to perform different functions for the resources. This meant that we could override the default requestor class for the library, changing the standard get to make a post call if the filtering params are present.

class FilterOverrideRequestor < JsonApiClient::Query::Requestor
  def get(params = {})
    if params[:filter].present?
      path = "#{resource_path(params}/filter"
      request(:post, path, body: params)
    else
      super
    end
  end
end

With the new requestor class in place, we could update the class attribute on our base JsonApiClient::Resource class to use our new requestor, instead of the default one.

class BaseResource < JsonApiClient::Resource
  # …
  self.requestor_class = FilterOverrideRequestor
  # … 
end

The last step in getting our filtering working was updating the routes to allow the POST request.

namespace :local_api do
  jsonapi_resources :foo do
    post :filter, to: "foo#index", on: :collection
  end
end

With those changes, we no longer had to worry about ever exceeding query string limitations.

A weird behavior we noticed from the JSONAPI::Resources was that when we were querying for a resource by an array of IDs, if the array contained any nils, nothing was returned, even if there should have been matches for the other IDs in the array.

When we first noticed this, our reaction was to always call compact on the array of IDs we were sending to the API, but that wasn’t really great from a development perspective -- we knew we’d wind up forgetting to add the compact, leaving us wondering why our API responses were always empty.

Our solution was to make the API deal with it for us. We created a base JSONAPI::Resource class, and add our logic for filtering by ID, and for cleaning up the IDs, there.

module LocalAPI
  class BaseResource < JSONAPI::Resource
    filter :id, apply: lambda { |records, ids, _options|
      records.where(id: normalize_ids(ids))
    }

    def normalize_ids(ids)
      Array(ids).flatten.compact
    end
  end
end

By making the ID cleanup a method, we were able to use it in other places, such as when filtering a resource by an associated resource id.

# JSONAPI Resource
module LocalAPI
  class FooResource < BaseResource
    filter :bar_id, apply: lambda { |records, bar_ids, _options|
      records.where(bar_id: normalize_ids(bar_ids))
    }
  end
end

It continued to be a boon to us that the libraries we were using were very customizable, allowing us to override what we needed to in order to make our API work how we wanted it to.

Check out the rest of this series!

• Part 1: Our Approach
• Part 2: Solving Pagination
• Part 4: Custom Sorting
• Part 5: Polymorphism
• Part 6: Mimicking ActiveRecord
• Part 7: Testing