by Jake Worth on January 5, 2017

Let's write an integration test for Phoenix using Wallaby.

Integration tests are used for behavior description and feature delivery. From the test-writer's perspective, they are often seen as a capstone, or, to the outside-in school, an initial 10,000-foot description of the landscape. At Hashrocket we've been perusing the integration testing landscape for Elixir, and recently wrote a suite using Wallaby.

In this post, I'll walk through a test we wrote for a recent Phoenix project.

Two emerging tools in this space are Hound and Wallaby; they differ in many ways, enumerated in this Elixir Chat thread.

The Wallaby team describes the project as follows:

Wallaby helps you test your web applications by simulating user interactions. By default it runs each TestCase concurrently and manages browsers for you.

We chose Wallaby for our project– an ongoing Rails-to-Phoenix port of Today I Learned (available here)– because we liked the API. It's similar to Ruby's Capybara.

Here are the basic steps we took to create our first Wallaby test.

Wallaby concurrently powers multiple PhantomJS headless browsers. To leverage that feature we'll need PhantomJS:

$ npm install -g phantomjs

Next, add Wallaby to your Phoenix dependencies:

# mix.exs

def deps do
  [{:wallaby, "~> 0.14.0"}]
end

As always, install the dependencies with mix deps.get.

Ensure that Wallaby is properly started, using pattern matching, in your test_helper.exs:

# test/test_helper.exs

{:ok, _} = Application.ensure_all_started(:wallaby)

If you're using Ecto, enable concurrent testing by adding the Phoenix.Ecto.SQL.Sandbox plug to your endpoint (this requires Ecto v2.0.0-rc.0 or newer). Put this is at the top of endpoint.ex, before any other plugs.

# lib/tilex/endpoint.ex

if Application.get_env(:your_app, :sql_sandbox) do
  plug Phoenix.Ecto.SQL.Sandbox
end

# config/test.exs

# Make sure Phoenix is setup to serve endpoints
config :tilex, Tilex.Endpoint,
  server: true

config :tilex, :sql_sandbox, true

Use test_helper.exs for any further configuration, like so:

# test/test_helper.exs

Application.put_env(:wallaby, :base_url, "http://localhost:4001")

We also enabled a feature which saves a screenshot on every failure. This is crucial when testing with a headless browser:

# config/test.exs

config :wallaby, screenshot_on_failure: true

That's the basic setup. If you get stuck, here's the pull request where we made all these changes.

Anytime we're writing a test, we want to extract shared logic into a central place. Phoenix accomplishes this with the IntegrationCase concept. This module does a lot of things, including importing other modules, defining aliases, and assigning variables.

I don't want to dig too deeply into this file, but will include it here in its entirety so all our setup is clear. A few noteworthy points are import Tilex.TestHelpers, which will let us build custom helper functions, and the setup tags block, which I copied directly from the Wallaby docs.

# test/support/integration_case.ex

defmodule Tilex.IntegrationCase do

  use ExUnit.CaseTemplate

  using do
    quote do
      use Wallaby.DSL

      alias Tilex.Repo
      import Ecto
      import Ecto.Changeset
      import Ecto.Query

      import Tilex.Router.Helpers
      import Tilex.TestHelpers
    end
  end

  setup tags do
    :ok = Ecto.Adapters.SQL.Sandbox.checkout(Tilex.Repo)

    unless tags[:async] do
      Ecto.Adapters.SQL.Sandbox.mode(Tilex.Repo, {:shared, self()})
    end

    metadata = Phoenix.Ecto.SQL.Sandbox.metadata_for(Tilex.Repo, self())
    {:ok, session} = Wallaby.start_session(metadata: metadata)
    {:ok, session: session}
  end
end

Let's test!

We're going to assert about a user's (or a developer's, in the language of this app) ability to create a new post through a form. Forms are fantastic subjects for integration tests because there are many edge cases.

First, create a test file. Here's the structure, which defines our test module and includes our IntegrationCase module. We use async: true to configure this test case to run in parallel with other test cases.

# test/features/developer_create_post_test.exs

defmodule DeveloperCreatesPostTest do
  use Tilex.IntegrationCase, async: true
end

I start by creating an empty test to confirm my setup.

# test/features/developer_create_post_test.exs

defmodule DeveloperCreatesPostTest do
  use Tilex.IntegrationCase, async: true

  test "fills out form and submits", %{session: session} do
  end
end

Run it with mix test:

$ mix test

warning: variable session is unused
  test/features/developer_creates_post_test.exs:6

.

Finished in 1.3 seconds
1 test, 0 failures

Okay, our test runs. Also, notice that Elixir compile-time warning, variable session is unused? That's the kind of perk I love about writing in this language. We'll keep the variable, because it will prove useful.

What now? Let's fill out the form, which looks like this:

<% # web/templates/post/form.html.eex %>

<%= form_for @changeset, post_path(@conn, :create), fn f -> %>
  <dl>
    <dt>
      <%= label f, :title %>
    </dt>
    <dd>
      <%= text_input f, :title %>
    </dd>
  </dl>
  <dl>
    <dt>
      <%= label f, :body %>
    </dt>
    <dd>
      <%= textarea f, :body %>
    </dd>
    <dt>
      <%= label f, :channel_id, "Channel" %>
    </dt>
    <dd>
      <%= select f, :channel_id, @channels, prompt: "" %>
    </dd>
  </dl>
  <%= submit "Submit" %>
<% end %>

Our test will use Ecto Factory to generate a channel struct, which we'll need to select the channel on the form. All the test code that follows is implied to be inside our test block.

EctoFactory.insert(:channel, name: "phoenix")

visit(session, "/posts/new")
h1_heading = get_text(session, "main header h1")
assert h1_heading == "Create Post"

So we create a channel, use the visit/2 function to visit our new post path, and then get the text from the header and make an assertion about it.

get_text/2 is a custom helper function, available in a test_helpers.exs file that we already imported via our IntegrationCase module. It extracts a common task: getting the inner text from an HTML selector. Here's the definition:

# test/support/test_helpers.ex

defmodule Tilex.TestHelpers do

  use Wallaby.DSL

  def get_text(session, selector) do
    session |> find(selector) |> text
  end
end

Extract as many helpers as your tolerance for abstraction allows.

Okay, time to fill in the form. Here, our session variable and Elixir's pipe operator (|>) shine.

session
|> fill_in("Title", with: "Example Title")
|> fill_in("Body", with: "Example Body")
|> Actions.select("Channel", option: "phoenix")
|> click_on('Submit')

We'll alias Wallaby.DSL.Actions to Actions for convenience. Why be so verbose at all? Because select/3 is ambiguous with Ecto.Query.

Let's assert about our submission. If it worked, we should be on the post index page, which looks like this:

<% # web/templates/post/index.html.eex %>

<section id="home">
  <%= for post <- @posts do %>
    <%= render Tilex.SharedView, "post.html", conn: @conn, post: post %>
  <% end %>
</section>

And here's the post partial it references, which we'll assert about:

<% # web/templates/shared/post.html.eex %>

<article class="post">
  <section>
    <div class="post__content copy">
      <h1>
        <%= link(@post.title, to: post_path(@conn, :show, @post)) %>
      </h1>
      <%= raw Tilex.Markdown.to_html(@post.body) %>
      <footer>
        <p>
          <br/>
          <%= link(display_date(@post), to: post_path(@conn, :show, @post), class: "post__permalink") %>
        </p>
      </footer>
    </div>
    <aside>
      <ul>
        <li>
          <%= link("##{@post.channel.name}", to: channel_path(@conn, :show, @post.channel.name), class: "post__tag-link") %>
        </li>
      </ul>
    </aside>
  </section>
</article>

Okay, on to the assertions:

index_h1_heading = get_text(session, "header.site_head div h1")
post_title       = get_text(session, ".post h1")
post_body        = get_text(session, ".post .copy")
post_footer      = get_text(session, ".post aside")

assert index_h1_heading =~ ~r/Today I Learned/i
assert post_title       =~ ~r/Example Title/
assert post_body        =~ ~r/Example Body/
assert post_footer      =~ ~r/#phoenix/i

Once again we use our helper function get_text/2 to capture the text on the page. Then, we use ExUnit to assert about the copy we found.

Here it is all together:

# test/features/developer_create_post_test.exs

defmodule DeveloperCreatesPostTest do
  use Tilex.IntegrationCase, async: true

  alias Wallaby.DSL.Actions

  test "fills out form and submits", %{session: session} do

    EctoFactory.insert(:channel, name: "phoenix")

    visit(session, "/posts/new")
    h1_heading = get_text(session, "main header h1")
    assert h1_heading == "Create Post"

    session
    |> fill_in("Title", with: "Example Title")
    |> fill_in("Body", with: "Example Body")
    |> Actions.select("Channel", option: "phoenix")
    |> click_on('Submit')

    index_h1_heading = get_text(session, "header.site_head div h1")
    post_title       = get_text(session, ".post h1")
    post_body        = get_text(session, ".post .copy")
    post_footer      = get_text(session, ".post aside")

    assert index_h1_heading =~ ~r/Today I Learned/i
    assert post_title       =~ ~r/Example Title/
    assert post_body        =~ ~r/Example Body/
    assert post_footer      =~ ~r/#phoenix/i
  end
end

And the final test run:

$ mix test

.

Finished in 4.5 seconds
1 test, 0 failures

Randomized with seed 433617

Don't be alarmed by the 4.5 second run time. At the time of this post's publication, Tilex has twenty-two unit and integration tests, and the entire suite often runs about that fast on a normal laptop. Setup and teardown is the big cost.

If you haven't read Plataformatec's post about integration testing with Hound, check it out. Both Hound and Wallaby seem great.

Integration testing will be a big part of developing in Phoenix, as developers build more and more complex applications. I hope this post gave you a new tool for developing your Phoenix apps from the supportive harness of a test suite. Please let me know if you've used Wallaby, and how it worked for you.

^{Photo Credit: NASA/Landsat8, Acadia National Park, flickr.com, https://www.flickr.com/photos/gsfc/29143744581. Accessed 2 January 2017.}