Hashrocket.com / blog

Large elixir with love

Elixir With Love

posted on and written by Micah Woods in

Image 100x100 micah woods

Elixir Alchemists love the pipe (|>) operator, and with good reason: it enables transformation of data in ways that are very expressive.

These expressive declarations of code – or "pipelines" – can seem like a magic bullet, but let’s look at an example where the pipeline becomes cumbersome and unwieldy. Then we’ll refactor to use the new with macro introduced in Elixir 1.2, and fall in love with pipelines again.

Take a look at the following code. If you've been using Elixir, you've written code like this (or been tempted to).

defmodule Example.Pipeline do

  def transform_data do
    get_data_from_internet
    |> put_data_in_storage
    |> make_second_api_request
    |> put_data_in_storage
    |> transform_result
  end

  def get_data_from_internet do
    {:ok, :got_it}
  end

  def put_data_in_storage({:ok, thing_to_store})do
    {:ok, thing_to_store}
  end

  def make_second_api_request({:ok, _value}) do
    {:ok, :second_request}
  end

  def transform_result({:ok, value}) do
    value
  end
end

Start a mix REPL (iex -S mix) and try it out. This code is great: the transform_data/0 function expresses exactly what the intention of the code is. But this pattern isn't without problems: imagine if any or all of the functions in the pipeline were to produce an error. Let’s refactor the code to produce an error in any of the functions twenty percent of the time:

defmodule Example.Pipeline do

  def transform_data do
    get_data_from_internet
    |> put_data_in_storage
    |> make_second_api_request
    |> put_data_in_storage
    |> transform_result
  end

  def get_data_from_internet do
    random_failure({:ok, :got_it}, {:error, :getting_data})
  end

  def put_data_in_storage({:ok, thing_to_store})do
    random_failure({:ok, thing_to_store}, {:error, :data})
  end

  def make_second_api_request({:ok, _value}) do
    random_failure({:ok, :second_request}, {:error, :second_data})
  end

  def transform_result({:ok, value}) do
    value
  end

  defp random_failure(pass, fail) do
    if Enum.random(1..10) > 2 do
      pass
    else
      fail
    end
  end
end

If you try to run this code in the REPL, you will quickly get a function clause error, explaining that no functions match the error states. In order to fix this we need to add a function head for every error state. For this simple example, this isn’t so hard, but it can become very complex if you have multiple types of errors that can occur. Here are the function heads for this simple example:

defmodule Example.Pipeline do

  def transform_data do
    get_data_from_internet
    |> put_data_in_storage
    |> make_second_api_request
    |> put_data_in_storage
    |> transform_result
  end

  def get_data_from_internet do
    random_failure({:ok, :got_it}, {:error, :getting_data})
  end

  def put_data_in_storage({:ok, thing_to_store})do
    random_failure({:ok, thing_to_store}, {:error, :data})
  end
  def put_data_in_storage({:error, _} = error) do
    error
  end

  def make_second_api_request({:ok, _value}) do
    random_failure({:ok, :second_request}, {:error, :second_data})
  end
  def make_second_api_request({:error, _} = error) do
    error
  end

  def transform_result({:ok, value}) do
    value
  end
  def transform_result({:error, _} = error) do
    error
  end

  defp random_failure(pass, fail) do
    if Enum.random(1..10) > 2 do
      pass
    else
      fail
    end
  end
end

Now if you run the code again, you get the result or the error. This code works, but once again, what if we add another type of error that can occur? Now we have to write another function head all the way down the pipeline. Thankfully Elixir has given us the with macro. Here is our refactored code using with.

defmodule Example.Pipeline do

  def transform_data do
    with {:ok, value} <- get_data_from_internet,
         {:ok, value} <- put_data_in_storage(value),
         {:ok, value} <- make_second_api_request(value),
         {:ok, value} <- put_data_in_storage(value),
    do: transform_result(value)
  end

  def get_data_from_internet do
    random_failure({:ok, :got_it}, {:error, :getting_data})
  end

  def put_data_in_storage(thing_to_store)do
    random_failure({:ok, thing_to_store}, {:error, :data})
  end

  def make_second_api_request(_value) do
    random_failure({:ok, :second_request}, {:error, :second_data})
  end

  def transform_result(value) do
    value
  end

  defp random_failure(pass, fail) do
    if Enum.random(1..10) > 2 do
      pass
    else
      fail
    end
  end
end

This works because with tries to match the thing on the left of the <- and then continues the pipeline. When it doesn’t match the left, it stops the pipeline and returns the thing on the right. For example, if get_data_from_internet/0 returned {:error, :dont_match}, it would not match {:ok, value}. In this case the pipeline would stop and {:error, :dont_match} would immediately return.

Woohoo, pipelines are back! Even if we had multiple types of errors, this code would just continue to work. Alchemists can once again write elegant, declarative code.

Posted in Elixir