Elixir Streams for Lazy Evaluation
Lazy evaluation is a great way to delay the execution of larger datasets. In a typical enumeration each item is evaluated one-by-one. This isn't a problem with smaller sets but as those sets get larger the amount time to process grows exponentially. Every function has to evaluate the entire set before the next function can execute. Elixir's
Stream module allows us to compose our enumerations prior to execution. Let's look at some examples.
First will take a look at using the
Enum module to filter a range of numbers and take from the list.
iex(1)> Enum.filter(1..10000000000000000, &(rem(&1, 3) == 0 || rem(&1, 5) == 0)) |> Enum.take 5
Wait for it... No really because it's going to take awhile. See, the
Enum.filter function has to complete its iteration over the entire set before it can pipe the filtered list to the
take function. However, if we use the
Stream module we can compose the computation prior to executing.
iex(1)> Stream.filter(1..10000000000000000, &(rem(&1, 3) == 0 || rem(&1, 5) == 0)) |> Enum.take 5 [3, 5, 6, 9, 10]
When you compose a chain of streams it's easy to see how the execution differs from the typical chain of
Enum functions. We can borrow the next example from the
Stream module docs.
First we look at an example with piping the
Enum functions together. You will note that each map is applied in order so first all the numbers are printed then they are doubled and printed.
1..3 |> Enum.map(&IO.inspect(&1)) |> Enum.map(&(&1 * 2)) |> Enum.map(&IO.inspect(&1)) 1 2 3 2 4 6 #=> [2,4,6]
Now we can look at the same example composed with Streams.
stream = 1..3 |> Stream.map(&IO.inspect(&1)) |> Stream.map(&(&1 * 2)) |> Stream.map(&IO.inspect(&1)) Enum.to_list(stream) 1 2 2 4 3 6 #=> [2,4,6]
Now we see that the each number is completely evaluated before moving to the next number in the enumeration.
That's just a quick look at the power of streams in Elixir for lazy evaluation.