How to keep piping in Elixir?

Don’t break the pipeline, even if you need an intermediate result.

In this article, I’d like to share my recent discovery about piping in Elixir. I’m surprised how simple it is and that I have never used this before. I guess most of you already know it but I want to share this approach with those who will find it useful like I did.

Meet the pipe

Elixir pipe operator |> is not a new thing in programming. If we take, for example, the old good Ocaml, we can meet there a similar approach:

let sum x y = x + y;;
sum 3 (sum 1 2);;
# - : int = 6
1
|> sum 2 
|> sum 3;;
# - : int = 6

But let’s focus on Elixir now. Imagine you want to process a string in a way to get a list of all uppercased words. Normally, you would do:

iex(1)> String.split(String.upcase("Ruby is dead"))
["RUBY", "IS", "DEAD"]

However, thanks to the pipe operator, we can do:

iex(2)> "Ruby is dead"
...(2)> |> String.upcase()
...(2)> |> String.split()
["RUBY", "IS", "DEAD"]

This gives the same result in a more elegant way because Elixir pipe operator passes the result of an expression as the first parameter of another expression.

Understand the pipe

There’s an interesting function in the Macro module I’d like to summon here. Macros are compile-time constructs that are invoked with Elixir’s AST as input and a superset of Elixir’s AST as output. We are interested in theunpipe/1 function, in particular, which breaks a pipeline expression into a list.

Have a look at our previous example. Firstly, we need to get an AST representation of our pipeline:

iex(3)> ast = quote do
...(3)> "Ruby is dead"
...(3)> |> String.upcase()
...(3)> |> String.split()
...(3)> end
{:|>, [context: Elixir, import: Kernel],
 [
   {:|>, [context: Elixir, import: Kernel],
    [
      "Ruby is dead",
      {{:., [], [{:__aliases__, [alias: false], [:String]}, :upcase]}, [], []}
    ]},
   {{:., [], [{:__aliases__, [alias: false], [:String]}, :split]}, [], []}
 ]}

As you can see, ”Ruby is dead” string is the first argument passed to theString.upcase/2 and the result becomes an input to the String.split/1 function.

Now, let’s unpipe the AST:

iex(4)> Macro.unpipe(ast)
[
  {"Ruby is dead", 0},
  {{{:., [], [{:__aliases__, [alias: false], [:String]}, :upcase]}, [], []}, 0},
  {{{:., [], [{:__aliases__, [alias: false], [:String]}, :split]}, [], []}, 0}
]

We get a list that follows the pipeline directly:

1. The AST of ”Ruby is dead” string.
2. String.upcase/2 function
3. String.split/1 function

0 means that each element (or the result of execution starting from the second one) will be passed as the first attribute to the consecutive functions.

Use the pipe

Have a look now at a more complex pipeline. Our use case is as follows:

1. We have structs that we want to serialize.
2. The struct contains one association.
3. The given association should be serialized as well if it exists.
4. All the rest irrelevant fields should be dropped, only the meaningful ones need to be dumped.
5. A pure map has to represent the final structure to be stored.

Here comes the implementation:

Almost everything is achieved with the implementation above, but we are not encoding the associated structure. Exactly for that, we will inject into a pipeline:

Take a look at the case clause within the pipeline and pattern matching there. We modified the intermediate result along the way and passed it to the rest of the pipeline.

Alright, I know this is pretty unrealistic:

• Normally, you would extend Video serialization with encoding its associations.
• Alternatively, you could pipe in a function that would pattern match on the argument (function as a named case expression) and do the same.

However, for the sake of this article, I wanted to find a ridiculously simple way to show you the simple trick you may have not known about.

To pipe or not to pipe?

Now, you may think exactly like:

Because why not? But please stop for a moment and consider pros and cons.

When you take a look at Credo’s style guide, you will see this:

It is preferred to start pipe chains with a “pure” value rather than a function call.

# preferred way - very readable due to the clear flow of data
username
|> String.strip()
|> String.downcase()
# also okay - but often slightly less readable
String.strip(username)
|> String.downcase()

And I strongly agree with that — either use pipes in the entire expression or nothing — don’t mix the syntaxes!

Secondly, in my opinion, it’s an overdesign to use the pipe operator only once per expression. I don’t see a reason to use username |> String.strip() instead of just String.strip(username).

Vertical or Horizontal pipes?

Recently, I found an interesting discussion on Elixir forum regarding code formatting in terms of pipes. The author was wondering whether to use horizontal or vertical pipes:

"Ruby is dead"
|> String.upcase()
|> String.split()

vs.

"Ruby is dead" |> String.upcase() |> String.split()

In the Elixir codebase, it’s more common to see the first approach, nevertheless, the second one can be found as well.

When a line is short (usually less than 80 characters), it’s a matter of personal preference or company/project guidelines. Of course, when it comes to long lines, the formatter will align them vertically.

What is your preferred style BTW? I, personally, like to have all expressions line by line (considering that I never use only one pipe).

Bonus: Fault-tolerant pipe

With these who have already known all the mentioned things, I’d like to share yet another one, to leave you with at least some value.

Have you thought so far how it would be to redefine or alias the pipe operator? I’m sure you haven’t since you are sane, but I took the opportunity.

A function (or actually a macro) will be extremely simple:

defmacro left ~> right do
  quote do
    unquote(left) |> unquote(right)
  end
end

and you can use it in the same way:

iex(1)> 1..10 ~> Enum.random()
6

However, it’s not very useful now to have yet another alias for the same functionality. Let’s extend it a little:

defmacro left ~> right do
  quote do
    case unquote(left) do
      result ->
        result |> unquote(right)
    end
  end
end

I haven’t changed it that much but I introduced the possibility of pattern matching the result from the left side before passing it to the function on the right. How can we leverage that? For example, to validate the previous result we can do:

defmacro left ~> right do
  quote do
    case unquote(left) do
      {:error, error} ->
        {:error, error}
result ->
        result |> unquote(right)
    end
  end
end

In the case above we are checking whether the previous operation produced {:error, error} tuple, and we don’t process the result in such a case.

Finally, I can present a practical example:

Notice that parse/1 function (and any subsequent one) doesn’t have to handle {:error, error} tuple! It’s matched inside our Pipe macro and we can focus only on the meaningful logic without unnecessary error handling.

Summary

I hope you find this article useful and you will get a real value which you can add to your own project. Pipes are really helpful constructs that can make your code more readable if used wisely — keep that in mind when designing flows in your applications.

Leave your e-mail to get the latest content immediately:
https://tinyletter.com/KamilLelonek

And if you have ever considered proposing any change to the pipe operator, just think twice whether it will really simplify the things:

Anyhow, happy piping!

See? Plumbing is an easy thing!