For loops and comprehensions in Elixir - transforming imperative code

In this article, we’ll cover some common uses of for loops and comprehensions in Python, how to analyze an existing loop, and how to transform them into their equivalent expressions in Elixir, using the functions in the Enum module and comprehensions.

We’ll focus on:

• transforming a collection of data through a function (map)

• filtering values into or out of a collection (filter)

• producing a single aggregate value or structure, such as an average (reduce or fold)

We’ll finish off with a basic example that combines all three!

Python

For loops

In Python, for loops typically feature interleaved processing - the steps are combined together into the same clause or body. Here’s an example that squares the first two even numbers:

1result = 0
2
3for num in [1, 2, 3, 4, 5]:
4    if num % 2 == 0:
5        result += num ** 2
6
7print(result)  # Output: 20

One challenge of this interleaved body is to:

1. identify each step, and…

2. work out what type of step it is.

Breaking apart each step allows you to understand the transformations taking place, eliminate any unnecessary ones, and rewrite those steps into another language construct or higher-level function.

Annotating the function above results in:

1result = 0
2
3for num in [1, 2, 3, 4, 5]:
4    ## Filter
5    if num % 2 == 0:
6	    ## Reduce (result += ) and Map (num ** 2)
7        result += num ** 2
8
9print(result)  # Output: 20

The steps

As a result - the order of steps are:

1. Filter “out” odd numbers/”in” even numbers

2. Map numbers (e.g. 2) to their corresponding square number (e.g. 4)

3. Reduce to a sum of the squared even numbers

Comprehensions

Comprehensions in Python are simple ways to map and filter collections like lists and dictionaries. They don’t offer a way to reduce the result, but we can use built-in functions like sum to transform the above to process the result of the comprehension:

1result = sum(num ** 2 for num in [1, 2, 3, 4, 5] if num % 2 == 0)
2print(result)  # Output: 20

With comprehensions, the expression divides the map step (num ** 2) and filter step (if num % 2 == 0) clearly. sum is the reduce step here.

It’s easy to skim through these comprehension expressions in Python, and it places a useful upper limit on the complexity of a comprehension.

With this background, and a better understanding of the structure and limitations of Python’s processing constructs, let’s proceed to rewriting the above Python code using Elixir’s comprehensions and Enum pipelines!

Mapping: Enum.map and generators

How can we write the step to square numbers. In Elixir, it’s simple!

Using Enum.map:

1Enum.map([1, 2, 3, 4, 5], & &1 ** 2)

and using comprehensions (for):

1for n <- [1, 2, 3, 4, 5], do: n ** 2

The <- represents a generator expression, generating values to be used in the body of the for expression, after do:

Filtering: Enum.filter and filters

Easy to do with Enum.filter (or Enum.reject):

1[1, 2, 3, 4, 5]
2|> Enum.filter(& rem(&1, 2) == 0)
3|> Enum.map(& &1 ** 2)

We’ll want to filter out odd numbers before they are squared, so we place it in the right place in the pipeline - before Enum.map.

Using comprehensions, we can add a second expression to the head of the comprehension, a filter, which is a boolean test:

1for n <- [1, 2, 3, 4, 5], rem(n, 2) == 0, do: n ** 2

The rem(n, 2) == 0 expression then discards any elements that return false (or nil), leaving [2, 4] as the numbers that are actually passed to the body (do: n ** 2) of the comprehension.

Reduce -> Enum.reduce and reduce:

Using Enum.reduce/2, we can convert a list of squared numbers into their sum by adding to an accumulator. The first element is used as the initial value of the accumulator if we don’t specify an initial value for the accumulator (Enum.reduce/3), and that’s handy here:

1[1, 2, 3, 4, 5]
2|> Enum.filter(& rem(&1, 2) == 0)
3|> Enum.map(& &1 ** 2)
4|> Enum.reduce(& &1 + &2)

With comprehensions, we have even more power than the Python equivalent. We can add a reduce step by adding another clause to the head:

1for n <- [1, 2, 3, 4, 5], rem(n, 2) == 0, reduce: 0, do: acc -> acc + n ** 2

making two changes here:

1. adding a reduce: 0 clause to the head, to specify that we will accumulate a value whose initial value is 0

2. changing the for body to capture an acc value (the accumulator) that we can add the current squared value to.

Built-in functions: Enum.sum

As a general rule, we should express the data we want to transform in the highest-level way possible. It’s useful to think of Enum.reduce as the lowest level functional transformation, since all other data processing can be rewritten in terms of it.

The Enum module contains plenty of higher-level functions, typically involving reducing a list of values to a single aggregate value, like a sum, maximum or minimum. In this case, we’d like the sum of the elements.

For Enum pipelines, this is straightforward:

1[1, 2, 3, 4, 5]
2|> Enum.filter(& rem(&1, 2) == 0)
3|> Enum.map(& &1 ** 2)
4|> Enum.sum()

There is not a way to represent these high-level aggregate functions in comprehensions, so we can pipe the output of the comprehension into a Enum.sum call like so, similar to how we did in Python:

1(for n <- [1, 2, 3, 4, 5], rem(n, 2) == 0, do: n ** 2) |> Enum.sum()

Mixing different forms should generally be avoided, especially if the transformation is a simple one, as it results in less mental load for the reader - the reduce: form above is actually clearer to read despite being lower-level.

Which Elixir expression is better?

To summarise, we’ve ended up with two forms which could be considered idiomatic. For Enum pipelines:

1[1, 2, 3, 4, 5]
2|> Enum.filter(& rem(&1, 2) == 0)
3|> Enum.map(& &1 ** 2)
4|> Enum.sum()

and comprehensions:

1for n <- [1, 2, 3, 4, 5], rem(n, 2) == 0, reduce: 0, do: acc -> acc + n ** 2

Easy to read code should be straightforward to scan through, without ambiguity or stumbling over expressions. I think both forms fill that criteria, as:

1. they follow a single consistent form - either Enum pipelines or comprehensions

2. each expression corresponds to a single processing step

3. it can be read top-to-bottom or left-to-right without interruption

Conclusion

Writing these transformations can be done in several different ways in Elixir, and it is easy for a codebase to vary styles, especially as code is changed and processing becomes more complicated over time.

PureType can break down and analyze Enum pipelines and comprehensions to represent them in their clearest and most idiomatic form, learning your preferences and increasing your code’s readability and clarity for others on the team. Try it out today!