Serialization is the Secret

If a value mutates in the forest with no one to see it, does it really mutate?

One of the major elements that sets Elixir apart from most other programming languages is immutability. But what does that actually mean? What does it do for us?

The word immutable is defined by Merriam-Webster as “not capable of or susceptible to change”. If nothing ever observes a change, it may as well have not happened anyway, so we will be discussing mutation as two discrete concepts: mutation, and the observation of mutation.

Here is a simple example of a mutable program in Javascript.

Here is a simple example of an immutable program in Elixir.

They look very similar. And in fact, they both have consistent behavior if the next line of the program was to print out the value of counter. We are guaranteed in both cases to see the value 1 printed. It would appear that the observation of variables behaves the same.

Rebinding vs Mutation

Before I explain the benefits of immutability, it’s important to disambiguate something that folks often initially believe is mutation.

In the Javascript example, our reference to the counter is mutated. However, in the Elixir example, what we’ve done is rebound the variable. No pointers or values have changed. In Javascript, primitive values are immutable, but a variable’s reference is not. In later examples we’ll see how Javascript allows mutating other values i.e objects, called reference values.

The first is an actual mutation of memory. The second is an allocation of new memory. The old memory is unaffected. One could argue that this is a form of mutation. This mutation, however, is syntactic.

This is a convenience that the language provides. When you rebind a variable, it is equivalent to creating a new variable, and altering references to that variable in the same scope to use that new name instead. Let’s look at what this means in practice.

In Javascript, we mutate the actual underlying memory. What this looks like in practice:

Notice how the value for counter is now 1. In Elixir, however, this behaves differently.

How can this be? We’ve clearly reassigned counter. The reason for this is that rebinding is equivalent to creating a new variable. Just like in Javascript, a variable is not available outside of the scope it was defined in. The Elixir code is effectively rewritten to this:

In this way, no reference to a given variable can ever truly be mutated in Elixir. You only ever create new bindings with new values. There is just a language level construct that allows you to reuse the same variable name.

Cool, so why immutability?

Now that you understand that rebinding isn’t mutation, you can see the fact that the value of a given variable will never change. This lends itself to well to understandable code. For example, in Javascript, objects are mutable, and are passed to functions by reference. Which means that calling functions can do “surprising stuff”:

After executing this code, profile has mutated. Before calling newScore, profile.score was 3. Afterwards, it is 7. This would be a pretty beginner mistake to make, but it is still something that you have to contend with. You will always have to consider using things like Object.freeze or Object.seal, or just trusting that your other code makes no mutations to the parameters that you pass in.

In Elixir, it is just not possible for that to happen. If I call a function, it can do all kinds of side effects, but it will never be able to affect the values of the variables that are currently in scope for me.

This is where observation of mutation comes into play. In the Javascript example, mutation can be observed effectively at any time. If I started an async function, for example, and passed in a mutable variable, it could change at any time.

Time: the most frustrating part of programming

And I don’t mean dealing with time zones. Although maybe that actually is the most frustrating part of programming.

What I mean is handling the passage of time. Let’s go back to our Javascript example, and introduce some concurrency. When concurrency comes into play, we can no longer think in terms of a discrete, causal series of events. Instead, we have to think of “state over time”.

The value of profile.score at the end of this program is entirely dependent on how long it takes to doSomethingElse(). That means that sequential runs of this program can produce different results. There are absolutely steps to mitigate this. For instance, you can await newScore. You can make doSomethingElse() an async function, and await them both using Promise.all().

But imagine that you had passed `profile` into both of those now-async functions? And each one modified it at a different time? Handling race conditions like this can be very difficult, and oftentimes immutable data structures are the solution. You can use things like locks and mutexes, but that is just as easy to get wrong as any other thing.

So, to summarize, mutation can occur at any time and can also be observed at any time in Javascript.

Elixir is not, by all definitions, completely immutable

In Elixir, everything runs in a process. It is a garbage collected language. Each process has its own stack, and its own heap. This garbage collection means that the memory location of a variable can technically change at any time. So from a pedantic standpoint, you could have two variables whose memory location swaps at some point while running your program. This is transparent to the programmer (which is good).

With this in mind, an argument could be made that anything stored in a process’s heap is mutable state. For example, let’s take a simple, GenServer-backed counter in Elixir.

When I increment this counter, new memory is allocated for the new number to live in. When I access this counter, I’m reading from that memory location. At some point, a garbage collection comes through, cleaning up the references to the old variables, and potentially moving the location of the new variable in memory.

So this process, while indirect, actually has all of the hallmarks of mutation! The same variable changing its location in memory in a way that is completely out of the developer’s control. Other variables being removed from memory or moved around at any time. SpoOoOoky!

So Elixir is mutable then? By some definitions, sure. In Elixir, values are immutable, but ultimately there is state, somewhere in our system. And it even changes over time! So what gives?

There are two secret sauces in the Elixir restaurant. They are:

• Function calling as the only way to observe mutating state

• Serialized access to mutating state

Function calling as the only way to observe mutating state

Something we have available to us in Elixir is the “process dictionary”. By many definitions, the process dictionary is mutable state. I would argue that, from the perspective of our program, it is not more or less mutable than any other thing. The reason for this, is that in Elixir, all mutating state requires calling a function to observe it.

Want to know the current time? You must call DateTime.now(). You can change the process dictionary with Process.put. You can even call a function that makes this change. Lets look at an example based on the Javascript “mutating profile” example from before:

Here we are “mutating” the process dictionary state. It was one thing, and now it is something else. However, we cannot observe this mutation without calling Process.get. The profile variable will always, unambiguously, equal %{score: 3}. The only way that could change is by reassigning the variable, and calling a function. For example:

Our variable values are always safe in Elixir 🥳

Serialized access to mutating state

This is the most important part of the whole equation. We’ve shown that we can accomplish mutating state in Elixir, albeit in a different way than you might have seen elsewhere. But what about the race condition example from before? We can’t really write an equivalent example to that in Elixir. Our concurrency primitive is to start another process, which has its own isolated process dictionary. There is no way to modify another process’s state.

So we’ll base this on our counter example, which is how you’d actually do This Kind Of Thing™ in Elixir. Each process handles messages serially, one-at-a-time. This means that, from the perspective of the counter, there is no such thing as a race condition! And callers, in order to interact with the counter, must call functions to send it a message, or to observe its changing state. Nothing the counter does can modify the state of the callers.

By forcing the mutation of state to be serialized through a process’s mailbox, and limiting the observation of mutating state to calling functions, our programs are more understandable, and less surprising.

What’s next?

There is a lot more to say on this topic, and I fully intend to do so! Understandability and predictability are not the only benefits of the concepts described above. Many of the benefits only even come into the picture when it’s time to scale, or to model graceful failure of our application components, and its those properties that I’ll be writing about next.