Recently I’ve spent some time learning Elixir: My First Week With Elixir As A Rubyist, and I am really enjoying it. I want to share with you what I have learned about an Elixir concept called GenServer, and how I am approaching writing tests for it—with test-driven development.
GenServer contains code that abstracts away the boilerplate of process communication, state management, and message ordering. This enables us to quickly create processes that manage state in an Elixir program using a declarative style.
Why use a GenServer?
In Elixir everything is a process. Each Elixir process runs a function. Functions can do three things: recurse to emulate a loop, perform some calculation, or wait on messages—to arrive in their mailbox from other processes.
By adding a receive block to the function that a process is running, it can read the messages that match a predetermined pattern. Pattern matching is very important in Elixir.
The downside of messaging in Elixir is that both two-way communication between processes and maintaining message order can be tricky. As a result of this pain point, the Core Elixir team wrote this common functionality into a module called GenServer.
GenServers provide two types of message patterns to clients, synchronous and asynchronous. Synchronous can be used to perform an operation where the caller needs the result now. Asynchronous messages are great to kick off long-running work, or computations for which the client doesn’t need the result—at least not yet.
The synchronous interaction starts by invoking the function named GenServer.call/3 (pronounced “GenServer call three”). I like to think of the call function as a “phone call.” A sender picks up a phone, sends a message, and waits for a response.
Asynchronous messages are created by calling the GenServer.cast/2 function. “Cast” is like sending an email from a noreply address. Senders cannot tell when the receiver gets the message, and there is no response from the receiver to the sender.
Creating a GenServer with TDD
So now that we know about GenServer, let’s build one using a common technique in our industry, test-driven development (TDD).
When I say TDD, I am referring to a style of programming where we write failing tests before we write any production code. We will write just enough code to make the tests pass. Once the tests are green, we refactor—change the structure of the code, not the behavior.
A more exhaustive overview of TDD is available here: “The Cycles of TDD” by Robert C. Martin.
In this tutorial, we will make a GenServer that models a bank account. This will use both types of messages—calls and casts. There is one nice surprise, as well: we can hide the GenServer functions inside a module, allowing the interaction to look like an object in other programming languages.
Here are the features we will build together:
Backlog- Check that initial account balance is zero- Add money to an account- Remove money from an account
Setting up a project
With our backlog defined, we are ready to create our project and our first test. Create a new Elixir project using the following mix command (mix is like Rake for Elixir).
$ mix new bank
Once we have our new project, open it up in your editor of choice. Inside the test directory, remove the ‘bank_test.exs’ that has been generated, and create a new test called ‘account_test.exs’. I usually add a simple failing test to verify that I have configured the test correctly. Add the following code to our new test file:
># test/account_test.exsdefmodule AccountTest do use ExUnit.Case test "fails" do assert true == false endend
We should be able to see this test fail by running the tests. From a terminal in the bank directory, execute:
$ mix test
We should see the following failure:
1) test fails (AccountTest) test/account_test.exs:4 Assertion with == failed code: assert true == false left: true right: false stacktrace: test/account_test.exs:5: (test)Finished in 0.02 seconds1 test, 1 failure
Starting with a zero balance
Now that we are up and running, let’s add a real test. We want to check that the initial balance is zero. Modify the test in “account_test.exs” as follows:
defmodule AccountTest do use ExUnit.Case test "initial balance is zero" do {:ok, pid} = Account.start_link endend
Here we are following a common pattern for starting GenServers by invoking the Account.start_link/0 function. This behaves similarly to a constructor in other languages.
Running our tests, we see a failure due the Account module’s not yet being defined.
** (UndefinedFunctionError) function Account.start_link/0 is undefined (module Account is not available)
We can resolve this by creating an Account module. Create a file named ‘account.ex’ inside the ‘bank/lib’ directory.
# bank/lib/account.exdefmodule Accountend
Now running the tests will give an error about the start_link/0 function’s being undefined.
** (UndefinedFunctionError) function Account.start_link/0 is undefined or private
Let’s add the function to the module, with no implementation.
# lib\account.exdefmodule Account do def start_link do endend
The next error we see from the tests is that nil doesn’t match the pattern of Tuple :ok and a process id.
** (MatchError) no match of right hand side value: nil
Here we want something that returns a Tuple with :ok and the process ID. Update the implementation of your Account to start a GenServer. Add the following code to make the test pass:
defmodule Account do use GenServer def start_link do GenServer.start_link(__MODULE__, :ok) endend
Returning a Tuple with a status atom and some data is a very common pattern in Elixir.
You will notice that we did two things to the module. First, we added a statement use GenServer. This injects the GenServer code into our Account module.
The second change is the invocation of GenServer.start_link/3. This function takes three arguments: the name of the module, the argument to pass to the init callback—more on this in a second—and a keyword list of options, which defaults to empty list.
The __MODULE__ references the name of the current module. The second argument :ok is just an arbitrary value.
When we run the tests, they pass. However, we also get two warnings: one that the module doesn’t implement the init/1 callback, and another that says that our pid variable in the test is unused.
Callbacks are an aspect that take some getting used to. The GenServer code we invoked via the GenServer.start_link/3 expects functions to exist our Account module.
This is acting a lot like the template method pattern—a super class that calls methods on it’s subclass, in order to provide variations in an algorithm. So, we will have to add a few more callbacks during this tutorial.
Since we will use the pid in a moment, please ignore the warning about it. Let’s implement the init/1 function so that our other warning goes away.
defmodule Account do # ... def init(:ok) do endend
Our init/1 is pattern-matching the argument when its value is :ok. Arguments other than the matching pattern will case an error.
When we run the tests, we receive an error about the init/1 function returning ‘nil’.
** (EXIT from #PID<0.148.0>) bad return value: nil
This is because the GenServer implementation expects our init/1 to return a Tuple of :ok and the initial state of the process. Let’s set the initial state to an empty Map.
defmodule Account do # ... def init(:ok) do {:ok, %{}} endend
Now our test passes, and we can add an assertion that the balance is zero. Modify the test as follows:
# test\account_test.exsdefmodule AccountTest do use ExUnit.Case test "initial balance is zero" do {:ok, pid} = Account.start_link assert 0 == Account.get_balance(pid) endend
Our next error shows us that the get_balance/1 function is not defined on our module. Add the get_balance/1 function with an empty implementation.
defmodule Account do # ... def get_balance(pid) do endend
We now receive a failure that nil is not equal to 0.
1) test initial balance is zero (AccountTest) test/account_test.exs:4 Assertion with == failed code: assert 0 == Account.get_balance(pid) left: 0 right: nil stacktrace: test/account_test.exs:7: (test)
The implementation is not simple, so let’s return a constant value. Change the get_balance/1 function to return 0.
When practicing TDD, if I believe the implementation to get a test passing is trivial, I will make an attempt to write it. If I can’t visualize what the implementation is, or if it takes longer than 20 seconds to write, I will make it pass by hard-coding a value and then writing another test to force me to make that value dynamic.
defmodule Account do # ... def get_balance(pid) do 0 endend
Now that we are at green, let’s refactor to a better implementation. Change the Map inside the init/1 function to contain a key of ‘balance’ with a value of ‘0’
defmodule Account do # ... def init(:ok) do {:ok, %{balance: 0}} end # ...end
Next, modify the get_balance/1 function to invoke the GenServer.call/3 function. The first argument is the process ID, the second is the request that is passed to the handle_call/3 function, and the last is an optional timeout.
defmodule Account do # ... def get_balance(pid) do GenServer.call(pid, :get_balance) endend
Our tests now show an error about how the handle_call/3 function does not exist in our module.
20:34:55.424 [error] GenServer #PID<0.149.0> terminating** (RuntimeError) attempted to call GenServer #PID<0.149.0> but no handle_call/3 clause was provided (bank) lib/gen_server.ex:754: Account.handle_call/3 (stdlib) gen_server.erl:661: :gen_server.try_handle_call/4 (stdlib) gen_server.erl:690: :gen_server.handle_msg/6 (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3Last message (from #PID<0.148.0>): :get_balanceState: %{balance: 0}Client #PID<0.148.0> is alive
We can resolve this by adding the function with its implementation.
defmodule Account do # ... def handle_call(:get_balance, _from, state) do {:reply, Map.get(state, :balance), state} endend
The tests now pass, but there is a lot to discuss here. First handle_call/3 matches only requests with :get_balance as the first argument. The second argument begins with an underscore—signifying that it is not used. Lastly, the state of the process is passed to the function, so we can update it or look up values.
The return value is a Tuple—which we have seen before—but this one is different. Instead of :ok it begins with :reply. This is one of a set of values that GenServer’s implementation can react to—check the documentation for more info. The second value in the Tuple is that of the :balance key from the state Map. Finally the state is passed unchanged as the third element in the Tuple.
As we rerun our tests, we see that they still pass, telling us that our refactoring hasn’t changed the expected behavior.
Let’s mark off what we have done on our backlog.
Backlog-Check that initial account balance is zero- Add money to an account- Remove money from an account
Adding money to our account
With our users able to check their balances, let’s give them the ability to deposit money to their accounts. Add a new test that calls Account.deposit/2 with $10.
defmodule AccountTest do use ExUnit.Case # ... test 'depositing money changes the balance' do {:ok, pid} = Account.start_link Account.deposit(pid, 10.0) endend
Notice that the Account.deposit/2 function doesn’t return anything. This is an arbitrary design choice in this tutorial. In order to find the balance we must invoke get_balance/2.
When we run the tests, we get an error that Account.deposit/2 is undefined.
** (UndefinedFunctionError) function Account.deposit/2 is undefined or private
Add the deposit function to our Account module.
defmodule Account def deposit(pid, amount) do endend
The test passes, but we receive a warning about unused variables (pid and amount); we can ignore those, for the moment. Next we finish writing the test by asserting that the balance is now 10.0.
defmodule AccountTest do # ... test 'depositing money changes the balance' do {:ok, pid} = Account.start_link Account.deposit(pid, 10.0) assert 10.0 == Account.get_balance(pid) endend
Running the test reveals that our balance is still zero.
Assertion with == failed code: assert 10.0 == Account.get_balance(pid) left: 10.0 right: 0 stacktrace: test/account_test.exs:15: (test)
In order to change the balance, we need to invoke the GenServer.cast/2 function and implement the handle_cast/2 in our Account module.
First, invoking the GenServer.cast/2 function. We pass cast/2 a Tuple with the atom :deposit and the amount we want to deposit.
defmodule Account do # ... def deposit(pid, amount) do GenServer.cast(pid, {:deposit, amount}) endend
Our tests now tell us that we need to implement the handle_cast/2 callback.
07:29:43.927 [error] GenServer #PID<0.149.0> terminating** (RuntimeError) attempted to cast GenServer #PID<0.149.0> but no handle_cast/2 clause was provided (bank) lib/gen_server.ex:785: Account.handle_cast/2 (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4 (stdlib) gen_server.erl:711: :gen_server.handle_msg/6 (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3Last message: {:"$gen_cast", {:deposit, 10.0}}State: %{balance: 0}
Here we add an empty implementation with some pattern matching on the first argument. Our handle_cast/2 pattern is able to destructure the Tuple and bind the amount to a variable.
defmodule Account do # ... def handle_cast({:deposit, amount}, state) do endend
Running the tests gives us a bad return type error, as the GenServer is expecting a Tuple with :noreply and the new state.
** (EXIT from #PID<0.139.0>) bad return value: nil
Return a Tuple that contains :noreply and the state argument, so that we can get the code to fail in the expected way.
It may be tempting to jump to the real implementation, but it is very important that we see the test fail in an expected way. This prevents false positives.
defmodule Account do # ... def handle_cast({:deposit, amount}, state) do {:noreply, state} endend
Now we see the expected error, that 0 is not equal to 10.
Assertion with == failed code: assert 10.0 == Account.get_balance(pid) left: 10.0 right: 0 stacktrace: test/account_test.exs:15: (test)
We can now make the test pass by looking up the balance of the Account and then updating the key.
defmodule Account do # ... def handle_cast({:deposit, amount}, state) do balance = Map.get(state, :balance) {:noreply, Map.put(state, :balance, balance + amount)} endend
And now our tests pass! Wooo! It’s time for some refactoring. There is a Map function that allows us to both get and update a key. It is called Map.get_and_update/3. Let’s change our implementation to use that function.
defmodule Account do # ... def handle_cast({:deposit, amount}, state) do {_value, balance_with_deposit} = Map.get_and_update(state, :balance, fn balance -> {balance, balance + amount} end) {:noreply, balance_with_deposit} endend
When we run the tests, we see they still pass.
The get_and_update/3 takes an anonymous function, which returns the original value and our new value for the :balance key.
Next, the get_and_update/3 returns a Tuple with the original value and the updated Map. So we use destructuring to get the balance containing the deposit. Lastly, we pass the updated balance into our :noreply Tuple.
Updating our backlog, we see that we have one final feature to write: removing money.
Backlog-Check that initial account balance is zero-Add money to an account- Remove money from an account
Spending our money
So far, our users can check their balances and add money. Now for the fun part: spending.
Let’s write a test for removing money from an Account. Our test will verify that money is subtracted from the balance when withdraw/2 is called.
Add the following test:
defmodule AccountTest do # ... test 'withdraw money reduces the balance' do {:ok, pid} = Account.start_link() Account.withdraw(pid, 52.34) endend
Executing the tests, we see a failure because the Account.withdraw/2 does not exist.
** (UndefinedFunctionError) function Account.withdraw/2 is undefined or private
Add an empty implementation in order to get the program to compile.
defmodule Account do # ... def withdraw(pid, amount) do end # ...end
With the code now compiling, we can add the assertion to verify the balance.
defmodule AccountTest do # ... test 'withdraw money reduces the balance' do {:ok, pid} = Account.start_link() Account.withdraw(pid, 52.34) assert -52.34 == Account.get_balance(pid) endend
When run, the tests give us a meaningful failure that the balance is 0 but expected -52.34. Let’s take a step toward making this pass by invoking GenServer.cast/2, but we are going to pass a Tuple with :withdraw and the amount as the last argument. This is similar to what we wrote for the deposit/2 function.
defmodule Account do # ... def withdraw(pid, amount) do GenServer.cast(pid, {:withdraw, amount}) end # ...end
When the tests are run, we see an error that there is no matching handle_cast/2 function.
20:37:50.448 [error] GenServer #PID<0.153.0> terminating** (FunctionClauseError) no function clause matching in Account.handle_cast/2 (bank) lib/account.ex:28: Account.handle_cast({:withdraw, 52.34}, %{balance: 0}) (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4 (stdlib) gen_server.erl:711: :gen_server.handle_msg/6 (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3Last message: {:"$gen_cast", {:withdraw, 52.34}}State: %{balance: 0}
We can resolve this by adding a new handle_cast/2 implementation that matches on the :withdraw message. Our initial implementation will return the :noreply Tuple and the unchanged state. This will allow us to see the same failure message on the balance.
defmodule Account do # ... def handle_cast({:deposit, amount}, state) do # ... end def handle_cast({:withdraw, amount}, state) do {:noreply, state} endend
We can make the test pass by subtracting the current balance rather than adding it.
defmodule Account do # ... def handle_cast({:deposit, amount}, state) do {_value, balance_with_deposit} = Map.get_and_update(state, :balance, fn balance -> {balance, balance + amount} end) {:noreply, balance_with_deposit} end def handle_cast({:withdraw, amount}, state) do {_value, balance_with_deposit} = Map.get_and_update(state, :balance, fn balance -> {balance, balance - amount} end) {:noreply, balance_with_deposit} endend
With all the tests passing, you will notice that the only difference between these implementations is subtraction and addition. Let’s do one final refactoring to clean up this code.
First, we are going to separate what is different in these two implementations from what is the same.
“What is different?” you ask. Well, the way it changes the balance. So we extract an anonymous function to change the balance.
defmodule Account do # ... def handle_cast({:deposit, amount}, state) do # Extracted anonymous function update_balance = fn balance, amount -> balance + amount end {_value, balance_with_deposit} = Map.get_and_update(state, :balance, fn balance -> # we now call the function instead of doing a calculation {balance, update_balance.(balance, amount)} end) {:noreply, balance_with_deposit} end # similar changes were made to this function def handle_cast({:withdraw, amount}, state) do update_balance = fn balance, amount -> balance - amount end {_value, balance_with_deposit} = Map.get_and_update(state, :balance, fn balance -> {balance, update_balance.(balance, amount)} end) {:noreply, balance_with_deposit} endend
With our anonymous function extracted, we run the tests and see that they still pass. There is one other difference between these functions: the name of the Map in the pattern matched from the result of Map.get_and_update/3. Let’s change the name in both functions to updated_balance.
defmodule Account do # ... def handle_cast({:deposit, amount}, state) do update_balance = fn balance, amount -> balance + amount end # variable 'update_balance' changed to 'updated_balance' {_value, updated_balance} = Map.get_and_update(state, :balance, fn balance -> {balance, update_balance.(balance, amount)} end) # variable name changed {:noreply, updated_balance} end def handle_cast({:withdraw, amount}, state) do update_balance = fn balance, amount -> balance - amount end # variable name changed {_value, updated_balance} = Map.get_and_update(state, :balance, fn balance -> {balance, update_balance.(balance, amount)} end) # variable name changed {:noreply, updated_balance} endend
Now that we have an identical set of code, we can extract a function to update the balance. We will start with the handle_cast/2 that matches the withdraw function.
We are going to create a new private function, change_balance/3. The first argument is going to be the state, the second is the amount to change the balance, and the third is our anonymous function.
You will notice that the names of our variables and functions feel inconsistent at the moment. That is part of the process; we don’t quite know where everything is going to land. Once the structures are in place, we will change the names.
defmodule Account do # ... def handle_cast({:withdraw, amount}, state) do update_balance = fn balance, amount -> balance - amount end {:noreply, change_balance(state, amount, update_balance)} end # extracted function defp change_balance(state, amount, update_balance) do {_value, updated_balance} = Map.get_and_update(state, :balance, fn balance -> {balance, update_balance.(balance, amount)} end) updated_balance endend
Next, in the handle_cast/2 function, we should inline that anonymous function, since it is so short.
defmodule Account do # ... def handle_cast({:withdraw, amount}, state) do {:noreply, change_balance(state, amount, &(&1 - &2))} endend
We can use the & syntax, which allows us to create a smaller anonymous function, and reference each argument by its index &N—where N is some index.
With our function extracted, let’s make the names more consistent. Change the name of the third argument in change_balance/3 to calculate_balance.
defmodule Account do # ... defp change_balance(state, amount, calculate_balance) do {_value, updated_balance} = Map.get_and_update(state, :balance, fn balance -> {balance, calculate_balance.(balance, amount)} end) updated_balance endend
Now we can rename the change_balance/3 to get_updated_balance/3.
defmodule Account do # ... def handle_cast({:withdraw, amount}, state) do {:noreply, get_updated_balance(state, amount, &(&1 - &2))} end defp get_updated_balance(state, amount, calculate_balance) do {_value, updated_balance} = Map.get_and_update(state, :balance, fn balance -> {balance, calculate_balance.(balance, amount)} end) updated_balance endend
With our extracted get_updated_balance/3 function complete, we can change handle_cast/2 for deposits to use our new function as well.
defmodule Account do # ... # notice the only different is in the anonymous function def handle_cast({:deposit, amount}, state) do {:noreply, get_updated_balance(state, amount, &(&1 + &2))} end def handle_cast({:withdraw, amount}, state) do {:noreply, get_updated_balance(state, amount, &(&1 - &2))} end #...end
There you have it! All of our tests pass, and we have some clean code. Here is the final version of the Account GenServer:
defmodule Account do use GenServer def start_link do GenServer.start_link(__MODULE__, :ok) end def init(:ok) do {:ok, %{balance: 0}} end # API def get_balance(pid) do GenServer.call(pid, :get_balance) end def deposit(pid, amount) do GenServer.cast(pid, {:deposit, amount}) end def withdraw(pid, amount) do GenServer.cast(pid, {:withdraw, amount}) end # Callbacks def handle_call(:get_balance, _from, state) do {:reply, Map.get(state, :balance), state} end def handle_cast({:deposit, amount}, state) do {:noreply, get_updated_balance(state, amount, &(&1 + &2))} end def handle_cast({:withdraw, amount}, state) do {:noreply, get_updated_balance(state, amount, &(&1 - &2))} end # private defp get_updated_balance(state, amount, calculate_balance) do {_value, updated_balance} = Map.get_and_update(state, :balance, fn balance -> {balance, calculate_balance.(balance, amount)} end) updated_balance endend
We are all done with our current backlog!
Backlog-Check that initial account balance is zero-Add money to an account-Remove money from an account
Conclusion
In this post, I showed you how I have been approaching using test-driven development to create GenServers. We started with a backlog of three functions we wanted our Account to perform, and we took small steps, following the mantra of “Red, Green, Refactor” to arrive at our implementation.
Along the way, we learned how important pattern matching is to Elixir code. We also learned what GenServers are, and how we can use their call and cast functionality.
We observed:
TDD
- How to write just enough of a test to fail
- Getting to green fast—by hard-coding
- Refactoring to a real implementation
Elixir
- Pattern matching is very important
- Functions often return status Tuples
- GenServers behave a lot like objects in other languages
I hope that you found this article informative. If you enjoyed it, please leave a like or a comment below. Thanks, and Happy Coding!
Steven Solomon
