Random Notes

Elixir: Time for Some Configuration

Configuring Elixir applications is a common problem point for new developers and I’ve seen many questions about it in the community chats. So I thought to write down my knowledge in case it helps anyone.

The usual way of getting configured values in your application is using Application.get_env/3. Environment variables are fetched with System.get_env/2. But when and where you use them matters. To simplify, I consider there to be three phases where you can configure your application: build time, startup time, and runtime.

Build time Startup time Runtime
  • Regular config (config/config.exs etc.)
  • Code outside functions
  • config/releases.exs
  • Config providers
  • Init callbacks
  • Runtime get_env calls

Build time

The usual way of configuring an application is through configuration files in the config directory, starting with config.exs, that may import other files based on the current environment. But configurations set in the these files are set at build time. This means that if you use code such as System.get_env/2, it will use the environment as it was when the project was built. During development when you run the program with mix run or iex -S mix this is fine, as it is constantly recompiled. But for production using Mix releases, you usually don’t want to use the values as they were on the build machine.

This also applies for metaprogramming, i.e. code outside function bodies. See the following example:

defmodule Test do
  @attr Application.get_env(:my_app, :foo) # Build time

  def fun() do
    Application.get_env(:my_app, :foo) # Runtime
  end
end

Macros can muddle the waters somewhat, because they can be evaluated at build or runtime depending on the macro. For example the following in a Phoenix router would be evaluated at build time:

defmodule Router do
  scope "/" do
    get(Application.get_env(:my_app, :index_route), Ctrl, :index)
  end
end

But the following task macro’s body (from MBU) would be runtime instead:

defmodule Mix.Tasks.Foo do
  task _ do
    IO.inspect(System.get_env("FOO"))
  end
end

So with macros you will have to look up the macro’s documentation and/or source code.

Startup time

Mix releases have a concept of startup time configuration (called runtime in Mix’s documentation). Startup time configuration is processed when the Erlang runtime system is booting. In technical terms a small subset of applications in the runtime environment are started (those needed to process the configuration), then the configuration is applied, and after that the rest of the applications are started. You can use the config/releases.exs file for this purpose. If you call System.get_env/2 here, it will use the environment in the target machine at startup time.

Mix releases also support configuration providers that allow for extending config/releases.exs and loading configuration from e.g. JSON files on the target machine. You can read more about Mix release configuration and its limitations in the Mix documentation. Notably, you cannot call Mix at startup time – or runtime for that matter – as it is not available in the compiled application.

Runtime

I tend to separate runtime from startup time as it’s slightly different. At startup time only a minimal set of applications is running to process the configuration. After that, the VM starts the rest of the applications and subsequently starts running their init callbacks. As far as the VM is concerned, this is just normal operation, so I call this runtime.

Many libraries offer such init callbacks that are used to configure their operation at runtime. When for example System.get_env/2 is used here, it will get the environment on the target machine at that time. Finally, you can always just call Application.get_env/3 and System.get_env/2 in any regular function at runtime and it will use whatever value is current at that time (both application environment and environment variables can be changed at runtime).

Here’s an example of an init callback from a real project of mine:

defmodule CodeStatsWeb.Geolix do
  @moduledoc """
  Module for initialising Geolix databases at runtime instead of build time.
  """

  @spec init() :: :ok
  def init() do
    db_dir = Application.app_dir(:code_stats, "priv")

    databases = [
      %{
        id: :city,
        adapter: Geolix.Adapter.MMDB2,
        source: Path.join([db_dir, "geoip-cities.gz"])
      },
      %{
        id: :country,
        adapter: Geolix.Adapter.MMDB2,
        source: Path.join([db_dir, "geoip-countries.gz"])
      }
    ]

    Application.put_env(:geolix, :databases, databases)
  end
end

As you can see, I’ve taken care to call functions such as Application.app_dir/2 only inside the init/0 function body so that the values will be evaluated at runtime. This function will be called by the Geolix library when it is starting up. Different libraries have different methods of configuration but this is a popular one.

Note about Application.compile_env/3

In Elixir 1.10, there is a new function to retrieve configured values. Since the mistake of using Application.get_env/3 at build time is so common, there is a new function Application.compile_env/3 that works a bit differently. It is used to explicitly read configuration at build time. When Elixir starts up, it checks if there is a different configuration value available than the one that was compiled, and raises an error if this happens. This is meant to help you avoid mistakes and surprises when you have compiled in one configuration, but set up another configuration in the target environment, and are wondering why you are seeing the wrong value.

You can read more information about the function from its documentation or the announcement blog post.

Additional resources and notes

So I hope that’s a simple explanation into the different configuration methods. Here are some additional links for reading:

The techniques mentioned here are quite recent. Mix releases are available from Elixir 1.9 onwards, in older versions you need to use Distillery releases and its own configuration providers.

Full example and practise

Let’s imagine a project using Mix releases that has been built with the following config/config.exs:

import Config

config :my_app,
  db_user: "Jeff",
  db_name: "devdb",
  db_dir: "/tmp/db"

Now it has been deployed with the following config/releases.exs:

import Config

config :my_app,
  db_user: "Joan",
  db_name: System.get_env("DB_NAME"),
  db_dir: "/var/lib/postgres"

When starting up, the environment DB_NAME=proddb DB_DIR=/home/kari/db is given to the program.

The project contains this module:

defmodule Foo do
  require Logger

  @db_user Application.get_env(:my_app, :db_user)

  def init(db_name \\ Application.get_env(:my_app, :db_name)) do
    db_dir = Application.get_env(:my_app, :db_dir)

    Logger.debug("DB_DIR=" <> System.get_env("DB_DIR"))

    %{
      user: @db_user,
      name: db_name,
      dir: db_dir
    }
  end
end

Can you figure out what is returned from the init/1 function, assuming it is given no arguments? And what is logged to the debug console?

If you think you’ve got it, or you’re just eager to get the answers, here they are: