Elixir: Time for Some Configuration
Configuring Elixir applications is a common problem point for new developers and I’ve seen many questions about it in the community chats. So I thought to write down my knowledge in case it helps anyone.
tl;dr for those on Elixir v1.11: Use config/runtime.exs
for your configuration needs. Read on for
more details / nuance.
UPDATED 6th Oct 2020:
Elixir 1.11 was released
and the post has been updated with information about config/runtime.exs
.
The usual way of getting configured values in your application is using
Application.get_env/3
. Environment
variables are fetched with System.get_env/2
.
Configuration is set using the config
functions in
Config or
Mix.Config (deprecated).
But when and where you use these matters. To simplify, I consider there to be three phases where you
can configure your application: build time, startup time, and runtime.
Build time | Startup time | Runtime |
---|---|---|
|
|
|
Build time
The usual way of configuring an application is through configuration files in the config
directory,
starting with config.exs
, that may import other files based on the current environment. But
configurations set in the these files are set at build time. This means that if you
use code such as System.get_env/2
, it will use the environment as it was when the project was
built. During development when you run the program with mix run
or iex -S mix
this is fine, as
it is constantly recompiled. But for production using Mix releases, you usually don’t want to use the
values as they were on the build machine.
This also applies for metaprogramming, i.e. code outside function bodies. See the following example:
defmodule Test do
@attr Application.get_env(:my_app, :foo) # Build time
def fun() do
Application.get_env(:my_app, :foo) # Runtime
end
end
Macros can muddle the waters somewhat, because they can be evaluated at build or runtime depending on the macro. For example the following in a Phoenix router would be evaluated at build time:
defmodule Router do
scope "/" do
get(Application.get_env(:my_app, :index_route), Ctrl, :index)
end
end
But the following task macro’s body (from MBU) would be runtime instead:
defmodule Mix.Tasks.Foo do
task _ do
IO.inspect(System.get_env("FOO"))
end
end
So with macros you will have to look up the macro’s documentation and/or source code.
Startup time
To avoid the issue of the configuration being compiled in at build time and unchangeable, you can configure the system at startup time (called runtime in Mix’s documentation). Startup time configuration is processed when the Erlang runtime system is booting. In technical terms a small subset of applications in the runtime environment are started (those needed to process the configuration), then the configuration is applied, and finally the system is restarted with the updated configuration.
From Elixir v1.9 to v1.10, the place to do this was config/releases.exs
. This was only used when a
Mix release was starting up. Since Elixir v1.11, there is a new file called config/runtime.exs
that is always processed, whether in dev, test, or production environment. This allows you to have a
truly unified configuration file for all environments.
Mix releases also support configuration providers that allow for extending these files and loading configuration from e.g. JSON files on the target machine. You can read more about Mix release configuration and its limitations in the Mix documentation.
NOTE: Since config/releases.exs
and config/runtime.exs
are also used in releases, where Mix is
not available, you cannot call any Mix functions in them. Instead functions from the
Config module should be used. Luckily (since v1.11) they
include the new
config_env/0 and
config_target/0 functions to get the
configuration environment and target respectively.
Runtime
I tend to separate runtime from startup time as it’s slightly different. At startup time only a minimal set of applications is running to process the configuration. After that, the VM restarts with the rest of the applications and subsequently starts running their init callbacks. As far as the VM is concerned, this is just normal operation, so I call this runtime.
Many libraries offer such init callbacks that are used to configure their operation at runtime. When
for example System.get_env/2
is used here, it will get the environment on the target machine at
that time. Finally, you can always just call Application.get_env/3
and System.get_env/2
in any
regular function at runtime and it will use whatever value is current at that time (both application
environment and environment variables can be changed at runtime).
Here’s an example of an init callback from a real project of mine:
defmodule CodeStatsWeb.Geolix do
@moduledoc """
Module for initialising Geolix databases at runtime instead of build time.
"""
@spec init() :: :ok
def init() do
db_dir = Application.app_dir(:code_stats, "priv")
databases = [
%{
id: :city,
adapter: Geolix.Adapter.MMDB2,
source: Path.join([db_dir, "geoip-cities.gz"])
},
%{
id: :country,
adapter: Geolix.Adapter.MMDB2,
source: Path.join([db_dir, "geoip-countries.gz"])
}
]
Application.put_env(:geolix, :databases, databases)
end
end
As you can see, I’ve taken care to call functions such as Application.app_dir/2
only inside the
init/0
function body so that the values will be evaluated at runtime. This function will be called
by the Geolix library when it is starting up. Different libraries have different methods of
configuration but this is a popular one.
Note about Application.compile_env/3
In Elixir 1.10, there is a new function to retrieve configured values. Since the mistake of using
Application.get_env/3
at build time is so common, there is a new function
Application.compile_env/3
that works a bit differently. It is used to explicitly read configuration
at build time. When Elixir starts up, it checks if there is a different configuration value available
than the one that was compiled, and raises an error if this happens. This is meant to help you avoid
mistakes and surprises when you have compiled in one configuration, but set up another configuration
in the target environment, and are wondering why you are seeing the wrong value.
You can read more information about the function from its documentation or the announcement blog post.
Additional resources and notes
So I hope that’s a simple explanation into the different configuration methods. Here are some additional links for reading:
- Configuration in Elixir guide
- Mix release configuration — especially check section “customization and configuration summary”
- Information on
config/runtime.exs
The techniques mentioned here are quite recent. Mix releases are available from Elixir 1.9 onwards, in older versions you need to use Distillery releases and its own configuration providers.
Full example and practise
Let’s imagine a project using Mix releases that has been built with the following
config/config.exs
:
import Config
config :my_app,
db_user: "Jeff",
db_name: "devdb",
db_dir: "/tmp/db"
Now it has been deployed with the following config/runtime.exs
:
import Config
config :my_app,
db_user: "Joan",
db_name: System.get_env("DB_NAME"),
db_dir: "/var/lib/postgres"
When starting up, the environment DB_NAME=proddb DB_DIR=/home/kari/db
is given to the program.
The project contains this module:
defmodule Foo do
require Logger
@db_user Application.get_env(:my_app, :db_user)
def init(db_name \\ Application.get_env(:my_app, :db_name)) do
db_dir = Application.get_env(:my_app, :db_dir)
Logger.debug("DB_DIR=" <> System.get_env("DB_DIR"))
%{
user: @db_user,
name: db_name,
dir: db_dir
}
end
end
Can you figure out what is returned from the init/1
function, assuming it is given no arguments?
And what is logged to the debug console?
If you think you’ve got it, or you’re just eager to get the answers, here they are:
The return value from the function will be a map containing:
user
:"Jeff"
– The value is resolved at build time, because it is outside the function. If the attribute was usingApplication.compile_env/3
instead, it would raise an error when starting up the release.name
:"proddb"
– The value is set inconfig.exs
but overridden from the environment inruntime.exs
. It is set at startup time and fetched in theinit/1
function at runtime, because the call toApplication.get_env/3
is as a default argument (simplified: default arguments are evaluated at runtime, when the function is called).dir
:"/var/lib/postgres"
– Same as above, this time theApplication.get_env/3
call is explicitly inside the function body. The environment variable does not override the value asSystem.get_env/2
was not used inruntime.exs
.
- The function logs
DB_DIR=/home/kari/db
, as the environment variable is fetched at runtime inside the function. - Note: The application configuration and environment variables can be changed – with some caveats –
at any time, for example with the functions
Application.put_env/4
andSystem.put_env/2
. Thus technically they could be changed by other code in the project before theinit/1
function is called and the values there would be different. But this is rare and likely not something that most will bump into.