Simple Configuration Setup for Elixir Projects (v1.11+)
I’ve written about Elixir configuration
in an earlier post,
describing the differences in the configuration styles. If you don’t know how Elixir
configuration works, I suggest reading it also. In this post, I will demonstrate a system
for configuring an Elixir project using the config/runtime.exs
system introduced in
Elixir 1.11. This system is how I configure my projects, so feel free take it as inspiration, but
it’s not a law that you have to follow.
Aim
The aims of this system are the following:
- It should be (nearly) unified in development and production, i.e. between running with Mix directly, and running a Mix release on the target system
- It should allow configuring most things even after compiling the release
- It should allow configuring using environment variables, casting to internal datatypes
- It should use the standard Elixir configuration system and not change the way configuration is retrieved in the code
We accomplish these by using these parts:
config/config.exs
for compile time configuration (mostly)config/runtime.exs
for all other configuration.env
file for handy configuration value tweaking in development- A config helpers module to cast environment variables to the correct datatypes
Do note that I use this method in a relatively simple, single-node deployment that does not use Docker or Kubernetes or the other modern fancy-pants stuff. So maybe those have a different way of doing things. But at least this is one way and maybe food for thought.
The examples in this post are from a Phoenix project I have. Phoenix still generates a project
skeleton with the “traditional” config/{config,dev,prod,test}.exs
files, so it requires a bit
of a cleanup to move to this system.
Compile config
The classic config files in Elixir are config/config.exs
, and all the files you import from
it, usually things like config/dev.exs
, config/dev.secret.exs
, and so on. These are for compile
time configuration. When developing, the app is constantly recompiled, so putting configuration
here can work and is tempting, but it will bite you when you build a release from your project, as
the values can no longer be changed after compilation.
Let’s start with config/config.exs
. It should only contain configuration required at
compile time (such as settings that will affect the compilation of modules), and may also contain
configuration that we can also say with certainty will not be needed to change once the project has
been deployed. You can import other compile time config files, but personally I think it’s not
needed for most projects. The config/*.secret.exs
style should generally not be used, instead
configuration using secrets should live in config/runtime.exs
, which I will talk about later.
Example file:
import Config
# Configures the endpoint
config :code_stats, CodeStatsWeb.Endpoint,
render_errors: [accepts: ~w(html json)],
pubsub_server: CodeStats.PubSub,
code_reloader: Config.config_env() == :dev,
debug_errors: Config.config_env() == :dev,
check_origin: Config.config_env() == :prod
config :code_stats,
ecto_repos: [CodeStats.Repo]
config :geolix,
init: {CodeStatsWeb.Geolix, :init}
# Store mix env used to compile app, since Mix won't be available in release
config :code_stats, compile_env: Mix.env()
# Store app version and commit hash at compile time
config :code_stats,
commit_hash: System.cmd("git", ["rev-parse", "--verify", "--short", "HEAD"]) |> elem(0),
version: Mix.Project.config()[:version]
# Use Jason for JSON parsing in Phoenix
config :phoenix, :json_library, Jason
You can see there are some settings that affect Phoenix compilation, such as the code reloader setting. There are also settings that I don’t expect to change, such as the Ecto repo, Geolix initialisation module, and JSON parsing library. There’s also values that I deliberately want to pick at compile time, such as the Mix environment and git version hash.
Runtime config
Runtime configuration is the meat of this post. With Elixir 1.11, we have the magnificent
config/runtime.exs
file, that has a couple of useful properties. Firstly, it is evaluated at
release startup time, so it can use the system environment as it exists on the target system. And
secondly, since all the modules have already been compiled, we can call code from our project and
dependencies. This will come in handy later.
Let’s start with a simple example:
import Config
config :code_stats,
site_name: System.get_env("SITE_NAME", "Code::Stats")
Since this is evaluated at startup time, we can now change the site name without recompiling the project. Just change the environment variable and restart.
As .exs
files are just regular Elixir, we can also use code:
with key when is_binary(key) <- System.get_env("BAMBOO_API_KEY") do
config :code_stats, CodeStats.Mailer,
adapter: Bamboo.MailgunAdapter,
api_key: key,
domain: System.get_env("BAMBOO_DOMAIN")
else
nil ->
IO.puts(:stderr, "Mailer key not configured, using LocalAdapter.")
config :code_stats, CodeStats.Mailer, adapter: Bamboo.LocalAdapter
end
This allows for some handy things, but I would limit it to basic conditionals to avoid going overboard. Don’t make your config Turing complete.
Finally, if some configuration only needs to be applied in certain environments, you can use
Config.config_env/0
:
case Config.config_env() do
:prod ->
config :code_stats, CodeStatsWeb.Endpoint,
secret_key_base: System.get_env("SECRET_KEY_BASE"),
server: true
:dev ->
config :code_stats, CodeStatsWeb.Endpoint,
watchers: [mix: ["frontend.watch", cd: Path.expand("../", __DIR__)]]
# And so on
end
This is also the weakness with config/runtime.exs
: you cannot import other files, so the file may
get lengthy. In my projects it has stayed manageable, but your mileage may vary. Additionally, you
might have noticed that the watcher configuration above could have just as well been in the compile
time config files. It’s true, but personally I like to keep as much of it in the same file as I can,
to make things easier to find. Just find your own style. :)
.env and config helpers
Now that we have a proper config/runtime.exs
, we have a unified configuration that runs both when
we are developing locally, and in our production release. We have two issues still, though.
The first is that setting all of these environment variables when working locally can be tedious.
There are different ways to do it, such as having a script file with lots of export FOO=1
statements, and remembering to source .env
often enough, but for myself I wanted something that
didn’t require extra work. That’s why I wrote DotenvParser,
a library to parse a .env
file and load its values to the system environment.
With DotenvParser, we can do the following trick at the start of the file:
import Config
if Config.config_env() == :dev do
DotenvParser.load_file(".env") # Works because the dependency is already compiled
end
# Continue with configuration
Now after this block, all the environment variables defined in .env
have been loaded and can be
retrieved with System.get_env/2
. Add it to .gitignore
to be able to play with the values without
hassle (just document it elsewhere for other devs).
The other issue that you always hit with environment variables is that they are all strings. It
would be nice to have real datatypes in our configuration. Since config/runtime.exs
can call our
modules, we can just write some helper code to deal with this. This is such a simple module that I
haven’t even packaged it, but you can find it
in my project’s codebase:
defmodule CodeStats.ConfigHelpers do
@type config_type :: :string | :integer | :boolean | :json
@doc """
Get value from environment variable, converting it to the given type if needed.
If no default value is given, or `:no_default` is given as the default, an error is raised if the variable is not
set.
"""
@spec get_env(String.t(), :no_default | any(), config_type()) :: any()
def get_env(var, default \\ :no_default, type \\ :string)
def get_env(var, :no_default, type) do
System.fetch_env!(var)
|> get_with_type(type)
end
def get_env(var, default, type) do
with {:ok, val} <- System.fetch_env(var) do
get_with_type(val, type)
else
:error -> default
end
end
@spec get_with_type(String.t(), config_type()) :: any()
defp get_with_type(val, type)
defp get_with_type(val, :string), do: val
defp get_with_type(val, :integer), do: String.to_integer(val)
defp get_with_type("true", :boolean), do: true
defp get_with_type("false", :boolean), do: false
defp get_with_type(val, :json), do: Jason.decode!(val)
defp get_with_type(val, type), do: raise("Cannot convert to #{inspect(type)}: #{inspect(val)}")
end
The caveat to remember here is that since the code is called at startup time, no applications are running. So we wouldn’t be able to query the database, for example. But we can call regular, non-app-specific code like that.
Now when we want to retrieve configuration in the file, we can do the following:
import Config
import CodeStats.ConfigHelpers, only: [get_env: 3]
if Config.config_env() == :dev do
DotenvParser.load_file(".env")
end
config :code_stats,
beta_mode: get_env("BETA_MODE", false, :boolean),
max_xp_per_pulse: get_env("MAX_XP_PER_PULSE", 1_000, :integer),
cors_allowed_origins: get_env("CORS_ALLOWED_ORIGINS", ["http://localhost"], :json)
So now we can easily get environment variables and cast them to the correct types. The only thing
left to do is to set the environment variables in the production system. I use systemd, so I tend to
set them in the unit file as Environment=FOO=val
:
[Unit]
After=postgresql.service
[Service]
Type=simple
User=release_user
Restart=always
RestartSec=10
Environment=PORT=51110
Environment=HOST=codestats.net
Environment=HOST_PORT=443
# And so on...
ExecStart=/path/to/bin/code_stats start
WorkingDirectory=/path/to/wrk/dir
[Install]
WantedBy=multi-user.target
Conclusion
What we have set up in this post:
- Only compile time and very static configs in
config/config.exs
- All other config in
config/runtime.exs
, which will be evaluated both in development and production - Local development values parsed from
.env
file in project root - Handy config helpers for casting environment variables to the correct types
You can see this system in action in my project Code::Stats:
If you have questions or feedback about this system, please add a comment below, or contact me:
- @AmNicd on Twitter
- Nicd on the Elixir Discord
- Nicd- on #elixir-lang on Freenode