Random Notes

FBU: My First Build Tool™

EDIT 2017-04-04: I have since renamed the project to MBU: Mix Build Utilities and published it on Hex.pm: hex.pm/packages/mbu. I have edited the links and code examples in this post to reflect that.


tl;dr I wrote my own build tool using Elixir’s Mix: Nicd/mbu.

It’s no secret that I somewhat dislike the state of modern web development. JavaScript is its own terrible world, but one of the sad parts of it is the ecosystem and tooling around it. There’s a lot of innovation and hard work going on in very many fragmented projects, resulting in reimplementations of already solved problems and a ton of half working, alpha quality, 0.x versioned packages with unknown support status. With these packages, you start your project by building an elaborate house of cards that is the build system. And you dread the day when you need to touch it again.

Motivations

A lot of text about my motivations ahead, if you just want to see the end result, click here to jump to the code examples.

Things that irk me the most in the build tools I have used – those being Grunt, Gulp, Brunch, and Webpack (though it’s not meant to really be one, it can be used as one):

So, what is the answer? Writing my own build tool, of course!

Standards
Somewhat relevant xkcd. © Randall Munroe, licensed under CC-BY-NC 2.5.

I took a look at some alternatives like NPM scripts, Make, and a couple of Python based build tools, but none of them really got me interested. I also wanted to avoid adding dependencies in another language, since I was writing in Elixir. Suddenly I had a thought: doesn’t Elixir have its own build tool already? Elixir comes with a swiss army knife called Mix that is used for compiling, testing, running tasks, and managing dependencies. So I thought, maybe I’ll write my own build tool using custom Mix tasks, just to see what it would look like. Because I’m not an imaginative person, I call it Frontend Build Utilities, or FBU for short.

From the start I knew it would not be a silver bullet that solves everyone’s problems, but rather a tool very tailored to my own needs. I knew it would be bad in some ways, but at least I would have more control over the ways it was bad in. And most of all, I just wanted to write it for the heck of it, to see what it would be like.

How it Works

Here’s how I set out to fix the problems above:

Example Usage

FBU is basically a collection of utilities to help with writing Mix tasks that do the building. A basic task looks like this:

defmodule Mix.Tasks.Frontend.Build.Css.Copy do
  use MBU.BuildTask
  import MebeWeb.FrontendConfs

  @shortdoc "Copy compiled CSS to target dir"

  @deps [
    "frontend.build.css.compile"
  ]

  task _ do
    # Ensure target path exists
    out_path = Path.join([dist_path(), "css"])
    File.mkdir_p!(out_path)

    File.cp_r!(Mix.Tasks.Frontend.Build.Css.Compile.out_path(), out_path)
  end
end

The relevant parts above are the @deps attribute and the task block. @deps defines a list of tasks this task depends on. They will be automatically run in parallel before the task itself is run. This allows for easily building hierarchies of tasks that run when one target is executed, like in Gulp or Make.

The task block is a macro that is converted to the usual run/1 function of a Mix task. It takes in an optional list of arguments, prints out logging information and runs the dependencies unless deps: false was given in the argument list. Inside the task there’s just regular Elixir code with nothing special about it. This task creates the output folder and copies files into it. Simple.

For the tasks themselves, FBU contains a few utility functions. The most important are exec, watch and listen. Here is an example of using exec and listen (with only the relevant parts):

def bin(), do: node_bin("babel")

def out_path(), do: Path.join([tmp_path(), "transpiled", "js"])

def args(), do: [
  Path.join([src_path(), "js"]),
  "--out-dir",
  out_path(),
  "--source-maps",
  "inline"
]

task _ do
  bin() |> exec(args()) |> listen()
end

exec takes in a path to a binary (given here by node_bin inside bin) and the command line arguments to give to that binary as a list. It executes the binary and returns a %ProgramSpec struct that can be passed to listen.

listen takes in a single spec (can be a program spec or a watch spec that you will see later) or a list of them and listens to their output. Output is logged to the console. By default the program name is used for prefixing the logs, but it can be changed by passing an additional argument name: "foo" to exec. When the last program has stopped, listen will return.

Watching source files is an important feature of build tools. Many tools also have builtin watch capabilities. With FBU you can leverage them by calling exec and giving the right command line switches to make the tool enter watch mode, usually by giving a switch -w. If the tool does not have watch capabilities, or you wish to run custom Elixir code when watching, you can use the watch function. It starts a custom filesystem watcher that can call any callback, and returns a %WatchSpec to put into listen.

Here’s this blog’s watch task:

defmodule Mix.Tasks.Frontend.Watch do
  use MBU.BuildTask
  import MBU.TaskUtils
  alias Mix.Tasks.Frontend.Build.Js.Transpile, as: TranspileJS
  alias Mix.Tasks.Frontend.Build.Js.Bundle, as: BundleJS
  alias Mix.Tasks.Frontend.Build.Js.Copy, as: CopyJS
  alias Mix.Tasks.Frontend.Build.Css.Compile, as: CompileCSS
  alias Mix.Tasks.Frontend.Build.Css.Copy, as: CopyCSS

  @shortdoc "Watch frontend and rebuild when necessary"

  @deps [
    "frontend.build"
  ]

  task _ do
    [
      exec(
        TranspileJS.bin(),
        TranspileJS.args() ++ ["-w"]
      ),

      watch(
        "JSBundle",
        TranspileJS.out_path(),
        BundleJS
      ),

      watch(
        "JSCopy",
        BundleJS.out_path(),
        CopyJS
      ),

      exec(
        CompileCSS.bin(),
        CompileCSS.args() ++ [
          "-w",
          CompileCSS.scss_file()
        ]
      ),

      watch(
        "CSSCopy",
        CompileCSS.out_path(),
        fn events -> IO.inspect(events); run_task(CopyCSS, deps: false) end
      )
    ]
    |> listen(watch: true)
  end
end

As you can see above, the watch task consists of two normal executions (babel and node-sass have their own builtin watchers) and three custom watchers. The watch calls take the name of the watcher as the first argument (for logging purposes), the path or paths to watch as the second and a callback function or task module that is called when file events happen. The callback receives the a list of events as the argument, here we inspect it (for debugging purposes) and then run the CopyCSS task manually. Note that we give deps: false to the tasks in the CopyCSS watcher, to avoid it running its dependencies, which would not be useful when watching. This is done automatically if you just provide the task module instead of a callback function.

After the specs, again, the listen function is called, but this time with an argument watch: true. This instructs the function to start a key listener that will stop the started programs when the user presses the enter key. That way the watch can be stopped gracefully.

Conclusion

So I built a frontend build tool and all I got was this crappy blog post. I’m withholding judgement over whether the tool is good or not before I have used it for a while. By my first impressions, it turned out to be magic free and explicit, but at the same time very verbose and minimal. I kind of like how it looks, but only time will tell if this whole thing made any sense.

If you’d like to check it out, it’s open source and can be found on Github at Nicd/mbu or on Hex.pm. For example usage, see my blog’s build setup on BitBucket.