Bazel is the open source variant of Google’s internal Blaze build system. It aims to provide fast and deterministic build/test, scaling to extremely large code bases with billions of lines of code, for example Google’s monorepo. Beyond the basics of performance and correctness, some examples of the advanced features that Bazel brings to the table include:

As a Googler, I’ve experienced first hand the benefits of working with Blaze and find the internal build experience to be close to magic; it works seamlessly and allows the developer to focus on their core mission of developing great software, mostly without the hassles and complexity that I’ve encountered in other build systems such as GNU Make, SCons, Xcode, CMake and autoconf/automake.

In the OSS Envoy project, we on-boarded Bazel in the Spring of 2017, converting the project to Bazel from its extant CMake based build system. There were a number of motivating factors behind this shift, including providing a scalable build/test environment that would grow with the project. Embracing Bazel also had the added benefit of providing better integration with Google’s internal build infrastructure.

A key issue in Bazel, that does not exist in Blaze, is the management of external dependencies. A monorepo by definition provides complete encapsulation of all dependencies in a single repository and build system. Third party dependencies must be converted to Blaze in order to build. In the OSS world, projects typically depend on other project repositories, together with their build artifacts, via a combination of implicit and explicit external dependencies. These are often captured in requirements manifests, git submodules, setup scripts, README.md instructions, vendoring, forking, Docker images, etc. A project written in Bazel might depend on 5 other projects, which in turns might each depend on a number of other projects, each using a completely different non-Bazel canonical build system.

While converting Envoy to Bazel, we evaluated and implemented a number of strategies for managing our external dependencies. These dependencies were mostly C and C++ and had a mixture of upstream supported build system types (autoconf/automake, CMake, shell scripts, Bazel). While there is official Bazel documentation for managing external dependencies, it is language independent and does not deal in depth with the specific challenges that exist for C/C++ dependency management. It also does not provide comprehensive guidance for the common situation in which dependencies do not have a native Bazel build system. We hope that the experiences documented below provide for a useful account of how a medium sized C++ OSS project can deal in practice with the external dependency management problem in Bazel.

Read the rest →