Build system#

Building software for embedded devices is a complex process. Projects often have custom toolchains, target different hardware platforms, and require additional configuration and post-processing of artifacts.

As a modern embedded framework, Pigweed’s goal is to collect these embedded use cases into a powerful and flexible build system, then extend it with support for modern software development practices.

See Pigweed’s GN Python Build for information about Python build automation with Pigweed.

What’s in a build system?#

Pigweed AI summary: Pigweed's build system offers a range of features beyond compiling code, including simple toolchain configuration, multi-target builds, multi-language support, custom scripting, size reporting, documentation, and unit testing. The system also includes a file system watcher, called pw watch, which triggers builds for quick iteration. Pigweed's build system is designed with embedded development in mind and offers support for packaging Python within a build system, running size reports, and creating high-quality, up-to-date references for developers.

A quality build system provides a variety of features beyond compiling code. Throughout our experience with embedded development, we’ve found several build features to be especially useful, and designed Pigweed’s build system with them in mind.

Simple toolchain configuration#

Pigweed AI summary: This section discusses the importance of simple toolchain configuration for embedded projects, emphasizing the need for ease in both initial setup and later adjustments. Custom build toolchains are often used for specific hardware in these projects.

Embedded projects often use custom build toolchains for their specific hardware. Configuring these should be a simple process, both in their initial setup and later adjustments.

Multi-target builds#

Pigweed AI summary: The process of building firmware for different boards or MCUs can be complex and involves changing compiler flags and firmware libraries. Pigweed's build system is designed to support multiple target configurations simultaneously, making the process easier.

Virtually every consumer product has firmware that targets different boards or MCUs during development. While building for a single board is simple enough, the complexity of supporting different targets ranges from changing compiler flags to swapping out entire libraries of firmware and drivers. This is often done by running multiple builds, configuring each one accordingly. In Pigweed, we’ve designed our build system with first-class multi-target support in mind, allowing any number of target configurations to be built simultaneously.

Multi-language support#

Pigweed AI summary: This section discusses how embedded projects are usually written in C, C++, and assembly, but can also be written in other languages like Rust. Host-side tooling can also be written in various languages, and having all of these build together can save a lot of time.

Embedded projects are typically written in C, C++, and assembly. However, it is possible to have firmware written in other languages, such as Rust. Additionally, projects may have host-side tooling written in a wide variety of languages. Having all of these build together proves to be a large time saver.

Custom scripting#

Pigweed AI summary: Custom scripting is often necessary for post-processing build artifacts in embedded projects, including extracting ELF sections, injecting metadata, image signing, creating debugging symbol databases, and extracting string tokens. These steps are run during the build process and facilitated by the build system. The pw_python_action module is a related resource.

Embedded projects often require post-processing of build artifacts; these may include:

Extracting ELF sections into a different container
Injecting metadata into firmware images
Image signing
Creating databases of symbols for debugging
Extracting string tokens into a database (for example, with pw_tokenizer)

These are run as steps during a build, facilitated by the build system.

Python#

Pigweed AI summary: Python is a popular scripting language used by many development teams, including Pigweed. While it works well for local development, issues can arise when scripts need to be distributed. Proper support for packaging Python within a build system can alleviate these concerns and allow teams to focus on writing code.

Python is a favorite scripting language of many development teams, and here at Pigweed, we’re no exception. Much of Pigweed’s host-side tooling is written in Python. While Python works great for local development, problems can arise when scripts need to be packaged and distributed for vendors or factory teams. Having proper support for packaging Python within a build system allows teams to focus on writing code instead of worrying about distribution.

Size reporting#

Pigweed AI summary: The article discusses the importance of memory usage on embedded devices and the need for custom tooling to determine the amount of flash and RAM space used by firmware. Running size reports as part of a build ensures that the information is always up-to-date and allows for tracking of space usage over time. The article also includes a reference to a tool called pw_bloat for further reading.

On embedded devices, memory is everything. Most projects have some sort of custom tooling to determine how much flash and RAM space their firmware uses. Being able to run size reports as part of a build ensures that they are always up-to-date and allows space usage to be tracked over time.

Documentation#

Pigweed AI summary: Pigweed, a software development platform, emphasizes the importance of documentation for future maintainers of a project. They have integrated documentation that builds alongside its code and combines with other build features to provide high-quality, up-to-date references for developers. Additionally, they recommend using their pw_docgen module for documentation purposes.

An oft-neglected part of software development, documentation is invaluable for future maintainers of a project. As such, Pigweed has integrated documentation which builds alongside its code and combines with other build features, such as size reports, to provide high quality, up-to-date references for developers.

Unit testing#

Pigweed AI summary: Unit testing is crucial for maintaining consistent functionality of code and avoiding accidental regressions. Host-side unit tests can be run on a desktop, while device-side unit tests are necessary for developing actual firmware. Pigweed has a test framework and build integration for running tests on physical devices. See also: pw_unit_test and pw_target_runner.

Unit tests are essential to ensure that the functionality of code remains consistent as changes are made to avoid accidental regressions. Running unit tests as part of a build keeps developers constantly aware of the impact of their changes.

Host-side unit tests#

Pigweed AI summary: The Pigweed software targets embedded devices, but much of its code can be tested on a host desktop by using different backends. This is useful during development because it allows tests to run consistently without the need to flash a device.

Though Pigweed targets embedded devices, a lot of its code can be run and tested on a host desktop by swapping out backends to host platform libraries. This is highly beneficial during development, as it allows tests to consistently run without having to go through the process of flashing a device.

Device-side unit tests#

Pigweed AI summary: The importance of running tests on actual hardware for firmware development is emphasized in this section. Pigweed has developed a test framework and build integration to facilitate running tests across physical devices as part of the build process.

As useful as host-side tests are, they are not sufficient for developing actual firmware, and it is critical to run tests on the actual hardware. Pigweed has invested into creating a test framework and build integration for running tests across physical devices as part of a build.

Bonus: pw watch#

Pigweed AI summary: The article discusses the benefits of having a file system watcher in web development, which triggers a build for quick iteration when source file changes occur. This feature allows for unit tests and size reports to re-run whenever dependent code is modified. Pigweed's watcher, initially seen as a gimmick, has become a staple of Pigweed development, with most users having it permanently running in a terminal window. The article also includes a "See also" section with a reference to the pw_watch module.

In web development, it is common to have a file system watcher listening for source file changes and triggering a build for quick iteration. When combined with a fast incremental build system, this becomes a powerful feature, allowing things such as unit tests and size reports to re-run whenever any dependent code is modified.

While initially seen as somewhat of a gimmick, Pigweed’s watcher has become a staple of Pigweed development, with most Pigweed users having it permanently running in a terminal window.

Pigweed’s build systems#

Pigweed AI summary: Pigweed offers multiple build systems, including GN, CMake, and Bazel, to allow projects to choose the most suitable one for them. GN is the most full-featured and has become Pigweed's primary build system, while CMake and Bazel are also supported. The word "target" is overloaded within GN/Bazel and Pigweed, so the former is referred to as "GN/Bazel targets" and the latter as "Pigweed targets." GN is a meta-build

Pigweed can be used either as a monolith or à la carte, slotting into an existing project. To this end, Pigweed supports multiple build systems, allowing Pigweed-based projects to choose the most suitable one for them.

Of the supported build systems, GN is the most full-featured, followed by CMake, and finally Bazel.

Note

A quick note on terminology: the word “target” is overloaded within GN/Bazel (and Pigweed)—it can refer to either a GN/Bazel build target, such as a source_set or executable, or to an output platform (e.g. a specific board, device, or system).

To avoid confusing the two, we refer to the former as “GN/Bazel targets” and the latter as “Pigweed targets”.

GN#

Pigweed AI summary: GN (Generate Ninja) is a meta-build system that outputs Ninja build files, originally designed for use in Chromium. Pigweed has adopted GN as its primary build system due to its speed and simplicity, and it is recommended for Pigweed-based projects where possible.

A perhaps unfamiliar name, GN (Generate Ninja) is a meta-build system that outputs Ninja build files, originally designed for use in Chromium. Pigweed first experimented with GN after hearing about it from another team, and we quickly came to appreciate its speed and simplicity. GN has become Pigweed’s primary build system; it is used for all upstream development and strongly recommended for Pigweed-based projects where possible.

The GN build#

Pigweed AI summary: This paragraph provides an overview of Pigweed's GN build structure, including information on the GN build file, the BUILDCONFIG.gn file, top-level GN targets, configuration files, module variables, GN target type wrappers, Pigweed targets, and other build files. It also mentions the support for CMake and Bazel as alternative build systems.

This section describes Pigweed’s GN build structure, how it is used upstream, build conventions, and recommendations for Pigweed-based projects. While containing some details about how GN works in general, this section is not intended to be a guide on how to use GN. To learn more about the tool itself, refer to the official GN reference.

Entrypoint: .gn#

Pigweed AI summary: The GN build system uses a file called ".gn" to define a project's root directory. This file must point to the location of a "BUILDCONFIG.gn" file for the project. In Pigweed upstream, the ".gn" file only serves this purpose, but downstream projects can use it to set global overrides for Pigweed's build arguments. This is done by defining a "default_args" scope containing the overrides, such as configuring the protobuf libraries used. An example of

The entrypoint to a GN build is the .gn file, which defines a project’s root directory (henceforth //).

.gn must point to the location of a BUILDCONFIG.gn file for the project. In Pigweed upstream, this is its only purpose.

Downstream projects may additionally use .gn to set global overrides for Pigweed’s build arguments, which apply across all Pigweed targets. For example, a project could configure the protobuf libraries that it uses. This is done by defining a default_args scope containing the overrides.

# The location of the BUILDCONFIG file.
buildconfig = "//BUILDCONFIG.gn"

# Build arguments set across all Pigweed targets.
default_args = {
  dir_pw_third_party_nanopb = "//third_party/nanopb-0.4.2"
}

Configuration: BUILDCONFIG.gn#

Pigweed AI summary: The BUILDCONFIG.gn file configures the GN build by defining global variables and options. It is conventionally placed at the root of the build tree and evaluated before any other GN files. Variables defined within it are placed into GN's global scope and are available in every file without requiring imports. BUILDCONFIG.gn is evaluated for every GN toolchain, allowing it to set different options for each Pigweed target. It also allows defining arbitrary variables, unlike .gn files

The file BUILDCONFIG.gn configures the GN build by defining any desired global variables/options. It can be located anywhere in the build tree, but is conventionally placed at the root. .gn points GN to this file.

BUILDCONFIG.gn is evaluated before any other GN files, and variables defined within it are placed into GN’s global scope, becoming available in every file without requiring imports.

The options configured in this file differ from those in .gn in two ways:

BUILDCONFIG.gn is evaluated for every GN toolchain (and Pigweed target), whereas .gn is only evaluated once. This allows BUILDCONFIG.gn to set different options for each Pigweed target.
In .gn, only GN build arguments can be overridden. BUILDCONFIG.gn allows defining arbitrary variables.

Generally, it is preferable to expose configuration options through build args instead of globals in BUILDCONFIG.gn (something Pigweed’s build previously did), as they are more flexible, greppable, and easier to manage. However, it may make sense to define project-specific constants in BUILDCONFIG.gn.

Pigweed’s upstream BUILDCONFIG.gn does not define any variables; it just sets Pigweed’s default toolchain, which GN requires.

Top-level GN targets: //BUILD.gn#

Pigweed AI summary: The root BUILD.gn file in a Pigweed project defines all the libraries, images, tests, and binaries built by the project. The file is evaluated immediately after BUILDCONFIG.gn, with the active toolchain. The BUILD.gn file is responsible for enumerating each of the Pigweed targets built by a project. Pigweed projects are recommended to create a top-level group for each of their Pigweed targets that builds a common GN target with the appropriate toolchain. The root

The root BUILD.gn file defines all of the libraries, images, tests, and binaries built by a Pigweed project. This file is evaluated immediately after BUILDCONFIG.gn, with the active toolchain (which is the default toolchain at the start of a build).

//BUILD.gn is responsible for enumerating each of the Pigweed targets built by a project. This is done by instantiating a version of each of the project’s GN target groups with each Pigweed target’s toolchain. For example, in upstream, all of Pigweed’s GN targets are contained within the pigweed_default group. This group is instantiated multiple times, with different Pigweed target toolchains.

These groups include the following:

host – builds pigweed_default with Clang or GCC, depending on the platform
host_clang – builds pigweed_default for the host with Clang
host_gcc – builds pigweed_default for the host with GCC
stm32f429i – builds pigweed_default for STM32F429i Discovery board
docs – builds the Pigweed documentation and size reports

Pigweed projects are recommended to follow this pattern, creating a top-level group for each of their Pigweed targets that builds a common GN target with the appropriate toolchain.

It is important that no dependencies are listed under the default toolchain within //BUILD.gn, as it does not configure any build parameters, and therefore should not evaluate any other GN files. The pattern that Pigweed uses to achieve this is to wrap all dependencies within a condition checking the toolchain.

group("my_application_images") {
  deps = []  # Empty in the default toolchain.

  if (current_toolchain != default_toolchain) {
    # This is only evaluated by Pigweed target toolchains, which configure
    # all of the required options to build Pigweed code.
    deps += [ "//images:evt" ]
  }
}

# The images group is instantiated for each of the project's Pigweed targets.
group("my_pigweed_target") {
  deps = [ ":my_application_images(//toolchains:my_pigweed_target)" ]
}

Warning

Pigweed’s default toolchain is never used, so it is set to an empty toolchain which doesn’t define any tools. //BUILD.gn contains conditions which check that the current toolchain is not the default before declaring any GN target dependencies to prevent the default toolchain from evaluating any other BUILD files. All GN targets added to the build must be placed under one of these conditional scopes.

“default” group#

Pigweed AI summary: The BUILD.gn file can have a group named "default" which will be built by Ninja when no arguments are given. Defining this group makes using "pw watch" easier.

The root BUILD.gn file can define a special group named default. If present, Ninja will build this group when invoked without arguments.

Tip

Defining a default group makes using pw watch simple!

Optimization levels#

Pigweed AI summary: This section discusses the optimization levels in Pigweed's //BUILD.gn file. The pw_DEFAULT_C_OPTIMIZATION_LEVEL build argument specifies the optimization level for default groups, with supported values including debug, size_optimized, and speed_optimized. Pigweed also defines versions of its groups for each optimization level in the pw_C_OPTIMIZATION_LEVELS list, allowing for direct building at the desired optimization level. Examples of group names with optimization levels include host_clang_debug and stm32f429i

Pigweed’s //BUILD.gn defines the pw_DEFAULT_C_OPTIMIZATION_LEVEL build arg, which specifies the optimization level to use for the default groups (host, stm32f429i, etc.). The supported values for pw_DEFAULT_C_OPTIMIZATION_LEVEL are:

debug – create debugging-friendly binaries (-Og)
size_optimized – optimize for size (-Os)
speed_optimized – optimized for speed, without increasing code size (-O2)

Pigweed defines versions of its groups in //BUILD.gn for each optimization level specified in the pw_C_OPTIMIZATION_LEVELS list. Rather than relying on pw_DEFAULT_C_OPTIMIZATION_LEVEL, you may directly build a group at the desired optimization level: <group>_<optimization>. Examples include host_clang_debug, host_gcc_size_optimized, and stm32f429i_speed_optimized.

Upstream GN target groups#

Pigweed AI summary: This document describes the logical groups of GN targets used in Pigweed's upstream development. The groups include applications, host tools, runtime sanitizers, coverage, Pigweed modules, Pigweed module tests, and Pigweed default. The document also provides information on how to build each group and lists the supported runtime sanitizers. Downstream projects are advised not to use Pigweed's top-level BUILD.gn file but instead use the pw_modules and pw_module_tests variables in modules.gni to pull all of

In upstream, Pigweed splits its top-level GN targets into a few logical groups, which are described below. In order to build a GN target, it must be listed in one of the groups in this file.

Important

Pigweed’s top-level BUILD.gn file should not be used by downstream projects. Projects that wish to pull all of Pigweed’s code into their build may use the pw_modules and pw_module_tests variables in modules.gni.

apps#

Pigweed AI summary: The "apps" section in Pigweed defines the application images built, including common images for all Pigweed targets and specific images for each target through the "pw_TARGET_APPLICATIONS" build argument. This group includes modules' example executables.

This group defines the application images built in Pigweed. It lists all of the common images built across all Pigweed targets, such as modules’ example executables. Each Pigweed target can additionally provide its own specific images through the pw_TARGET_APPLICATIONS build arg, which is included by this group.

host_tools#

Pigweed AI summary: The "host_tools" section pertains to the host-side tooling binaries that are built for Pigweed.

This group defines host-side tooling binaries built for Pigweed.

runtime_sanitizers#

Pigweed AI summary: The "runtime_sanitizers" group in Clang defines host-side build targets for various runtime sanitizers, including AddressSanitizer, MemorySanitizer, UndefinedBehaviorSanitizer, and ThreadSanitizer. The UndefinedBehaviorSanitizer also has additional heuristic checks for integer behavior, floating point division by zero, implicit conversions, and nullability. Building this group results in "host_clang_<sanitizer>" build directories with "pw_module_tests" per supported sanitizer. Links to documentation for each sanitizer are

This group defines host-side build targets for Clang runtime sanitizers. Next runtime sanitizers supported:

asan – AddressSanitizer is a fast memory error detector.
msan – MemorySanitizer is a detector of uninitialized reads.
ubsan – UndefinedBehaviorSanitizer is a fast undefined behavior detector.
ubsan_heuristic – UndefinedBehaviorSanitizer with the following additional checks enabled:
- integer: Checks for undefined or suspicious integer behavior.
- float-divide-by-zero: Checks for floating point division by zero.
- implicit-conversion: Checks for suspicious behavior of implicit conversions.
- nullability: Checks for null as function arg, lvalue and return type.
These additional checks are heuristic and may not correspond to undefined behavior.
tsan – ThreadSanitizer is a tool that detects data races.

Results of building this group are host_clang_<sanitizer> build directories with pw_module_tests per supported sanitizer.

coverage#

Pigweed AI summary: This section titled "coverage" defines the host-side build target for Clang source-based code coverage.

This group defines host-side build target for Clang source-based code coverage.

pw_modules#

Pigweed AI summary: The "pw_modules" section contains a list of the main libraries for all of Pigweed's modules. These modules are listed in the "pw_modules" variable, which is provided by "modules.gni".

This group lists the main libraries for all of Pigweed’s modules.

The modules in the pw_modules group are listed in the pw_modules variable, which is provided by modules.gni.

pw_module_tests#

Pigweed AI summary: This paragraph explains that all unit tests for modules are located in the "pw_module_tests" section, allowing them to be run simultaneously. The test groups within this section are listed in the "pw_module_tests" variable provided by "modules.gni".

All modules’ unit tests are collected here, so that they can all be run at once.

The test groups in pw_module_tests group are listed in the pw_module_tests variable, which is provided by modules.gni.

pigweed_default#

Pigweed AI summary: The "pigweed_default" group is responsible for defining everything built in a Pigweed build invocation by collecting other groups and depending on them based on the active Pigweed target's configuration. It is recommended to not add new dependencies to this group and instead use one of the groups listed above. This group is instantiated for each upstream Pigweed target's toolchain.

This group defines everything built in a Pigweed build invocation by collecting the above groups and conditionally depending on them based on the active Pigweed target’s configuration. Generally, new dependencies should not be added here; instead, use one of the groups listed above.

The pigweed_default group is instantiated for each upstream Pigweed target’s toolchain.

Pigweed target instantiations#

Pigweed AI summary: This section discusses Pigweed target instantiations, which are groups that wrap pigweed_default with a specific target toolchain. They are named after the Pigweed target, such as host_clang and stm32f429i.

These groups wrap pigweed_default with a specific target toolchain. They are named after the Pigweed target, e.g. host_clang, stm32f429i, etc.

Other BUILD files: //**/BUILD.gn#

Pigweed AI summary: This section discusses the other BUILD.gn files in a Pigweed project, which define libraries, configurations, and build arguments for each module.

The rest of the BUILD.gn files in the tree define libraries, configs, and build args for each of the modules in a Pigweed project.

Project configuration: //build_overrides/pigweed.gni#

Pigweed AI summary: This section discusses the requirement for a Pigweed configuration file in each Pigweed project, which must be located in a specific location in the GN build tree. The file currently only contains one build argument, which must be set to the GN build path to the root of the Pigweed repository within the project.

Each Pigweed project must contain a Pigweed configuration file at a known location in the GN build tree. Currently, this file only contains a single build argument, which must be set to the GN build path to the root of the Pigweed repository within the project.

Module variables#

Pigweed AI summary: Pigweed is designed to be a subcomponent of a larger project, so it cannot assume where it or its modules are located. Therefore, Pigweed's upstream BUILD.gn files use variables defined relative to a project-specific "dir_pigweed" instead of absolute paths. To depend on Pigweed modules from GN code, import Pigweed's overrides file and reference these module variables. The import statement must be separated by a blank line from other imports to prevent gn format from reordering it.

As Pigweed is intended to be a subcomponent of a larger project, it cannot assume where it or its modules is located. Therefore, Pigweed’s upstream BUILD.gn files do not use absolute paths; instead, variables are defined pointing to each of Pigweed’s modules, set relative to a project-specific dir_pigweed.

To depend on Pigweed modules from GN code, import Pigweed’s overrides file and reference these module variables.

# This must be imported before .gni files from any other Pigweed modules. To
# prevent gn format from reordering this import, it must be separated by a
# blank line from other imports.

import("//build_overrides/pigweed.gni")

GN target type wrappers#

Pigweed AI summary: Pigweed has defined wrappers around GN target types like source_set and executable to make it easier to inject global configuration options. These wrappers are located in $dir_pw_build/target_types.gni. To fully utilize Pigweed's target configuration system, it is recommended to use pw_* target types instead of GN builtins in your BUILD.gn files.

To facilitate injecting global configuration options, Pigweed defines wrappers around builtin GN target types such as source_set and executable. These are defined within $dir_pw_build/target_types.gni.

Note

To take advantage of Pigweed’s flexible target configuration system, use pw_* target types (e.g. pw_source_set) in your BUILD.gn files instead of GN builtins.

Pigweed targets#

Pigweed AI summary: Pigweed targets are GN toolchains that are used to build for a specific hardware platform. These targets set special arguments for Pigweed to build. For more information on Pigweed's target system, refer to the Hardware targets documentation.

To build for a specific hardware platform, builds define Pigweed targets. These are essentially GN toolchains which set special arguments telling Pigweed how to build. For information on Pigweed’s target system, refer to Hardware targets.

The empty toolchain#

Pigweed AI summary: Pigweed's default toolchain is set to an "empty" toolchain that doesn't specify any compilers or override build arguments. Downstream projects are advised to do the same and configure builds for each of their Pigweed targets. The use of an empty toolchain is necessary for Pigweed's advanced build features that require the ability to generate new toolchains on the fly. It also avoids inconsistencies with GN's treatment of default and non-default toolchains. While it's possible to build Pigweed

Pigweed’s BUILDCONFIG.gn sets the project’s default toolchain to an “empty” toolchain which does not specify any compilers or override any build arguments. Downstream projects are recommended to do the same, following the steps described in Top-level GN targets: //BUILD.gn to configure builds for each of their Pigweed targets.

Why use an empty toolchain?

To support some of its advanced (and useful!) build features, Pigweed requires the ability to generate new toolchains on the fly. This requires having knowledge of the full configuration of the current toolchain (which is easy if it’s all defined within a scope), something which is impractical to achieve using the default toolchain.

Additionally, there are some cases where GN treats default and non-default toolchains differently. By not using the default toolchain, we avoid having to deal with these inconsistencies.

It is possible to build Pigweed using only the default toolchain, but it requires a more complicated setup to get everything working and should be avoided unless necessary (for example, when integrating with a large existing GN-based project).

Upstream development examples#

Pigweed AI summary: This section provides examples of building for upstream Pigweed, specifically for creating a custom executable/app image. The process involves defining an executable GN target using the pw_executable template, adding it to the apps group in the root BUILD.gn file, and running the ninja build to compile the executable for every supported Pigweed target. The compiled binary can then be retrieved from the out directory using a specific path format.

If developing for upstream Pigweed, some common build use cases are described below.

Building a custom executable/app image#

Pigweed AI summary: This section provides instructions for building a custom executable/app image using the GN target and ninja build. The process involves defining the executable GN target using the pw_executable template, adding the target to the apps group in the root BUILD.gn file, and running the ninja build to compile the executable. The compiled binary can be retrieved from the out directory using a specific path format that includes the Pigweed target, GN path, and executable name.

Define your executable GN target using the pw_executable template.

# //foo/BUILD.gn
pw_executable("foo") {
  sources = [ "main.cc" ]
  deps = [ ":libfoo" ]
}

In the root BUILD.gn file, add the executable’s GN target to the apps group.

# //BUILD.gn
group("apps") {
  deps = [
    # ...
    "//foo",  # Shorthand for //foo:foo
  ]
}

Run the ninja build to compile your executable. The apps group is built by default, so there’s no need to provide a target. The executable will be compiled for every supported Pigweed target.
```
ninja -C out
```
Alternatively, build your executable by itself by specifying its path to Ninja. When building a GN target manually, the Pigweed target for which it is built must be specified on the Ninja command line.

For example, to build for the Pigweed target host_gcc_debug:
```
ninja -C out host_gcc_debug/obj/foo/bin/foo
```
Note

The path passed to Ninja is a filesystem path within the out directory, rather than a GN path. This path can be found by running gn outputs.
Retrieve your compiled binary from the out directory. It is located at the path
```
out/<pw_target>/obj/<gn_path>/{bin,test}/<executable>
```
where pw_target is the Pigweed target for which the binary was built, gn_path is the GN path to the BUILD.gn file defining the executable, and executable is the executable’s GN target name (potentially with an extension). Note that the executable is located within a bin subdirectory in the module (or test for unit tests defined with pw_test).

For example, the foo executable defined above and compiled for the Pigweed target stm32f429i_disc1_debug is found at:
```
out/stm32f429i_disc1_debug/obj/foo/bin/foo
```

CMake#

Pigweed AI summary: CMake is a popular tool used in C/C++ development and is widely used in various projects, including embedded devices. Pigweed offers CMake support for projects that already have a CMake build and want to integrate Pigweed modules.

A well-known name in C/C++ development, CMake is widely used by all kinds of projects, including embedded devices. Pigweed’s CMake support is provided primarily for projects that have an existing CMake build and wish to integrate Pigweed modules.

Bazel#

Pigweed AI summary: Bazel is an open source build system that was originally developed by Google for internal use. It has gained popularity in the open source community and is being used by both open source and proprietary projects. Its modular structure allows for customizable usage. However, Bazel support is still experimental and may not be stable, so users are advised to use GN for a stable set of build APIs.

The open source version of Google’s internal build system. Bazel has been growing in popularity within the open source world, as well as being adopted by various proprietary projects. Its modular structure makes it a great fit for à la carte usage.

Note

Bazel support is experimental and only for the brave for now. If you are looking for stable set of build API’s please use GN.

The Bazel build#

Pigweed AI summary: This section provides an overview of Pigweed's Bazel build structure and conventions. It explains how to use Bazel for Pigweed-based projects and provides recommendations. The section also mentions that it is not a guide on how to use Bazel and directs readers to the official Bazel reference for more information. It covers general usage of Bazel, building and testing with Bazel, test tag conventions, code coverage, configuration concepts in Bazel, and Pigweed's Bazel configuration. It also provides

This section describes Pigweed’s Bazel build structure, how it is used upstream, build conventions, and recommendations for Pigweed-based projects. While containing some details about how Bazel works in general, this section is not intended to be a guide on how to use Bazel. To learn more about the tool itself, refer to the official Bazel reference.

General usage#

Pigweed AI summary: The paragraph discusses the general usage of Bazel for targeting embedded platforms. It mentions the commonly used commands in Bazel, such as building, testing, and code coverage. It also provides instructions on how to build and test Bazel targets for specific Pigweed targets. Additionally, it explains how to run tests on an embedded target using Bazel's "--run_under" flag and mentions the test tag conventions used in Pigweed. The paragraph concludes by explaining how to use the code coverage functionality in Bazel

While described in more detail in the Bazel docs there a few Bazel features that are of particular importance when targeting embedded platforms. The most commonly used commands used in bazel are;

bazel build //your:target
bazel test //your:target
bazel coverage //your:target

Note

Code coverage support is only available on the host for now.

Building#

Pigweed AI summary: To build and test a Bazel target for a specific Pigweed target, such as stm32f429i-discovery, a slight variation is required. The command to use is "bazel build //your:target --platforms=@pigweed//pw_build/platforms:stm32f429i-disc1". For more information on creating your own platforms, refer to the Bazel platforms reference. Examples of constraints and platforms can be found in the ‘//pw_build/platforms’ and ‘

When it comes to building/testing your Bazel target for a specific Pigweed target (e.g. stm32f429i-discovery) a slight variation is required.

bazel build //your:target \
  --platforms=@pigweed//pw_build/platforms:stm32f429i-disc1

For more information on how to create your own platforms refer to the official Bazel platforms reference. You may also find helpful examples of constraints and platforms in the ‘//pw_build/platforms’ and ‘//pw_build/constraints’ directories.

Testing#

Pigweed AI summary: Running tests on an embedded target with Bazel is possible but experimental. The easiest way to do this is by using Bazel's '--run_under' flag. To make it work, create a Bazel target that uploads the elf to your Pigweed target, connects with your target using serial or other communication method, listens to the communication transport for keywords like "PASSED" or "FAIL", and returns 0 or 1 respectively. To run the test, use the command 'bazel

Running tests on an embedded target with Bazel is possible although support for this is experimental. The easiest way of achieving this at the moment is to use Bazel’s ‘–run_under’ flag. To make this work create a Bazel target (‘//your_handler’) that;

Takes a single argument (the path to the elf) and uploads the elf to your Pigweed target.
Connects with your target using serial or other communication method.
Listens to the communication transport for the keywords (“PASSED”, “FAIL”) and returns (0, 1) respectively if one of the keywords is intercepted. (This step assumes you are using the pw_unit_test package and it is configured for your target).

Run;

bazel test //your:test --platforms=//your/platform --run_under=//your_handler

Test tag conventions#

Pigweed AI summary: Pigweed follows the standard Bazel test tag conventions and also uses additional tags. The "integration" tag is used for large, slow integration tests in upstream Pigweed. These tests can be skipped using the "--test_tag_filters" option. For example, running "bazel test --test_tag_filters=-integration //..." will run all tests except for the integration tests.

Pigweed observes the standard Bazel test tag conventions. We also use the following additional tags:

integration: large, slow integration tests in upstream Pigweed are given the integration tag. You can skip running these tests using –test_tag_filters. For example,
```
bazel test --test_tag_filters=-integration //...
```
will run all tests except for these integration tests.

Code Coverage#

Pigweed AI summary: This section provides instructions on how to use the code coverage functionality in Bazel. The process involves adding lines to the ‘.bazelrc’ file, generating a combined lcov coverage report, and viewing the results using the command line utility ‘lcov’. The instructions are presented in a series of enumerated lists and code blocks.

Making use of the code coverage functionality in Bazel is straightforward.

Add the following lines to your ‘.bazelrc’.

coverage --experimental_generate_llvm_lcov
coverage --combined_report=lcov

Generate a combined lcov coverage report. This will produce a combined lcov coverage report at the path ‘bazel-out/_coverage/_coverage_report.dat’. e.g.

bazel coverage //pw_log/...

View the results using the command line utility ‘lcov’.

lcov --list bazel-out/_coverage/_coverage_report.dat

Configuration#

Pigweed AI summary: Bazel's configuration API consists of three primary concepts: selects, compatibility lists, and flags/build settings. Selects are used to specify different dependencies or source code based on the platform being targeted. Compatibility lists allow you to specify which platforms your targets are compatible with. Flags/build settings are useful for injecting dependencies from the command line without creating new constraints. Pigweed's Bazel configuration API is designed to be distributed across the repository and allows for easy configuration of dependencies and backends. The configuration system determines

Generally speaking there are three primary concepts that make up Bazel’s configuration API.

Selects
Compatibility lists
Flags/Build settings

Selects#

Pigweed AI summary: Selects in Bazel are used to specify different dependencies or sources based on the platform being targeted. This can be useful for creating platform-specific libraries. The Bazel selects reference provides more information on how to use selects in Bazel. An example of using selects in Bazel is shown in the provided code snippet.

Selects are useful for specifying different dependencies/source depending on the platform that is currently being targeted. For more information on this please see the Bazel selects reference. e.g.

pw_cc_library(
  name = "some_platform_dependant_library",
  deps = select({
    "@platforms//cpu:armv7e-m": [":arm_libs"],
    "//conditions:default": [":host_libs"],
  }),
)

Compatibility lists#

Pigweed AI summary: Compatibility lists in Bazel allow you to specify which platforms your targets are compatible with. You can use the target_compatible_with attribute to specify which platforms are supported and which are not. This feature allows you to create compatibility matrices without affecting your ability to build your entire repository for a given Pigweed target. For more information on how to use the target_compatible_with attribute, refer to the Bazel target_compatible_with reference.

Compatibility lists allow you to specify which platforms your targets are compatible with. Consider an example where you want to specify that a target is compatible with only a host os;

pw_cc_library(
  name = "some_host_only_lib",
  srcs = ["host.cc"],
  target_compatible_with = select({
    "@platforms//os:windows": [],
    "@platforms//os:linux": [],
    "@platforms//os:ios": [],
    "@platforms//os:macos": [],
    "//conditions:default": ["@platforms//:incompatible"],
  }),
)

In this case building from or for either Windows/Linux/Mac will be supported, but other OS’s will fail if this target is explicitly depended on. However if building with a wild card for a non-host platform this target will be skipped and the build will continue. e.g.

bazel build //... --platforms=@pigweed//pw_build/platforms:cortex_m0

This allows for you to easily create compatibility matricies without adversely affecting your ability build your entire repo for a given Pigweed target. For more detailed information on how to use the target_compatible_with attribute please see Bazel target_compatible_with reference.

Flags/build settings#

Pigweed AI summary: This section discusses the use of flags/build settings in Bazel, which allow for quick injection of dependencies from the command line without creating new constraints. A label_flag example is provided, where a default target can be overridden from the command line. More detailed information on flags/settings can be found in the Bazel config reference.

Flags/build settings are particularly useful in scenarios where you may want to be able to quickly inject a dependency from the command line but don’t necessarily want to create an entirely new set of constraints to use with a select statement.

Note

The scope for what is possible with build flags/settings goes well beyond what will be described here. For more detailed information on flags/settings please see Bazel config reference.

A simple example of when it is useful to use a label_flag is when you want to swap out a single dependency from the command line. e.g.

pw_cc_library(
  name = "some_default_io",
  srcs = ["default_io.cc"],
)

pw_cc_library(
  name = "some_other_io",
  srcs = ["other_io.cc"],
)

label_flag(
  name = "io",
  default_build_setting = ":some_default_io",
)

pw_cc_library(
  name = "some_target_that_needs_io",
  deps = [":io"],
)

From here the label_flag by default redirects to the target “:some_default_io”, however it is possible to override this from the command line. e.g.

bazel build //:some_target_that_needs_io --//:io=//:some_other_io

Pigweed’s Bazel configuration#

Pigweed AI summary: Pigweed's Bazel configuration API is designed to be distributed across the Pigweed repository and/or your downstream repository. It is the transpose of GN's centralized configuration API. The configuration interface is designed to start simple and then grow with your project. Currently, the only configurations available under the Bazel+Pigweed configuration API is the ability to switch facade backends. To set up your workspace with Pigweed, you need to fetch the Pigweed repository and add the necessary dependencies. The Pig

Pigweeds Bazel configuration API is designed to be distributed across the Pigweed repository and/or your downstream repository. If you are coming from GN’s centralized configuration API it might be useful to think about Pigweed+Bazel’s configuration as the transpose of GN’s configuration. The configuration interface that is supported by Pigweed is designed to start simple and then grow with your project.

Note

There are plans to extend the configuration API for Bazel. However, currently the only configurations that are available under the Bazel+Pigweed configuration API is the ability to switch facade backends. For more information on what this is please see the Facades section of Module Structure.

Consider a scenario that you are building a flight controller for a spacecraft. But have very little experience with Pigweed and you have just landed here. First things first you would;

Set up your WORKSPACE to fetch the Pigweeds repository. Then add the dependencies that you need from Pigweeds WORKSPACE.
Add a pigweed_config rule to your WORKSPACE, using Pigweed’s default configuration.

# WORKSPACE ...
load("//pw_build:target_config.bzl", "pigweed_config")

# Configure Pigweeds backend.
pigweed_config(
    name = "pigweed_config",
    build_file = "@pigweed//targets:default_config.BUILD",
)

Note

We are aware, that the experience of setting up your WORKSPACE file to work with pigweed is less than ideal. This API is under construction, but we are working on this!

Now to explain what is going on here. Housed under the “pigweed_config” remote repository is a set of configuration flags. These can be used to inject dependencies into facades to override the default backend.

Continuing on with our scenario, consider that you maybe want to try using the ‘//pw_chrono’ module. So you create a target in your repository like so;

# BUILD
pw_cc_library(
  name = "time_is_relative",
  srcs = ["relative_time_on_earth.cc"],
  deps = ["@pigweed//pw_chrono"],
)

Now this should work out of the box for any host operating system. e.g. Running;

bazel build //:time_is_relative

will produce a working library. But as your probably here because Pigweed offers a set of embedded libraries you might be interested in running your code on some random micro-controller/FPGA combined with an RTOS. For now let’s assume that by some coincidence you are using FreeRTOS and are happy to make use of our default ‘//pw_chrono’ backend for FreeRTOS. You could build the following with;

bazel build //:time_is_relative \
--platforms=@pigweed//pw_build/platforms:freertos

There is a fair bit to unpack here in terms of how our configuration system is determining which dependencies to choose for your build. The dependency tree (that is important for configuration) in a project such as this would look like.

@pigweed//pw_chrono:pw_chrono_facade <-----------.
 ^                                               |
 |                            @pigweed//pw_chrono_freertos:system_clock
 |                            (Actual backend)
 |                                               ^
 |                                               |
 |                            @pigweed//pw_chrono:system_clock_backend_multiplexer
 |                            Select backend based on OS:
 |                            [FreeRTOS (X), Embos ( ), STL ( ), Threadx ( )]
 |                                               ^
 |                                               |
@pigweed//pw_chrono  -------> @pigweed_config//:pw_chrono_system_clock_backend
 ^                            (Injectable)
 |
//:time_is_relative

So when evaluating this setup Bazel checks the dependencies for ‘//pw_chrono’ and finds that it depends on “@pigweed_config//:pw_chrono_system_clock_backend” which looks like this;

# pw_chrono config.
label_flag(
    name = "pw_chrono_system_clock_backend",
    build_setting_default = "@pigweed//pw_chrono:system_clock_backend_multiplexer",
)

Looking at the ‘build_setting_default’ we can see that by default it depends back on the target “@pigweed//pw_chrono:system_clock_backend_multiplexer”. If you only had one backend you could actually just change the ‘build_setting_default’ to point directly to your backend. However because we have four different backends we have to use the select semantics to choose the right one. In this case it looks like;

pw_cc_library(
  name = "system_clock_backend_multiplexer",
  visibility = ["@pigweed_config//:__pkg__"],
  deps = select({
      "@pigweed//pw_chrono_freertos:system_clock_backend":
          ["//pw_chrono_freertos:system_clock"],
      "@pigweed//pw_chrono_embos:system_clock_backend":
          ["//pw_chrono_embos:system_clock"],
      "@pigweed//pw_chrono_threadx:system_clock_backend":
          ["//pw_chrono_threadx:system_clock"],
      "//conditions:default": ["//pw_chrono_stl:system_clock"],
  }),
)

Intuitively you can see that the first option was selected, which terminates the configuration chain.

Continuing on with our scenario let’s say that you have read Module Structure and now want to implement your own backend for ‘//pw_chrono’ using a hardware RTC. In this case you would create a new directory ‘pw_chrono_my_hardware_rtc’. To ensure that your new backend compiles with the facade an easy and temporary way to override the dependency tree is to override the label flag in ‘@pigweed_config’. For example;

bazel build //:time_is_relative \
  --@pigweed_config//pw_chrono_system_clock_backend=//pw_chrono_my_hardware_rtc:system_clock

This temporarily modifies the build graph to look something like this;

@pigweed//pw_chrono:pw_chrono_facade <-----.
 ^                                         |
 |                      @your_workspace//pw_chrono_my_hardware_rtc:system_clock
 |                      (Actual backend)
 |                                         ^
 |                                         |
@pigweed//pw_chrono  -> @pigweed_config//:pw_chrono_system_clock_backend
 ^                      (Injectable)
 |
//:time_is_relative

Now while this is a nice temporary change, but you might find yourself in need of a more permanent configuration. Particularly if you want to override multiple different backends. In other words if you had several backends to override, that would translate to several different command line flags (one for each override). This problem further compounds as you have multiple Pigweed targets all requiring different combinations of different backends as you can’t even reuse your command line entries. Instead you would have to memorize the correct combination of backends for each of your targets.

So continuing on with our scenario, let’s say we add a backup microcontroller to our spacecraft. But this backup computer doesn’t have a hardware RTC. We still want to share the bulk of the code between the two computers but now we need two separate implementations for our pw_chrono facade. Let’s say we choose to keep the primary flight computer using the hardware RTC and switch the backup computer over to use Pigweed’s default FreeRTOS backend. In this case we might want to do something similar to @pigweed//pw_chrono:system_clock_backend_multiplexer and create selectable dependencies for the two different computers:

Create a constraint value corresponding to your custom backend:

# //pw_chrono_my_hardware_rtc/BUILD.bazel
constraint_value(
  name = "system_clock_backend",
  constraint_setting = "//pw_chrono:system_clock_constraint_setting",
)

Create a set of platforms that can be used to switch constraint values. For example:

# //platforms/BUILD.bazel
platform(
  name = "primary_computer",
  constraint_values = ["//pw_chrono_my_hardware_rtc:system_clock_backend"],
)

platform(
  name = "backup_computer",
  constraint_values = ["@pigweed//pw_chrono_freertos:system_clock_backend"],
)

Create a target multiplexer that will select the right backend depending on which computer you are using. For example:

# //pw_chrono/BUILD
load("//pw_build:pigweed.bzl", "pw_cc_library")

pw_cc_library(
  name = "system_clock_backend_multiplexer",
  deps = select({
    "//pw_chrono_my_hardware_rtc:system_clock_backend": [
      "//pw_chrono_my_hardware_rtc:system_clock",
    ],
    "@pigweed//pw_chrono_freertos:system_clock_backend": [
      "@pigweed//pw_chrono_freertos:system_clock",
    ],
    "//conditions:default": [
      "@pigweed//pw_chrono_stl:system_clock",
    ],
  }),
)

Add a build setting override for the pw_chrono_system_clock_backend label flag to your .bazelrc file that points to your new target multiplexer.
```
# //.bazelrc
build --@pigweed_config//:pw_chrono_system_clock_backend=@your_workspace//pw_chrono:system_clock_backend_multiplexer
```

Building your target now will result in slightly different build graph. For example, running;

bazel build //:time_is_relative --platforms=//platforms:primary_computer

Will result in a build graph that looks like;

@pigweed//pw_chrono:pw_chrono_facade <---.
 ^                                        |
 |                     @your_workspace//pw_chrono_my_hardware_rtc:system_clock
 |                     (Actual backend)
 |                                        ^
 |                                        |
 |                     @your_workspace//pw_chrono:system_clock_backend_multiplexer
 |                     Select backend based on OS:
 |                     [Primary (X), Backup ( ), Host only default ( )]
 |                                        ^
 |                                        |
@pigweed//pw_chrono -> @pigweed_config//:pw_chrono_system_clock_backend
 ^                     (Injectable)
 |
//:time_is_relative

Build system#

What’s in a build system?#

Simple toolchain configuration#

Multi-target builds#

Multi-language support#

Custom scripting#

See also#

Python#

Size reporting#

See also#

Documentation#

See also#

Unit testing#

Host-side unit tests#

Device-side unit tests#

See also#

Bonus: pw watch#

See also#

Pigweed’s build systems#

GN#

The GN build#

Entrypoint: .gn#

Configuration: BUILDCONFIG.gn#

Top-level GN targets: //BUILD.gn#

“default” group#

Optimization levels#

Upstream GN target groups#

apps#

host_tools#

runtime_sanitizers#

coverage#

pw_modules#

pw_module_tests#

pigweed_default#

Pigweed target instantiations#

Other BUILD files: //**/BUILD.gn#

Project configuration: //build_overrides/pigweed.gni#

Module variables#

GN target type wrappers#

Pigweed targets#

The empty toolchain#

Upstream development examples#

Building a custom executable/app image#

CMake#

Bazel#

The Bazel build#

General usage#

Building#

Testing#

Test tag conventions#

Code Coverage#

Configuration#

Selects#

Compatibility lists#

Flags/build settings#

Pigweed’s Bazel configuration#