Getting Started#

Pigweed AI summary: This paragraph provides an overview of the Pigweed development workflow. It mentions that Pigweed can be used as a library in existing projects and that the documentation and sample project provide guidance on how to do so. It also mentions that the Pigweed team is available to help with project setup through their chat room or mailing list. The paragraph then goes on to explain the express setup for Pigweed, which involves configuring the GN build and starting the watcher. It also provides instructions on how to make a test fail

This guide will walk you through the typical upstream development workflow.

Note

This documentation and the sample project show how to use Pigweed as a library in your existing project. Using Pigweed as the foundation for new projects is our intended use case, but you may need more guidance than this documentation provides to do that right now.

We’re happy to help you get your project setup; just drop in our chat room or send a note to the mailing list.

Express setup#

Pigweed AI summary: To quickly set up Pigweed, you can follow the express setup guide. This setup configures Pigweed's watcher for three targets: Host (Mac, Linux, or Windows), Device/STM32F429 (build only, with the option to run tests on the STM32F429I-DISC1 kit), and Docs (builds the Pigweed docs). To get started, make sure you have Git and Python installed and on your path. Then, clone Pigweed and bootstrap the

If you’d like to skip the detailed explanations, below is the shorter version of getting setup for Pigweed. If you run into trouble, look at the more in-depth guide below, starting at Prerequisites. The express setup configures Pigweed’s watcher for three targets to give a taste of Pigweed:

  1. Host - Mac, Linux, or Windows. Builds and runs tests

  2. Device/STM32F429 - Build only; Optionally, the STM32F429I-DISC1 kit to follow along later in the guide to run tests directly on said device(s)

  3. Docs - Builds the Pigweed docs

To get setup:

  1. Make sure you have Git and Python installed and on your path.

  2. Clone Pigweed and bootstrap the environment (compiler setup & more). Be patient, this step downloads ~1GB of LLVM, GCC, and other tooling.

    $ cd ~
$ git clone https://pigweed.googlesource.com/pigweed/pigweed
...
$ cd pigweed
$ source ./bootstrap.sh (On Linux & Mac)
$ bootstrap.bat         (On Windows)
...

  3. Configure the GN build.

    $ gn gen out
Done. Made 1047 targets from 91 files in 114ms

  4. Start the watcher. The watcher will invoke Ninja to build all the targets

    $ pw watch

 ▒█████▄   █▓  ▄███▒  ▒█    ▒█ ░▓████▒ ░▓████▒ ▒▓████▄
  ▒█░  █░ ░█▒ ██▒ ▀█▒ ▒█░ █ ▒█  ▒█   ▀  ▒█   ▀  ▒█  ▀█▌
  ▒█▄▄▄█░ ░█▒ █▓░ ▄▄░ ▒█░ █ ▒█  ▒███    ▒███    ░█   █▌
  ▒█▀     ░█░ ▓█   █▓ ░█░ █ ▒█  ▒█   ▄  ▒█   ▄  ░█  ▄█▌
  ▒█      ░█░ ░▓███▀   ▒█▓▀▓█░ ░▓████▒ ░▓████▒ ▒▓████▀

20200707 17:24:06 INF Starting Pigweed build watcher
20200707 17:24:06 INF Will build [1/1]: out
20200707 17:24:06 INF Attaching filesystem watcher to $HOME/wrk/pigweed/...
20200707 17:24:06 INF Triggering initial build...
...

  5. Congratulations, you’re ready to go! Now take Pigweed for a spin by making a test fail.

  6. With the watcher running in a separate window, edit pw_status/status_test.cc to make an expectation fail; for example, add EXPECT_EQ(0, 1); in a test.

  7. Save the file. Observe the watcher rebuild & retest, and fail. Restore the test if you feel like it.

  8. Open the generated docs in out/docs/gen/docs/html/index.html in your browser.

  9. Edit docs/getting_started.rst (this file!) and make any change. Save. See the watcher rebuild the docs. Reload your browser, and see the changes.

See below for equivalent Windows commands, and for more details on what each part does.

Note: After running bootstrap once, use source ./activate.sh (or activate.bat on Windows) to re-activate the environment without re-bootstrapping.

Prerequisites#

Pigweed AI summary: Most Linux installations should work without any additional manual installation of prerequisites. However, you may need to extend your system udev rules if you want to flash devices using OpenOCD. For macOS, you'll need to install XCode and relevant tools. If you encounter SSL certificate errors, try installing Python 3.11.3 and running a specific command. On Windows, you'll need to install Git, Python, and enable Developer Mode. Additionally, if you plan to flash devices with firmware,

Linux

Most Linux installations should work out of box, and not require any manual installation of prerequisites beyond basics like git and build-essential (or the equivalent for your distro).

To flash devices using OpenOCD, you may need to extend your system udev rules at /etc/udev/rules.d/. The OpenOCD repository has a good example udev rules file that includes many popular hardware debuggers.

macOS

To start using Pigweed on MacOS, you’ll need to install XCode. Download it via the App Store, then install the relevant tools from the command line.

$ xcode-select --install

If you get SSL certificate errors, try installing Python 3.11.3 <https://www.python.org/downloads/release/python-3113/> and then running /Applications/Python\ 3.11/Install\ Certificates.command.

Windows

To start using Pigweed on Windows, you’ll need to do the following:

  • Install Git. Git must be installed to run from the command line and third-party software or be added to PATH. Also, ensure that the Enable symbolic links option is selected.

  • Install Python.

  • Ensure that Developer Mode is enabled.

If you plan to flash devices with firmware, you’ll need to install OpenOCD and ensure it’s on your system path.

Bootstrap#

Pigweed AI summary: This section provides instructions on how to clone Pigweed and run the bootstrap to initialize the Pigweed virtual environment. The process may take several minutes to complete. The instructions are provided separately for Linux & macOS and Windows. A real-time demo is also included. Once completed, the user is ready to start using Pigweed.

Once you satisfied the prerequisites, you will be able to clone Pigweed and run the bootstrap that initializes the Pigweed virtual environment. The bootstrap may take several minutes to complete, so please be patient.

Linux & macOS

$ git clone https://pigweed.googlesource.com/pigweed/pigweed ~/pigweed
$ cd ~/pigweed
$ source ./bootstrap.sh

Windows

:: Run git commands from the shell you set up to use with Git during install.
> git clone https://pigweed.googlesource.com/pigweed/pigweed %HOMEPATH%\pigweed
> cd %HOMEPATH%\pigweed
> bootstrap.bat

Below is a real-time demo with roughly what you should expect to see as output:

build example using pw watch

Congratulations, you are now set up to start using Pigweed!

Activate your Pigweed Environment#

Pigweed AI summary: This section provides instructions on how to activate the Pigweed development environment in a new session after the initial setup process. The command to activate the environment is provided for Linux, macOS, and Windows. It is also noted that major changes may require re-running the bootstrap process.

After going through the initial setup process, your current terminal will be in the Pigweed development environment that provides all the tools you should need to develop on Pigweed. If you leave that session, you can activate the environment in a new session with the following command:

Linux & macOS

$ source ./activate.sh

Windows

> activate.bat

Some major changes may require triggering the bootstrap again, so if you run into host tooling changes after a pull it may be worth re-running bootstrap.

Build Pigweed for Host#

Pigweed AI summary: This section provides instructions on how to build Pigweed for the host using the GN/Ninja build system. It also recommends using simple, single directory build directories to avoid issues with finding source files while debugging and generating coverage reports. The section introduces the use of pw_watch to automatically build the directory and trigger a Ninja build whenever a source code file is saved. A demo of pw_watch in action is also provided.

Pigweed’s primary build system is GN/Ninja based. There are CMake and Bazel builds in-development, but they are incomplete and don’t have feature parity with the GN build. We strongly recommend you stick to the GN build system.

GN (Generate Ninja) just does what it says on the tin; GN generates Ninja build files.

The default GN configuration generates build files that allow you to build host binaries, device binaries, and upstream documentation all in one Ninja invocation.

Run GN as seen below:

$ gn gen out

Note

out is simply the directory the build files are saved to. Unless this directory is deleted or you desire to do a clean build, there’s no need to run GN again; just rebuild using Ninja directly.

Warning

Unless your build directory (the out in gn gen out) is exactly one directory away from the project root directory (the Pigweed repo root in this case), there will be issues finding source files while debugging and while generating coverage reports. This is due an issue in upstream LLVM reordering debug and coverage path mappings (tracked by b/278898014 and b/278906020). We recommend sticking to simple, single directory build directories for the time being.

Now that we have build files, it’s time to build Pigweed!

Now you could manually invoke the host build using ninja -C out every time you make a change, but that’s tedious. Instead, let’s use pw_watch.

Go ahead and start pw_watch:

$ pw watch

When pw_watch starts up, it will automatically build the directory we generated in out. Additionally, pw_watch watches source code files for changes, and triggers a Ninja build whenever it notices a file has been saved. You might be surprised how much time it can save you!

With pw watch running, try modifying pw_status/public/pw_status/status.h and watch the build re-trigger when you save the file.

See below for a demo of this in action:

build example using pw watch

Running Unit Tests#

Pigweed AI summary: The article explains that unit tests are automatically built and run when targeting the host, and they only output results when there's a failure. It suggests modifying a specific file to see a test failure and explains that running tests as part of the build isn't expensive because GN caches passing tests. The article also provides a command to manually run a specific test and notes that the compiler will default to either clang or gcc depending on the host OS.

Fun fact, you’ve been running the unit tests already! Ninja builds targeting the host automatically build and run the unit tests. Unit tests err on the side of being quiet in the success case, and only output test results when there’s a failure.

To see a test failure, modify pw_status/status_test.cc to fail by changing one of the strings in the “KnownString” test.

example test failure using pw watch

Running tests as part of the build isn’t particularly expensive because GN caches passing tests. Each time you build, only the tests that are affected (whether directly or transitively) by the code changes since the last build will be re-built and re-run.

Try running the pw_status test manually:

$ ./out/pw_strict_host_{clang,gcc}_debug/obj/pw_status/test/status_test

Depending on your host OS, the compiler will default to either clang or gcc.

Building for a Device#

Pigweed AI summary: The Pigweed "target" is a build configuration that includes a toolchain and default library configurations to result in binaries that run natively on the target. By default, the build invocation already builds for a device target in parallel with the host build. However, if you want to build just for the device, you can use the command "pw watch stm32f429i" or the Ninja invocation "ninja -C out stm32f429i".

A Pigweed “target” is a build configuration that includes a toolchain, default library configurations, and more to result in binaries that run natively on the target. With the default build invocation, you’re already building for a device target (the STMicroelectronics STM32F429I-DISC1) in parallel with the host build!

If you want to build JUST for the device, you can kick of watch with:

$ pw watch stm32f429i

This is equivalent to the following Ninja invocation:

$ ninja -C out stm32f429i

Running Tests on a Device#

Pigweed AI summary: Running tests on a device requires a few additional steps compared to running tests on a host. It is important to verify that tests pass on the device as well, as there may be unexpected bugs that can only be found through on-device testing. To run tests on a device, you need to connect the STM32F429I-DISC1 boards to your computer using the mini USB port. Pigweed will automatically detect the boards and distribute the tests across them. It is important to ensure that you have

While tests run automatically on the host, it takes a few more steps to get tests to run automatically on a device, too. Even though we’ve verified tests pass on the host, it’s crucial to verify the same with on-device testing. We’ve encountered some unexpected bugs that can only be found by running the unit tests directly on the device.

1. Connect Device(s)#

Pigweed AI summary: To use Pigweed, users need to connect one or more STM32F429I-DISC1 boards to their computer using the mini USB port on the board. Pigweed will automatically detect the boards and distribute the tests across them, resulting in faster tests with more boards. However, users may need to make some environment-specific updates to ensure they have permissions to use the USB device, such as updating udev rules and ensuring they are in the plugdev and dialout groups on Linux. An image

Connect any number of STM32F429I-DISC1 boards to your computer using the mini USB port on the board (not the micro USB). Pigweed will automatically detect the boards and distribute the tests across the devices. More boards = faster tests! Keep in mind that you may have to make some environment specific updates to ensure you have permissions to use the USB device. For example, on Linux you may need to update your udev rules and ensure you’re in the plugdev and dialout groups.

development boards connected via USB

2. Launch Test Server#

Pigweed AI summary: To run tests on multiple devices, Ninja sends test requests to a server running in the background. The server can be launched in another window using the command "stm32f429i_disc1_test_server" after activating the Pigweed environment. It is important to note that if any additional boards are attached or detached, the server will need to be relaunched.

To allow Ninja to run tests on an arbitrary number of devices, Ninja will send test requests to a server running in the background. Launch the server in another window using the command below (remember, you’ll need to activate the Pigweed environment first).

$ stm32f429i_disc1_test_server

Note: If you attach or detach any more boards to your workstation you’ll need to relaunch this server.

3. Configure GN#

Pigweed AI summary: To configure GN, enable a build argument specific to the stm32f429i-disc1 target, which tells GN to use the testing server. Append the line "pw_use_test_server = true" to the file that opens in your editor when running GN.

Tell GN to use the testing server by enabling a build arg specific to the stm32f429i-disc1 target.

$ gn args out
# Append this line to the file that opens in your editor to tell GN to run
# on-device unit tests.
pw_use_test_server = true

Done!#

Pigweed AI summary: This paragraph explains that when code changes are made and a build is triggered, all unit tests will be run on attached boards. The paragraph also mentions a demo that shows what this process looks like, which includes an image of the demo.

Whenever you make code changes and trigger a build, all the affected unit tests will be run across the attached boards!

See the demo below for an example of what this all looks like put together:

pw watch running on-device tests

Building the Documentation#

Pigweed AI summary: Pigweed has RST files that generate HTML documentation, which can be found at https://pigweed.dev/. The documentation is built as part of the default build invocation, and changes can be easily made and viewed. The rendered HTML documentation can be found at out/docs/gen/docs/html. The command to explicitly build the documentation is "ninja -C out docs".

In addition to the markdown documentation, Pigweed has a collection of information-rich RST files that are used to generate HTML documentation. All the docs are hosted at https://pigweed.dev/, and are built as a part of the default build invocation. This makes it easier to make changes and see how they turn out. Once built, you can find the rendered HTML documentation at out/docs/gen/docs/html.

You can explicitly build just the documentation with the command below.

$ ninja -C out docs

This concludes the introduction to developing for upstream Pigweed.

Building Tests Individually#

Pigweed AI summary: The article explains how to build tests individually using GN's built-in tool, gn outputs, which translates a GN build step into a Ninja build step. It is important to specify which target to build the test under and this can be done by appending the GN path to the target toolchain in parenthesis after the desired GN build step label. The article also provides an example of using xargs to turn this into a single command in macOS and Linux.

Sometimes it’s faster to incrementally build a single test target rather than waiting for the whole world to build and all tests to run. GN has a built-in tool, gn outputs, that will translate a GN build step into a Ninja build step. In order to build and run the right test, it’s important to explicitly specify which target to build the test under (e.g. host, SM32F529I-DISC1). This can be done by appending the GN path to the target toolchain in parenthesis after the desired GN build step label as seen in the example below.

$ gn outputs out "//pw_status:status_test.run(//targets/host/pigweed_internal:pw_strict_host_clang_debug)"
pw_strict_host_clang_debug/obj/pw_status/status_test.run.pw_pystamp

$ ninja -C out pw_strict_host_clang_debug/obj/pw_status/status_test.run.pw_pystamp
ninja: Entering directory `out'
[4/4] ACTION //pw_status:status_test.run(//targets/host/pigweed_internal:pw_strict_host_clang_debug)

The .run following the test target name is a sub-target created as part of the pw_test GN template. If you remove .run, the test will build but not attempt to run.

In macOS and Linux, xargs can be used to turn this into a single command:

$ gn outputs out "//pw_status:status_test.run(//targets/host/pigweed_internal:pw_strict_host_clang_debug)" | xargs ninja -C out

Next steps#

Pigweed AI summary: If you want to explore more of what Pigweed has to offer, you can check out the Module Guides. There is also a sample project available that demonstrates how to use Pigweed in your own project. Additionally, there was a talk given at Hackaday's 2021 supercon about Pigweed, and improvements have been made since then. If you want to set up Pigweed for your own project, there is currently limited documentation available, but the sample project can provide some guidance. If you

Check out other modules#

Pigweed AI summary: The paragraph suggests that if one wants to explore more of Pigweed's offerings, they should check out the Module Guides.

If you’d like to see more of what Pigweed has to offer, dive into the Module Guides.

Check out the sample project#

Pigweed AI summary: There is a sample project available that shows how to use Pigweed in your own project. It can be found at this link: [sample project](https://pigweed.googlesource.com/pigweed/sample_project/+/main/README.md). It is important to note that there are multiple ways to utilize Pigweed and the sample project is just one example.

We have a sample project that demonstrates how to use Pigweed in your own project. Note that there are many ways to leverage Pigweed and the sample project is one approach.

Check out the Hackaday Supercon talk about Pigweed#

Pigweed AI summary: The article discusses a talk given at Hackaday's 2021 supercon about Pigweed, an embedded unit testing library integration for the command line. The talk is titled "Give Pigweed a Whirl" and the article mentions that improvements have been made since the talk, including the addition of RTOS primitives.

We gave a talk at Hackaday’s 2021 supercon, Give Pigweed a Whirl

We’ve made improvements since we gave the talk; for example, we now have RTOS primitives.

Set up Pigweed for your own project#

Pigweed AI summary: The documentation for setting up Pigweed in a separate project is not yet comprehensive. The sample project provides some guidance on using Pigweed as a library in a broader project, but additional assistance may be required. The quickest way to get help is by joining the chat room, or you can reach out to the mailing list.

We don’t yet have thorough documentation for leveraging Pigweed in a separate project (our intended use case!). The sample project shows how to use Pigweed as a library in your broader project, but you may need further guidance.

Dropping into our chat room is the most immediate way to get help. Alternatively, you can send a note to the mailing list.