Automated analysis#

Pigweed AI summary: Pigweed's source code is continuously verified for correctness and style using a suite of automated tools. These tools can also be used to verify the code of projects using Pigweed. The tools used for verification include pylint, mypy, clang-tidy, AddressSanitizer (asan), ThreadSanitizer (tsan), UndefinedBehaviorSanitizer (ubsan), and OSS-Fuzz. The document provides detailed information on each of these tools and how to use them in your own project. Additionally, it

The correctness and style of Pigweed’s source code is continuously verified using a suite of automated tools. We also make it easy to use the same tools to verify the code of projects using Pigweed.

Summary#

Pigweed AI summary: The Pigweed project is verified using various tools such as pylint, mypy, clang-tidy, AddressSanitizer, ThreadSanitizer, UndefinedBehaviorSanitizer, and OSS-Fuzz during presubmit or in CI. The document provides a detailed discussion of these tools and their configuration, as well as how to use them in other projects.

On presubmit or in CI we verify Pigweed using:

  • pylint

  • mypy

  • clang-tidy

  • AddressSanitizer (asan)

  • ThreadSanitizer (tsan)

  • UndefinedBehaviorSanitizer (ubsan)

  • OSS-Fuzz

The rest of this document discusses these tools and their configuration in greater detail, and how to use them in your own project.

Analysis tools#

Pigweed AI summary: This section discusses various analysis tools used in Pigweed, including PyLint for Python code, Mypy for enforcing type annotations, clang-tidy for C++ code, and clang sanitizers for detecting memory errors, data races, and undefined behavior. The section also mentions fuzz testing with OSS-fuzz to uncover potential vulnerabilities in Pigweed. The tools are run with various presubmit steps and configurations, and the section provides links to documentation and resources for each tool.

Static analysis#

Pigweed AI summary: The article discusses various static analysis tools used in Pigweed, an open-source embedded development platform. PyLint is a customizable Python linter that detects problems such as unused arguments/variables and mutable default parameter values. Mypy enforces type annotations and helps find bugs like passing a string into a function that expects a list of strings. Clang-tidy is a C++ linter and static analysis tool that identifies bug-prone patterns and non-idiomatic usage. Pigweed enables over 50 relevant

PyLint#

Pigweed AI summary: PyLint is a customizable Python linter that detects problems such as overly broad catch statements, unused arguments/variables, and mutable default parameter values. It can be run with various presubmit steps and is included in Pigweed. To learn how to run PyLint on your Pigweed-based project, refer to the "Enabling analysis for your project" section.

PyLint is a customizable Python linter. Pigweed complies with almost all the default checks; see .pylintrc for details. PyLint detects problems such as overly broad catch statements, unused arguments/variables, and mutable default parameter values.

For upstream Pigweed, PyLint can be run with ninja python.lint.pylint or ninja python.lint. It’s also included in a variety of presubmit steps, like static_analysis and python_checks.gn_python_check. See the Enabling analysis for your project section to learn how to run PyLint on your Pigweed-based project.

Mypy#

Pigweed AI summary: Mypy is an analysis tool that enforces type annotations in Python 3 code, which can catch bugs that may not be caught by unit tests. It helps find bugs such as passing a string into a function that expects a list of strings. Mypy can be run with various presubmit steps and is included in the ninja python.lint command.

Python 3 allows for type annotations for variables, function arguments, and return values. Most, but not all, of Pigweed’s Python code has type annotations, and these annotations have caught real bugs in code that didn’t yet have unit tests. Mypy is an analysis tool that enforces these annotations.

Mypy helps find bugs like when a string is passed into a function that expects a list of strings—since both are iterables this bug might otherwise be hard to track down.

Mypy can be run with ninja python.lint.mypy or ninja python.lint. It’s also included in a variety of presubmit steps, like static_analysis and python_checks.gn_python_check.

clang-tidy#

Pigweed AI summary: Clang-tidy is a C++ static analysis tool that identifies bug-prone patterns, non-idiomatic usage, and performance issues. While powerful, it defines a large number of checks, many of which are special-purpose or have high false positive rates. Pigweed enables over 50 relevant checks for embedded C/C++ libraries with good signal-to-noise ratios. Clang Static Analyzers are not currently enabled due to false positives and time-consuming manual verification. Clang-tidy can be run

clang-tidy is a C++ “linter” and static analysis tool. It identifies bug-prone patterns (e.g., use after move), non-idiomatic usage (e.g., creating std::unique_ptr with new rather than std::make_unique), and performance issues (e.g., unnecessary copies of loop variables).

While powerful, clang-tidy defines a very large number of checks, many of which are special-purpose (e.g., only applicable to FPGA HLS code, or code using the Abseil library) or have high false positive rates. Pigweed enables over 50 checks which are relevant to an embedded C/C++ library and have good signal-to-noise ratios. The full list of Pigweed’s checks is in .clang-tidy.

We do not currently enable the Clang Static Analyzers because they suffer from false positives, and their findings are time-consuming to manually verify.

clang-tidy can be run with ninja static_analysis or pw presubmit --step static_analysis. Note that as a static analysis tool, clang-tidy will not produce any runnable binaries: it simply analyzes the source files.

Clang sanitizers#

Pigweed AI summary: This section discusses the use of Clang sanitizers in Pigweed's unit tests, including asan, msan, tsan, and ubsan, which detect memory errors, uninitialized memory, data races, and undefined behavior. The configurations used for these sanitizers are provided, and their current status can be viewed in the Pigweed CI console. Unlike clang-tidy, the Clang sanitizers require runtime instrumentation to detect issues. Pigweed does not currently support msan. More information about

We run all of Pigweed’s unit tests with the additional instrumentation described in this section. For more detail about these sanitizers, see the Github documentation.

Note

Pigweed does not currently support msan. See https://issuetracker.google.com/234876100 for details.

The exact configurations we use for these sanitizers are in pw_toolchain/host_clang/BUILD.gn. You can see the current status of the sanitizer builds in the Pigweed CI console, as pigweed-linux-san-*.

Unlike clang-tidy, the clang sanitizers are runtime instrumentation: the instrumented binary needs to be run for issues to be detected.

Fuzzers#

Pigweed AI summary: The Fuzzers section discusses the use of fuzz testing to detect errors in software by providing randomly generated inputs. The article mentions the use of OSS-fuzz to continuously uncover potential vulnerabilities in Pigweed and provides a link to a dashboard with the latest results. The pw_fuzzer module documentation is also referenced for more details.

Fuzz testing detects errors in software by providing it with randomly generated inputs. We use OSS-fuzz to continuously uncover potential vulnerabilities in Pigweed. Dashboard with Pigweed’s latest results. See the pw_fuzzer module documentation for more details.

Enabling analysis for your project#

Pigweed AI summary: This article discusses various tools and techniques for enabling analysis in your project. It covers PyLint and Mypy, which can be configured to run during the build process, as well as Clang-tidy, which can be enabled through GN args or by using specific toolchains. The article also briefly touches on fuzzers and provides links to relevant documentation.

PyLint and Mypy#

Pigweed AI summary: PyLint and Mypy can be set up to run during project builds by adding "python.lint" to the default build group, or individually using "python.lint.mypy" and "python.lint.pylint". They can also be added to individual presubmit steps, and the "python_checks.gn_python_lint" presubmit step can be directly included. Examples of this can be found at the provided links.

PyLint and Mypy can be configured to run every time your project is built by adding python.lint to your default build group. (You can also add one or both individually using python.lint.mypy and python.lint.pylint.) Likewise, these can be added to individual presubmit steps (examples). You can also directly include the python_checks.gn_python_lint presubmit step.

clang-tidy#

Pigweed AI summary: The article discusses how to use the pw_static_analysis_toolchain template to create a build group for static analysis using clang-tidy. It also mentions that a .clang-tidy file can be placed at the root of the repository to control which checks are executed. The article provides a reference to the clang documentation for more information on how the tool chooses which .clang-tidy files to apply.

pw_toolchain/static_analysis_toolchain.gni provides the pw_static_analysis_toolchain template that can be used to create a build group performing static analysis. See pw_toolchain documentation for more details. This group can then be added as a presubmit step using pw_presubmit.

You can place a .clang-tidy file at the root of your repository to control which checks are executed. See the clang documentation for a discussion of how the tool chooses which .clang-tidy files to apply when run on a particular source file.

Clang sanitizers#

Pigweed AI summary: This article explains how to enable Clang sanitizers for your build in two ways. The first method involves setting the gn arg "pw_toolchain_SANITIZERS" to the desired subset of ["address", "thread", "undefined"] if you are already building your tests with certain toolchains. The second method involves building your tests with the appropriate toolchain from a list of sanitizer toolchains if you are not using the aforementioned toolchains. The article also provides an example of how to run a

There are two ways to enable sanitizers for your build.

GN args on debug toolchains#

Pigweed AI summary: This article explains how to enable the clang sanitizers for tests built with certain toolchains. By setting the gn arg "pw_toolchain_SANITIZERS" to a desired subset of ["address", "thread", "undefined"], the sanitizers can be enabled. An example is provided for running the "pw_executable" target "sample_binary" with asan using the toolchain "host_clang_debug".

If you are already building your tests with one of the following toolchains (or a toolchain derived from one of them):

  • pw_toolchain_host_clang.debug

  • pw_toolchain_host_clang.speed_optimized

  • pw_toolchain_host_clang.size_optimized

you can enable the clang sanitizers simply by setting the gn arg pw_toolchain_SANITIZERS to the desired subset of ["address", "thread", "undefined"].

Example#

Pigweed AI summary: This paragraph provides instructions on how to run a specific target in a project's BUILD.gn file with asan using a defined toolchain called "host_clang_debug". The instructions involve running a command in the terminal using the "gn gen" and "ninja" commands.

If your project defines a toolchain host_clang_debug that is derived from one of the above toolchains, and you’d like to run the pw_executable target sample_binary defined in the BUILD.gn file in examples/sample with asan, you would run,

gn gen out --args='pw_toolchain_SANITIZERS=["address"]'
ninja -C out host_clang_debug/obj/example/sample/bin/sample_binary
out/host_clang_debug/obj/example/sample/bin/sample_binary

Sanitizer toolchains#

Pigweed AI summary: This section provides a list of toolchains that can be used to build tests instead of using "gn args". The toolchains listed are pw_toolchain_host_clang.asan, pw_toolchain_host_clang.ubsan, and pw_toolchain_host_clang.tsan. The documentation for Pigweed toolchains is also referenced.

Otherwise, instead of using gn args you can build your tests with the appropriate toolchain from the following list (or a toolchain derived from one of them):

  • pw_toolchain_host_clang.asan

  • pw_toolchain_host_clang.ubsan

  • pw_toolchain_host_clang.tsan

See the pw_toolchain module documentation for more about Pigweed toolchains.

Fuzzers#

Pigweed AI summary: This paragraph refers to the "Fuzzers" section and directs the reader to the "pw_fuzzer" module documentation.

See the pw_fuzzer module documentation.