Pigweed AI summary: The pw_fuzzer tool helps developers find bugs in their C++ code by using state-of-the-art tools to automatically detect them with just a few lines of code. Fuzzing is a complex process that requires interactions between compiler-added instrumentation, compiler runtimes, code under test, and the fuzzer code itself. pw_fuzzer makes it easier to write, build, run, and deploy fuzzers, and provides integration with two fuzzing engines: libFuzzer and FuzzTest. It is

Unstable C++14 C++17 C++20

Use state of the art tools to automatically find bugs in your C++ code with 5 lines of code or less!

FUZZ_TEST(MyTestSuite, TestMyInterestingFunction)
  .WithDomains(Arbitrary<size_t>(), AsciiString());

Background#

Pigweed AI summary: The article discusses the potential for bugs in code paths and inputs that unit tests may not cover, and introduces the concept of fuzzing as a solution. However, fuzzing requires complex interactions between various components, including compiler-added instrumentation and fuzzer code, and needs to be part of a continuous fuzzing infrastructure to be reliably useful. The article recommends reading "pw_fuzzer: Concepts" to learn more about the different components of a fuzzer and how they work together to discover hard-to-find bugs.

You’ve written some code. You’ve written unit tests for that code. The unit tests pass. But could there be bugs in inputs or code paths the unit tests do not cover? Fuzzing can help!

However, fuzzing requires some complex interactions between compiler-added instrumentation, compiler runtimes, code under test, and the fuzzer code itself. And to be reliably useful, fuzzers need to be part of a continuous fuzzing infrastructure, adding even more complexity.

See pw_fuzzer: Concepts to learn more about the different components of a fuzzer and how they work together to discover hard-to-find bugs.

Our solution#

Pigweed AI summary: The pw_fuzzer tool simplifies the process of writing, building, running, and deploying fuzzers. It integrates with two fuzzing engines, libFuzzer and FuzzTest, and can produce artifacts for continuous fuzzing infrastructures like ClusterFuzz and OSS-Fuzz. This tool makes it easy to create fuzzers from unit tests in just a few lines of code.

pw_fuzzer makes it easier to write, build, run, and deploy fuzzers. It provides convenient integration with two fuzzing engines:

  • libFuzzer, allowing integration of legacy fuzzers.

  • FuzzTest, allowing easy creation of fuzzers from unit tests in around 5 lines of code.

Additionally, it produces artifacts for continuous fuzzing infrastructures such as ClusterFuzz and OSS-Fuzz, and provides the artifacts those systems need.

Who this is for#

Pigweed AI summary: The "pw_fuzzer" tool is useful for C++ code authors who have written unit tests and want to find actionable bugs in their code. It is particularly effective for testing code that receives inputs from untrusted sources, has complex algorithms, or handles high volumes of inputs and unreliable dependencies. Fuzzing works best with code that handles inputs deterministically and resembles unit tests in terms of scope and isolation.

pw_fuzzer is useful for authors of wide range of C++ code who have written unit tests and are interested in finding actionable bugs in their code.

In particular, coverage-guided is effective for testing and finding bugs in code that:

  • Receives inputs from untrusted sources and must be secure.

  • Has complex algorithms with some equivalence, e.g. compress and decompress, and must be correct.

  • Handles high volumes of inputs and/or unreliable dependencies and must be stable.

Fuzzing works best when code handles inputs deterministically, that is, given the same input it behaves the same way. Fuzzing will be less effective with code that modifies global state or has some randomness, e.g. depends on how multiple threads are scheduled. Simply put, good fuzzers typically resemble unit tests in terms of scope and isolation.

Is it right for you?#

Pigweed AI summary: The "pw_fuzzer" tool currently only supports fuzzers that run on a host and are built with Clang. Sanitizer runtimes like AddressSanitizer are not suitable for running on a device due to significant memory overhead, and the supported engines assume the presence of certain POSIX features.

Currently, pw_fuzzer only provides support for fuzzers that:

  • Run on host. Sanitizer runtimes such as AddressSanitizer add significant memory overhead and are not suitable for running on device. Additionally, the currently supported engines assume the presence of certain POSIX features.

  • Are built with Clang. The instrumentation used in fuzzing is added by clang.

Getting started#

Pigweed AI summary: This paragraph discusses the first step in adding a fuzzer, which is determining which fuzzing engine to use. Pigweed currently supports two fuzzing engines: FuzzTest and libFuzzer. FuzzTest is the recommended engine as it makes it easy to create fuzzers from existing unit tests, but it requires additional third-party dependencies and at least C++17. On the other hand, libFuzzer is a mature, proven engine that is a part of LLVM and requires code authors to implement

The first step in adding a fuzzer is to determine what fuzzing engine should you use. Pigweed currently supports two fuzzing engines:

  • FuzzTest is the recommended engine. It makes it easy to create fuzzers from your existing unit tests, but it does requires additional third party dependencies and at least C++17. See pw_fuzzer: Adding Fuzzers Using FuzzTest for details on how to set up a project to use FuzzTest and on how to create and run fuzzers with it.

  • libFuzzer is a mature, proven engine. It is a part of LLVM and requires code authors to implement a specific function, LLVMFuzzerTestOneInput. See pw_fuzzer: Adding Fuzzers Using LibFuzzer for details on how to write fuzzers with it.

Roadmap#

Pigweed AI summary: The pw_fuzzer is currently being developed with plans to add improvements such as documenting workflows for analyzing coverage and improving fuzzers, adding CMake support for FuzzTest, developing an OSS-Fuzz and ClusterFuzz workflow templates for downstream projects, and adding a pw_cli plugin for fuzzing. The section also includes various targets for further reading on topics such as fuzzing, instrumentation, and engines.

pw_fuzzer is under active development. There are a number of improvements we would like to add, including:

  • b/282560789 - Document workflows for analyzing coverage and improving fuzzers.

  • b/280457542 - Add CMake support for FuzzTest.

  • b/281138993 - Add a pw_cli plugin for fuzzing.

  • b/281139237 - Develop OSS-Fuzz and ClusterFuzz workflow templates for downtream projects.