pw_docgen#

Pigweed AI summary: The docgen module provides tools to generate documentation for Pigweed-based projects, as well as for Pigweed itself. It allows for the combination of all relevant documentation into one place, making it easier for downstream consumers to access. The documentation generation is integrated into the build system, allowing for automatic regeneration whenever the code is changed. Pigweed documentation files are written in reStructuredText format and rendered to HTML using Sphinx. The module also includes Sphinx extensions for rendering diagrams and generating module metadata. The documentation generation

The docgen module provides tools to generate documentation for Pigweed-based projects, and for Pigweed itself.

Pigweed-based projects typically use a subset of Pigweed’s modules and add their own product-specific modules on top of that, which may have product-specific documentation. Docgen provides a convenient way to combine all of the relevant documentation for a project into one place, allowing downstream consumers of release bundles (e.g. factory teams, QA teams, beta testers, etc.) to have a unified source of documentation early on.

The documentation generation is integrated directly into the build system. Any build target can depend on documentation, which allows it to be included as part of a factory release build, for example. Additionally, documentation itself can depend on other build targets, such as report cards for binary size/profiling. Any time the code is changed, documentation will be regenerated with the updated reports.

Documentation Overview#

Pigweed AI summary: The Pigweed module offers documentation that explains its features, usage, and programming API. The documentation also includes report cards that provide an overview of the module's size, cost, and performance benchmarks. These report cards help potential users assess the impact of incorporating the module into their projects.

Each Pigweed module provides documentation describing its functionality, use cases, and programming API.

Included in a module’s documentation are report cards which show an overview of the module’s size cost and performance benchmarks. These allow prospective users to evaluate the impact of including the module in their projects.

Build Integration#

Pigweed AI summary: Pigweed documentation files are written in reStructuredText format and rendered to HTML using Sphinx through Pigweed’s GN build system. Additional Sphinx plugins are used for rendering diagrams within reStructuredText files. Documentation source and asset files are placed alongside code within a module and registered as a pw_doc_group target within a BUILD.gn file. The pw_doc_gen template creates a target which renders complete HTML documentation for a project. All source files listed under a pw_doc_gen target and its pw_doc_group dependencies

Pigweed documentation files are written in reStructuredText format and rendered to HTML using Sphinx through Pigweed’s GN build system.

There are additonal Sphinx plugins used for rendering diagrams within reStructuredText files including:

mermaid via the sphinxcontrib-mermaid package.

Documentation source and asset files are placed alongside code within a module and registered as a pw_doc_group target within a BUILD.gn file. These groups become available for import within a special documentation generation target, which accumulates all of them and renders the resulting HTML. This system can either be used directly within Pigweed, or integrated into a downstream project.

GN Templates#

Pigweed AI summary: The GN Templates section provides information on two templates used for defining documentation files: pw_doc_group and pw_doc_gen. The pw_doc_group template is used to group documentation source files and assets, and each Pigweed module is expected to provide at least one pw_doc_group target defining the module's documentation. The pw_doc_gen template creates a target that renders complete HTML documentation for a project, depending on registered pw_doc_group targets and requiring a conf.py file and a top-level index.rst file. Both

pw_doc_group#

Pigweed AI summary: The "pw_doc_group" template is used to group documentation source files and assets for each Pigweed module. Each module should have at least one "pw_doc_group" target, which can depend on other groups. The template has several arguments, including sources, inputs, group_deps, report_deps, and other_deps. An example of using the template is provided.

The main template for defining documentation files is pw_doc_group. It is used to logically group a collection of documentation source files and assets. Each Pigweed module is expected to provide at least one pw_doc_group target defining the module’s documentation. A pw_doc_group can depend on other groups, causing them to be built with it.

Arguments

sources: RST documentation source files.
inputs: Additional resources required for the docs (images, data files, etc.)
group_deps: Other pw_doc_group targets required by this one.
report_deps: Report card generating targets (e.g. pw_size_diff) on which the docs depend.
other_deps: Any other GN targets that should be run before this pw_doc_group runs that is not included in one of the above dep categories.

Example

pw_doc_group("my_doc_group") {
  sources = [ "docs.rst" ]
  inputs = [ "face-with-tears-of-joy-emoji.svg" ]
  group_deps = [ ":sub_doc_group" ]
  report_deps = [ ":my_size_report" ]
}

pw_doc_gen#

Pigweed AI summary: The pw_doc_gen template is used to create a target that generates complete HTML documentation for a project. It requires a conf.py file and a top-level index.rst file to tie everything together. The template has several arguments, including conf, index, output_directory, deps, and python_metadata_deps. An example of how to use the template is provided in the code block.

The pw_doc_gen template creates a target which renders complete HTML documentation for a project. It depends on registered pw_doc_group targets and creates an action which collects and renders them.

To generate the complete docs, the template also requires a conf.py file configuring Sphinx’s output, and a top level index.rst for the main page of the documentation. These are added at the root level of the built documentation to tie everything together.

Arguments

conf: Path to the conf.py to use for Sphinx.
index: Path to the top-level index.rst file.
output_directory: Directory in which to render HTML output.
deps: List of all pw_doc_group targets required for the documentation.
python_metadata_deps: Python-related dependencies that are only used as deps for generating Python package metadata list, not the overall documentation generation. This should rarely be used by non-Pigweed code.

Example

pw_doc_gen("my_docs") {
  conf = "//my_docs/conf.py"
  index = "//my_docs/index.rst"
  output_directory = target_gen_dir
  deps = [
    "//my_module:my_doc_group",
  ]
}

Generating Documentation#

Pigweed AI summary: This section explains how documentation is generated in Pigweed. All source files listed under a "pw_doc_gen" target and its "pw_doc_group" dependencies are copied into a directory structure that mirrors the original layout of the modules in which the sources appear. The documentation tree created in this way is passed to Sphinx to build HTML output. Relative imports within documentation files must be relative to this structure, except for the top-level "index.rst" file's imports, which must start from the project's

All source files listed under a pw_doc_gen target and its pw_doc_group dependencies get copied out into a directory structure mirroring the original layout of the modules in which the sources appear. This is demonstrated below using a subset of Pigweed’s core documentation.

Consider the following target in $dir_pigweed/docs/BUILD.gn:

pw_doc_gen("docs") {
  conf = "conf.py"
  index = "index.rst"
  output_directory = target_gen_dir
  deps = [
    "$dir_pw_bloat:docs",
    "$dir_pw_docgen:docs",
    "$dir_pw_preprocessor:docs",
  ]
}

A documentation tree is created under the output directory. Each of the sources and inputs in the target’s dependency graph is copied under this tree in the same directory structure as they appear under the root GN build directory ($dir_pigweed in this case). The conf.py and index.rst provided directly to the pw_doc_gen template are copied in at the root of the tree.

out/gen/docs/pw_docgen_tree/
├── conf.py
├── index.rst
├── pw_bloat
│   ├── bloat.rst
│   └── examples
│       └── simple_bloat.rst
├── pw_docgen
│   └── docgen.rst
└── pw_preprocessor
    └── docs.rst

This is the documentation tree which gets passed to Sphinx to build HTML output. Imports within documentation files must be relative to this structure. In practice, relative imports from within modules’ documentation groups are identical to the project’s directory structure. The only special case is the top-level index.rst file’s imports; they must start from the project’s build root.

Sphinx Extensions#

Pigweed AI summary: This paragraph discusses the Sphinx Extensions module in Pigweed's documentation generator. It explains how to use the pigweed-module directive to register module metadata and provides an example. It also lists the options available for the directive. Additionally, it mentions the google_analytics extension, which adds a Google Analytics tracking tag to the documentation pages. The paragraph concludes with instructions on how to debug Pigweed's Sphinx extensions using pdb.

This module houses Pigweed-specific extensions for the Sphinx documentation generator. Extensions are included and configured in docs/conf.py.

module_metadata#

Pigweed AI summary: The Pigweed module documentation follows a standard format outlined in SEED-0102. The "pigweed-module" Sphinx directive is used to register module metadata that can be used in the Sphinx build. To achieve title and subtitle formatting, the directive should be added after the document title and a class should be added to the document title. The directive options include specifying the module name, a short tagline, the module's status (experimental, unstable, or stable), whether the module is deprecated, the

Per SEED-0102, Pigweed module documentation has a standard format. The pigweed-module Sphinx directive provides that format and registers module metadata that can be used elsewhere in the Sphinx build.

We need to add the directive after the document title, and add a class to the document title to achieve the title & subtitle formatting. Here’s an example:

.. rst-class:: with-subtitle

=========
pw_string
=========

.. pigweed-module::
   :name: pw_string
   :tagline: Efficient, easy, and safe string manipulation
   :status: stable
   :languages: C++14, C++17, Rust
   :code-size-impact: 500 to 1500 bytes
   :get-started: module-pw_string-get-started
   :design: module-pw_string-design
   :guides: module-pw_string-guide
   :api: module-pw_string-api

   Module sales pitch goes here!

Directive options#

Pigweed AI summary: The given paragraph provides a summary of the options available for a directive. These options include the module name, a short tagline, the status of the module, whether it is deprecated or not, the languages it supports, the code size impact, references to the getting started section, tutorials, guides, design considerations, concepts documentation, and API documentation.

name: The module name (required)
tagline: A very short tagline that summarizes the module (required)
status: One of experimental, unstable, and stable (required)
is-deprecated: A flag indicating that the module is deprecated
languages: A comma-separated list of languages the module supports. If the language has API docs (Rust), they will be linked from the metadata block.
code-size-impact: A summarize of the average code size impact
get-started: A reference to the getting started section (required)
tutorials: A reference to the tutorials section
guides: A reference to the guides section
design: A reference to the design considerations section (required)
concepts: A reference to the concepts documentation
api: A reference to the API documentation

google_analytics#

Pigweed AI summary: The Google Analytics extension for Sphinx adds a tracking tag to each page of documentation when rendered to HTML, if a Google Analytics ID is set in the Sphinx configuration. The ID can be set automatically based on the GN argument "pw_docs_google_analytics_id", which allows for control over whether tracking is enabled or not in the build configuration. This feature is typically only enabled for documentation builds intended for deployment on the web.

When this extension is included and a google_analytics_id is set in the Sphinx configuration, a Google Analytics tracking tag will be added to each page of the documentation when it is rendered to HTML.

By default, the Sphinx configuration’s google_analytics_id is set automatically based on the value of the GN argument pw_docs_google_analytics_id, allowing you to control whether tracking is enabled or not in your build configuration. Typically, you would only enable this for documentation builds intended for deployment on the web.

Debugging Pigweed’s Sphinx extensions#

Pigweed AI summary: To debug Pigweed's Sphinx extensions, you can use the pdb module to step through your extension code. First, set a breakpoint in your extension code using the "breakpoint()" function. Then, build the python.install to install the code change into the bootstrap venv. Finally, manually invoke Sphinx to build the docs and trigger your breakpoint. You should see build output from Sphinx, and the build should pause at your breakpoint, where you will see pdb's prompt.

To step through your Pigweed extension code with pdb:

Set a breakpoint in your extension code:
```
breakpoint()
```
Build python.install to install the code change into the bootstrap venv (environment/pigweed-venv/lib/python3.8/site-packages/pw_docgen):
```
ninja -C out python.install
```
Manually invoke Sphinx to build the docs and trigger your breakpoint:
```
cd out
sphinx-build -W -b html -d docs/gen/docs/help docs/gen/docs/pw_docgen_tree docs/gen/docs/html -v -v -v
```
You should see build output from Sphinx. The build should pause at your breakpoint and you should then see pdb’s prompt ((Pdb)).