pw_ide#

This module provides tools for supporting code editor and IDE features for Pigweed projects.

Usage#

Setup#

Pigweed AI summary: To get started, using "pw ide sync" is usually sufficient.

Most of the time, pw ide sync is all you need to get started.

Configuration#

pw_ide has a built-in default configuration. You can create a configuration file if you need to override those defaults.

A project configuration can be defined in .pw_ide.yaml in the project root. This configuration will be checked into source control and apply to all developers of the project. Each user can also create a .pw_ide.user.yaml file that overrides both the default and project settings, is not checked into source control, and applies only to that checkout of the project. All of these files have the same schema, in which these options can be configured:

property PigweedIdeSettings.working_dir: Path#

Path to the pw_ide working directory.

The working directory holds C++ compilation databases and caches, and other supporting files. This should not be a directory that’s regularly deleted or manipulated by other processes (e.g. the GN out directory) nor should it be committed to source control.

Default: .pw_ide

property PigweedIdeSettings.compdb_search_paths: List[Tuple[Path, str]]#

Paths to directories to search for compilation databases.

If you’re using a build system to generate compilation databases, this may simply be your build output directory. However, you can add additional directories to accommodate compilation databases from other sources.

Entries can be just directories, in which case the default target inference pattern will be used. Or entries can be tuples of a directory and a target inference pattern. See the documentation for target_inference for more information.

Default: ['out']

property PigweedIdeSettings.targets: List[str]#

The list of targets that should be enabled for code analysis.

In this case, “target” is analogous to a GN target, i.e., a particular build configuration. By default, all available targets are enabled. By adding targets to this list, you can constrain the targets that are enabled for code analysis to a subset of those that are available, which may be useful if your project has many similar targets that are redundant from a code analysis perspective.

Target names need to match the name of the directory that holds the build system artifacts for the target. For example, GN outputs build artifacts for the pw_strict_host_clang_debug target in a directory with that name in its output directory. So that becomes the canonical name for the target.

Default: []

property PigweedIdeSettings.target_inference: str#

A glob-like string for extracting a target name from an output path.

Build systems and projects have varying ways of organizing their build directory structure. For a given compilation unit, we need to know how to extract the build’s target name from the build artifact path. A simple example:

clang++ hello.cc -o host/obj/hello.cc.o

The top-level directory host is the target name we want. The same compilation unit might be used with another build target:

gcc-arm-none-eabi hello.cc -o arm_dev_board/obj/hello.cc.o

In this case, this compile command is associated with the arm_dev_board target.

When importing and processing a compilation database, we assume by default that for each compile command, the corresponding target name is the name of the top level directory within the build directory root that contains the build artifact. This is the default behavior for most build systems. However, if your project is structured differently, you can provide a glob-like string that indicates how to extract the target name from build artifact path.

A * indicates any directory, and ? indicates the directory that has the name of the target. The path is resolved from the build directory root, and anything deeper than the target directory is ignored. For example, a glob indicating that the directory two levels down from the build directory root has the target name would be expressed with */*/?.

Note that the build artifact path is relative to the compilation database search path that found the file. For example, for a compilation database search path of {project dir}/out, for the purposes of target inference, the build artifact path is relative to the {project dir}/out directory. Target inference patterns can be defined for each compilation database search path. See the documentation for compdb_search_paths for more information.

Default: ?

property PigweedIdeSettings.default_target: Optional[str]#

The default target to use when calling --set-default.

This target will be selected when pw ide cpp --set-default is called. You can define an explicit default target here. If that command is invoked without a default target definition, pw_ide will try to infer the best choice of default target. Currently, it selects the target with the broadest compilation unit coverage.

Default: None

property PigweedIdeSettings.cascade_targets: bool#

Mix compile commands for multiple targets to maximize code coverage.

By default (with this set to False), the compilation database for each target is consistent in the sense that it only contains compile commands for one build target, so the code intelligence that database provides is related to a single, known compilation artifact. However, this means that code intelligence may not be provided for every source file in a project, because some source files may be relevant to targets other than the one you have currently set. Those source files won’t have compile commands for the current target, and no code intelligence will appear in your editor.

If this is set to True, compilation databases will still be separated by target, but compile commands for all other targets will be appended to the list of compile commands for that target. This will maximize code coverage, ensuring that you have code intelligence for every file that is built for any target, at the cost of consistency—the code intelligence for some files may show information that is incorrect or irrelevant to the currently selected build target.

The currently set target’s compile commands will take priority at the top of the combined file, then all other targets’ commands will come after in order of the number of commands they have (i.e. in the order of their code coverage). This relies on the fact that clangd parses the compilation database from the top down, using the first compile command it encounters for each compilation unit.

Default: False

property PigweedIdeSettings.sync: List[str]#

A sequence of commands to automate IDE features setup.

pw ide sync should do everything necessary to get the project from a fresh checkout to a working default IDE experience. This defines the list of commands that makes that happen, which will be executed sequentially in subprocesses. These commands should be idempotent, so that the user can run them at any time to update their IDE features configuration without the risk of putting those features in a bad or unexpected state.

Default: ['pw --no-banner ide cpp --process']

property PigweedIdeSettings.clangd_additional_query_drivers: List[str]#

Additional query driver paths that clangd should use.

By default, pw_ide supplies driver paths for the toolchains included in Pigweed. If you are using toolchains that are not supplied by Pigweed, you should include path globs to your toolchains here. These paths will be given higher priority than the Pigweed toolchain paths.

Default: []

property PigweedIdeSettings.editors: Dict[str, bool]#

Enable or disable automated support for editors.

Automatic support for some editors is provided by pw_ide, which is accomplished through generating configuration files in your project directory. All supported editors are enabled by default, but you can disable editors by adding an '<editor>': false entry.

Default: vscode: true

C++ Code Intelligence#

Pigweed AI summary: The C++ Code Intelligence feature provided by clangd, a language server, can be used with any editor that supports the language server protocol. It uses a compilation database, which is a JSON file containing compile commands for the project. Multiple targets and toolchains require separate compilation databases. The pw_ide tool can be used to manage these databases. The "pw ide sync" command finds and analyzes compilation databases, creating new ones for each target toolchain if necessary. The available target toolchains can be

clangd is a language server that provides C/C++ code intelligence features to any editor that supports the language server protocol (LSP). It uses a compilation database, a JSON file containing the compile commands for the project. Projects that have multiple targets and/or use multiple toolchains need separate compilation databases for each target toolchain. pw_ide provides tools for managing those databases.

Assuming you have one or more compilation databases that have been generated by your build system, start with:

pw ide sync

This command will:

Find every compilation database in your build directory
Analyze each database
- If a database is internally consistent (i.e., it only contains valid compile commands for a single target), it will use that database as-is for the target toolchain that database pertains to. This is the typical case for CMake builds.
- Otherwise, if a database contains commands for multiple target toolchains and/or contains invalid compile commands, the database will be processed, yielding one new compilation database for each target toolchain. Those databases will be used instead of the original.
Link each target to its respective compilation database

Now, you can list the available target toolchains with:

pw ide cpp --list

Then set the target toolchain that clangd should use with:

pw ide cpp --set <selected target name>

clangd will now work as designed since it is configured to use a compilation database that is consistent to just a single target toolchain.

clangd must be run with arguments that provide the Pigweed environment paths to the correct toolchains and sysroots. One way to do this is to launch your editor from the terminal in an activated environment (e.g. running vim from the terminal), in which case nothing special needs to be done as long as your toolchains are in the Pigweed environment or $PATH. But if you launch your editor outside of the activated environment (e.g. launching Visual Studio Code from your GUI shell’s launcher), you can generate the command that invokes clangd with the right arguments with:

pw ide cpp --clangd-command

Python Code Intelligence#

Pigweed AI summary: This section explains that any Python language server can be used with Pigweed projects as long as it is configured to use the Pigweed virtual environment, and provides a command to output the path to the virtual environment.

Any Python language server should work well with Pigweed projects as long as it’s configured to use the Pigweed virtual environment. You can output the path to the virtual environment on your system with:

pw ide python --venv

Docs Code Intelligence#

Pigweed AI summary: The esbonio language server can provide code intelligence for RestructuredText and Sphinx, and works well with Pigweed projects when pointed to Pigweed's Python virtual environment. To use it with Visual Studio Code, install the esbonio extension after setting up pw_ide, and allow it to be installed in the Pigweed Python environment. After indexing, code intelligence will be available.

The esbonio language server will provide code intelligence for RestructuredText and Sphinx. It works well with Pigweed projects as long as it is pointed to Pigweed’s Python virtual environment. For Visual Studio Code, simply install the esbonio extension, which will be recommended to you after setting up pw_ide. Once it’s installed, a prompt will ask if you want to automatically install esbonio in your Pigweed Python environment. After that, give esbonio some time to index, then you’re done!

Command-Line Interface Reference#

Pigweed AI summary: The Command-Line Interface Reference provides CLI tools for pw_ide. The available sub-commands include "sync", "setup", "cpp", "python", and "vscode". The "sync" command sets up or syncs the Pigweed project IDE features, creating the necessary directories and settings files. The "setup" command is deprecated and should be replaced with "pw ide sync". The "cpp" command configures C/C++ code intelligence support, allowing for the selection of target toolchains

CLI tools for pw_ide.

usage: pw ide [-h] {sync,setup,cpp,python,vscode} ...

Sub-commands#

Pigweed AI summary: The given text is a documentation for the Pigweed project's IDE features. It provides information on various sub-commands available in Pigweed IDE, such as "sync", "setup", "cpp", "python", and "vscode". Each sub-command is explained along with its usage and options. The documentation also includes instructions on how to configure C/C++ and Python code intelligence support, as well as support for Visual Studio Code. It provides details on how to set up the development environment, process

sync#

Pigweed AI summary: The "sync" section of Pigweed project allows users to set up or sync their IDE features with the command "pw ide sync". This command creates a working directory and settings files for all supported editors, and can be customized with additional setup steps in .pw_ide.yaml. Users can re-run this command in the future to add new IDE features without overwriting existing configurations.

Setup or sync your Pigweed project IDE features.

pw ide sync [-h]

This will automatically set up your development environment with all the features that Pigweed IDE supports, with sensible defaults.

At minimum, this command will create the .pw_ide working directory and create settings files for all supported editors. Projects can define additional setup steps in .pw_ide.yaml.

When new IDE features are introduced in the future (either by Pigweed or your downstream project), you can re-run this command to set up the new features. It will not overwrite or break any of your existing configuration.

setup#

Pigweed AI summary: This paragraph is informing the reader that the "pw ide setup" command is deprecated and should be replaced with "pw ide sync". The command is used for setting up an IDE and the paragraph includes the syntax for the command.

Deprecated! Please use pw ide sync.

pw ide setup [-h]

cpp#

Pigweed AI summary: The given text is a documentation for configuring C/C++ code intelligence support using the Pigweed tool. It explains how to use the "pw ide cpp" command to configure the target toolchain for C/C++ language analysis. It provides various options such as listing available toolchains, setting the target toolchain, processing a compilation database, and generating the clangd command for code analysis. The documentation also mentions the importance of build configurations and how Pigweed handles the compilation database. It concludes by explaining the

Configure C/C++ code intelligence support.

pw ide cpp [-h] [-l] [-g] [-s TARGET] [--set-default] [-p] [--clangd-command]
           [--clangd-command-for SYSTEM]

Named Arguments#

Pigweed AI summary: The paragraph describes the named arguments available for C/C++ language analysis. These arguments include options to list the target toolchains, print the current target toolchain, set the target toolchain, set the default toolchain, process files in the clang compilation database format, and print the command to run clangd in the Pigweed environment. Each option has a default value of False.

-l, --list

list the target toolchains available for C/C++ language analysis

Default: False

-g, --get

print the current target toolchain used for C/C++ language analysis

Default: False

-s, --set

set the target toolchain to use for C/C++ language server analysis

--set-default

set the C/C++ analysis target toolchain to the default defined in pw_ide settings

Default: False

-p, --process

process a file or several files matching the clang compilation database format

Default: False

--clangd-command

print the command for your system that runs clangd in the activated Pigweed environment

Default: False

--clangd-command-for

print the command for the specified system that runs clangd in the activated Pigweed environment

Code intelligence can be provided by clangd or other language servers that use the clangd compilation database format, defined at: https://clang.llvm.org/docs/JSONCompilationDatabase.html

Pigweed projects define their build configuration(s) via a build system, usually GN, Bazel, or CMake. Based on build configurations, the build system generates commands to compile each translation unit in the project. clangd uses those commands to parse the build graph and provide rich code intelligence.

Pigweed projects often target multiple devices & architectures, and use multiple compiler toolchains. As a result, there may be more than one way to compile each translation unit. Your build system ensures that it only invokes a single compiler command for each translation unit which is consistent with the toolchain and target appropriate to that build, which we refer to as a “target toolchain”.

We need to do the same thing with the compilation database that clangd uses. We handle this by:

Processing the compilation database produced the build system into multiple internally-consistent compilation databases, one for each target toolchain.
Providing commands to select which target toolchain you want to use for code analysis.

Refer to the Pigweed documentation or your build system’s documentation to learn how to produce a clangd compilation database. Once you have one, run this command to process it (or provide a glob to process multiple):

pw ide cpp --process {path to compile_commands.json}

You can now examine the target toolchains that are available to you:

pw ide cpp --list

… and select the target toolchain you want to use:

pw ide cpp --set host_clang

As long as your editor or language server plugin is properly configured, you will now get code intelligence features relevant to that particular target toolchain.

You can see what target toolchain is selected by running:

pw ide cpp

Whenever you switch to a target toolchain you haven’t used before, clangd will index the build, which may take several minutes. This process is not blocking, so you can take advantage of code analysis immediately even while the indexing is in progress. These indexes are cached, so you can switch between targets without re-indexing each time.

If your build configuration changes significantly (e.g. you add a new file to the project), you will need to re-process the compilation database for that change to be recognized and manifested in the target toolchain. Your target toolchain selection will not change, and your index will only need to be incrementally updated.

You can generate the clangd command your editor needs to run with:

pw ide cpp --clangd-command

If your editor uses JSON for configuration, you can export the same command in that format:

pw ide cpp --clangd-command-for json

python#

Pigweed AI summary: This paragraph discusses how to configure Python code intelligence support using the Pigweed Python virtual environment. It provides a command to generate the path to the virtual environment interpreter and an option to print the path to the virtual environment.

Configure Python code intelligence support.

pw ide python [-h] [--venv]

Named Arguments#

Pigweed AI summary: This section describes a named argument called "--venv" which, when used, will print the path to the Pigweed Python virtual environment. The default value for this argument is False.

--venv

print the path to the Pigweed Python virtual environment

Default: False

You can generate the path to the Python virtual environment interpreter that your editor/language server should use with:

pw ide python --venv

vscode#

Pigweed AI summary: This article explains how to configure support for Visual Studio Code (VSC) using the "pw ide vscode" command. This command replaces the current VSC settings for a project with the Pigweed default settings, project-specific settings, and personal settings. The article also mentions that the command manages VSC tasks and extensions and provides options to control which settings types are modified. It is advised not to manually edit the VSC settings file, and support for VSC can be disabled or enabled at the project

Configure support for Visual Studio Code.

pw ide vscode [-h] [--include SETTINGS_TYPE [SETTINGS_TYPE ...]]
              [--exclude SETTINGS_TYPE [SETTINGS_TYPE ...]]

Named Arguments#

Pigweed AI summary: This paragraph discusses named arguments and their usage in updating settings types. The options "--include" and "--exclude" are provided as examples of named arguments that can be used to specify which settings types to update or not update.

--include: update only these settings types
--exclude: do not update these settings types

This will replace your current Visual Studio Code (VSC) settings for this project (in .vscode/settings.json, etc.) with the following sets of settings, in order:

The Pigweed default settings
Your project’s settings, if any (in .vscode/pw_project_settings.json)
Your personal settings, if any (in .vscode/pw_user_settings.json)

In other words, settings files lower on the list can override settings defined by those higher on the list. Settings defined in the sources above are not active in VSC until they are merged and output to the current settings file by running:

pw ide vscode

Refer to the Visual Studio Code documentation for more information about these settings: https://code.visualstudio.com/docs/getstarted/settings

This command also manages VSC tasks (.vscode/tasks.json) and extensions (.vscode/extensions.json). You can explicitly control which of these settings types (“settings”, “tasks”, and “extensions”) is modified by this command by using the --include or --exclude options.

Your current VSC settings will never change unless you run pw ide commands. Since the current VSC settings are an artifact built from the three settings files described above, you should avoid manually editing that file; it will be replaced the next time you run pw ide vscode. A backup of your previous settings file will be made, and you can diff it against the new file to see what changed.

These commands will never modify your VSC user settings, which are stored outside of the project repository and apply globally to all VSC instances.

The settings files are system-specific and shouldn’t be checked into the repository, except for the project settings (those with pw_project_), which can be used to define consistent settings for everyone working on the project.

Note that support for VSC can be disabled at the project level or the user level by adding the following to .pw_ide.yaml or .pw_ide.user.yaml respectively:

editors:
  vscode: false

Likewise, it can be enabled by setting that value to true. It is enabled by default.

Design#

Supporting `clangd` for Embedded Projects#

There are three main challenges that often prevent clangd from working out-of-the-box with embedded projects:

Embedded projects cross-compile using alternative toolchains, rather than using the system toolchain. clangd doesn’t know about those toolchains by default.
Embedded projects (particularly Pigweed project) often have multiple targets that use multiple toolchains. Most build systems that generate compilation databases put all compile commands in a single database, meaning a single file can have multiple, conflicting compile commands. clangd will typically use the first one it finds, which may not be the one you want.
Pigweed projects have build steps that use languages other than C/C++. These steps are not relevant to clangd but many build systems will include them in the compilation database anyway.

To deal with these challenges, pw_ide processes the compilation database you provide, yielding one or more compilation databases that are valid, consistent, and specific to a particular target toolchain. This enables code intelligence and navigation features that reflect that build.

After processing a compilation database, pw_ide knows what target toolchains are available and provides tools for selecting which target toolchain is active. These tools can be integrated into code editors, but are ultimately CLI-driven and editor-agnostic. Enabling code intelligence in your editor may be as simple as configuring its language server protocol client to use the clangd command that pw_ide can generate for you.

When to provide additional configuration to support your use cases#

Pigweed AI summary: The default configuration for clangd in pw_ide should work fine if you're using Pigweed's toolchains and your native host toolchain. However, if you're using other toolchains, you'll need to provide additional configuration. Clangd needs a path to the compiler and the same compiler reflected in the query driver argument. When using pw_ide with external toolchains, you can add a path to the compiler to clangd_additional_query_drivers in your project's pw_ide.yaml

The default configuration for clangd in pw_ide should work without additional configuration as long as you’re using only toolchains provided by Pigweed and your native host toolchain. If you’re using other toolchains, keep reading.

clangd needs two pieces of information to use a toolchain:

A path to the compiler, which will be taken from the compile command.
The same compiler to be reflected in the query driver argument provided when running clangd.

When using pw_ide with external toolchains, you only need to add a path to the compiler to clangd_additional_query_drivers in your project’s pw_ide.yaml file. When processing a compilation database, pw_ide will use the query driver globs to find your compiler and configure clangd to use it.

Compiler wrappers#

Pigweed AI summary: The article mentions that if you are using a compiler wrapper command like ccache that is configured using the KEY=VALUE pattern, it will work without any additional configuration.

If you’re using ccache or any other wrapper command that is configured using ccache’s’ KEY=VALUE pattern, it will work out of the box.

Selected API Reference#

Configure C/C++ IDE support for Pigweed projects.

We support C/C++ code analysis via clangd, or other language servers that are compatible with the clangd compilation database format.

While clangd can work well out of the box for typical C++ codebases, some work is required to coax it to work for embedded projects. In particular, Pigweed projects use multiple toolchains within a distinct environment, and almost always define multiple targets. This means compilation units are likely have multiple compile commands and the toolchain executables are unlikely to be in your path. clangd is not equipped to deal with this out of the box. We handle this by:

Processing the compilation database produced by the build system into multiple internally-consistent compilation databases, one for each target (where a “target” is a particular build for a particular system using a particular toolchain).
Creating unambiguous paths to toolchain drivers to ensure the right toolchain is used and that clangd knows where to find that toolchain’s system headers.
Providing tools for working with several compilation databases that are spiritually similar to tools like pyenv, rbenv, etc.

In short, we take the probably-broken compilation database that the build system generates, process it into several not-broken compilation databases in the pw_ide working directory, and provide a stable symlink that points to the selected active target’s compliation database. If clangd is configured to point at the symlink and is set up with the right paths, you’ll get code intelligence.

class pw_ide.cpp.ClangdSettings(settings: PigweedIdeSettings)#

Makes system-specific settings for running clangd with Pigweed.

__init__(settings: PigweedIdeSettings)#

command(system: str = 'Linux') → str#: Return the command that runs clangd with Pigweed paths.

class pw_ide.cpp.CppCompilationDatabase(root_dir: Optional[Path] = None, file_path: Optional[Path] = None, source_file_path: Optional[Path] = None, target_inference: Optional[str] = None)#

A representation of a clang compilation database.

See: https://clang.llvm.org/docs/JSONCompilationDatabase.html

__init__(root_dir: Optional[Path] = None, file_path: Optional[Path] = None, source_file_path: Optional[Path] = None, target_inference: Optional[str] = None) → None#

add(*commands: CppCompileCommand)#: Add compile commands to the compilation database.

classmethod load(compdb_to_load: Union[List[Dict[str, Any]], str, TextIOBase, Path], root_dir: Path, target_inference: Optional[str] = None) → CppCompilationDatabase#

Load a compilation database.

You can provide a JSON file handle or path, a JSON string, or a native Python data structure that matches the format (list of dicts).

merge(other: CppCompilationDatabase) → None#

Merge values from another database into this one.

This will not overwrite a compile command that already exists for a particular file.

process(settings: PigweedIdeSettings, *, default_path: Optional[Path] = None, path_globs: Optional[List[str]] = None, strict: bool = False, always_output_new: bool = False) → Optional[CppCompilationDatabasesMap]#

Process a clangd compilation database file.

Given a clang compilation database that may have commands for multiple valid or invalid targets/toolchains, keep only the valid compile commands and store them in target-specific compilation databases.

If this finds that the processed file is functionally identical to the input file (meaning that the input file did not require processing to be used successfully with clangd), then it will return None, indicating that the original file should be used. This behavior can be overridden by setting always_output_new, which will ensure that a new compilation database is always written to the working directory and original compilation databases outside the working directory are never made available for code intelligence.

to_file(path: Path)#: Write the compilation database to a JSON file.

to_json() → str#: Output the compilation database to a JSON string.

class pw_ide.cpp.CppCompilationDatabasesMap(settings: PigweedIdeSettings)#

Container for a map of target name to compilation database.

__init__(settings: PigweedIdeSettings)#

classmethod merge(*db_sets: CppCompilationDatabasesMap) → CppCompilationDatabasesMap#

Merge several sets of processed compilation databases.

If you process N compilation databases produced by a build system, you’ll end up with N sets of processed compilation databases, containing databases for one or more targets each. This method merges them into one set of databases with one database per target.

The expectation is that the vast majority of the time, each of the raw compilation databases that are processed will contain distinct targets, meaning that the keys of each CppCompilationDatabases object that’s merged will be unique to each object, and this operation is nothing more than a shallow merge.

However, this also supports the case where targets may overlap between CppCompilationDatabases objects. In that case, we prioritize correctness, ensuring that the resulting compilation databases will work correctly with clangd. This means not including duplicate compile commands for the same file in the same target’s database. The choice of which duplicate compile command ends up in the final database is unspecified and subject to change. Note also that this method expects the settings value to be the same between all of the provided CppCompilationDatabases objects.

test_write() → None#

Test writing to file.

This will raise an exception if the file is not JSON-serializable.

write() → None#: Write compilation databases to target-specific JSON files.

class pw_ide.cpp.CppCompileCommand(file: str, directory: str, command: Optional[str] = None, arguments: Optional[List[str]] = None, output: Optional[str] = None)#

A representation of a clang compilation database compile command.

See: https://clang.llvm.org/docs/JSONCompilationDatabase.html

__init__(file: str, directory: str, command: Optional[str] = None, arguments: Optional[List[str]] = None, output: Optional[str] = None) → None#

process(*, default_path: Optional[Path] = None, path_globs: Optional[List[str]] = None, strict: bool = False) → Optional[CppCompileCommand]#

Process a compile command.

At minimum, a compile command from a clang compilation database needs to be correlated with its target, and this method returns the target name with the compile command. But it also cleans up other things we need for reliable code intelligence:

Some targets may not be valid C/C++ compile commands. For example, some build systems will naively include build steps for Python or for linting commands. We want to filter those out.
Some compile commands don’t provide a path to the compiler executable (referred to by clang as the “driver”). In that case, clangd is very unlikely to find the executable unless it happens to be in $PATH. The --query-driver argument to clangd allowlists executables/drivers for use its use, but clangd doesn’t use it to resolve ambiguous paths. We bridge that gap here. Any executable without a path will be either placed in the provided default path or searched for in the query driver globs and be replaced with a path to the executable.

class pw_ide.cpp.CppIdeFeaturesState(pw_ide_settings: PigweedIdeSettings)#

Container for IDE features state data.

__init__(pw_ide_settings: PigweedIdeSettings) → None#

pw_ide.cpp.path_to_executable(exe: str, *, default_path: Optional[Path] = None, path_globs: Optional[List[str]] = None, strict: bool = False) → Optional[Path]#

Return the path to a compiler executable.

In a clang compile command, the executable may or may not include a path. For example:

/usr/bin/clang      <- includes a path
../path/to/my/clang <- includes a path
clang               <- doesn't include a path

If it includes a path, then clangd will have no problem finding the driver, so we can simply return the path. If the executable doesn’t include a path, then clangd will search $PATH, and may not find the intended driver unless you actually want the default system toolchain or Pigweed paths have been added to $PATH. So this function provides two options for resolving those ambiguous paths:

Provide a default path, and all executables without a path will be re-written with a path within the default path.
Provide the a set of globs that will be used to search for the executable, which will normally be the query driver globs used with clangd.

By default, if neither of these options is chosen, or if the executable cannot be found within the provided globs, the pathless executable that was provided will be returned, and clangd will resort to searching $PATH. If you instead pass strict=True, this will raise an exception if an unambiguous path cannot be constructed.

This function only tries to ensure that all executables have a path to eliminate ambiguity. A couple of important things to keep in mind:

This doesn’t guarantee that the path exists or an executable actually exists at the path. It only ensures that some path is provided to an executable.
An executable being present at the indicated path doesn’t guarantee that it will work flawlessly for clangd code analysis. The clangd --query-driver argument needs to include a path to this executable in order for its bundled headers to be resolved correctly.

This function also filters out invalid or unsupported drivers. For example, build systems will sometimes naively include build steps for Python or other languages in the compilation database, which are not usable with clangd. As a result, this function has four possible end states:

It returns a path with an executable that can be used as a clangd driver.
It returns None, meaning the compile command was invalid.
It returns the same string that was provided (as a Path), if a path couldn’t be resolved and strict=False.
It raises an UnresolvablePathException if the executable cannot be placed in an unambiguous path and strict=True.

Automated Support for Code Editors & IDEs#

pw_ide provides a consistent framework for automatically applying settings for code editors, where default settings can be defined within pw_ide, which can be overridden by project settings, which in turn can be overridden by individual user settings.

Visual Studio Code#

Pigweed AI summary: The article explains how to generate settings for Visual Studio Code using the "pw ide sync" command. The settings can be customized at the project or user level using "pw_project_settings.json" and "pw_user_settings.json". The article also discusses how to access tasks and configurations for running and debugging projects using "tasks.json" and "launch.json". Finally, there is a tip about the difference between "Change C++ Code Analysis Target" and "Set C++ Code Analysis Target".

Running pw ide sync will automatically generate settings for Visual Studio Code. pw_ide comes with sensible defaults for Pigweed projects, but those can be augmented or overridden at the project level or the user level using pw_project_settings.json and pw_user_settings.json respectively. The generated settings.json file is essentially a build artifact and shouldn’t be committed to source control.

The same pattern applies to tasks.json, which provides Visual Studio Code tasks for pw_ide commands. Access these by opening the command palette (Ctrl/Cmd-Shift-P), selecting Tasks: Run Task, then selecting the desired task.

The same pattern also applies to launch.json, which is used to define configurations for running and debugging your project. Create a pw_project_launch.json with configurations that conform to the Visual Studio Code debugger configuration format.

Tip

What’s the difference between “Change C++ Code Analysis Target” and “Set C++ Code Analyis Target”? “Set” will automatically restart the clangd language server for you to pick up the changed target immediately, while “Change” will not.

Selected API Reference#

Framework for configuring code editors for Pigweed projects.

Editors and IDEs vary in the way they’re configured and the options they provide for configuration. As long as an editor uses files we can parse to store its settings, this framework can be used to provide a consistent interface to managing those settings in the context of a Pigweed project.

Ideally, we want to provide three levels of editor settings for a project:

User settings (specific to the user’s checkout)
Project settings (included in source control, consistent for all users)
Default settings (defined by Pigweed)

… where the settings on top can override (or cascade over) settings defined below.

Some editors already provide mechanisms for achieving this, but in ways that are particular to that editor, and many other editors don’t provide this mechanism at all. So we provide it in a uniform way here by adding a fourth settings level, active settings, which are the actual settings files the editor uses. Active settings are built (rather than edited or cloned) by looking for user, project, and default settings (which are defined by Pigweed and ignored by the editor) and combining them in the order described above. In this way, Pigweed can provide sensible defaults, projects can define additional settings to provide a uniform development experience, and users have the freedom to make their own changes.

class pw_ide.editors.EditorSettingsDefinition(pw_ide_settings: Optional[PigweedIdeSettings] = None, data: Optional[Callable[[PigweedIdeSettings], OrderedDict[str, Any]]] = None)#

Provides access to a particular group of editor settings.

A particular editor may have one or more settings types (e.g., editor settings vs. automated tasks settings, or separate settings files for each supported language). pw_ide also supports multiple settings levels, where the “active” settings are built from default, project, and user settings. Each combination of settings type and level will have one EditorSettingsDefinition, which may be in memory (e.g., for default settings defined in code) or may be backed by a file (see EditorSettingsFile).

Settings are accessed using the modify context manager, which provides you a dict-like data structure to manipulate.

Initial settings can be provided in the constructor via a callback that takes an instance of PigweedIdeSettings and returns a settings dict. This allows the initial settings to be dependent on overall IDE features settings.

__init__(pw_ide_settings: Optional[PigweedIdeSettings] = None, data: Optional[Callable[[PigweedIdeSettings], OrderedDict[str, Any]]] = None)#

build() → Generator[OrderedDict[str, Any], None, None]#

Expose a settings file builder.

You get an empty dict when entering the content, then you can build up settings by using sync_to to merge other settings dicts into this one, as long as everything is JSON-serializable. Example:

with settings_definition.modify() as settings:
    some_other_settings.sync_to(settings)

This data is not persisted to disk.

get() → OrderedDict[str, Any]#: Return the settings as an ordered dict.

sync_to(settings: OrderedDict[str, Any]) → None#: Merge this set of settings on top of the provided settings.

class pw_ide.editors.EditorSettingsFile(settings_dir: Path, name: str, file_format: _StructuredFileFormat)#

Provides access to an editor settings defintion stored in a file.

It’s assumed that the editor’s settings are stored in a file format that can be deserialized to Python dicts. The settings are represented by an ordered dict to make the diff that results from modifying the settings as easy to read as possible (assuming it has a plain text representation).

This represents the concept of a file; the file may not actually be present on disk yet.

__init__(settings_dir: Path, name: str, file_format: _StructuredFileFormat) → None#

build() → Generator[OrderedDict[str, Any], None, None]#

Expose a settings file builder.

You get an empty dict when entering the content, then you can build up settings by using sync_to to merge other settings dicts into this one, as long as everything is JSON-serializable. Example:

with settings_file.modify() as settings:
    some_other_settings.sync_to(settings)

After modifying the settings and leaving this context, the file will be written. If a failure occurs while writing the new file, it will be deleted.

get() → OrderedDict[str, Any]#

Read a settings file into an ordered dict.

This does not keep the file context open, so while the dict is mutable, any changes will not be written to disk.

class pw_ide.editors.EditorSettingsManager(pw_ide_settings: PigweedIdeSettings, settings_dir: Optional[Path] = None, file_format: Optional[_StructuredFileFormat] = None, types_with_defaults: Optional[Dict[_SettingsTypeT, Callable[[PigweedIdeSettings], OrderedDict[str, Any]]]] = None)#

Manages all settings for a particular editor.

This is where you interact with an editor’s settings (actually in a subclass of this class, not here). Initializing this class sets up access to one or more settings files for an editor (determined by _SettingsTypeT, fulfilled by an enum that defines each of an editor’s settings files), along with the cascading settings levels.

__init__(pw_ide_settings: PigweedIdeSettings, settings_dir: Optional[Path] = None, file_format: Optional[_StructuredFileFormat] = None, types_with_defaults: Optional[Dict[_SettingsTypeT, Callable[[PigweedIdeSettings], OrderedDict[str, Any]]]] = None)#

active(settings_type: _SettingsTypeT)#: Active settings for the provided settings type.

default(settings_type: _SettingsTypeT)#: Default settings for the provided settings type.

delete_all_active_settings() → None#: Delete all active settings files.

delete_all_backups() → None#: Delete all backup files.

project(settings_type: _SettingsTypeT)#: Project settings for the provided settings type.

user(settings_type: _SettingsTypeT)#: User settings for the provided settings type.

Configure Visual Studio Code (VSC) for Pigweed projects.

VSC recognizes three sources of configurable settings:

Project settings, stored in {project root}/.vscode/settings.json
Workspace settings, stored in (workspace root)/.vscode/settings.json; a workspace is a collection of projects/repositories that are worked on together in a single VSC instance
The user’s personal settings, which are stored somewhere in the user’s home directory, and are applied to all instances of VSC

This provides three levels of settings cascading:

Workspace <- Project <- User

… where the arrow indicates the ability to override.

Out of these three, only project settings are useful to Pigweed projects. User settings are essentially global and outside the scope of Pigweed. Workspaces are seldom used and don’t work well with the Pigweed directory structure.

Nonetheless, we want a three-tiered settings structure for Pigweed projects too:

Default settings provided by Pigweed, configuring VSC to use IDE features
Project-level overrides that downstream projects may define
User-level overrides that individual users may define

We accomplish all of that with only the project settings described in #1 above.

Default settings are defined in this module. Project settings can be defined in .vscode/pw_project_settings.json and should be checked into the repository. User settings can be defined in .vscode/pw_user_settings.json and should not be checked into the repository. None of these settings have any effect until they are merged into VSC’s settings (.vscode/settings.json) via the functions in this module. Those resulting settings are system-specific and should also not be checked into the repository.

We provide the same structure to both tasks and extensions as well. Defaults are provided by Pigweed, can be augmented or overridden at the project level with .vscode/pw_project_tasks.json and .vscode/pw_project_extensions.json, can be augmented or overridden by an individual developer with .vscode/pw_user_tasks.json and .vscode/pw_user.extensions.json, and none of this takes effect until they are merged into VSC’s active settings files (.vscode/tasks.json and .vscode/extensions.json) by running the appropriate command.

class pw_ide.vscode.VscSettingsManager(pw_ide_settings: PigweedIdeSettings, settings_dir: Optional[Path] = None, file_format: Optional[_StructuredFileFormat] = None, types_with_defaults: Optional[Dict[_SettingsTypeT, Callable[[PigweedIdeSettings], OrderedDict[str, Any]]]] = None)#: Manages all settings for Visual Studio Code.

class pw_ide.vscode.VscSettingsType(value)#

Visual Studio Code settings files.

VSC supports editor settings (settings.json), recommended extensions (extensions.json), tasks (tasks.json), and launch/debug configurations (launch.json).

pw_ide#

Usage#

Setup#

Configuration#

C++ Code Intelligence#

Python Code Intelligence#

Docs Code Intelligence#

Command-Line Interface Reference#

Sub-commands#

sync#

setup#

cpp#

Named Arguments#

python#

Named Arguments#

vscode#

Named Arguments#

Design#

Supporting clangd for Embedded Projects#

When to provide additional configuration to support your use cases#

Compiler wrappers#

Selected API Reference#

Automated Support for Code Editors & IDEs#

Visual Studio Code#

Selected API Reference#

Supporting `clangd` for Embedded Projects#