pw_blob_store#
pw_blob_store is a storage container library for storing a single blob of
data. BlobStore is a flash-backed persistent storage system with integrated
data integrity checking that serves as a lightweight alternative to a file
system.
Usage#
Pigweed AI summary: The usage of a BlobStore involves using BlobReader and BlobWriter objects that are constructed using the BlobStore. The HasData() method can be used to check the data state of a blob without needing to open a reader or writer. BlobStore uses a write buffer to allow for smaller or unaligned writes, and can be used with a zero-size write buffer to reduce memory requirements. BlobWriter objects are Writer compatible but do not support reading the blob's contents, and there is currently no mechanism for
Most operations on a BlobStore are done using BlobReader and
BlobWriter objects that have been constructed using a BlobStore. Though
a BlobStore may have multiple open BlobReader objects, no other
readers/writers may be active if a BlobWriter is opened on a blob store.
The data state of a blob can be checked using the HasData() method.
The method returns true if the blob is currenty valid and has at least one data
byte. This allows checking if a blob has stored data without needing to
instantiate and open a reader or writer.
Write buffer#
Pigweed AI summary: BlobStore uses a write buffer to allow for smaller and unaligned writes to the flash write alignment. The write buffer can also be used for deferred writes that can be written to flash at a later time or by a different thread/context. A zero-size write buffer can be used to reduce memory requirements, but the user must maintain write sizes that are a multiple of the flash write size. If a non-zero sized write buffer is used, the write buffer size must be a multiple of the flash write size
BlobStore uses a write buffer to allow writes smaller than and/or unaligned to the flash write aligment. BlobStore also supports using the write buffer for deferred writes that can be enqueued and written to flash at a later time or by a different thread/context.
BlobStore can be used with a zero-size write buffer to reduce memory requirements. When using zero-size write buffer, the user is required to write maintain write sizes that are a multiple of the flash write size the blob is configured for.
If a non-zero sized write buffer is used, the write buffer size must be a multiple of the flash write size.
Writing to a BlobStore#
Pigweed AI summary: This section discusses writing to a BlobStore using BlobWriter objects, which are compatible with pw::stream::Writer but do not support reading the blob's contents. Opening a BlobWriter on a BlobStore that already contains data will discard any existing data if certain methods are called, and there is no way to append to existing data. The code example shows how to use BlobWriterWithBuffer to write data to a BlobStore and manually close the BlobWriter to enable error handling.
BlobWriter objects are pw::stream::Writer compatible, but do not support
reading any of the blob’s contents. Opening a BlobWriter on a BlobStore
that contains data will discard any existing data if Discard(), Write
(), or Erase() are called. There is currently no mechanism to allow
appending to existing data.
BlobStore::BlobWriterWithBuffer writer(my_blob_store);
writer.Open();
writer.Write(my_data);
// ...
// A close is implied when a BlobWriter is destroyed. Manually closing a
// BlobWriter enables error handling on Close() failure.
writer.Close();
Erasing a BlobStore#
Pigweed AI summary: This section explains the two methods for erasing the contents of a BlobStore: Discard() and Erase(). Discard() is the faster option and simply marks the BlobStore as empty, while Erase() performs a flash erase of the underlying partition and is useful for manually controlling when a flash erase is performed.
There are two distinctly different mechanisms to “erase” the contents of a BlobStore:
Discard(): Discards any ongoing writes and ensuresBlobReaderobjects see theBlobStoreas empty. This is the fastest way to logically erase aBlobStore.Erase(): Performs an explicit flash erase of theBlobStore’s underlying partition. This is useful for manually controlling when a flash erase is performed before aBlobWriterstarts to write data (as flash erase operations may be time-consuming).
Naming a BlobStore’s contents#
Pigweed AI summary: This article discusses how to name the contents of a BlobStore, which is similar to naming a file. This allows for easy identification of data stored in a shared BlobStore. The article also provides information on how to read from a BlobStore using a BlobReader object, which can be opened and closed and is seekable. The article includes code examples for naming and reading from a BlobStore.
Data in a BlobStore May be named similarly to a file. This enables
identification of a BlobStore’s contents in cases where different data may be
stored to a shared blob store. This requires an additional RAM buffer that can
be used to encode the BlobStore’s KVS metadata entry. Calling
MaxFileNameLength() on a BlobWriter will provide the max file name
length based on the BlobWriter’s metadata encode buffer size.
SetFileName() performs a copy of the provided file name, meaning it’s safe
for the std::string_view to be invalidated after the function returns.
constexpr size_t kMaxFileNameLength = 48;
BlobStore::BlobWriterWithBuffer<kMaxFileNameLength> writer(my_blob_store);
writer.Open();
writer.SetFileName("stonks.jpg");
writer.Write(my_data);
// ...
writer.Close();
Reading from a BlobStore#
Pigweed AI summary: This section explains how to read from a BlobStore, which can have multiple open BlobReader objects. However, no other readers or writers can be active if a BlobWriter is opened on the store. To read from a BlobStore, you need to create a BlobReader instance, open it using BlobReader::Open(), and then use BlobReader::Read() or BlobReader::GetMemoryMappedBlob() to read the data. The BlobReader is also seekable, so you can use BlobReader
A BlobStore may have multiple open BlobReader objects. No other
readers/writers may be open/active if a BlobWriter is opened on a blob
store.
Create BlobReader instance
BlobReader::Open()
Read data using BlobReader::Read() or BlobReader::GetMemoryMappedBlob(). BlobReader is seekable. Use BlobReader::Seek() to read from a desired offset.
BlobReader::Close()
FileSystem RPC integration#
Pigweed AI summary: The pw_blob_store offers a FileSystemEntry implementation that can be used with pw_file's FlatFileSystemService. This makes it easier to list BlobStore objects as files through pw_file's FileSystem RPC service.
pw_blob_store provides an optional FileSystemEntry implementation for
use with pw_file’s FlatFileSystemService. This simplifies the process of
enumerating BlobStore objects as files via pw_file’s FileSystem RPC
service.
Size report#
Pigweed AI summary: The size report provides information on the memory usage of the blob store. It includes a table that shows the memory usage for different segments of the blob store. The table includes columns for the label, segment, and delta. The report also includes information on the memory usage of the blob store with deferred write. The documentation for this module is currently incomplete.
The following size report showcases the memory usage of the blob store.
Label |
Segment |
Delta |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
BlobStore |
FLASH
|
+3,872 |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
BlobStore with deferred write |
FLASH
|
+4,160 |
Note
The documentation for this module is currently incomplete.