Flash Circular Buffer (FCB)¶

Flash circular buffer provides an abstration through which you can treat flash like a FIFO. You append entries to the end, and read data from the beginning.

Description
Usage
Data structures
API

Description ¶

Elements in the flash contain the length of the element, the data within the element, and checksum over the element contents.

Storage of elements in flash is done in a FIFO fashion. When user requests space for the next element, space is located at the end of the used area. When user starts reading, the first element served is the oldest element in flash.

Elements can be appended to the end of the area until storage space is exhausted. User has control over what happens next; either erase oldest block of data, thereby freeing up some space, or stop writing new data until existing data has been collected. FCB treats underlying storage as an array of flash sectors; when it erases old data, it does this a sector at a time.

Elements in the flash are checksummed. That is how FCB detects whether writing element to flash completed ok. It will skip over entries which don’t have a valid checksum.

Usage ¶

To add an element to circular buffer:

Call fcb_append() to get the location where data can be written. If this fails due to lack of space, you can call fcb_rotate() to make some. And then call fcb_append() again.
Use flash_area_write() to write element contents.
Call fcb_append_finish() when done. This completes the entry by calculating the checksum.

To read contents of the circular buffer: * Call fcb_walk() with a pointer to your callback function. * Within callback function copy in data from the element using flash_area_read(). You can tell when all data from within a sector has been read by monitoring returned element’s area pointer. Then you can call fcb_rotate(), if you’re done with that data.

Alternatively: * Call fcb_getnext() with 0 in element offset to get the pointer to oldest element. * Use flash_area_read() to read element contents. * Call fcb_getnext() with pointer to current element to get the next one. And so on.

Data structures ¶

This data structure describes the element location in the flash. You would use it figure out what parameters to pass to flash_area_read() to read element contents. Or to flash_area_write() when adding a new element.

struct fcb_entry {
    struct flash_area *fe_area;
    uint32_t fe_elem_off;
    uint32_t fe_data_off;
    uint16_t fe_data_len;
};

Element	Description
fe_area	Pointer to info about the flash sector. Pass this to flash_area_x x() routines.
fe_elem_ off	Byte offset from the start of the sector to beginning of element.
fe_data_ off	Byte offset from start of the sector to beginning of element data. Pass this to to flash_area_x x() routines.
fe_data_ len	Number of bytes in the element.

The following data structure describes the FCB itself. First part should be filled in by the user before calling fcb_init(). The second part is used by FCB for its internal bookkeeping.

struct fcb {
    /* Caller of fcb_init fills this in */
    uint32_t f_magic;           /* As placed on the disk */
    uint8_t f_version;          /* Current version number of the data */
    uint8_t f_sector_cnt;       /* Number of elements in sector array */
    uint8_t f_scratch_cnt;      /* How many sectors should be kept empty */
    struct flash_area *f_sectors; /* Array of sectors, must be contiguous */

    /* Flash circular buffer internal state */
    struct os_mutex f_mtx;      /* Locking for accessing the FCB data */
    struct flash_area *f_oldest;
    struct fcb_entry f_active;
    uint16_t f_active_id;
    uint8_t f_align;            /* writes to flash have to aligned to this */
};

Element	Description
f_magic	Magic number in the beginning of FCB flash sector. FCB uses this when determining whether sector contains valid data or not.
f_version	Current version number of the data. Also stored in flash sector header.
f_sector_cnt	Number of elements in the f_sectors array.
f_scratch _cnt	Number of sectors to keep empty. This can be used if you need to have scratch space for garbage collecting when FCB fills up.
f_sectors	Array of entries describing flash sectors to use.
f_mtx	Lock protecting access to FCBs internal data.
f_oldest	Pointer to flash sector containing the oldest data. This is where data is served when read is started.
f_active	Flash location where the newest data is. This is used by fcb_append() to figure out where the data should go to.
f_active_id	Flash sectors are assigned ever-increasin g serial numbers. This is how FCB figures out where oldest data is on system restart.
f_align	Some flashes have restrictions on alignment for writes. FCB keeps a copy of this number for the flash here.

API ¶

typedef int (*fcb_walk_cb)(struct fcb_entry *loc, void *arg)¶

Walk over all log entries in FCB, or entries in a given flash_area.

cb gets called for every entry. If cb wants to stop the walk, it should return non-zero value.

Entry data can be read using flash_area_read(), using loc->fe_area, loc->fe_data_off, and loc->fe_data_len as arguments.

typedef int (*fcb_walk_cb)(struct fcb_entry *loc, void *arg)

Walk over all log entries in FCB, or entries in a given flash_area.

cb gets called for every entry. If cb wants to stop the walk, it should return non-zero value.

Entry data can be read using fcb_read().

struct flash_area *fe_area¶

uint32_t fe_elem_off¶

uint32_t fe_data_off¶

uint16_t fe_data_len¶

uint32_t f_magic¶

uint8_t f_version¶

uint8_t f_sector_cnt¶

uint8_t f_scratch_cnt¶

struct flash_area *f_sectors¶

struct os_mutex f_mtx¶

struct flash_area *f_oldest¶

struct fcb_entry f_active¶

uint16_t f_active_id¶

uint8_t f_align¶

struct fcb_entry flb_entry¶

uint32_t flb_index¶

struct fcb_log_bmark *fls_bmarks¶: Array of bookmarks.

int fls_cap¶: The maximum number of bookmarks.

int fls_size¶: The number of currently usable bookmarks.

int fls_next¶: The index where the next bookmark will get written.

struct fcb fl_fcb¶

uint8_t fl_entries¶

uint32_t fl_watermark_off¶

struct fcb_log_bset fl_bset¶

struct fcb *fe_fcb¶

struct flash_sector_range *fe_range¶

uint16_t fe_sector¶

uint16_t fe_entry_num¶

uint8_t f_range_cnt¶

uint16_t f_sector_cnt

struct flash_sector_range *f_ranges¶

uint16_t f_oldest_sec¶

uint16_t f_sector_entries¶

struct flash_sector_range *si_range¶

uint32_t si_sector_offset¶

uint16_t si_sector_in_range¶

int fcb_init(struct fcb *fcb)¶

int fcb_append(struct fcb*, uint16_t len, struct fcb_entry *loc)¶

fcb_append() appends an entry to circular buffer.

When writing the contents for the entry, use loc->fl_area and loc->fl_data_off with flash_area_write(). When you’re finished, call fcb_append_finish() with loc as argument.

When writing the contents for the entry, fcb_write() with fcb_entry filled by. fcb_append(). When you’re finished, call fcb_append_finish() with loc as argument.

int fcb_append_finish(struct fcb*, struct fcb_entry *append_loc)¶

int fcb_walk(struct fcb*, struct flash_area*, fcb_walk_cb cb, void *cb_arg)¶

int fcb_getnext(struct fcb*, struct fcb_entry *loc)¶

int fcb_rotate(struct fcb*)¶: Erases the data from oldest sector.

int fcb_append_to_scratch(struct fcb*)¶: Start using the scratch block.

int fcb_free_sector_cnt(struct fcb *fcb)¶: How many sectors are unused.

int fcb_is_empty(struct fcb *fcb)¶: Whether FCB has any data.

int fcb_offset_last_n(struct fcb *fcb, uint8_t entries, struct fcb_entry *last_n_entry)¶: Element at offset entries from last position (backwards).

int fcb_clear(struct fcb *fcb)¶: Clears FCB passed to it.

int fcb_area_info(struct fcb *fcb, struct flash_area *fa, int *elemsp, int *bytesp)¶

Usage report for a given FCB area.

Returns number of elements and the number of bytes stored in them.

void fcb_log_init_bmarks(struct fcb_log *fcb_log, struct fcb_log_bmark *buf, int bmark_count)¶

Bookmarks are an optimization to speed up lookups in FCB-backed logs.

The concept is simple: maintain a set of flash area+offset pairs corresponding to recently found log entries. When we perform a log lookup, the walk starts from the bookmark closest to our desired entry rather than from the beginning of the log.

Bookmarks are stored in a circular buffer in the fcb_log object. Each time the log is walked, the starting point of the walk is added to the set of bookmarks.

FCB rotation invalidates all bookmarks. It is up to the client code to clear a log’s bookmarks whenever rotation occurs.

Configures an fcb_log to use the specified buffer for bookmarks.

Parameters

fcb_log: The log to configure.
bmarks: The buffer to use for bookmarks.
bmark_count: The bookmark capacity of the supplied buffer.

void fcb_log_clear_bmarks(struct fcb_log *fcb_log)¶

Erases all bookmarks from the supplied fcb_log.

Parameters

fcb_log: The fcb_log to clear.

const struct fcb_log_bmark *fcb_log_closest_bmark(const struct fcb_log *fcb_log, uint32_t index)¶

Searches an fcb_log for the closest bookmark that comes before or at the specified index.

Return

The closest bookmark on success; NULL if the log has no applicable bookmarks.

Parameters

fcb_log: The log to search.
index: The index to look for.

void fcb_log_add_bmark(struct fcb_log *fcb_log, const struct fcb_entry *entry, uint32_t index)¶

Inserts a bookmark into the provided log.

Parameters

fcb_log: The log to insert a bookmark into.
entry: The entry the bookmark should point to.
index: The log entry index of the bookmark.

int fcb_init_flash_area(struct fcb *fcb, int flash_area_id, uint32_t magic, uint8_t version)¶

int fcb_write(struct fcb_entry *loc, uint16_t off, void *buf, uint16_t len)¶

int fcb_append_finish(struct fcb_entry *append_loc)

int fcb_walk(struct fcb*, int sector, fcb_walk_cb cb, void *cb_arg)

int fcb_read(struct fcb_entry *loc, uint16_t off, void *buf, uint16_t len)¶

int fcb_get_total_size(const struct fcb *fcb)¶

Get total size of FCB.

return FCB’s size in bytes

Parameters

fcb: FCB to use

int fcb_area_info(struct fcb *fcb, int sector, int *elemsp, int *bytesp)

Usage report for a given FCB sector.

Returns number of elements and the number of bytes stored in them.

FCB_MAX_LEN¶

FCB_OK¶: Error codes.

FCB_ERR_ARGS¶

FCB_ERR_FLASH¶

FCB_ERR_NOVAR¶

FCB_ERR_NOSPACE¶

FCB_ERR_NOMEM¶

FCB_ERR_CRC¶

FCB_ERR_MAGIC¶

FCB_ERR_VERSION¶

FCB_MAX_LEN

FCB_SECTOR_OLDEST¶

FCB_ENTRY_SIZE¶

FCB_CRC_LEN¶

FCB_OK: Error codes.

FCB_ERR_ARGS

FCB_ERR_FLASH

FCB_ERR_NOVAR

FCB_ERR_NOSPACE

FCB_ERR_NOMEM

FCB_ERR_CRC

FCB_ERR_MAGIC

FCB_ERR_VERSION

struct fcb_entry¶

#include <fcb.h>

Entry location is pointer to area (within fcb->f_sectors), and offset within that area.

Entry location point to sector, and offset within that sector.

struct fcb¶: #include <fcb.h>

struct fcb_log_bmark¶: #include <fcb.h>
An individual fcb log bookmark.

struct fcb_log_bset¶

#include <fcb.h>

A set of fcb log bookmarks.

Public Members

struct fcb_log_bmark *fls_bmarks¶: Array of bookmarks.

int fls_cap¶: The maximum number of bookmarks.

int fls_size¶: The number of currently usable bookmarks.

int fls_next¶: The index where the next bookmark will get written.

struct fcb_log¶: #include <fcb.h>
fcb_log is needed as the number of entries in a log

struct fcb_sector_info¶: #include <fcb.h>

An OS to build, deploy and securely manage billions of devices

Latest News:

Flash Circular Buffer (FCB)¶

Description ¶

Usage ¶

Data structures ¶

API ¶

An OS to build, deploy and securely manage billions of devices

Latest News:

Flash Circular Buffer (FCB)¶

Description¶

Usage¶

Data structures¶

API¶

Description ¶

Usage ¶

Data structures ¶

API ¶