OrgZephyr/zephyr
OrgZephyr
/
zephyr
mirror of https://github.com/zephyrproject-rtos/zephyr.git
1
0
Fork 0
Code Issues Releases Wiki Activity
zephyr/subsys/fs/nvs/nvs_priv.h

53 lines
1.1 KiB
C
Raw Normal View History

subsys: fs: Add Non Volatile Storage (NVS) for zephyr Zephyr supports fatfs, nffs and fcb as storage layer. fatfs and nffs are less suited for application in memory restricted IC's. fcb has a smaller footprint but has a complex api. The proposed module is a module with a even smaller footprint compared to fcb and a simple interface for reading and writing entries. The module provides wear levelling of flash. This allows the module to be used not only to store configuration settings but to store device state (e.g. state of a light switch over reboots) of a zephyr device. Fixes buffer overflow by introducing maximum read length in nvs_read() and nvs_read_hist(). Fixes nvs_write() not to reflash the same data. Allows the user to do call nvs_write() for all defined entries without worries about flash wear. Fixes garbage collection error where wrong data could be copied. Add nvs_delete() to allow deleting a stored entry. A deleted entry will not be copied to a new flash sector Include flash wear information in the README.md documentation 0/25 Update module after reviewers remarks, added documentation to nvs.h, removed README.md by nvs.rst in doc/subsystems folder 04/26 Update module after reviewers remarks, updated nvs.rst, added more documentation to samples/subsys/nvs/src/main.c, updated doxygen info in nvs.h (hope this time it works). 04/26 Update subsystems.rst to include nvs.restart 04/27 Updated nvs.c and nvs.h to avoid a possible flash deletion loop when the file system is full. 04/29 Updated nvs_write to detect and ignore deletes of non-existing items 05/06 Update NVS module to return standard error codes, removed low level API, added configuration options. NVS now uses the board dts to determine the flash storage location (FLASH_AREA_STORAGE_OFFSET). 05/06 Update nvs.rst. Updated intendation and added intermediate variables in nvs.c to make the code easier to read. 05/06 Update nvs.rst. 05/07 Update nvs.rst 05/08 Changed the API to a more standard file system API. 05/08 Removed cnt_max from nvs_read() as it is not used. 05/08 Removed #ifdef(CONFIG_NVS_LOG) from nvs_priv.h, now the module can be build with debugging off. 05/09 Removed configuration options for SECTOR_SIZE, SECTOR_COUNT and MAX_ELEM_SIZE. It is now easy to support multiple NVS filesystems on one or multiple devices. Changed logging to support newlib systems. Thanks to Olivier Martin for reporting and proposed changes. Signed-off-by: Laczen JMS <laczenjms@gmail.com>
2018-03-02 05:09:17 +08:00
/* NVS: non volatile storage in flash
*
* Copyright (c) 2018 Laczen
*
* SPDX-License-Identifier: Apache-2.0
*/
#ifndef __NVS_PRIV_H_
#define __NVS_PRIV_H_
#ifdef __cplusplus
extern "C" {
#endif
#define SYS_LOG_DOMAIN "fs/nvs"
#define SYS_LOG_LEVEL CONFIG_NVS_LOG_LEVEL
#include <logging/sys_log.h>
/*
subsys: fs/nvs: Rewrite for improved robustness On flash NVS was stored one entry after another including the metadata of each entry. This has the disadvantage that when an incomplete write is performed (e.g. due to power failure) the complete sector had to be rewritten to get a completely functional system. The present rewrite changed the storage in flash of the data. For each sector the data is now written as follows: the data itself at the beginning of the sector (one after the other), the metadata (id, length, data offset in the sector, and a crc of the metadata) is written from the end of the sector. The metadata is of fixed size (8 byte) and for a sector that is completely occupied a metadata entry of all zeros is used. Writing data to flash always is done by: 1. Writing the data, 2. Writing the metadata. If an incomplete write is done NVS will ignore this incomplete write. At the same time the following improvements were done: 1. NVS now support 65536 sectors of each 65536 byte. 2. The sector size no longer requires to be a power of 2 (but it still needs to be a multiple of the flash erase page size). 3. NVS now also keeps track of the free space available. Signed-off-by: Laczen JMS <laczenjms@gmail.com>
2018-07-09 22:50:57 +08:00
* MASKS AND SHIFT FOR ADDRESSES
* an address in nvs is an u32_t where:
* high 2 bytes represent the sector number
* low 2 bytes represent the offset in a sector
subsys: fs: Add Non Volatile Storage (NVS) for zephyr Zephyr supports fatfs, nffs and fcb as storage layer. fatfs and nffs are less suited for application in memory restricted IC's. fcb has a smaller footprint but has a complex api. The proposed module is a module with a even smaller footprint compared to fcb and a simple interface for reading and writing entries. The module provides wear levelling of flash. This allows the module to be used not only to store configuration settings but to store device state (e.g. state of a light switch over reboots) of a zephyr device. Fixes buffer overflow by introducing maximum read length in nvs_read() and nvs_read_hist(). Fixes nvs_write() not to reflash the same data. Allows the user to do call nvs_write() for all defined entries without worries about flash wear. Fixes garbage collection error where wrong data could be copied. Add nvs_delete() to allow deleting a stored entry. A deleted entry will not be copied to a new flash sector Include flash wear information in the README.md documentation 0/25 Update module after reviewers remarks, added documentation to nvs.h, removed README.md by nvs.rst in doc/subsystems folder 04/26 Update module after reviewers remarks, updated nvs.rst, added more documentation to samples/subsys/nvs/src/main.c, updated doxygen info in nvs.h (hope this time it works). 04/26 Update subsystems.rst to include nvs.restart 04/27 Updated nvs.c and nvs.h to avoid a possible flash deletion loop when the file system is full. 04/29 Updated nvs_write to detect and ignore deletes of non-existing items 05/06 Update NVS module to return standard error codes, removed low level API, added configuration options. NVS now uses the board dts to determine the flash storage location (FLASH_AREA_STORAGE_OFFSET). 05/06 Update nvs.rst. Updated intendation and added intermediate variables in nvs.c to make the code easier to read. 05/06 Update nvs.rst. 05/07 Update nvs.rst 05/08 Changed the API to a more standard file system API. 05/08 Removed cnt_max from nvs_read() as it is not used. 05/08 Removed #ifdef(CONFIG_NVS_LOG) from nvs_priv.h, now the module can be build with debugging off. 05/09 Removed configuration options for SECTOR_SIZE, SECTOR_COUNT and MAX_ELEM_SIZE. It is now easy to support multiple NVS filesystems on one or multiple devices. Changed logging to support newlib systems. Thanks to Olivier Martin for reporting and proposed changes. Signed-off-by: Laczen JMS <laczenjms@gmail.com>
2018-03-02 05:09:17 +08:00
*/
subsys: fs/nvs: Rewrite for improved robustness On flash NVS was stored one entry after another including the metadata of each entry. This has the disadvantage that when an incomplete write is performed (e.g. due to power failure) the complete sector had to be rewritten to get a completely functional system. The present rewrite changed the storage in flash of the data. For each sector the data is now written as follows: the data itself at the beginning of the sector (one after the other), the metadata (id, length, data offset in the sector, and a crc of the metadata) is written from the end of the sector. The metadata is of fixed size (8 byte) and for a sector that is completely occupied a metadata entry of all zeros is used. Writing data to flash always is done by: 1. Writing the data, 2. Writing the metadata. If an incomplete write is done NVS will ignore this incomplete write. At the same time the following improvements were done: 1. NVS now support 65536 sectors of each 65536 byte. 2. The sector size no longer requires to be a power of 2 (but it still needs to be a multiple of the flash erase page size). 3. NVS now also keeps track of the free space available. Signed-off-by: Laczen JMS <laczenjms@gmail.com>
2018-07-09 22:50:57 +08:00
#define ADDR_SECT_MASK 0xFFFF0000
#define ADDR_SECT_SHIFT 16
#define ADDR_OFFS_MASK 0x0000FFFF
subsys: fs/nvs: Improved nvs for larger blocksizes The nvs module has some disadvantages for larger block size. The data header and slot are taking up to much space. A rewrite is proposed that reduces the used storage space for systems with write block size > 4. The data storage in flash is now one unit consisting of: data_length, data_id, data and data_length again in a multiple of the write block size. The data_length at the end is used to validate the correctness of the flash write and also allows to travel backwards in the filesystem. As a comparison, on a system with block size 8 byte, a 32 bit values now fits 1 block including the metadata (length and id). This used to be 3 blocks. The data_length will occupy 1 byte if the data length is less than 128 byte, it will occupy 2 byte if the data length is 128 byte or more. The data length is limited to 16383 byte. Each write to flash is verified by a read back of the data. The read performance is improved because reading is done backwards so the latest items are found first. When the filesystem is locked it can be unlocked by calling reinit(), this will clear flash and setup everything for storage. add sample documentation - README.rst Update dtsi to include erase_block_size, use erase_block_size in sample Update prj.conf to include CONFIG_MPU_ALLOW_FLASH_WRITE Signed-off-by: Laczen JMS <laczenjms@gmail.com>
2018-06-01 22:13:54 +08:00
subsys: fs: Add Non Volatile Storage (NVS) for zephyr Zephyr supports fatfs, nffs and fcb as storage layer. fatfs and nffs are less suited for application in memory restricted IC's. fcb has a smaller footprint but has a complex api. The proposed module is a module with a even smaller footprint compared to fcb and a simple interface for reading and writing entries. The module provides wear levelling of flash. This allows the module to be used not only to store configuration settings but to store device state (e.g. state of a light switch over reboots) of a zephyr device. Fixes buffer overflow by introducing maximum read length in nvs_read() and nvs_read_hist(). Fixes nvs_write() not to reflash the same data. Allows the user to do call nvs_write() for all defined entries without worries about flash wear. Fixes garbage collection error where wrong data could be copied. Add nvs_delete() to allow deleting a stored entry. A deleted entry will not be copied to a new flash sector Include flash wear information in the README.md documentation 0/25 Update module after reviewers remarks, added documentation to nvs.h, removed README.md by nvs.rst in doc/subsystems folder 04/26 Update module after reviewers remarks, updated nvs.rst, added more documentation to samples/subsys/nvs/src/main.c, updated doxygen info in nvs.h (hope this time it works). 04/26 Update subsystems.rst to include nvs.restart 04/27 Updated nvs.c and nvs.h to avoid a possible flash deletion loop when the file system is full. 04/29 Updated nvs_write to detect and ignore deletes of non-existing items 05/06 Update NVS module to return standard error codes, removed low level API, added configuration options. NVS now uses the board dts to determine the flash storage location (FLASH_AREA_STORAGE_OFFSET). 05/06 Update nvs.rst. Updated intendation and added intermediate variables in nvs.c to make the code easier to read. 05/06 Update nvs.rst. 05/07 Update nvs.rst 05/08 Changed the API to a more standard file system API. 05/08 Removed cnt_max from nvs_read() as it is not used. 05/08 Removed #ifdef(CONFIG_NVS_LOG) from nvs_priv.h, now the module can be build with debugging off. 05/09 Removed configuration options for SECTOR_SIZE, SECTOR_COUNT and MAX_ELEM_SIZE. It is now easy to support multiple NVS filesystems on one or multiple devices. Changed logging to support newlib systems. Thanks to Olivier Martin for reporting and proposed changes. Signed-off-by: Laczen JMS <laczenjms@gmail.com>
2018-03-02 05:09:17 +08:00
/*
* Status return values
*/
#define NVS_STATUS_NOSPACE 1
subsys: fs/nvs: Rewrite for improved robustness On flash NVS was stored one entry after another including the metadata of each entry. This has the disadvantage that when an incomplete write is performed (e.g. due to power failure) the complete sector had to be rewritten to get a completely functional system. The present rewrite changed the storage in flash of the data. For each sector the data is now written as follows: the data itself at the beginning of the sector (one after the other), the metadata (id, length, data offset in the sector, and a crc of the metadata) is written from the end of the sector. The metadata is of fixed size (8 byte) and for a sector that is completely occupied a metadata entry of all zeros is used. Writing data to flash always is done by: 1. Writing the data, 2. Writing the metadata. If an incomplete write is done NVS will ignore this incomplete write. At the same time the following improvements were done: 1. NVS now support 65536 sectors of each 65536 byte. 2. The sector size no longer requires to be a power of 2 (but it still needs to be a multiple of the flash erase page size). 3. NVS now also keeps track of the free space available. Signed-off-by: Laczen JMS <laczenjms@gmail.com>
2018-07-09 22:50:57 +08:00
#define NVS_BLOCK_SIZE 8
/* Allocation Table Entry */
struct nvs_ate {
u16_t id; /* data id */
subsys: fs/nvs: improve some comments Use "within sector" instead of "sector", as it could be confused with the unit otherwise. Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
2018-09-15 18:02:05 +08:00
u16_t offset; /* data offset within sector */
u16_t len; /* data len within sector */
subsys: fs/nvs: Rewrite for improved robustness On flash NVS was stored one entry after another including the metadata of each entry. This has the disadvantage that when an incomplete write is performed (e.g. due to power failure) the complete sector had to be rewritten to get a completely functional system. The present rewrite changed the storage in flash of the data. For each sector the data is now written as follows: the data itself at the beginning of the sector (one after the other), the metadata (id, length, data offset in the sector, and a crc of the metadata) is written from the end of the sector. The metadata is of fixed size (8 byte) and for a sector that is completely occupied a metadata entry of all zeros is used. Writing data to flash always is done by: 1. Writing the data, 2. Writing the metadata. If an incomplete write is done NVS will ignore this incomplete write. At the same time the following improvements were done: 1. NVS now support 65536 sectors of each 65536 byte. 2. The sector size no longer requires to be a power of 2 (but it still needs to be a multiple of the flash erase page size). 3. NVS now also keeps track of the free space available. Signed-off-by: Laczen JMS <laczenjms@gmail.com>
2018-07-09 22:50:57 +08:00
u8_t part; /* part of a multipart data - future extension */
u8_t crc8; /* crc8 check of the entry */
subsys: fs/nvs: declare the ate structure as packed The allocation table entry should be as small as possible in the flash, so declare it as packed to avoid that the compiler pads it. Note that this doesn't change anything on ARM, but it might help for other (future) architectures. Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
2018-09-15 18:02:05 +08:00
} __packed;
subsys: fs: Add Non Volatile Storage (NVS) for zephyr Zephyr supports fatfs, nffs and fcb as storage layer. fatfs and nffs are less suited for application in memory restricted IC's. fcb has a smaller footprint but has a complex api. The proposed module is a module with a even smaller footprint compared to fcb and a simple interface for reading and writing entries. The module provides wear levelling of flash. This allows the module to be used not only to store configuration settings but to store device state (e.g. state of a light switch over reboots) of a zephyr device. Fixes buffer overflow by introducing maximum read length in nvs_read() and nvs_read_hist(). Fixes nvs_write() not to reflash the same data. Allows the user to do call nvs_write() for all defined entries without worries about flash wear. Fixes garbage collection error where wrong data could be copied. Add nvs_delete() to allow deleting a stored entry. A deleted entry will not be copied to a new flash sector Include flash wear information in the README.md documentation 0/25 Update module after reviewers remarks, added documentation to nvs.h, removed README.md by nvs.rst in doc/subsystems folder 04/26 Update module after reviewers remarks, updated nvs.rst, added more documentation to samples/subsys/nvs/src/main.c, updated doxygen info in nvs.h (hope this time it works). 04/26 Update subsystems.rst to include nvs.restart 04/27 Updated nvs.c and nvs.h to avoid a possible flash deletion loop when the file system is full. 04/29 Updated nvs_write to detect and ignore deletes of non-existing items 05/06 Update NVS module to return standard error codes, removed low level API, added configuration options. NVS now uses the board dts to determine the flash storage location (FLASH_AREA_STORAGE_OFFSET). 05/06 Update nvs.rst. Updated intendation and added intermediate variables in nvs.c to make the code easier to read. 05/06 Update nvs.rst. 05/07 Update nvs.rst 05/08 Changed the API to a more standard file system API. 05/08 Removed cnt_max from nvs_read() as it is not used. 05/08 Removed #ifdef(CONFIG_NVS_LOG) from nvs_priv.h, now the module can be build with debugging off. 05/09 Removed configuration options for SECTOR_SIZE, SECTOR_COUNT and MAX_ELEM_SIZE. It is now easy to support multiple NVS filesystems on one or multiple devices. Changed logging to support newlib systems. Thanks to Olivier Martin for reporting and proposed changes. Signed-off-by: Laczen JMS <laczenjms@gmail.com>
2018-03-02 05:09:17 +08:00
subsys: fs/nvs: simplify crc8 computation Instead of forcing the crc8 entry to 0xff for the crc8 computation, just ignore this field in the computation as it is the last one. This avoid having to set it back to the original value for _nvs_ate_crc8_check. Add a build assertion to ensure crc8 is kept last. Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
2018-09-15 18:02:05 +08:00
BUILD_ASSERT_MSG(offsetof(struct nvs_ate, crc8) ==
sizeof(struct nvs_ate) - sizeof(u8_t),
"crc8 must be the last member");
subsys: fs: Add Non Volatile Storage (NVS) for zephyr Zephyr supports fatfs, nffs and fcb as storage layer. fatfs and nffs are less suited for application in memory restricted IC's. fcb has a smaller footprint but has a complex api. The proposed module is a module with a even smaller footprint compared to fcb and a simple interface for reading and writing entries. The module provides wear levelling of flash. This allows the module to be used not only to store configuration settings but to store device state (e.g. state of a light switch over reboots) of a zephyr device. Fixes buffer overflow by introducing maximum read length in nvs_read() and nvs_read_hist(). Fixes nvs_write() not to reflash the same data. Allows the user to do call nvs_write() for all defined entries without worries about flash wear. Fixes garbage collection error where wrong data could be copied. Add nvs_delete() to allow deleting a stored entry. A deleted entry will not be copied to a new flash sector Include flash wear information in the README.md documentation 0/25 Update module after reviewers remarks, added documentation to nvs.h, removed README.md by nvs.rst in doc/subsystems folder 04/26 Update module after reviewers remarks, updated nvs.rst, added more documentation to samples/subsys/nvs/src/main.c, updated doxygen info in nvs.h (hope this time it works). 04/26 Update subsystems.rst to include nvs.restart 04/27 Updated nvs.c and nvs.h to avoid a possible flash deletion loop when the file system is full. 04/29 Updated nvs_write to detect and ignore deletes of non-existing items 05/06 Update NVS module to return standard error codes, removed low level API, added configuration options. NVS now uses the board dts to determine the flash storage location (FLASH_AREA_STORAGE_OFFSET). 05/06 Update nvs.rst. Updated intendation and added intermediate variables in nvs.c to make the code easier to read. 05/06 Update nvs.rst. 05/07 Update nvs.rst 05/08 Changed the API to a more standard file system API. 05/08 Removed cnt_max from nvs_read() as it is not used. 05/08 Removed #ifdef(CONFIG_NVS_LOG) from nvs_priv.h, now the module can be build with debugging off. 05/09 Removed configuration options for SECTOR_SIZE, SECTOR_COUNT and MAX_ELEM_SIZE. It is now easy to support multiple NVS filesystems on one or multiple devices. Changed logging to support newlib systems. Thanks to Olivier Martin for reporting and proposed changes. Signed-off-by: Laczen JMS <laczenjms@gmail.com>
2018-03-02 05:09:17 +08:00
#ifdef __cplusplus
}
#endif
#endif /* __NVS_PRIV_H_ */