TFDB
v0.0.7 Release
Tiny Flash Database for MCU.
In the daily development of microcontrollers, some information is always needed. At this time, a solution for microcontroller FLASH storage is needed. Currently, there are many solutions for microcontroller storage, such as: EASYFLASH, FLASHDB, OSAL_NV, etc. Their programs are very large and are not worth storing few variables. And it is rare to consider flash write errors.
In actual products, the flash write of embedded products may be affected by various factors (battery powered, unexpected power outage, temperature, etc.) and is not very stable. Once an error occurs, it will lead to a series of product problems.
Unlike many other KV-type databases, each variable that needs to be stored in TinyFlashDB will be allocated a separate microcontroller flash sector, and the variable length is immutable.
Therefore, TinyFlashDB is only suitable for storing several key variables (such as: IAP jump flag, system power outage time, etc.), and is not suitable for large-scale data storage (large-scale data storage can use EASYFLASH, etc.).
TinyFlashDB was designed to consider the impact of write errors, pursue security guarantees within its ability, resource occupation as much as possible (less than 1kb code occupies), and universality as possible (can be ported to 8-bit computers such as 51, stm32L4 series that cannot be written in reverse order, some flash-encrypted microcontrollers and other ordinary 32-bit computers).
const tfdb_index_t test_index = {
. end_byte = 0x00 ,
. flash_addr = 0x4000 ,
. flash_size = 256 ,
. value_length = 2 ,
}; /* c99写法，如果编译器不支持，可自行改为c89写法 */
tfdb_addr_t addr = 0 ; /*addr cache*/
uint8_t test_buf [ TFDB_ALIGNED_RW_BUFFER_SIZE ( 2 , 1 )]; /*aligned_value_size*/
uint16_t test_value ;
void main ()
{
TFDB_Err_Code result ;
result = tfdb_set ( & test_index , test_buf , & addr , & test_value );
if ( result == TFDB_NO_ERR )
{
printf ( "set ok, addr:%xn" , addr );
}
addr = 0 ; /* reset addr cache, to see tfdb_get. */
result = tfdb_get ( & test_index , test_buf , & addr , & test_value );
if ( result == TFDB_NO_ERR )
{
printf ( "get ok, addr:%x, value:%xn" , addr , test_value );
}
} typedef struct _tfdb_index_struct {
tfdb_addr_t flash_addr ; /* the start address of the flash block */
uint16_t flash_size ; /* the size of the flash block */
uint8_t value_length ; /* the length of value that saved in this flash block */
uint8_t end_byte ; /* must different to TFDB_VALUE_AFTER_ERASE */
/* 0x00 is recommended for end_byte, because almost all flash is 0xff after erase. */
} tfdb_index_t ;Structure function: In TinyFlashDB, API operations require the specified parameter index. This index structure stores the address of flash, the size of flash, the length of stored variables, and the end flag. This information will be checked when reading the flash sector.
TFDB_Err_Code tfdb_get ( const tfdb_index_t * index , uint8_t * rw_buffer , tfdb_addr_t * addr_cache , void * value_to ); Function function: Get a variable with a specified variable length in index from the sector pointed to by index . Flash header data verification error will not re-initialize flash.
Parameter index : index pointer for tfdb operation.
Parameter rw_buffer : write and read cache. All flash operations will finally copy the sorted data into the buffer, and then call tfdb_port_write or tfdb_port_read for reading and writing. When the chip has special requirements for the written data area cache (such as 4-byte alignment, 256-byte alignment, etc.), the variable pointer that meets the requirements can be passed to the function for use through this parameter. At least 4 bytes in length.
Parameter addr_cache : can be NULL , or a pointer to the address cache variable. When addr_cache is not NULL and is not 0, addr_cache is considered to have been initialized successfully, no longer check the flash header, and data is read directly from the address of addr_cache .
Parameter value_to : The address to store the data content.
Return value: TFDB_NO_ERR succeeds, others fail.
TFDB_Err_Code tfdb_set ( const tfdb_index_t * index , uint8_t * rw_buffer , tfdb_addr_t * addr_cache , void * value_from ); Function function: Write a variable with a specified variable length in the index into the sector pointed to by index , and flash header data verification error is errored and flash is reinitialized.
Parameter index : index pointer for tfdb operation.
Parameter rw_buffer : write and read cache. All flash operations will finally copy the sorted data into the buffer, and then call tfdb_port_write or tfdb_port_read for reading and writing. When the chip has special requirements for the written data area cache (such as 4-byte alignment, 256-byte alignment, etc.), the variable pointer that meets the requirements can be passed to the function for use through this parameter. At least 4 bytes in length.
Parameter addr_cache : can be NULL , or a pointer to the address cache variable. When addr_cache is not NULL and is not 0, addr_cache is considered to have been initialized successfully, no longer check the flash header, and data is read directly from the address of addr_cache .
Parameter value_from : The data content to be stored.
Return value: TFDB_NO_ERR succeeds, others fail.
The tfdb dual API is encapsulated based on tfdb_set and tfdb_get . tfdb dual will call tfdb_set and tfdb_get and add two bytes of seq to the front of the data, so in the tfdb dual, the longest supported storage variable length is 253 bytes.
At the same time, the tfdb dual API needs to provide two buffers, and it needs to be aligned_value_size increases the length of the two-byte variable and then recalculates it.
typedef struct _my_test_params_struct
{
uint32_t aa [ 2 ];
uint8_t bb [ 16 ];
} my_test_params_t ;
my_test_params_t my_test_params = {
1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18
};
tfdb_dual_index_t my_test_tfdb_dual = {
. indexes [ 0 ] = {
. end_byte = 0x00 ,
. flash_addr = 0x08077000 ,
. flash_size = 256 ,
. value_length = TFDB_DUAL_VALUE_LENGTH ( sizeof ( my_test_params_t )),
},
. indexes [ 1 ] = {
. end_byte = 0x00 ,
. flash_addr = 0x08077100 ,
. flash_size = 256 ,
. value_length = TFDB_DUAL_VALUE_LENGTH ( sizeof ( my_test_params_t )),
},
};
tfdb_dual_cache_t my_test_tfdb_dual_cache = { 0 };
void my_test_tfdb_dual_func ()
{
uint32_t rw_buffer [ TFDB_DUAL_ALIGNED_RW_BUFFER_SIZE ( TFDB_DUAL_VALUE_LENGTH ( sizeof ( my_test_params_t )), 4 )];
uint32_t rw_buffer_bak [ TFDB_DUAL_ALIGNED_RW_BUFFER_SIZE ( TFDB_DUAL_VALUE_LENGTH ( sizeof ( my_test_params_t )), 4 )];
TFDB_Err_Code err ;
for ( uint8_t i = 0 ; i < 36 ; i ++ )
{
err = tfdb_dual_get ( & my_test_tfdb_dual , ( uint8_t * ) rw_buffer , ( uint8_t * ) rw_buffer_bak , & my_test_tfdb_dual_cache , & my_test_params );
if ( err == TFDB_NO_ERR )
{
printf ( "read okncache seq1:0x%04x, seq2:0x%04xnaddr1:0x%08x, addr2:0x%08xn" , my_test_tfdb_dual_cache . seq [ 0 ], my_test_tfdb_dual_cache . seq [ 1 ], my_test_tfdb_dual_cache . addr_cache [ 0 ], my_test_tfdb_dual_cache . addr_cache [ 1 ]);
}
else
{
printf ( "read err:%dn" , err );
}
my_test_params . aa [ 0 ] ++ ;
my_test_params . aa [ 1 ] ++ ;
for ( uint8_t i = 0 ; i < 16 ; i ++ )
{
my_test_params . bb [ i ] ++ ;
}
memset ( & my_test_tfdb_dual_cache , 0 , sizeof ( my_test_tfdb_dual_cache )); /* 测试无地址缓存写入 */
err = tfdb_dual_set ( & my_test_tfdb_dual , ( uint8_t * ) rw_buffer , ( uint8_t * ) rw_buffer_bak , & my_test_tfdb_dual_cache , & my_test_params );
if ( err == TFDB_NO_ERR )
{
printf ( "write okncache seq1:0x%04x, seq2:0x%04xnaddr1:0x%08x, addr2:0x%08xn" , my_test_tfdb_dual_cache . seq [ 0 ], my_test_tfdb_dual_cache . seq [ 1 ], my_test_tfdb_dual_cache . addr_cache [ 0 ], my_test_tfdb_dual_cache . addr_cache [ 1 ]);
}
else
{
printf ( "write err:%dn" , err );
}
memset ( & my_test_tfdb_dual_cache , 0 , sizeof ( my_test_tfdb_dual_cache )); /* 测试无地址缓存读取 */
}
} typedef struct _tfdb_dual_index_struct
{
tfdb_index_t indexes [ 2 ];
} tfdb_dual_index_t ;
typedef struct _tfdb_dual_cache_struct
{
tfdb_addr_t addr_cache [ 2 ];
uint16_t seq [ 2 ];
} tfdb_dual_cache_t ; Structure function: In the TinyFlashDB dual, API operations require the specified parameter index , index stores two tfdb_index_t .
TFDB_Err_Code tfdb_dual_get ( const tfdb_dual_index_t * index , uint8_t * rw_buffer , uint8_t * rw_buffer_bak , tfdb_dual_cache_t * cache , void * value_to );Function function: Get a variable with a specified variable length in index from the sector pointed to by index. Flash header data verification error will not re-initialize flash.
Parameter index : index pointer for tfdb operation.
Parameter rw_buffer : write and read cache. All flash operations will finally copy the sorted data into the buffer, and then call tfdb_port_write or tfdb_port_read for reading and writing. When the chip has special requirements for the written data area cache (such as 4-byte alignment, 256-byte alignment, etc.), the variable pointer that meets the requirements can be passed to the function for use through this parameter. At least 4 bytes in length.
Parameter rw_buffer_bak : write and read cache. All flash operations will finally copy the sorted data into the buffer, and then call tfdb_port_write or tfdb_port_read for reading and writing. When the chip has special requirements for the written data area cache (such as 4-byte alignment, 256-byte alignment, etc.), the variable pointer that meets the requirements can be passed to the function for use through this parameter. At least 4 bytes in length.
Parameter cache : cannot be NULL , it must be a cached pointer defined by tfdb_dual_cache_t . When the data in cache is legal, it is considered that cache has been initialized successfully and the data is read directly from the flash block and address of cache .
Parameter value_to : The address to store the data content.
Return value: TFDB_NO_ERR succeeds, others fail.
TFDB_Err_Code tfdb_dual_set ( const tfdb_dual_index_t * index , uint8_t * rw_buffer , uint8_t * rw_buffer_bak , tfdb_dual_cache_t * cache , void * value_from );Function function: Write a variable with a specified variable length in the index into the sector pointed to by index, and flash header data verification error is errored and flash is reinitialized.
Parameter index : index pointer for tfdb operation.
Parameter rw_buffer : write and read cache. All flash operations will finally copy the sorted data into the buffer, and then call tfdb_port_write or tfdb_port_read for reading and writing. When the chip has special requirements for the written data area cache (such as 4-byte alignment, 256-byte alignment, etc.), the variable pointer that meets the requirements can be passed to the function for use through this parameter. At least 4 bytes in length.
Parameter rw_buffer_bak : write and read cache. All flash operations will finally copy the sorted data into the buffer, and then call tfdb_port_write or tfdb_port_read for reading and writing. When the chip has special requirements for the written data area cache (such as 4-byte alignment, 256-byte alignment, etc.), the variable pointer that meets the requirements can be passed to the function for use through this parameter. At least 4 bytes in length.
Parameter cache : cannot be NULL , it must be a cached pointer defined by tfdb_dual_cache_t . When the data in cache is legal, it is considered that cache has been initialized successfully and the data is read directly from the flash block and address of cache .
Parameter value_from : The data content to be stored.
Return value: TFDB_NO_ERR succeeds, others fail.
Observing the above code, you can find that TinyFlashDB operations require index parameters defined by tfdb_index_t .
After Flash is initialized, the header information is 4 bytes, so only flash with 1, 2, 4, and 8 byte operations is supported:
The header will be read when the header is initialized, so the first requirement for the data pointed to by rw_buffer in the function is at least 4 bytes. If the minimum write unit is 8 bytes, the first requirement is at least 8 bytes.
| First byte | Second byte | Byte 3 | Fourth byte and other aligned bytes |
|---|---|---|---|
| flash_size higher 8-bit bytes | flash_size low 8-bit bytes | value_length | end_byte |
When data is stored, it will be aligned according to byte operations supported by flash, so the second data pointed to by rw_buffer in the function is required to be at least aligned_value_size bytes calculated in the following function:
aligned_value_size = index -> value_length + 2 ; /* data + verify + end_byte */
#if ( TFDB_WRITE_UNIT_BYTES == 2 )
/* aligned with TFDB_WRITE_UNIT_BYTES */
aligned_value_size = (( aligned_value_size + 1 ) & 0xfe );
#elif ( TFDB_WRITE_UNIT_BYTES == 4 )
/* aligned with TFDB_WRITE_UNIT_BYTES */
aligned_value_size = (( aligned_value_size + 3 ) & 0xfc );
#elif ( TFDB_WRITE_UNIT_BYTES == 8 )
/* aligned with TFDB_WRITE_UNIT_BYTES */
aligned_value_size = (( aligned_value_size + 7 ) & 0xf8 );
#endif| First value_length bytes | value_length+1 byte | value_length+2 bytes | Other aligned bytes |
|---|---|---|---|
| value_from data content | value_from sum verification | end_byte | end_byte |
After each write, it will be read out for verification. If the verification fails, it will continue to try to write at the next address. Until the maximum number of writes (TFDB_WRITE_MAX_RETRY) is reached or the header verification error is wrong.
When reading data, it will also be calculated and checked. If it does not pass, it will continue to read until the latest data that has passed the verification is returned, or the reading fails.
There are only 3 legal values for the two-byte seq in the front of the data, 0x00ff->0x0ff0->0xff00.
This cycle repeats itself, by reading the seq of the latest variable in the two blocks, we can determine which flash sector is the latest value stored in the flash sector.
When the latest value is stored in the first sector, the next write will be written in the second sector, and vice versa.
TFDB_Err_Code tfdb_port_read ( tfdb_addr_t addr , uint8_t * buf , size_t size );
TFDB_Err_Code tfdb_port_erase ( tfdb_addr_t addr , size_t size );
TFDB_Err_Code tfdb_port_write ( tfdb_addr_t addr , const uint8_t * buf , size_t size ); /* use string.h or self functions */
#define TFDB_USE_STRING_H 1
#if TFDB_USE_STRING_H
#include "string.h"
#define tfdb_memcpy memcpy
#define tfdb_memcmp memcmp
#define TFDB_MEMCMP_SAME 0
#else
#define tfdb_memcpy
#define tfdb_memcmp
#define TFDB_MEMCMP_SAME
#endif
#define TFDB_DEBUG printf
/* The data value in flash after erased, most are 0xff, some flash maybe different.
* if it's over 1 byte, please be care of little endian or big endian. */
#define TFDB_VALUE_AFTER_ERASE 0xff
/* The size of TFDB_VALUE_AFTER_ERASE, only support 1 / 2 / 4.
* This value must not bigger than TFDB_WRITE_UNIT_BYTES. */
#define TFDB_VALUE_AFTER_ERASE_SIZE 1
/* the flash write granularity, unit: byte
* only support 1(stm32f4)/ 2(CH559)/ 4(stm32f1)/ 8(stm32L4) */
#define TFDB_WRITE_UNIT_BYTES 8 /* @note you must define it for a value */
/* @note the max retry times when flash is error ,set 0 will disable retry count */
#define TFDB_WRITE_MAX_RETRY 32
/* must not use pointer type. Please use uint32_t, uint16_t or uint8_t. */
typedef uint32_t tfdb_addr_t ;After removing DEBUG printing information, the resource occupies as follows:
keil -o2 compilation optimization options
Code ( inc . data ) RO Data RW Data ZI Data Debug Object Name
154 0 0 0 0 2621 tfdb_port . o
682 0 0 0 0 4595 tinyflashdb . ogcc -os compilation optimization options
. text . tfdb_port_read
0x00000000000039b4 0x1a ./ Drivers / TFDB / tfdb_port . o
0x00000000000039b4 tfdb_port_read
. text . tfdb_port_erase
0x00000000000039ce 0x46 ./ Drivers / TFDB / tfdb_port . o
0x00000000000039ce tfdb_port_erase
. text . tfdb_port_write
0x0000000000003a14 0x5c ./ Drivers / TFDB / tfdb_port . o
0x0000000000003a14 tfdb_port_write
. text . tfdb_check
0x0000000000003a70 0x56 ./ Drivers / TFDB / tinyflashdb . o
0x0000000000003a70 tfdb_check
. text . tfdb_init
0x0000000000003ac6 0x56 ./ Drivers / TFDB / tinyflashdb . o
0x0000000000003ac6 tfdb_init
. text . tfdb_set
0x0000000000003b1c 0x186 ./ Drivers / TFDB / tinyflashdb . o
0x0000000000003b1c tfdb_set
. text . tfdb_get
0x0000000000003ca2 0x11c ./ Drivers / TFDB / tinyflashdb . o
0x0000000000003ca2 tfdb_get Bare Metal Migration Routine, RT-Thread can be used with reference:
STM32F429IGT6
CH583
QQ communication group: 562090553
