mapping out eeprom space

[Simon]

I have need to write a few byte to EEPROM on a SAMC (this is actually a dedicated area of the flash memory but for clarity I will refer to it as EEPROM)

I can't constantly write this data as it will wear the memory out. I would copy the data from the EEPROM at start up and hold it in a user buffer to then write it back to EEPROM on shut down.

Now in my instance I think all of my data will be uint32 anyway so I can create an array, but what about being able to deal with any data?

The only way I can think to do it is to create a structure with each variable defined.

I suppose I could do a union of my variable structure and my array.

This would probably be required anyway as a way to get data in and out by name rather than (array) address whilst the array still allows me to write code that can cycle through the memory space when writing or reading the EEPROM.

[Nominal Animal]

There was a related thread recently started by peter-h you missed, ideas for storing frequently updated data in a flash chip.

[Simon]

No I'm not asking about how to stop the wearing. As i said I have but a few bytes so I have made my plan. Read the data out at power up to RAM (put it into a variable). Update the variable during run as required. When power is about to be removed write the data back to the EEPROM.

So to handle the EEPROM it seems to make sense to deal in 32 bit chunks and make an array of these chunks. This means that I can keep track of pages and rows in a 3 dimensional array. The array can then be easily loaded up from the EEPROM or written to the EEPROM by cycling through each 32 bit chunk of data treating it as a uint32.

But in my program it makes sense to be able to use any data type and to use sensible names rather than eeprom[row][page][word] so I believe a union of that array and a structure would allow me to use the structured variables in the program and use the addressability of the array when I want to write the data or read it to EEPROM.

[Nominal Animal]

Ah, okay.

But in my program it makes sense to be able to use any data type and to use sensible names rather than eeprom[row][page][word] so I believe a union of that array and a structure would allow me to use the structured variables in the program and use the addressability of the array when I want to write the data or read it to EEPROM.

The other alternative is to write a pair of functions, pack and unpack, that copy the data between global variables and EEPROM.

Because this decouples the long-term storage format from the RAM format, it lets you pick more sensible data types for each variable, and optionally save some room.  For example, it makes sense for an often-accessed value to be a full register wide integer, but if it is actually limited to a much smaller range, say -50 to +205, you can actually store it in a single unsigned byte with a fixed offset (-50).

Another benefit from this is in that especially the pack function can compare the data to be stored and the existing data, and make an executive decision whether an update is necessary or not.

However.

On SAMC, the smallest erase unit is one row, or four 64 byte pages; i.e. 256 bytes.  Do you really need all those 2048 bits to store your state?  If not, then the aforementioned wear leveling thread is applicable.

Basically, if your data fits in 127, 84, 63, 50, 41, 34, 31, ... = floor(256/n)-1 bytes with n≥2  (easier if they fit in floor(64/n)-1 bytes, since write granularity is a page and not a row), and you reserve at least two rows (eight pages, or 512 bytes) of Flash, you can efficiently reduce the wear on your Flash storage by only erasing the other (next) row when the previous one is full, and using a single additional byte as a sequential counter to indicate which one was written last.  (AFAIK, SAMC too allows multiple writes to the same Flash page as long as the previously written data is unchanged.  This means that only the 0x00 and 0xFF counter values need to be reserved for erased pages (I can't be arsed to check which one SAMC erases to) for detecting "unused" record slots that can be filled without erasing the page.

[Simon]

The EEPROM will do 100'000 cycles, the machine will be powered on a few times a day, lets say 10 times.

100'000 / 10 = 10'000 days
10'000 /  5 = 2'000 weeks
2'000 / 50 = 40 years.....

i have a pack/unpack method:

Code: [Select]

void read_eeprom_to_user_buffer()
{
uint8_t wordcounter ;
uint8_t pagecounter  ;
uint8_t rowcounter = 0 ;

while ( rowcounter < d_eeprom_rows)
{
pagecounter = 0 ;

while ( pagecounter < d_eeprom_pages_per_row )
{
wordcounter = 0 ;

while (wordcounter < d_eeprom_words_per_page)
{
if ( REG_NVMCTRL_INTFLAG & 0x01 )
{
v_eeprom_user_buffer[rowcounter][pagecounter][rowcounter] = register32(FLASH_USER_PAGE_ADDR + wordcounter * 4 + pagecounter * FLASH_USER_PAGE_SIZE + rowcounter * FLASH_USER_PAGE_SIZE * d_eeprom_pages_per_row ) ;
wordcounter ++ ;
}
}

pagecounter ++ ;
}

rowcounter ++ ;
}
}

I do similar to write back and anther similar function goes row by row to erase the contents for the entire area ready for a bulk write.

I am using 1 256 byte row, yes they go that small. Obviously this will not work to control the entire available EEPROM space as there is 16kB of it and of RAM, this is just to store stuff that needs loading at power up and saving again at power down, like distance travelled, run time etc.

So to use that array in my program is clunky, I think I can union it with a struct so that I have variable names like eeprom.my_variable, I don't have to care where it goes in the EEPROM, the array covers the same space and will dealt with as above when it comes to storing.

so I am thinking of something like:

Code: [Select]


union {
volatile struct nv_variables
{
uint32_t metres_since_new ;
uint32_t metres_since_service ;
uint32_t decimetres_since_power_up ;
uint32_t decimetres_at_power_up ;
}  nv ;
volatile uint32_t v_eeprom_user_buffer[d_eeprom_rows][d_eeprom_pages_per_row][d_eeprom_words_per_page] ;
} nv ;


so what do I call these? nv.nv.metres_since_new  or nv.metres_since_new  ?

[Simon]

OK, so I think I have got this right:

Code: [Select]

union nv
{
volatile struct
{
uint32_t metres_since_new ;
uint32_t metres_since_service ;
uint32_t decimetres_since_power_up ;
uint32_t decimetres_at_power_up ;
} machine ;
volatile uint32_t v_eeprom_user_buffer[d_eeprom_rows][d_eeprom_pages_per_row][d_eeprom_words_per_page] ;
} nvv ;


So I can now use

Code: [Select]


nvv.v_eeprom_user_buffer[x][y][z]


or

Code: [Select]


nvv.machine.metres_since_new



[Nominal Animal]

What I meant by a pack/unpack routines was more like

Code: [Select]

/* The following are stored in EEPROM across reboots */
uint32_t  metres_since_new;
uint32_t  metres_since_service;
uint32_t  decimetres_since_power_up;
uint32_t  decimetres_at_power_up;

void load_globals(void)
{
    uint32_t  cache[3];

    /* Code that loads the most recently written 12 bytes from EEPROM to cache[] omitted */

    metres_since_new = cache[0];
    metres_since_service = cache[1];
    decimetres_since_power_up = 0;
    decimetres_at_power_up = cache[2];
}

void save_globals(void)
{
    uint32_t  cache[3] = { metres_since_new,  /* Or (metres_since_new + decimetres_since_power_up/10) ? */
                           metres_since_sevice, /* Or (metres_since_service + decimetres_since_power_up/10) ? */
                           decimetres_since_power_up + decimetres_at_power_up };

    /* Code that writes cache[] to EEPROM omitted */
}

Essentially, you have a cache of some fixed length copied to and from the EEPROM.  (If you don't use the REG_NVMCTL_INTFLAG fuse, you can use a counter at the beginning of each fixed size block.  You only need to divide the counter value into four blocks, A, B, C, D, so that values in A are less than those in B, those in B are less than those in C, those in C are less than those in D, and those in D are less than A, making the counter a cyclic one.)

Instead of copying the 32-bit values as-is like above, you can pack/extract them as sets of bits instead.

[Simon]

we are talking about broadly the same thing. Automated code works well with arrays but array data is not very verbose. So what I suggest is a union of a verbose structure and an array that is more useful for  mass treatment.

So at power up the EEPROM is copied to the array, the array occupies the same memory as the structured variables that are used in the program. When the device is to power down the array that has been updated by the variables in the structure being written to that are in the same memory location are written to the EEPROM, so one read and write from EEPROM on each power cycle and the variables reside in RAM during machine use.

[Nominal Animal]

Yep.  My suggestion differs in that that the values do not need to reside in a single structure, and can be basically any values in the global scope.

The cache[] is just a continuous chunk of bits/bytes/words, whatever is most appropriate, copied between RAM and EEPROM.  The actual variable values are packed to and from the cache[].  It is this extra step that can be useful.

For example, did you notice how in my example, at save time, the decimetres_since_power_up is folded to the value loaded into decimetres_at_power_up, and that load time, decimetres_since_power_up is always zeroed?  That really, only decimetres_since_power_up needs to be updated during runtime?
And that is why my version only really saves three of the four variables?

I don't see anything wrong in using a structure the way you are using it.  I just prefer the two-step unpack/pack function approach, since it seems more versatile/powerful to me.

[Simon]

Well I don't need to limit cycles, I can still move to another section after so many writes.

[bson]

If it's a small amount of data you can include a version and write a new record each time.  If it's say 256 bytes and the bank is erased you write it to 0x0000 as version 1.  The next time you write version 2 to 0x0100.  Then v3 to 0x0200.  And so on, until the bank is full.  At that point you erase the flash bank and write your current record, v256 to 0x0000.  On boot you scan the bank and locate the highest version number with a valid checksum.  It's especially quick and easy to scan if it's memory mapped.   It also requires being able to make arbitrary writes to a sector, but that's par for NOR flash or EEPROM.

[Simon]

well i will likely have a run time counter variable so the largest one automatically tells me which was the last unit of memory used, I can then move to the next page. But I am far from that at the moment. I am just trying to work out a way to manage in program variables like other variables whilst being able to copy them in and out of the EEPROM in quantity.

[Simon]

Well I am going in circles as C really does not seem to take kindly to unions of structures and arrays. I also realize that I am overthinking my array, it does not need to be 3 dimensional just 1, like the EEPROM space is.

That makes my reading/writing of the entire EEPROM function much simpler:

Code: [Select]


void read_eeprom_to_user_buffer()
{
uint8_t wordcounter = 0 ;
while (wordcounter < d_eeprom_words)
{
if ( REG_NVMCTRL_INTFLAG & 0x01 )
{
eeprom[wordcounter] = register32(FLASH_USER_PAGE_ADDR + wordcounter * 4  ) ;
wordcounter ++ ;
}
}
}


This also means that my array names are not so weird if I make defines of the indices. The only generic problem would be that the compiler will expect me to put data in there in whatever type the array is declare in.

[JPortici]

Well I am going in circles as C really does not seem to take kindly to unions of structures and arrays.

nah, i use it all the time

Code: [Select]

typedef struct {
  uint16_t param1;
  uint16_t param2;
  float param3;
  ...
} eeData_t;
typedef union {
  uint8_t byte[sizeof flash page or whatever];
  eeData_t eeData;
} eeDataInFlash_t;
firmware loads data in ram from eeprom (or dedicate flash page), changes the content in ram and set the condition for update (like update in 500ms, counter reset every time a new write is performed) then update eeprom/flash

[Simon]

How do you declare it in a header file so that other code can see the union?

[Siwastaja]

.h

Code: [Select]

typedef union
{
   struct __attribute__((packed))
   {
      int a;
      float b;
   };

    uint8_t u8[8];
} storage_t;

extern storage_t storage;

one of the .c files

Code: [Select]

storage_t storage;

Can be accessed from any .c file by including the .h.

You can add __attribute__((section(".something"))) to storage_t storage; definition, then use linker script to choose where to put it.

[Simon]

Don't you have things a bit mixed up there?

You mean:

Code: [Select]

*.c

typedef union
{
   struct __attribute__((packed))
   {
      int a;
      float b;
   };

    uint8_t u8[8];
} storage_t;

storage_t storage;



Code: [Select]


*.h

extern storage_t storage;


I thought I did that but it did not work

[Simon]

OK I see it. The typdef has to go in the header so that both the declaration and initialization of the variable can see it.

[Siwastaja]

You can of course put whatever you want in .c and .h files, and #include whatever, but the common and very good practice is like this:

Headers describe the interface and do not do anything that create actual memory-consuming objects. So just types, and declarations. Remember,
void func();
is a declaration, exactly same as
extern void func();

Declaration just means knowledge about something existing, and how it is used (types).

But
int var;
Is both a declaration and definition (actual storage);
extern int var;
is just the declaration (information about type, and the fact it exists somewhere else).

With this in mind,

module1.h

Code: [Select]

/*
   Comments about module1 and how it is used. Someone reading this might not even have module1.c
*/
#pragma once

typedef struct
{
    int a;
    int b;
} my_interface_type_t;

int interface_function(my_interface_type_t *param);

extern int module1_global_variable;

module1.c

Code: [Select]


int module1_global_variable; // memory is statically allocated once, here.

static int our_private_variable;

typedef struct
{
    int arr[57];
} our_own_type_t; // outside world doesn't need to know about this type, only module1.c uses it, so we didn't put it in .h file.

static void our_internal_function()
{
   // implementation of a function only module1.c uses
}

int interface_function(my_interface_type_t *param)
{
   // implementation of the exposed function, can be used from module1, but also elsewhere
}

and finally, module2.c

Code: [Select]

#include "module1.h"

static void our_internal_function() // can even use the same name, nothing to do with the previous one, only module2 knows about this
{
    module1_global_variable++; // we can access module1 global because it's not static, and because it's declared extern in the header file we included

    // we can access this type:
    my_interface_type_t ohyeah = {1, 2};
    // ... and call the function:

    int retval = interface_function(&ohyeah);
}

It's sad this isn't taught anywhere. (I'm thinking about university introductory CS classes where they taught completely bollocks, for example totally mixing up "global variables" and compilation unit scope static variables. They didn't know the latter exist, claimed they are globals and evil.)

Note that static controls the exposure of the variables and functions. Every function and variable which is used only from that module should be static.

[Simon]

It's sad this isn't taught anywhere. (I'm thinking about university introductory CS classes where they taught completely bollocks, for example totally mixing up "global variables" and compilation unit scope static variables. They didn't know the latter exist, claimed they are globals and evil.)

Oh you are lucky, my university tutors cannot even write comprehensible material in their own language, never mind getting any technical facts straight.