EEPROM wear levelling techniques

[HwAoRrDk]

I'm working on a project where I'm storing some persistent settings in my microcontroller's built-in EEPROM. Because the settings will be written every time the user changes something, which could be several times a day, I've decided to try and employ some wear levelling techniques so that the EEPROM doesn't wear out too quickly. The EEPROM in my microcontroller is rated for 10k write/erase cycles with 20 year retention, and 100k cycles with 1 year retention.

The primary technique I've decided to use is the common one of treating the entire EEPROM area as a cyclical array of entries, and each time the settings are written, it does so at the next entry to the last used one, wrapping around appropriately. In order to determine which entry is the latest, each entry has a monotonically incrementing sequence number, and the latest is whichever one has the greatest number (determined by scanning the entire EEPROM). I can store 16 entries, so that multiplies the most conservative rated cycle count to 160,000 writes. If the settings are written 20 times a day, that gives a lifespan of 8,000 days, or nearly 22 years, which should be suitable for my needs.

One other thing I am doing is that because the datasheet states that the granularity of the EEPROM is 4 bytes ("cycling is performed on 4 bytes even when a write/erase operation addresses a single byte"), I have made the sequence number a uint32_t, and the settings struct is padded to be a multiple of 4 bytes in size.

My first query is with the sequence number. Should I be concerned with roll-over? Now, a maximum possible value of 4,294,967,296 is capability for a lot of writes, and I'm confident that the EEPROM will have expired long before this sequence value ever gets near rolling over, but maybe there's something I'm not thinking of...

The other question I have is whether it is worth doing something like inverting (i.e. taking one's compliment of) the sequence number when writing/reading so that, for a long while at least, the majority of the bits in the sequence value are 1s rather than 0s. That is, if the sequence number is 2000 it gets written as 0xFFFFF82F rather than 0x000007D0. As I understand things, because the natural default value of an EEPROM is 0xFF (all 1s), when you write a value only the 0s perform a wear-inducing operation, and writing of 1s is essentially a 'no-op'. And so always writing the zero-value MSBs will wear things out quicker. Am I right? Should I do this? Or is it pointless, because the LSBs of the sequence numbers will still wear out first, and if they are corrupted, then it doesn't matter if the MSBs are still fine?

Also, is there any potential in using Gray code for the sequence number? IIRC, counting up with Gray code only ever changes one bit per increment.

Anyone got any other advice or insights?

[Psi]

My first query is with the sequence number. Should I be concerned with roll-over? Now, a maximum possible value of 4,294,967,296 is capability for a lot of writes, and I'm confident that the EEPROM will have expired long before this sequence value ever gets near rolling over, but maybe there's something I'm not thinking of...

Do you really need to store the sequence number?
Can you write them in order and then just scan for the "latest" one by its position when you do a read?
You'd need two blocks of eeprom for this approach, when one block is full you scan and copy all the "latest" entries into the other block (which becomes the new primary) and also erase the old block so it can be used later

[newbrain]

I have implemented something similar, but geared to the handling of a number (15+1) of presets.
Each preset has a fixed area in the EEPROM, and in each area wear leveling is done writing sequentially in 16 different slots.

Due to space constraints, I use an 8 bit sequence number.
Sequence numbers are checked at power-on, the highest is used as the current value, and the lowest to decide where to write next.
To avoid rollover confusion there is some conditional comparisons: if the difference between the max and the min is greater than 16, then a rollover has occurred, and the highest of the lower numbers is selected.

Presets can be manually loaded or stored by the user, with the exception of Preset 0, which is auto-saved 10 seconds after the last change to the settings.
The delay avoids a lot of useless writes if the user is tuning something or selecting some mode in sequence.

Each slot is CRC protected, to discard corrupted slots (e.g. if power went down while writing) when searching for the latest - they are rewritten at power-on with the latest values or with a default preset if all slots are corrupted.

You'd need two blocks of eeprom for this approach, when one block is full you scan and copy all the "latest" entries into the other block (which becomes the new primary) and also erase the old block so it can be used later

That's somewhat wasteful - in this way you always have an empty block, effectively halving the storage, which is in any case written twice (when the values were saved, and then at erasure) so double the write cycles per each position.
It's a bit simpler to implement, though.

[rteodor]

As I understand things, because the natural default value of an EEPROM is 0xFF (all 1s), when you write a value only the 0s perform a wear-inducing operation, and writing of 1s is essentially a 'no-op'

This is correct for the single cell EEPROM's (vast majority of them). There are also differential cells with insane long retention duration that read a random value when they are in the erased state.

And so always writing the zero-value MSBs will wear things out quicker. Am I right? Should I do this? Or is it pointless, because the LSBs of the sequence numbers will still wear out first

From what I'we worked with EEL (EEPROM Emulation Layer) there was no concern with leveling of individual bits.
Much effort was put into assuring the consistency of data, i.e.: if a reset occurs during a write, make sure that that data block is detected as corrupted. This was done setting a variable at the start of a block to 0x55555555 after all writing was done.

EEPROM's contain also a charge pump that raises voltages for writing/erasing. This circuit can sometimes fail a lot faster than individual bits.

And there is read/write disturbance as well. That is one more reason to roll over the entire EEPROM in blocks.

Presets can be manually loaded or stored by the user, with the exception of Preset 0, which is auto-saved 10 seconds after the last change to the settings.
The delay avoids a lot of useless writes if the user is tuning something or selecting some mode in sequence.

This makes a lot of sense in small apllications. One way is to keep a RAM image and only transfer that to flash in certain conditions: delay timer (as you said) or in an hourly/daily/weekly update.
But I would keep a dual copy in flash and update them alternately.

[AndyBeez]

NAND or NOR EEPROM?

In the past on a NOR EEPROM I did 'bump and zero' for a frequently altering value: Byte n was set to 'nil' (FF) and byte n+1 had the new value. Retrieval was simply reading forward from address 1 to find the valid byte. When the write address pointer exceeded the boundary, the pointer wrapped back to 1. EEPROM byte 0 was never used, which was a recommendation in the datasheet. Obviously FF was not a valid number to store, otherwise the array would iterate forever!

[HwAoRrDk]

Do you really need to store the sequence number?
Can you write them in order and then just scan for the "latest" one by its position when you do a read?

How exactly do I determine which one is the latest by position? It's cyclical, so the latest settings data might, for example, be in position 3, and positions 4-16 are old redundant data.

You'd need two blocks of eeprom for this approach, when one block is full you scan and copy all the "latest" entries into the other block (which becomes the new primary) and also erase the old block so it can be used later

That seems to me like it doesn't help. You have two blocks so you're only multiplying the write endurance by 2, but then halving it again by erasing the block when you stop using it. Unless I'm misunderstanding something, because I don't know what you mean by "one block is full".

Due to space constraints, I use an 8 bit sequence number.
Sequence numbers are checked at power-on, the highest is used as the current value, and the lowest to decide where to write next.
To avoid rollover confusion there is some conditional comparisons: if the difference between the max and the min is greater than 16, then a rollover has occurred, and the highest of the lower numbers is selected.

Ah, that's a good idea. I was wondering whether I could use a smaller sequence number to have more entries and thus multiply the endurance further, but then found out about the write/erase granularity of 4 bytes, so figured I might as well keep with the uint32_t sequence number.

Each slot is CRC protected, to discard corrupted slots (e.g. if power went down while writing) when searching for the latest - they are rewritten at power-on with the latest values or with a default preset if all slots are corrupted.

That's a good point, I think I might add a CRC and use up some of the padding space in my struct. A CRC-8 should be sufficient for a few bytes of data, right?

NAND or NOR EEPROM?

I don't know. It's only described in the datasheet as "true data EEPROM" - meaning that it's not flash emulating EEPROM.


I just realised that if I want to use an inverted sequence number, that's just a decrementing count from 0xFFFFFFFF! Then the logic to find the latest settings data is instead look for the smallest value, and I don't need to special-case ignoring sequence values of UINT32_MAX when scanning through (for handling case of blank EEPROM).

[newbrain]

That's a good point, I think I might add a CRC and use up some of the padding space in my struct. A CRC-8 should be sufficient for a few bytes of data, right?

CRC-8 is a bit meager, I would think (I remember it was used in ATM, but not on the full 53 octets packet, only on the first part of the header).
I used a CRC-16 inside a header that also contains the sequence counter and a version byte.

Code: [Select]

typedef struct Header_
{
    uint16_t crc16;   /*  0 CRC-16/IBM of whole block           */
    uint8_t  version; /*  2 Version of this block               */
    uint8_t  counter; /*  3 Sequential counter to pick latest   */
} Header;             /*  4 Total header size                   */
The version byte is there to make an automatic conversion if I update the settings data structure in an incompatible way (e.g. adding fields, changing their size or rearranging them) - or, at the very least, not to load invalid data.

[peter-h]

There are lots of ways to store data in a "rolling manner". Watch out for adjacent cell disturbance though; this can significantly reduce the apparent endurance.

Also consider other ways e.g.

Store the data in a RAM, and upon power loss save the data in the FLASH, into a dedicated page which has been pre-erased, so the (Adesto SPI FLASH case) 15ms programming time reduces to just 3ms which is easily achieved, without big capacitors and using a cheap VCC threshold detector

https://www.eevblog.com/forum/microcontrollers/ideas-for-storing-frequently-updated-data-in-a-flash-chip/msg3913820/#msg3913820

https://www.eevblog.com/forum/microcontrollers/flash-memory-write-without-pre-erase/msg4030507/#msg4030507

this shows you how in more detail
https://www.eevblog.com/forum/microcontrollers/how-fast-does-st-32f417-enter-standby-mode/msg4062652/#msg4062652

[HwAoRrDk]

CRC-8 is a bit meager, I would think (I remember it was used in ATM, but not on the full 53 octets packet, only on the first part of the header).

I think CRC-8 should be good for my needs. According to Wikipedia, the AUTOSAR variant is good for detecting up to 4 bit errors for payloads of up to 119 bits (~14 bytes), which covers enough payload data for me.

But yes, I think your observation holds in some cases - some CRC-8 implementations are only good for very limited quantities of payload data. I recall now reading a paper about that ages ago, IIRC the Koopman one.

I used a CRC-16 inside a header that also contains the sequence counter and a version byte.

Another good idea. Not sure if I will add a version number. If I need to change the settings structure, it'll almost certainly be because the hardware has changed, which will be de facto incompatible with previous firmware anyway.

[Doctorandus_P]

I'd also add an CRC over the whole memory block (for one setting).
Also, if your settings are expected to change multiple times a day, then you don't need years of retention.

This is the first time I've seen two different retention time values quoted for different number of write cycles. This does make sense as leakage increases with write cycles.

What do you do if the CRC fails? Do you go back to defaults, or do you attempt to read older settings (and generate some warning?)

Maybe it's an option to just erase old data blocks, so there is only one (or a few) valid block(s) in EEprom. If you do this you also don't need sequence numbers. You can use the end of empty memory as a start for your data.

If this is not good enough or you still have doubts, it's getting time to consider other technologies, such as Fram.

[HwAoRrDk]

I'd also add an CRC over the whole memory block (for one setting).
Also, if your settings are expected to change multiple times a day, then you don't need years of retention.

I think there's a mis-interpretation here. When I say "settings", I meant the settings, singular. I don't have multiple (e.g. a range of presets).

And the EEPROM settings will be basically persisting how the user had things set up last time they used it. They may be changing and saving it during usage, but it may be days, weeks, months between occasions they use the device, or using the device regularly but not changing the settings, so I do need long periods of retention.

What do you do if the CRC fails? Do you go back to defaults, or do you attempt to read older settings (and generate some warning?)

Perhaps revert to defaults, as it may be that older data is not what the user presently wants. Or perhaps just enter a general device-wide error state. Not sure yet.

[AndyBeez]

How mission critical is your application?

An alternate to EEPROM is Ferroelectric-RAM. FeRAM has very high RW endurance and has data retention measured in decades. Power consumption is battery friendly.

https://www.electronics-notes.com/articles/electronic_components/semiconductor-ic-memory/fram-ferroelectric-ram-memory.php

[eutectique]

Are you limited to internal EEPROM? For less than 2 quid you can have FRAM chip with 100 trillion (10^14) read/writes and 151-year data retention.

[SiliconWizard]

FRAM is great. But it *is* expensive compared to EEPROM. This $2 part is only... 512 *bytes*.
At this price you can get a decent 256 *Kbytes* EEPROM.

[woofy]

If you're only saving in a 16 entry set, you don't need 32 bits for the roll-over, a single byte is enough. The rollover, if there is one, is easily detected - its just a 0 and FF in the sequence numbers. So just pick the highest number before the break to the higher numbers. Do include a checksum/crc so you can fallback safely if there was a write failure.

[Psi]

You'd need two blocks of eeprom for this approach, when one block is full you scan and copy all the "latest" entries into the other block (which becomes the new primary) and also erase the old block so it can be used later

That's somewhat wasteful - in this way you always have an empty block, effectively halving the storage, which is in any case written twice (when the values were saved, and then at erasure) so double the write cycles per each position.
It's a bit simpler to implement, though.

Agreed, it depends what you value more.
Lots of advantages and disadvantages.

In many cases the additional number of writes you want over a single EEPROM cell is not that much more than that its rated for.
If it typically cell has 100,000 writes then a 10x leveling scheme to get 1mil writes is plenty for 90% of applications.

Even using a raw cell with no leveling, 100,000 writes gets you 27 writes per day for 10 years.

For an AVR with 100,000 cycle life, when tested by writing over and over again you get around 2-5million before actually seeing a failure.
Of course it does also depend on how long there is between write and read and doing a write loop test doesn't take that into account.

[SiliconWizard]

Don't confuse Flash with EEPROM though. Completely different thing and number of write cycles.
The ATMEGA series had a chunk of EEPROM (real EEPROM) separate from its Flash. That was also the case for older PICs.
Newer MCUs very, very rarely have true EEPROM embedded. The max number of write cycles for EEPROM is one to two orders of magnitude higher than Flash.

[Psi]

Yes, it's good to make that point clear.

I just assumed EEPROM because the topic is EEPROM.
However some software libs may offer EEPROM support on flash only MCUs and that isn't EEROM so max write cycles is much less, and entire pages need to be erased at once, which makes things more compilated.

[free_electron]

cyclic writes with tail-end erase.

before you can write an eeprom block you need to erase it.
so, if you write your structure and erase the NEXT block. you can do two things
you spread the data over multiple blocks for wear-leveling , you always have a freshly erased block ready for writing, and you know where the last configuration is (just before the empty block)

[newbrain]

before you can write an eeprom block you need to erase it.

Not for any EEPROM on my bench.
24C32, 24C64, 24C256.

That's, in fact, one of the major differences with flash.

[Psi]

before you can write an eeprom block you need to erase it.

Not for any EEPROM on my bench.
24C32, 24C64, 24C256.

That's, in fact, one of the major differences with flash.

Pretty sure he knows that and was just referring to saving a struct and needing to erase all the bytes in that struct sized block before writing it.

All EEPROM i've ever used is byte by byte erasable, unlike flash.
But maybe there are some that have block erase only ?

[SiliconWizard]

Usually writing byte by byte without erasing a whole page on EEPROM is much slower than erasing the page first and writing it after that. But yes, it is at least possible. That's something to be aware of though if write time is a concern.

[Nominal Animal]

For rolling (monotonically increasing) N-bit sequence numbers a and b,
    if ((intN_t)((uintN_t)(a) - (uintN_t)(b)) OP 0) {
        /* a OP b, but taking possible rollover into account */
    }
where OP is one of <, <=, ==, >=, >, or !=.
This works, because in C uintN_t arithmetic is modular, and intN_t uses twos complement.

In essence, it moves and wraps a and b so that b is at zero, and checks where a then is relative to b, positive or negative.  The uintN_t casts are not needed if a and b are declared as that type; but the intN_t cast should not be omitted.


For example, consider uint8_t a = 40 and uint8_t b = 200.  ((int8_t)(a - b)) = 96, so in terms of the monotonic sequence, a = b + 96.

If we have uint8_t a = 200 and uint8_t b = 40, then ((int8_t)(a - b)) = -96, so in terms of the monotonic sequence, a = b - 96.


If you have a cyclical buffer with L entries, each with a sequentially assigned unsigned integer (of size N bits, 2^N > L+1), with no holes or missing values, you can do a binary search to discover the latest (followed by oldest or unused) entry, as long as unused entries are assigned a special integer (either 0 or 2^N-1) and are all contiguous (or in two parts at the beginning and end).  In general, you need to examine about ⌈log₂ (L + M)⌉ + 2 entries, where M is the number of unused entries.  It is quite efficient, even when you do this for e.g. SD card sectors, without a filesystem.

[fchk]

Have you seen the new Microchip EERAM chips?

https://www.microchip.com/en-us/products/memory/serial-eeram

These are serial SRAM memory with automaic backup and restore to eeprom cells. These chips need a ceramic capacitor (22...68u) as shutdown energy storage.

This eliminates wearout issues completely.

fchk

[peter-h]

As would the software solution I suggested, and without dependence on some weird single sourced part which will one day bite you in the bum

As an extreme example, I can erase and program the bottom 32k of my 32F417 FLASH in 300ms, and with a decent size capacitor you could achieve that, too.