Arduino SPI EEPROM Array storage ideas...

[iampoor]

Hi
In my current project, I am planning on using an SPI EEPROM with an arduino. I am looking to use the CAT25256 EEPROM. It seems well documented, reliable, affordable, and is avaliable in 64k, 128k, 256k, and 512k sizes. This is important because I will want to reuse this code in future endeavors.

Anyways, I am currently trying to decide on how to store/arrange my data. I am building a system with 256 presets (0-255). I figured it would be easy to store each preset in a 64 byte page on the EEPROM, since, if I understand correctly, every time you write to the EEPROM, you must write the entire 64 byte page. Is this correct? Or does an EEPROM just need a memory write address and page write is a separate feature? 

Each preset will be storing the output state of 16 relays, and the associated present number in an array. I think the data structure for this will take up 3 bytes. 1 byte for assigned the channel number, and 2 bytes for all 16 relay states (Ie. 01010101 01010101 Channel 0 = off Channel 1 = on Channel 3 = off etc). I am planning on defining the memory location of each array in the eeprom, so data doesnt get lost. In addition, for each channel I would like to store assorted MIDI commands. To start, I will be sending MIDI beat clock, and program change commands. These are a maximum of 3 bytes each. However, I am not sure how to organize this as part of the data structure. Each preset I am thinking will be able to send 8 midi commands (More then I am ever likely to need). Would it make sense to just allocate 24 bytes at the end of every array and if no midi commands are present just set all the values to 0? Then each midi command is written in sequentially for recall?
Thank you for reading this and providing feedback, this is my first experience with EEPROM's, so I hope I am not messing up any of the fundamental concepts!

[promacjoe]

Have you looked into using FRAM (or Ferroelectric RAM) memory. It has a higher write speed, and A higher write cycle life. It also has better read/write characteristics.

Have a look at this website.

http://www.kerrywong.com/2012/01/15/using-fram-as-nonvolatile-memory-with-arduino/

or,

http://hackscribble.github.io/hackscribble-ferro-hardware/

[obiwanjacobi]

Yep, just layout the data you need for each preset, reserve a little extra space (per preset) for future enhancements -so the stored data remains a valid preset in any of your software versions- and off you go. You may also want to consider a checksum, per preset, per block or over the entire memory. Allows you to detect when the eeprom is on its way out...

You should program it in such a way that you generate the least amount of writes to the eeprom. So delay writes for an edited preset until the very last moment. Usually you reserve the size of a preset in RAM and use that as your working memory. When you load a preset, you copy it from eeprom to RAM. When you edit a preset you work in RAM. When you have to save a preset you copy it from RAM to eeprom.

You might also want to build in an option to dump all the preset data in a MIDI SysEx message so you can backup. An option to read that back in -if you have a midi in port- might also be handy.

[dannyf]

every time you write to the EEPROM, you must write the entire 64 byte page.

Depending on the particular chip. I cannot say this for all eeproms but they typically allow write/read on individual bytes, or in page format.

Check the datasheet to be sure.

[martinayotte]

every time you write to the EEPROM, you must write the entire 64 byte page.

Depending on the particular chip. I cannot say this for all eeproms but they typically allow write/read on individual bytes, or in page format.

Check the datasheet to be sure.

True !
On most of them, you can do single write, as long as it never been written before and the page has been previously erased.

[rstofer]

I'm going to pile on to the FRAM heap.  These devices are finally at a point where they can be used without concerns.  In fact, since no page operation is ever required, they are even easier to use than EEPROM.  They're also faster, truly random access and now available for 3.3V systems (and lower).

At DigiKey, just search for FRAM to see all the devices.  Next time I order something, I'll pick up a couple of these devices.

[martinayotte]

Yes ! I've used some FM25V10, and worked almost identically to W25Qxx, except when trying to pull out the JEDEC ID.

[iampoor]

Thanks for all the reply's!

I actually had never heard of FRAM until this thread. It looks VERY cool, and it essentially has an unlimited lifespan (well, compared to human life.  ). Since I think I will only need 16k or 32k of storage space at max, FRAM seems like a good option. The cost different doesnt really bother me.

Yep, just layout the data you need for each preset, reserve a little extra space (per preset) for future enhancements -so the stored data remains a valid preset in any of your software versions- and off you go. You may also want to consider a checksum, per preset, per block or over the entire memory. Allows you to detect when the eeprom is on its way out...

You should program it in such a way that you generate the least amount of writes to the eeprom. So delay writes for an edited preset until the very last moment. Usually you reserve the size of a preset in RAM and use that as your working memory. When you load a preset, you copy it from eeprom to RAM. When you edit a preset you work in RAM. When you have to save a preset you copy it from RAM to eeprom.

You might also want to build in an option to dump all the preset data in a MIDI SysEx message so you can backup. An option to read that back in -if you have a midi in port- might also be handy.

Thank you. So for each array, I will manually define the location in the eeprom correct? If I manually define the location and then use a for loop to pull out all the data, that would seem like the easiest solution. I am not 100% sure on how to store the midi messages. The 16 switch states are easy, but I will want the for loop to spit out the midi data to another location. Maybe separating the midi data and switch statements would make the most sense. Or separating them with a byte that has neither a midi command associated with it, or is easily recognized. Just think out loud here.

Yes, I will implement a checksum, that is a great idea! Not sure how to do a backup over sysex messages yet, but I will leave extra space for potential firmware updates and added functionality.

[obiwanjacobi]

I usually define static C-structs that represent the total data structure of a preset and have a (de)serialization routine that knows how to write the byte from/to the eeprom. Its basically a copy.

So for instance (assuming C)

Code: [Select]

struct MidiCommand
{
    uint8_t Status;
    uint8_t Param1;
    uint8_t Param2;
};
struct Preset
{
  char Name[17];
  MidiCommand MidiCommands[8];
  uint8_t Reserved[32];
};

This would define a preset with a name and room for 8 (short) midi commands and reserve 32 bytes for future enhancement per preset.
Use sizeof(Preset) to know how many bytes to copy to the eeprom. Determine the start address by multiplying the preset size times the preset-index (zero-based)...

Easy peasy...

EDIT: to minimize the number of eeprom writes, you can keep track of what fields were actually changed and save only those...

[Buriedcode]

every time you write to the EEPROM, you must write the entire 64 byte page.

Depending on the particular chip. I cannot say this for all eeproms but they typically allow write/read on individual bytes, or in page format.

Check the datasheet to be sure.

Every EEPROM I have used (SPI, I2C, the old parallel ones) can write to individual bytes.   It is flash that requires whole page erase/rewrite which is why flash chips have on board RAM that allows the emulation of one-byte/word writes and does the read and write-back on chip.