General > General Technical Chat
Preserving settings over a firmware upgrade
AndyC_772:
Some good ideas there folks, thank you all.
I think I'll go with:
- keep each new variant as similar as possible to what went before. Insert dummy variables where there used to be meaningful data, add anything new at the end. Everything stays in the same place as far as possible, at least until I run out of storage and have to do something drastic. Fortunately I have a lot more space available for code than I do for settings.
- add a revision number to the start of the nv struct, which increments each time there's a change that the code needs to know about. Come to think of it, I do this with register interfaces to FPGAs all the time, so it's hardly a foreign concept.
- move the magic number to the start, so it's always in the same place; we can still tell the difference between old config vs new config vs erased flash.
- at power up, check the revision number and run some one-time code to migrate settings as necessary. This might be as simple as setting factory defaults for new features, but it might include manipulating other data too. This is OK; better to update the settings once than forever have to cope with the fact that they might be stored in an out-of-date way.
tom66:
--- Quote from: nctnico on July 25, 2023, 07:28:45 pm ---You can use JSON on a small microcontroller just fine. In the end you only need a few simple text parsing functions to take the JSON data apart and extract the data you need.
--- End quote ---
The issue I forsee is updating one key possibly requires rewriting the whole JSON. That possibly requires enough RAM to recreate the JSON as a string, then you need to overwrite all of the flash pages that change. If you only have a couple of settings, no issue, but a large number of settings quickly becomes a bottleneck.
Not an issue on a Zynq running Linux. A headache on an STM32 with 32KB of RAM...
nctnico:
--- Quote from: tom66 on July 25, 2023, 07:33:45 pm ---
--- Quote from: nctnico on July 25, 2023, 07:28:45 pm ---You can use JSON on a small microcontroller just fine. In the end you only need a few simple text parsing functions to take the JSON data apart and extract the data you need.
--- End quote ---
The issue I forsee is updating one key possibly requires rewriting the whole JSON. That possibly requires enough RAM to recreate the JSON as a string, then you need to overwrite all of the flash pages that change. If you only have a couple of settings, no issue, but a large number of settings quickly becomes a bottleneck.
Not an issue on a Zynq running Linux. A headache on an STM32 with 32KB of RAM...
--- End quote ---
No problem at all. You erase the flash page you want to write the settings to (which needs to be done anyway *) and then create (snprintf) each piece of the JSON data one by-one and commit to flash. In case the flash can be written in chunks of several bytes (which is typical for ECC flash), then all you need is a small extra buffer to capture data before you have a full flash chunk. You'd need maybe 100 bytes at most for buffering (on stack) to have long key names and reasonable long text strings for settings.
* Ofcourse you have to implement a double / multi buffering system in order to always retain a valid record with settings in case there is a power outage during write or the flash wears out.
Veteran68:
My how things have changed since my early embedded systems days. Sure, compared to the bloat of something like XML, JSON is certainly much lighter, but it's still not what I'd call firmware light. I use it all the time in PC application code where you're storing or transferring lots of strings or human readable data, but I can't see incurring its data storage and parsing logic overhead for something like a firmware configuration.
High level markup languages are intended as self documenting data structures for data transfer and storage for high level business developers (many of which can barely code in Visual Basic or HTML/CSS/JS), not for an embedded systems developer dealing in bits and bytes, registers and interrupts. It's too easy to stream a series of bytes and lay them into a struct for use by the system. Then use the numerous tricks as discussed in this thread to manage the dynamic scalability of that data.
Then again the MCUs and SOCs we have today have mountains of storage and power compared to the old days when we counted every clock cycle and data bit, while hand-optimizing the assembler produced by C compilers (or just wrote it in ASM to begin with).
These kids today have it easy... :P
PlainName:
If the user is applying the upgrade then that implies some kind of connection (USB, WiFi, BT), in which case cannot the device send the config to the PC or phone (or website) that's sending the upgrade? In a perfect world it could save the config on demand (even if that meant emailing it to the user) so it could be saved 'just in case', or restored after the user has screwed it up.
Although the on-chip config is no doubt fine as blob (actually a defined structure), converting to and from JSON for the purposes of saving and restoring would save lots of hassle when the format changes, perhaps by thing being inserted or removed.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version