Author Topic: Preserving settings over a firmware upgrade  (Read 3309 times)

0 Members and 1 Guest are viewing this topic.

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 9336
  • Country: fi
Re: Preserving settings over a firmware upgrade
« Reply #25 on: July 28, 2023, 11:21:51 am »
That is the cost of backward compatibility. For sure everyone will agree that it is better to avoid needing such migrations but either way it is better to prepare the way the data is stored so that software can detect the settings are compatible or not. When the configuration data is incompatible between versions, the next question is whether to use sensible defaults or try to convert.

Yeah. And while it would be nice to plan all the future features/fixes in advance and come up with perfect set of settings from day 1, it will only happen for a really simple product or a product which does not need much updates. In which case OP would not be asking. Better mentally prepare for significant changes. On the other hand, I also prefer just-in-time principle for development: if you know you will be able to pull off migration code when needed, then do it only when needed. Simply, when the day comes that you have to break the compatibility, add the migration code then. Test it well before releasing the update in the wild. If you are lucky, you never need to do that. Even if you have to, you are then better off than trying to come up with the scheme 2 years earlier.

And simple structs lend themselves very well for such migration functions, as shown above in code examples by me or DavidAlfa: all you need are the constructs available in the programming language you are using in that project anyway.
 

Offline Kalvin

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
Re: Preserving settings over a firmware upgrade
« Reply #26 on: July 28, 2023, 12:30:09 pm »
The settings needs to be compatible both ways:

- The settings need to be preserved when upgrading to a newer version.
- The settings need to be backwards compatible when downgrading to an older version so that downgrading won't break those settings values that are shared between releases.

If the settings cannot be preserved over upgrade/downgrade, the firmware should provide good default values for those settings that cannot be preserved.
 

Offline DavidAlfa

  • Super Contributor
  • ***
  • Posts: 6403
  • Country: es
Re: Preserving settings over a firmware upgrade
« Reply #27 on: July 28, 2023, 12:31:04 pm »
50% simpler: Don't allow downgrading!
Hantek DSO2x1x            Drive        FAQ          DON'T BUY HANTEK! (Aka HALF-MADE)
Stm32 Soldering FW      Forum      Github      Donate
 

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 9336
  • Country: fi
Re: Preserving settings over a firmware upgrade
« Reply #28 on: July 28, 2023, 12:38:42 pm »
The settings needs to be compatible both ways:

- The settings need to be preserved when upgrading to a newer version.
- The settings need to be backwards compatible when downgrading to an older version so that downgrading won't break those settings values that are shared between releases.

If the settings cannot be preserved over upgrade/downgrade, the firmware should provide good default values for those settings that cannot be preserved.

I would not generally agree. It's a sensible decision not to allow downgrading, to KISS, especially in a cheap commodity product. (Much more limiting decisions are made all the time.) For a special industrial expensive gizmo, that might be desirable for some weird reason customer might have, but then you can sell the downgrade work, along with generating new settings, as specialized work and the customer would pay for it.

If downgrading is required for troubleshooting, then that can involve manually working out the correct settings. When the problem that required the downgrade is found, it can be applied to the new version so that the reason for downgrade goes away.
 

Offline Kalvin

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
Re: Preserving settings over a firmware upgrade
« Reply #29 on: July 28, 2023, 12:41:40 pm »
50% simpler: Don't allow downgrading!

If the settings management is done rightTM, then it really doesn't increase complexity that much.
 

Offline AndyC_772Topic starter

  • Super Contributor
  • ***
  • Posts: 4315
  • Country: gb
  • Professional design engineer
    • Cawte Engineering | Reliable Electronics
Re: Preserving settings over a firmware upgrade
« Reply #30 on: July 28, 2023, 12:53:16 pm »
I'm inclined to favour:

Code: [Select]
if (nv.settings_version < CURRENT_SETTINGS_VERSION)
  migrate_settings();

if (nv.settings_version > CURRENT_SETTINGS_VERSION)
  factory_default();

Downgrading firmware isn't something I'd look to explicitly lock out, but it doesn't need to be so graceful.
 
The following users thanked this post: thm_w, Siwastaja

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7509
  • Country: va
Re: Preserving settings over a firmware upgrade
« Reply #31 on: July 28, 2023, 02:46:09 pm »
I've upgraded a lot of stuff I wish could  be downgraded to fix new bugs or unwanted features. Altium is a great example: doesn't care what version of software you're running and will tell you if a new feature can't be handled in an old version (obviously, with an embedded device it can't do that, but it can ignore the new data). Altium 09 can be used to view Altium 23 documents perfectly well, despite the features introduced in the intervening 16 years.

I have similar software, and I love 'em. For the user they are pain free, and these things should be designed for the user, not the developer. In contrast, there is stuff I can't allow to be updated because there is no way back, and I hate 'em.

pfSense is middle of the road. You can export settings, update (or install somewhere else) and import. The worst I've had has been the Ethernet interfaces changing names, so the low level network IDs have to be frigged about with. But the bulk of the settings have been fine. But look at Tweetduck - minor update and it can't import settings from the previous version. Another one I used to not allow to update (pointless now since it no longer works, thanks to Musk).
 

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 9336
  • Country: fi
Re: Preserving settings over a firmware upgrade
« Reply #32 on: July 28, 2023, 02:52:15 pm »
50% simpler: Don't allow downgrading!

If the settings management is done rightTM, then it really doesn't increase complexity that much.

Settings management (how to store them) is the trivial part (unless you colossally fucked it up, of course). The problem is always in functionality (what the settings signify and how). There must be a reason why a new version with incompatible conf came out: it does something the old version did not do at all, or does some new thing better, and conf is somehow related to these changes. Now there also must be some reason why the new version is in use, and configured to do something. You would need to transform these new settings to something that does as close to the right thing as possible, using that old version. This is a lot of work for little gain, because going back in versions would be quite rare.

Therefore, I second AndyC's suggestion that downgrading would be handled with factory defaults. As downgrading is a special measure against something anyway, manually changing settings is not a big deal.
« Last Edit: July 28, 2023, 03:18:42 pm by Siwastaja »
 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7509
  • Country: va
Re: Preserving settings over a firmware upgrade
« Reply #33 on: July 28, 2023, 06:35:09 pm »
Quote
manually changing settings is not a big deal

Depends on how many, what they are, and the device. Some devices are so bleeding tedious to change anything you really don't want to do that very often, and with many devices you can't remember what you've changed since you spent that week setting it up 2 years ago (because, no export remember). That viewpoint - the user won't mind doing it all again - is what keeps stuff on old versions. If you're lucky the user will update once before finding out the best thing is to not do it again.
 

Offline thm_w

  • Super Contributor
  • ***
  • Posts: 7527
  • Country: ca
  • Non-expert
Re: Preserving settings over a firmware upgrade
« Reply #34 on: July 28, 2023, 09:14:43 pm »
Full source linked at the end?

Not licensed as open source.
Profile -> Modify profile -> Look and Layout ->  Don't show users' signatures
 

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 9336
  • Country: fi
Re: Preserving settings over a firmware upgrade
« Reply #35 on: July 29, 2023, 10:18:56 am »
Quote
manually changing settings is not a big deal

Depends on how many, what they are, and the device. Some devices are so bleeding tedious to change anything you really don't want to do that very often, and with many devices you can't remember what you've changed since you spent that week setting it up 2 years ago (because, no export remember). That viewpoint - the user won't mind doing it all again - is what keeps stuff on old versions. If you're lucky the user will update once before finding out the best thing is to not do it again.

You missed the context. The discussion was about downgrading. That is something which doesn't happen on its own. If adjusting settings is tedious to user, manual firmware changes in the opposite direction is surely tedious as well. There must be some specific reason why user wants to do that.

Your claim that difficulty in downgrading keeps people in old versions is weird. I think it's the opposite; difficulty in downgrading but ease of upgrading would drive people into trying the new versions instead to see if they fix whatever problem they have (or think they have).
« Last Edit: July 29, 2023, 10:21:06 am by Siwastaja »
 

Offline Kalvin

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
Re: Preserving settings over a firmware upgrade
« Reply #36 on: July 29, 2023, 10:40:11 am »
I'm inclined to favour:

Code: [Select]
if (nv.settings_version < CURRENT_SETTINGS_VERSION)
  migrate_settings();

if (nv.settings_version > CURRENT_SETTINGS_VERSION)
  factory_default();

Downgrading firmware isn't something I'd look to explicitly lock out, but it doesn't need to be so graceful.
This scheme has been working pretty well over the years:
Firmware settings are stored as C structs. The attachment shows three different versions of the same settings structure evolving over firmware versions.
  • When the firmware is run for the first time, the system doesn't find any existing settings, so it will just copy the default settings into the actual settings structure.
  • When the firmware is upgraded from version 1 to version 2, the new firmware version will be able to restore the version 1 settings. The firmware will then copy the default settings from the version 2 defaults to the version 2 settings.
  • When the firmware is upgraded from version 2 to version 3, the new firmware version will be able to restore the version 1 settings and version 2 settings. The firmware will then copy the default settings from the version 3 defaults to the version 3 settings.
  • When the firmware is downgraded from version 3 to version 2, the new firmware version will find the old settings version 1 settings, version 2 settings, and version 3 settings. However, it will only restore the version 1 settings and version 2 settings.
Typically, there are separate settings structs for different software components. This helps to keep the settings organized throughout the product's lifetime.

 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7509
  • Country: va
Re: Preserving settings over a firmware upgrade
« Reply #37 on: July 29, 2023, 02:15:49 pm »
Quote
manually changing settings is not a big deal

Depends on how many, what they are, and the device. Some devices are so bleeding tedious to change anything you really don't want to do that very often, and with many devices you can't remember what you've changed since you spent that week setting it up 2 years ago (because, no export remember). That viewpoint - the user won't mind doing it all again - is what keeps stuff on old versions. If you're lucky the user will update once before finding out the best thing is to not do it again.

You missed the context. The discussion was about downgrading. That is something which doesn't happen on its own. If

I was pointing out that changing settings can be a drag. Earlier, I pointed out that sometimes you want to downgrade because of new features/bugs. Both of these are in the context of "manually changing settings is not a big deal". Admittedly, the upgrade rant that flowed from that may have confused anyone not, ah, going with the flow and fixating on the part I didn't quote (because that quote was the entire context I was discussing).

Quote
Your claim that difficulty in downgrading keeps people in old versions is weird.

That's not what I claimed.
 

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 9336
  • Country: fi
Re: Preserving settings over a firmware upgrade
« Reply #38 on: July 29, 2023, 02:46:04 pm »
OK, so you just take words out of context to turn their meaning around, creating a strawman which you then argue against, without making it obvious you do so, wasting everyone's time. You are ignored from now on.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf