Custom STM32 bootloader guidance

[Tripplecheckuser]

I want a way to upgrade a custome pcb board for a product in the the field in case there is a bug of some sort. My board uses the STM32F030K6T6 ic. I haven't done any work with bootloaders so I started by reading through the available documentation ST provides (DataSheet, AN3155 and AN2606 ). From this, I gathered that I can access boot mode from the boot0 pin but can only flash an image via USART (pins pA10 and pA9 or pA14 and pA15). Problem with this is that the pcb has already been designed to use pA10 and pA9 for I2C communication with modules, pA14 is for SWD clk for flashing and pA15 is for a random LED. A full redesign isn't really possible as the product is set to launch in the next few months. My initial thought was to finish testing and functionality and work on upgradable capabilities later but it looks like bootloaders are a lot more complicated than I thought. On some of our older products, we use I2C to update our firmware but those products use Microchip ICs (PICs). With this issue in hand and a miniscule amount of experience, I came up with 2 possible solutions:

1. Make a custom bootloader for the stm32f030k6t6 that will allow for me to update via I2C

2.  switch pA9 and pA10 to USART peripherals, switch into boot mode via code (the boot0 pin was not considered when the circuit was made) and update firmware via USART.

Both these options seem way out of my scope of understanding, but I am willing to put in the work. What advice can you give me in terms of tackling this? Is there one option that is more practical than then the next given my experience level?

[Siwastaja]

There is absolutely nothing complicated in bootloaders, just some work and learning curve as always with anything.

Most robust and probably also easiest way to deal is dividing the flash into three parts: bootloader, application A and application B. Bootloader only verifies the validity of application A & B (against a checksum) and boots either one; a few dozen lines of code. Then application itself is responsible of receiving the firmware update data, erasing the other app and writing there. This way, power loss or other issues leave the working application intact, and the bootloader will just boot the old firmware again in case the new one fails at check.

This way, bootloader itself does not need any flash writing or I2C receiving functionality, and therefore, can be simple enough that bootloader itself does not need an update mechanism, either. I don't frankly understand why people prefer complicated bootloaders over doing most of the work in the application itself.

You have to:
* Learn how to use linker scripts to describe the memory addresses where application code is placed;
* Learn how to erase and program the flash memory - this is all explained in STM32 reference manual and simply involves some register writes and polling for status flags
* Learn the bootloader magic itself, which is just a few lines of code of inline assembly copy-pasted from the internetz.

[Psi]

The bootloader isnt very complicated, it's just like any other app

The most annoying bit is usually figuring out the correct settings for the linker script.
You usually also need to setup some variables no-init and have them at a fixed memory address.  (so the bootloader and app have some shared space to exchange flags bits and stuff)

[Tripplecheckuser]

So basically application A is my default ap that will act as a default (when I don't have an ap B or there is an error with app B's checksum) and application B would be the updated version of the application?

[Tripplecheckuser]

The bootloader isnt very complicated, it's just like any other app

The most annoying bit is usually figuring out the correct settings for the linker script.
You usually also need to setup some variables no-init and have them at a fixed memory address.  (so the bootloader and app have some shared space to exchange flags bits and stuff)

Hmmm linker scripts seem to be a big part of the process. I guess I will have to dive into what it takes to make one. I assume it is different than the bootloader code or application code?

[Siwastaja]

So basically application A is my default ap that will act as a default (when I don't have an ap B or there is an error with app B's checksum) and application B would be the updated version of the application?

No, the "default" app would alternate between A and B, because A must not write to A and B must not write to B in order to remain bootable if power is lost during update.

I deal with this writing an increasing counter value such that if app_a is now running with counter value 5, it writes app_b with counter value 6, and bootloader boots the app with larger counter value (if it passes the CRC check, that is).

[Siwastaja]

Hmmm linker scripts seem to be a bit part of the process. I guess I will have to dive into what it takes to make one. I assume it is different than the bootloader code or application code?

Yeah, maintain the bootloader and the application as two completely separate projects, with their own linker scripts, makefiles etc.

In my double buffered A-B scheme, I use two linker scripts for the app, producing two differently linked but otherwise identical application binaries.

[Psi]

Yeah, they are two separate projects and each produces it's own intel hex file as output. The intel hex file format includes the flash location to write the program. So the boot loader hex has different address range since its going to a different location in  the MCU flash block.
So you erase the chip, program one hex file and then program the 2nd hex file (or you can merge the two hex files together if you want to, since they are for different flash address ranges. just need a tool to merge intel hex files, there are many)

That is one of the things the linker script does, it sets up the locations in flash where each program will be and their sizes.
So you can have two projects and they cant step on each others toes.  If your bootloader gets too large your bootloader project will error out when you try to compile it saying it does not fit any more even when the MCU has plenty more free space.  Because you set the linker script to give it some space and are now trying to use more than that. etc..

It's more the 'syntax' of the linker script which can be annoying. but there are plenty of examples around the place.
Just need to read them and understand what they are doing so you can modify your one.

If you are doing the 'intermediate bootloader' thing where you have a block for the main app (APP) and a block of empty space (APP2) to store a new image to be updated to by bootloader (copy APP2 to APP). Then you need 3 blocks.
A small block for the bootloader, and two large blocks of equal size for APP and APP2.
The downside of a intermediate bootloader is that you waste bit under 1/2 the space of your MCUs flash memory for the APP2 block.    But if your project needs a block of flash for storing your own data/logs you can sometimes combine that block with APP2. It just means your data/logs has to be erased to perform a FW update.

[nctnico]

So basically application A is my default ap that will act as a default (when I don't have an ap B or there is an error with app B's checksum) and application B would be the updated version of the application?

No, the "default" app would alternate between A and B, because A must not write to A and B must not write to B in order to remain bootable if power is lost during update.

I deal with this writing an increasing counter value such that if app_a is now running with counter value 5, it writes app_b with counter value 6, and bootloader boots the app with larger counter value (if it passes the CRC check, that is).

Another option is to check the validity of B and overwrite A if B has a different (*) firmware version than A. That way you don't need to remap any of the memory to make a firmware image work from different base addresses (or have different firmware images to work from different addresses). Retry if the checksum for A is invalid after programming. Unless the flash is broken, this doesn't leave you with a bricked system.

I second the suggestion to do uploads through the application. This allows to update the update mechanism itself and makes the bootloader smaller. A downside is that you'll need to include a working firmware image into the bootloader image so the microcontroller can run the application straight away after production programming. Including a firmware binary is possible with an assembler file which includes a firmware binary which is then placed into the flash area where the application needs to sit (ld script again).

* checking for a different firmware version also allows to downgrade firmware by simply uploading an older version into the device.

[Siwastaja]

I second the suggestion to do uploads through the application. This allows to update the update mechanism itself and makes the bootloader smaller. A downside is that you'll need to include a working firmware image into the bootloader image so the microcontroller can run the application straight away after production programming.

I don't understand what you mean by "including" a firmware image into the bootloader. Quite obviously, when you do production programming, you can just program the whole thing with combination of bootloader + app.

[redkitedesign]

So basically application A is my default ap that will act as a default (when I don't have an ap B or there is an error with app B's checksum) and application B would be the updated version of the application?

No, the "default" app would alternate between A and B, because A must not write to A and B must not write to B in order to remain bootable if power is lost during update.

I second the suggestion to do uploads through the application. This allows to update the update mechanism itself and makes the bootloader smaller.

It also depends on how much flash you need for the actual product functionality.

If that is much larger that the update functionality, having two fullblown app's effectively halves your flash.

Thirdly, a bootloader with app-install/update functionality should not brick itself, and in extension the update mechanism when installing a new app, making this less risky.

In the A/B app scheme, it is possible to install an update that seems to work but for example has a bug preventing writes to the upper half of the flash.  Thus the update installs properly, passes CRC testing and runs fine.
But now any further updates are blocked, as this version cannot overwrite the old version! Prior testing can reduce this risk, but not totally eliminate it, especially if your business has grown and you now have multiple hardware versions with multiple customers with multiple firmware requirements.

(there are further mitigations possible, like starting the old version from the new to test if there is a way back, allowing an second update for the new app. But if you think it through properly, it is an unsolvable problem, you can only reduce the risk)

[Psi]

Yep, if you have the space, just have two app areas, one where your app is and one as sort of a 'temp storage area' where you app itself can assemble a new fw image based on data your APP receives, (uart/spi/lan/ble etc). When it's all there and any checksums confirmed good you set some flag variables and reboot into bootloader. The bootloader starts and sees from those flags that there's an image in the temp area that is ready for copying and it copies all of the temp storage area over the top of the main APP area, then reboots (app area now contains new fw).

Downside is if you intentionally flash an image that is bad you might brick the thing, since you can't load a new image into the temp space if the main app isn't running properly.
Upside is your bootloader is ultra simple, it doesn't need to know about anything, no comms, no reading packet of FW data or anything. All it does is start up and either copy a block of flash to another block of flash or not.


The other way of doing a bootloader is you duplicate all the code to talk to the uart/SPI, decode packets, understand protocols etc.. all in the bootloader itself.
So to do an update you have to first start the bootloader and then you talk to the bootloader to send blocks of FW for writing to flash.
It makes your bootloader more complicated and bigger but you can never brick the device because the bootloader is always available and always runs first so you can always connect to it and upload a new fw image.

[nctnico]

I second the suggestion to do uploads through the application. This allows to update the update mechanism itself and makes the bootloader smaller. A downside is that you'll need to include a working firmware image into the bootloader image so the microcontroller can run the application straight away after production programming.

I don't understand what you mean by "including" a firmware image into the bootloader. Quite obviously, when you do production programming, you can just program the whole thing with combination of bootloader + app.

Yes, and including a working firmware image into the bootloader image makes that easier. Just one file to program so no chance for mistakes.

[Siwastaja]

I use srec_cat to combine multiple intel hex files into one file which is easier to program on any device without having to think how one would use (possibly manufacturer-specific) programming tools to program multiple files. Highly useful small piece of software.

[nctnico]

So basically application A is my default ap that will act as a default (when I don't have an ap B or there is an error with app B's checksum) and application B would be the updated version of the application?

No, the "default" app would alternate between A and B, because A must not write to A and B must not write to B in order to remain bootable if power is lost during update.

I second the suggestion to do uploads through the application. This allows to update the update mechanism itself and makes the bootloader smaller.

It also depends on how much flash you need for the actual product functionality.

If that is much larger that the update functionality, having two fullblown app's effectively halves your flash.

Thirdly, a bootloader with app-install/update functionality should not brick itself, and in extension the update mechanism when installing a new app, making this less risky.

In the A/B app scheme, it is possible to install an update that seems to work but for example has a bug preventing writes to the upper half of the flash.  Thus the update installs properly, passes CRC testing and runs fine.
But now any further updates are blocked, as this version cannot overwrite the old version! Prior testing can reduce this risk, but not totally eliminate it, especially if your business has grown and you now have multiple hardware versions with multiple customers with multiple firmware requirements.

There is a simple mechanism to prevent this from happening and that is by only allowing firmware that is known to be compatible with the hardware. But in the end part of testing firmware updates on every hardware version needs to be part of the firmware release test.

[nctnico]

I use srec_cat to combine multiple intel hex files into one file which is easier to program on any device without having to think how one would use (possibly manufacturer-specific) programming tools to program multiple files. Highly useful small piece of software.

Including the binary image during compilation does the same but produces an ELF file (given the OP uses ST's CubeIDE and/or GCC). From there you can create any output format. It might not be possible with other tools. In the end it depends on your preference & situation.

[Tripplecheckuser]

That is one of the things the linker script does, it sets up the locations in flash where each program will be and their sizes.
So you can have two projects and they cant step on each others toes.  If your bootloader gets too large your bootloader project will error out when you try to compile it saying it does not fit any more even when the MCU has plenty more free space.  Because you set the linker script to give it some space and are now trying to use more than that. etc..

Funny you mention that. I was going through a simple bootloader tutorial by ST () and I think I encountered this issue of the bootloader suddenly being too large(specifically the heap). The issue with this is I don't know enough to modify the bootloader allotted memory properly so it takes the required space. In tutorial 1-4 (its a 6 part series) I had no problems (I'm using a different Nucleo board STM32F030R8) but as soon as I add the contents of tutorial 5, my debug session terminates immediately and erro is set to ENOMEM(in the sysmem source file) confirming I am running out of space. I tried doubling the length of memory for my bootloader (32->64kb) to no avail.

I tried checking the ST community but it looks like they are doing some website maintenance/revamp of some sort (just my luck). So I am kind of lost as to where to go next.

[Tripplecheckuser]

Another option is to check the validity of B and overwrite A if B has a different (*) firmware version than A. That way you don't need to remap any of the memory to make a firmware image work from different base addresses (or have different firmware images to work from different addresses). Retry if the checksum for A is invalid after programming. Unless the flash is broken, this doesn't leave you with a bricked system.

* checking for a different firmware version also allows to downgrade firmware by simply uploading an older version into the device.

So for a use case example:

• I have a LED BLINKING CODE (A) and an NEW LED BLINKING CODE (B).
• I flash my bootloader + A in my controller.
• My bootloader checks to see if there is any code in B and if the fw version is different. If not, bootloader runs A.
• I want to update my controller with NEW LED BLINKING CODE.
• I get NEW LED BLINKING CODE through A (using some i2c or uart communication GETNEWFW();) and store it in B then restart controller.
• Bootloader then checks if B is a different fw version than A (and it is now) so it overwrites A with B (NEW LED BLINKING CODE) and jumps to A (which now has NEW LED BLINKING CODE).

If the controller losses power...

• while A is storing NEW LED BLINKING LED in B, it will just retry when power is restored, allowing A to remain in tact
• when the bootloader is writing B to A, then A would be incomplete. I assume the controller would always have to always start in bootloader to prevent this from being a problem. So if the controller starts and sees A is invalid and B is (I assume it ignores the fm version in this case), it writes B to A.

Draw Backs:

• If the GETNEWFW function (that will be located in the currently running app) is implemented wrong on a new image, the device can not longer get new fw and will be bricked (maybe I could make that function modular and place it in RAM so it never gets changed)
• Double of my flash will be used (A + B) but only half of that will be in use at any given time (A)

Pros:

• the bootloader will be pretty robust and small
• bricking controller will be less likely
• can update with both older and new fw versions

 


Is this how this scenario would go?

[Tripplecheckuser]

If you are doing the 'intermediate bootloader' thing where you have a block for the main app (APP) and a block of empty space (APP2) to store a new image to be updated to by bootloader (copy APP2 to APP). Then you need 3 blocks.
A small block for the bootloader, and two large blocks of equal size for APP and APP2.
The downside of a intermediate bootloader is that you waste bit under 1/2 the space of your MCUs flash memory for the APP2 block.    But if your project needs a block of flash for storing your own data/logs you can sometimes combine that block with APP2. It just means your data/logs has to be erased to perform a FW update.

Wouldn't I still waste 1/2 the space if I went with the A/B switching scheme since I would need to account for APP and APP2  even though one app is used at at time? I am leaning towards the 'intermediate option'. My IC (STM32F030K6T6) has a max of 256kb of flash (according to the data sheet ) and my APP is 24kb so I think I should definitely have enough

[Siwastaja]

I think you got it right. The minor difference between what's proposed by me vs. nctnico is the copying step in bootloader. Copying B->A allows you to use single app binary linked to single address space, but the cost is somewhat increased complexity in bootloader (it has to erase and write flash, not bad at all but still something), halved write endurance of flash (probably not a big deal at all), and significant boot latency after the update (erasing is slow, often in seconds) - this might be a real problem.

IMHO having to link two binaries is a non-issue - you just run the linker twice with slightly different linker scripts -, but this is also matter of taste. If you do that, you can just choose whether to run A or B from the bootloader. Downside is having to either communicate, from the app, to the updater, which of the two files the MCU app wants - easy if you have some kind of periodic status message anyway; or always providing the two files concatenated each time, doubling the file transfer length through your update link.

If you want 100% robust flashing against power failures, and I think you should in $current_year, there is no way around effectively halving the flash, it happens in any scenario except the non-robust one.

[Tripplecheckuser]

Do you know any good resources that have the model of bootloader for the STM32boards? I plan on searching but would like a good place to start.

[krho]

Why halving the flash space available in MCU. Add SPI flash and have a backup copy there. In most cases it's cheaper than getting MCU with twice the flash. Especially if you need 1MiB+

[nctnico]

Another option is to check the validity of B and overwrite A if B has a different (*) firmware version than A. That way you don't need to remap any of the memory to make a firmware image work from different base addresses (or have different firmware images to work from different addresses). Retry if the checksum for A is invalid after programming. Unless the flash is broken, this doesn't leave you with a bricked system.

* checking for a different firmware version also allows to downgrade firmware by simply uploading an older version into the device.

So for a use case example:

• I have a LED BLINKING CODE (A) and an NEW LED BLINKING CODE (B).
• I flash my bootloader + A in my controller.
• My bootloader checks to see if there is any code in B and if the fw version is different. If not, bootloader runs A.
• I want to update my controller with NEW LED BLINKING CODE.
• I get NEW LED BLINKING CODE through A (using some i2c or uart communication GETNEWFW();) and store it in B then restart controller.
• Bootloader then checks if B is a different fw version than A (and it is now) so it overwrites A with B (NEW LED BLINKING CODE) and jumps to A (which now has NEW LED BLINKING CODE).

If the controller losses power...

• while A is storing NEW LED BLINKING LED in B, it will just retry when power is restored, allowing A to remain in tact
• when the bootloader is writing B to A, then A would be incomplete. I assume the controller would always have to always start in bootloader to prevent this from being a problem. So if the controller starts and sees A is invalid and B is (I assume it ignores the fm version in this case), it writes B to A.

Draw Backs:

• If the GETNEWFW function (that will be located in the currently running app) is implemented wrong on a new image, the device can not longer get new fw and will be bricked (maybe I could make that function modular and place it in RAM so it never gets changed)
• Double of my flash will be used (A + B) but only half of that will be in use at any given time (A)

Pros:

• the bootloader will be pretty robust and small
• bricking controller will be less likely
• can update with both older and new fw versions

 


Is this how this scenario would go?

Yes. And indeed, testing new firmware where it comes to receiving new firmware images is essential. OTOH: testing new firmware for bugs that affect existing firmware features is always necessary.

[Siwastaja]

Why halving the flash space available in MCU. Add SPI flash and have a backup copy there. In most cases it's cheaper than getting MCU with twice the flash. Especially if you need 1MiB+

It's still halving what you have available. You are adding more and halving that total. I don't agree with the general principle being good, because it adds cost and complexity. In some rare cases it might end up cheaper, but most microcontrollers I have ever worked with have had more than double the flash for the application, or if not, a next up model costs only a little bit more. (BOM line, PCB real estate, routing work, and loss of available IO also has an equivalent cost which can be more than the cost of a $0.50 flash chip.)

One notable exception is STM32H750 which is a stripped-down-flash version of STM32H743, and only has a single sector, so you definitely want external flash, not only due to the relatively small amount of flash, but also it being in one erase unit only, preventing failsafe firmware update. And the cost difference of these two parts is more than just $1 or $2.

[Psi]

Wouldn't I still waste 1/2 the space if I went with the A/B switching scheme since I would need to account for APP and APP2  even though one app is used at at time? I am leaning towards the 'intermediate option'.

Yes, the only way to not waste that space is to use the more traditional bootloader method.
Where you have a bootloader space and an one APP space. The bootloader runs and talks to the outside world to receive new FW data and write them to the APP area.

The disadvantage with this is that the bootloader is more complicated and larger in size.
It needs to understand RS232/SPI/TCP/BLE or whatever method you use for FW updates, and include libraries for them.  So there is more risk of bugs.

Because your bootloader needs the full communication stack you are now duplicated the stack in two places, in your app and in your bootloader.
Not a problem if you are using UART, but if your using TCP or BLE it gets annoying and wasteful. If in the future you update the app you may end up with a newer communication stack in your bootloader than in your app.  Which is fine but you need to support both, so has legacy support issues.

Check out this bootloader video that mikeselectricstuff did.
It's quite good at explaining things.