Generic solutions for remote firmware updates?

[e100]

This is a beginners question, do any of the newer (or older) micros from any manufacturer have specific onboard hardware to allow remote firmware updates without first resetting it and relying on the bootloader to shuffle data around. In other words, something like a co-processor that has the sole job of downloading and managing the firmware for the main processor. From what little I understand currently it mainly falls to the developer to add firmware download and upgrade code to their application which seems like a burden they could do without.

I've only ever used the Arduino which doesn't really support remote updates due to memory limitations, so I curious as to what other micros do.

Mike

[stfsux]

A "generic way" to upgrade a firmware is to use a "standard" such as JTAG (you can even debug with it).
You still need some specific hardware to talk with your target.
A jtag programmer/debugger wont work with all MCU which have jtag support.
There is openocd which tries to be a generic jtag debugger/programmer software (supports lots of hardware debugger).
There is clearly no generic way to upgrade a firmware, you have either to write your own bootloader or buy a specific hardware/software.

[mikeselectricstuff]

What is the issue ? - a bootloader doesn't have to start from reset ( though it should be able to do so as a fallback in case of a bad update )

[nctnico]

From what little I understand currently it mainly falls to the developer to add firmware download and upgrade code to their application which seems like a burden they could do without.

Unfortunately that is the way it is. Security and delivery methods vary so wildly that it is impossible to come up with a solution which fits all needs.

[mikeselectricstuff]

There are some that provide ROM based bootloaders, e.g. NXP, from serial and USB, but these may or may not be useful for end-product use

[nctnico]

There are some that provide ROM based bootloaders, e.g. NXP, from serial and USB, but these may or may not be useful for end-product use

I have good experience with NXP's bootloader in their ARM devices for field updates and customers doing their own programming/updates. Still the firmware files are plain hex files so no copy protection.

[rstofer]

The mbed platform does what you expect:  It downloads the new image over USB and stores it.  The new code will be loaded into the main processor when the board is reset.
To accomplish this, there is a separate MCU that deals with USB, downloading and flashing the main MCU.
The board itself looks like a mass storage device to the PC that downloads the image.

Look into the features of uboot and see if it will do something like what you want.

I would install updates via SD cards.  I still might have two MCUs but maybe not.

Most ARM MCUs have some kind of serial programming so a little AVR between an SD card and the main MCU could handle the updates.

[nctnico]

IMHO a solution with 2 MCUs is useless because you'd have to a) implement some kind of bootloader on the master anyway and b) you have to implement a protocol between the master and slave which is more trouble than it is worth.

[mikeselectricstuff]

IMHO a solution with 2 MCUs is useless because you'd have to a) implement some kind of bootloader on the master anyway and b) you have to implement a protocol between the master and slave which is more trouble than it is worth.

With a second MCU, the idea is it would use the main MCU's hardware programming interface.
A long time ago, before AVRs had self-programming, I did this in a product, using a PIC to program the main AVR

[alank2]

I've got a product that can remotely update AVR's over ISP.  It is a small, self contained programmer that you configure to handle one or more AVR's.  You can tell it to look for a specific AVR signature along with specific fuses, and you can configure it to only update X number of units before it will fail to update more.  I send it out to a customer with a prepaid return envelope so I get them back.  Here is my page at avrfreaks on it:

http://www.avrfreaks.net/forum/wts-standalone-pdiisp-auto-programmers

[e100]

The mbed platform does what you expect:  It downloads the new image over USB and stores it.  The new code will be loaded into the main processor when the board is reset.
To accomplish this, there is a separate MCU that deals with USB, downloading and flashing the main MCU.
The board itself looks like a mass storage device to the PC that downloads the image.

I think the Pyboard does a similar sort of thing emulating a mass storage device.
USB has a 25m length limitation (5 jumps of 5 meters) max due to timing issues.
I dream of having some kind of transparent bridge that can make a remote micro such as an mbed appear as a local device even though it is far away at the end of a long serial cable or on the either side of the world linked by TCPIP.

[Jeroen3]

Get an Segger J-Link PRO.
https://www.segger.com/jlink-pro.html

[technix]

I had an idea for AVR based on a Bluetooth module, an iPhone, and Optiboot. The Bluetooth module allowed iPhone to talk with the target AVR, preloaded with Optiboot, through BLE. Firmware updates can be packaged along with the mobile app, updated along with the app, and pushed into the user device upon connection.

[Psi]

Have a look at the nRF52 series. It's new and has BLE/RF radio on board.
But ignoring the RF, even as a generic MCU it has some pretty nice things going for it.
Really nice datasheets
Arm core
Any peripheral can be mapped to any of the gpio pins
A lot can be done in hardware. you can setup lists of i2C command in the easydma controller and have it talk to the I2C device to load stuff into memory all while the cpu is sleeping.
Plus it has remote firmware update build it, even over BLE. (Although it might be part of the softdevice binary)

[Bruce Abbott]

it mainly falls to the developer to add firmware download and upgrade code to their application which seems like a burden they could do without.

Developers having to write their own code? What a burden!

something like a co-processor that has the sole job of downloading and managing the firmware for the main processor.

A waste of hardware that does nothing most the time, when software can do the same job.

But if the 'burden' is too much for you, choose a platform that has in-circuit programming and interface to that. For example the Arduino Mini is programmed through a 'generic' TTL serial port. Interface that to a modem and boom - instant remote firmware updates!   

Use XBee for Wireless Arduino Programming

[diyaudio]

Hi, I was about to start a thread around the question (FOA) might as well join the conversation here.

I'm doing research around remote firmware update using a Arduino 328P (nano) platform, this is what I have thus far.

1) External Micron M25P16 Serial SPI Flash, I've allocated two firmware images a tested working recovery image and an area for firmware updates..
My idea's came from Mike's video which is really interesting from a theoretical point of view.
 

The issue I'm having is understanding how running "program code" copies firmware update code into memory code for the bootloader the programming the device.

Thanks to mike for the idea.

[mikeselectricstuff]

The issue I'm having is understanding how running "program code" copies firmware update code into memory code for the bootloader the programming the device.

Not sure what you don't understand - your main application code simply has some means to get data from whatever comms or storage interface it already uses and program it into the flash.

e.g. if your application already has a SD card filesystem, you add a routine that copies a file from the SD card to the flash. Then all teh bootloader has to do is  update the firmware from the flash, which is easy to access.

Intermediate memory bootloaders do not generally make sense for the simple case of updating from a simple serial port etc., though they might be useful in situations where the link may be unreliable - e.g radio - as you can ensure you have a complete and validated image before you start programming it. 

The whole point is that where the means of getting the new firmware image is complex ( filesystem, tcp/ip or whatever), you're greatly simplifying the job that the bootloader has to do, minimising its size and complexity.

The bootloader is typically difficult or impossible to update, so you want it to do as little as possible to reduce the likelihood of show-stopping bugs, and leave as much space as possible for your main application code.

[donotdespisethesnake]

@diyaudio I'm not sure I understand your question.

The application will write the firmware update to SPI Flash, then reset. After reset, the bootloader will read the SPI Flash to look for a new firmware update.

Using intermediate storage with a simple bootloader works quite well. There is an example here https://github.com/triffid/LPC17xx-DFU-Bootloader. The application presents a USB mass storage device and the firmware update is written to an SD card, the bootloader is still a little complex but the principle is similar.

[diyaudio]

The issue I'm having is understanding how running "program code" copies firmware update code into memory code for the bootloader the programming the device.

Intermediate memory bootloaders do not generally make sense for the simple case of updating from a simple serial port etc., though they might be useful in situations where the link may be unreliable - e.g radio - as you can ensure you have a complete and validated image before you start programming it. 

Hopefully I can modify Optiboot https://github.com/Optiboot/optiboot
to function as a 'Intermediate memory bootloader" using the following flow.

1) Program runs as per normal, go into update mode after a valid update firmware is found "after performing house keeping routines" i.e (CRC, Uncompres, Decrypt).
2) Go into DFU mode. Reset the device and run the boot loader.
3) Use a modified version of Optiboot bootloader to read the new firmware from external SPI flash (if a pending update is available) and perform the programming operation and exit.

I hope this is the correct technique to use.

[mikeselectricstuff]

The issue I'm having is understanding how running "program code" copies firmware update code into memory code for the bootloader the programming the device.

Intermediate memory bootloaders do not generally make sense for the simple case of updating from a simple serial port etc., though they might be useful in situations where the link may be unreliable - e.g radio - as you can ensure you have a complete and validated image before you start programming it. 

Hopefully I can modify Optiboot https://github.com/Optiboot/optiboot
to function as a 'Intermediate memory bootloader" using the following flow.

1) Program runs as per normal, go into update mode after a valid update firmware is found "after performing house keeping routines" i.e (CRC, Uncompres, Decrypt).
2) Go into DFU mode. Reset the device and run the boot loader.
3) Use a modified version of Optiboot bootloader to read the new firmware from external SPI flash (if a pending update is available) and perform the programming operation and exit.

I hope this is the correct technique to use.

Why would you mess about modifying something intended for a completely different scenario - most of thr code will be for comms which won;t be relevant - flash programming isn't hard - just read the  datasheet & write it from scratch.

[diyaudio]

The issue I'm having is understanding how running "program code" copies firmware update code into memory code for the bootloader the programming the device.

Intermediate memory bootloaders do not generally make sense for the simple case of updating from a simple serial port etc., though they might be useful in situations where the link may be unreliable - e.g radio - as you can ensure you have a complete and validated image before you start programming it. 

Hopefully I can modify Optiboot https://github.com/Optiboot/optiboot
to function as a 'Intermediate memory bootloader" using the following flow.

1) Program runs as per normal, go into update mode after a valid update firmware is found "after performing house keeping routines" i.e (CRC, Uncompres, Decrypt).
2) Go into DFU mode. Reset the device and run the boot loader.
3) Use a modified version of Optiboot bootloader to read the new firmware from external SPI flash (if a pending update is available) and perform the programming operation and exit.

I hope this is the correct technique to use.

Why would you mess about modifying something intended for a completely different scenario - most of thr code will be for comms which won;t be relevant - flash programming isn't hard - just read the  datasheet & write it from scratch.

Found a reference document.

Atmel devices have a feature called Self programming Program
memory. This feature enables an AVR device to reprogram the Flash
memory while executing the program. Such a feature is helpful for
applications that must self-update firmware or store parameters in Flash.
This application note provides the details about the C functions for accessing
the Flash memory.

http://www.atmel.com/images/atmel-2575-c-functions-for-reading-and-writing-to-flash-memory_applicationnote_avr106.pdf

[diyaudio]

The issue I'm having is understanding how running "program code" copies firmware update code into memory code for the bootloader the programming the device.

Not sure what you don't understand - your main application code simply has some means to get data from whatever comms or storage interface it already uses and program it into the flash.

e.g. if your application already has a SD card filesystem, you add a routine that copies a file from the SD card to the flash. Then all teh bootloader has to do is  update the firmware from the flash, which is easy to access.

Intermediate memory bootloaders do not generally make sense for the simple case of updating from a simple serial port etc., though they might be useful in situations where the link may be unreliable - e.g radio - as you can ensure you have a complete and validated image before you start programming it. 

The whole point is that where the means of getting the new firmware image is complex ( filesystem, tcp/ip or whatever), you're greatly simplifying the job that the bootloader has to do, minimising its size and complexity.

The bootloader is typically difficult or impossible to update, so you want it to do as little as possible to reduce the likelihood of show-stopping bugs, and leave as much space as possible for your main application code.

After a few hours of reading I have a better understanding.

Here is the catch, you cannot program flash memory from the main application code, The Store Program Memory (SPM) instruction performs self programming only from the Boot Flash section, what this means is and what I was trying to understand is application code will only be responsible for obtaining the firmware from the radio, then a highly stripped down version of optiboot will run read from SPI comms and perform the software upgrade.


I'm slowly going through the application notes and thus far it looks easy.

http://www.atmel.com/images/doc1644.pdf
http://www.atmel.com/Images/Atmel-0943-In-System-Programming_ApplicationNote_AVR910.pdf

Using SPM Instructions

The Flash memory may be programmed using the Store Program Memory (SPM) instruction. On devices
containing the Self Programming feature the program memory is divided into two main sections: (1)
Application Flash Section and (2) Boot Flash Section.

On devices with boot block, the SPM instruction has the ability to write to the entire Flash memory, but
can only be executed from the Boot section. Executing SPM from the Application section will have no
effect. On the smaller devices that do not have a boot block, the SPM instruction can be executed from
the entire flash memory.


 
 

[voltsandjolts]

You're making this hard for yourself.
The application code writes the new firmware file to a flash memory device which is NOT currently executing code.
Maybe an external flash chip that is solely dedicated to store firmware updates - they are cheap after all.

[diyaudio]

You're making this hard for yourself.
The application code writes the new firmware file to a flash memory device which is NOT currently executing code.
Maybe an external flash chip that is solely dedicated to store firmware updates - they are cheap after all.

I think there was a misunderstanding here.

1) The main application code obtains the software updates and stores it temporary inside external SPI flash.
2) The bootloader software loads the software from external flash, and programs "application flash" from "boot memory".

what do you mean I'm making it hard ?
     

[voltsandjolts]

Ahh, yes, I misunderstood, sorry.
I thought you were talking about programming the external flash.
You are a step ahead of me, you were actually talking about the bootloader programming the internal flash.
I'll get my coat on the way out....

[mikeselectricstuff]

then a highly stripped down version of optiboot will run read from SPI comms and perform the software upgrade.

err, no Optiboot is designed to do a totally different job - program using an external comms protocol. Most of the code will be the communication protocol, which is irrelevant.
You should probably look at the Optiboot code as a guide to how to use the flash programming instructions and configure the linker but it would make little sense to use any significant part of it.
Copying form SPI flash to code memory will be maybe 20-30 lines of code max including all setup.

 

[rainsee]

You're making this hard for yourself.
The application code writes the new firmware file to a flash memory device which is NOT currently executing code.
Maybe an external flash chip that is solely dedicated to store firmware updates - they are cheap after all.

[diyaudio]

You're making this hard for yourself.
The application code writes the new firmware file to a flash memory device which is NOT currently executing code.
Maybe an external flash chip that is solely dedicated to store firmware updates - they are cheap after all.

This was already concluded.

[Bruce Abbott]

Here is the catch, you cannot program flash memory from the main application code, The Store Program Memory (SPM) instruction performs self programming only from the Boot Flash section... application code will only be responsible for obtaining the firmware from the radio, then a highly stripped down version of optiboot will run read from SPI comms and perform the software upgrade.

While the SPM instruction itself must be in Boot Flash, the rest of the code doesn't have to be.

However if you want to be able to update the entire application space (including the code responsible for downloading the update from the radio) then you need external memory. Since this could cause the the device to be 'bricked' you also need a backup copy of the application that cannot be overwritten, and some way to restore it locally. The backup copy should be hardware write-protected to prevent an errant application from overwriting it.


 

[westfw]

You should probably look at the Optiboot code as a guide to how to use the flash programming instructions and configure the linker but it would make little sense to use any significant part of it.

I'm the main optiboot maintainer these days, and I agree with this.   More of optiboot is concerned with managing the serial communications than with programming the flash.

A lot of modern microcontrollers will include some sort of bootloader that runs from ROM, and will download code over some common interfaces (usually USB or Serial.)  This may or may not include "utility" functions to aid writing your own bootloader.  Typically, the built-in bootloader will not have the features that the user wants :-(  (Ie, it's good for the very first time loading a program, but doesn't quite do what they want in the end application.)

Several other microcontroller feature a dedicated section of flash for the bootloader.  Ie an ATmega AVR will expect you to use some of the normal program memory for a bootload, IF you want, but an ATXmega has a separate 4k of "boot memory" in addition to its advertised flash space.

The Teensy-3 arduino-like boards use a separate bootloading processor (typically another itty-bitty ARM.)  The code and exact technique are proprietary, but as far as I can figure out, it wiggles JTAG pins to load a bootloader into RAM of the target ARM, and then lets that run (from RAM.)  Thus it is completely non-invasive of the main ARM.  (This only works for chips that can execute out of RAM, of course.)

[krho]

Having external bootloader/programmer in production is not feasible. The problem is that You want to protect your code, so usually this means, that JTAG, loading external code to RAM and running it, integrated bootloader are all disabled otherwise it would be trivial to dump the firmware out of the MCU.

[e100]

it mainly falls to the developer to add firmware download and upgrade code to their application which seems like a burden they could do without.

Developers having to write their own code? What a burden!

something like a co-processor that has the sole job of downloading and managing the firmware for the main processor.

A waste of hardware that does nothing most the time, when software can do the same job.

But if the 'burden' is too much for you, choose a platform that has in-circuit programming and interface to that. For example the Arduino Mini is programmed through a 'generic' TTL serial port. Interface that to a modem and boom - instant remote firmware updates!   

Use XBee for Wireless Arduino Programming

In my experience wireless programming for the Arduino using this technique only works if you have 0% packet loss.
If you have a little bit of packet loss the transmission retry mechanism built into the 802.15.4 standard introduces delays that confuse the Optiboot bootloader and cause it to timeout. If you have infinite patience you can keep hitting the upload button until it succeeds, but your micro is dead in the water until then.

I would guess that memory constraints would prevent Optiboot ever becoming smarter at dealing with discontinuous or fragmented uploads.

[^_^]

Having external bootloader/programmer in production is not feasible. The problem is that You want to protect your code, so usually this means, that JTAG, loading external code to RAM and running it, integrated bootloader are all disabled otherwise it would be trivial to dump the firmware out of the MCU.

dude, you can just leave some buffer with random data in the bootloader and XOR the incoming firmware bytes recurringly with the buffer ;-)
no one will know unless they somehow dump the bootloader.
not to mention if you have some crypto hardware in the MCU 

[technix]

Having external bootloader/programmer in production is not feasible. The problem is that You want to protect your code, so usually this means, that JTAG, loading external code to RAM and running it, integrated bootloader are all disabled otherwise it would be trivial to dump the firmware out of the MCU.

dude, you can just leave some buffer with random data in the bootloader and XOR the incoming firmware bytes recurringly with the buffer ;-)
no one will know unless they somehow dump the bootloader.
not to mention if you have some crypto hardware in the MCU 

Or you can implement proper Elliptic Curve Diffie-Hellman key exchange, X.509 certificates and ChaCha20-Poly1305 (or AES-256-GCM) AEAD cipher in the boot loader, using uninitialized SRAM as cryptographically secure source of entropy for the crypto, and now you get reasonably good (if not too good) crypto for firmware in transit.

For crypto with firmware files in rest you need to figure out a way to burn a private key into the device (different for each device!) and keep a central registry of public keys on your server. Firmware files themselves are encrypted with ChaCha20-Poly1305 with a single key, but the key is not present in the distribution file (so you can use CDN) and is distributed from the server, encrypted with the public key if the device being upgraded using asymmetric crypto like ECDSS.

ECDHE, ECDSS, ChaCha20-Poly1305 and AES-256-GCM are all very strong cryptography that even NSA can get a headache trying to crack it if used properly. You don't need a separate checksum or hash algorithm if you used AEAD cipher or modes of operation like ChaCha20-Poly1305 or AES-256-GCM (as the Poly1305 addition to the ChaCha20 stream cipher, or the Galois-Counter Mode of a standard block cipher like AES-256, all provide built-in authentication) but if you need one use SHA-256 or SHA-512 if possible.

[westfw]

I would guess that memory constraints would prevent Optiboot ever becoming smarter at dealing with discontinuous or fragmented uploads.

You could increase optiboot's WDT timeout (normally 1s.)  It doesn't have any internal detection of "discontinuous" other than the top-level "if there are no error-free characters received withing the WD timeout, the chip will reset."

Or you can implement proper Elliptic Curve Diffie-Hellman key exchange, X.509 certificates and ChaCha20-Poly1305 (or AES-256-GCM) AEAD

Yeah.  Sure.   I'll point out that the original poster never made any requests at all WRT "protecting" their SW, and I find statements like "you must protect you code so well that bootloaders are out of the question" to be ... deeply debatable.

[diyaudio]

Having external bootloader/programmer in production is not feasible. The problem is that You want to protect your code, so usually this means, that JTAG, loading external code to RAM and running it, integrated bootloader are all disabled otherwise it would be trivial to dump the firmware out of the MCU.

dude, you can just leave some buffer with random data in the bootloader and XOR the incoming firmware bytes recurringly with the buffer ;-)
no one will know unless they somehow dump the bootloader.
not to mention if you have some crypto hardware in the MCU 

Or you can implement proper Elliptic Curve Diffie-Hellman key exchange, X.509 certificates and ChaCha20-Poly1305 (or AES-256-GCM) AEAD cipher in the boot loader, using uninitialized SRAM as cryptographically secure source of entropy for the crypto, and now you get reasonably good (if not too good) crypto for firmware in transit.

For crypto with firmware files in rest you need to figure out a way to burn a private key into the device (different for each device!) and keep a central registry of public keys on your server. Firmware files themselves are encrypted with ChaCha20-Poly1305 with a single key, but the key is not present in the distribution file (so you can use CDN) and is distributed from the server, encrypted with the public key if the device being upgraded using asymmetric crypto like ECDSS.

ECDHE, ECDSS, ChaCha20-Poly1305 and AES-256-GCM are all very strong cryptography that even NSA can get a headache trying to crack it if used properly. You don't need a separate checksum or hash algorithm if you used AEAD cipher or modes of operation like ChaCha20-Poly1305 or AES-256-GCM (as the Poly1305 addition to the ChaCha20 stream cipher, or the Galois-Counter Mode of a standard block cipher like AES-256, all provide built-in authentication) but if you need one use SHA-256 or SHA-512 if possible.

bootloaders should be lean, all of this just sounds like a maintenance mess, the truth is before this level of security is attempted, is the code firmware worth the effort to protect? in most cases no, avoiding a  maintenance mess is the first best effort principle.   

[technix]

Having external bootloader/programmer in production is not feasible. The problem is that You want to protect your code, so usually this means, that JTAG, loading external code to RAM and running it, integrated bootloader are all disabled otherwise it would be trivial to dump the firmware out of the MCU.

dude, you can just leave some buffer with random data in the bootloader and XOR the incoming firmware bytes recurringly with the buffer ;-)
no one will know unless they somehow dump the bootloader.
not to mention if you have some crypto hardware in the MCU 

Or you can implement proper Elliptic Curve Diffie-Hellman key exchange, X.509 certificates and ChaCha20-Poly1305 (or AES-256-GCM) AEAD cipher in the boot loader, using uninitialized SRAM as cryptographically secure source of entropy for the crypto, and now you get reasonably good (if not too good) crypto for firmware in transit.

For crypto with firmware files in rest you need to figure out a way to burn a private key into the device (different for each device!) and keep a central registry of public keys on your server. Firmware files themselves are encrypted with ChaCha20-Poly1305 with a single key, but the key is not present in the distribution file (so you can use CDN) and is distributed from the server, encrypted with the public key if the device being upgraded using asymmetric crypto like ECDSS.

ECDHE, ECDSS, ChaCha20-Poly1305 and AES-256-GCM are all very strong cryptography that even NSA can get a headache trying to crack it if used properly. You don't need a separate checksum or hash algorithm if you used AEAD cipher or modes of operation like ChaCha20-Poly1305 or AES-256-GCM (as the Poly1305 addition to the ChaCha20 stream cipher, or the Galois-Counter Mode of a standard block cipher like AES-256, all provide built-in authentication) but if you need one use SHA-256 or SHA-512 if possible.

bootloaders should be lean, all of this just sounds like a maintenance mess, the truth is before this level of security is attempted, is the code firmware worth the effort to protect? in most cases no, avoiding a  maintenance mess is the first best effort principle.   

If the main application also requires crypto, it can be built in the fashion that the copy of crypto routines used in the boot loader is shared by the application. In fact with just ECDHE, ECDSS, ChaCha20-Poly1305 and SHA-256 you can already get a good set of crypto, even useable when speaking TLS 1.2 to a server.