Generic solutions for remote firmware updates?

[e100]

This is a beginners question, do any of the newer (or older) micros from any manufacturer have specific onboard hardware to allow remote firmware updates without first resetting it and relying on the bootloader to shuffle data around. In other words, something like a co-processor that has the sole job of downloading and managing the firmware for the main processor. From what little I understand currently it mainly falls to the developer to add firmware download and upgrade code to their application which seems like a burden they could do without.

I've only ever used the Arduino which doesn't really support remote updates due to memory limitations, so I curious as to what other micros do.

Mike

[stfsux]

A "generic way" to upgrade a firmware is to use a "standard" such as JTAG (you can even debug with it).
You still need some specific hardware to talk with your target.
A jtag programmer/debugger wont work with all MCU which have jtag support.
There is openocd which tries to be a generic jtag debugger/programmer software (supports lots of hardware debugger).
There is clearly no generic way to upgrade a firmware, you have either to write your own bootloader or buy a specific hardware/software.

[mikeselectricstuff]

What is the issue ? - a bootloader doesn't have to start from reset ( though it should be able to do so as a fallback in case of a bad update )

[nctnico]

From what little I understand currently it mainly falls to the developer to add firmware download and upgrade code to their application which seems like a burden they could do without.

Unfortunately that is the way it is. Security and delivery methods vary so wildly that it is impossible to come up with a solution which fits all needs.

[mikeselectricstuff]

There are some that provide ROM based bootloaders, e.g. NXP, from serial and USB, but these may or may not be useful for end-product use

[nctnico]

There are some that provide ROM based bootloaders, e.g. NXP, from serial and USB, but these may or may not be useful for end-product use

I have good experience with NXP's bootloader in their ARM devices for field updates and customers doing their own programming/updates. Still the firmware files are plain hex files so no copy protection.

[rstofer]

The mbed platform does what you expect:  It downloads the new image over USB and stores it.  The new code will be loaded into the main processor when the board is reset.
To accomplish this, there is a separate MCU that deals with USB, downloading and flashing the main MCU.
The board itself looks like a mass storage device to the PC that downloads the image.

Look into the features of uboot and see if it will do something like what you want.

I would install updates via SD cards.  I still might have two MCUs but maybe not.

Most ARM MCUs have some kind of serial programming so a little AVR between an SD card and the main MCU could handle the updates.

[nctnico]

IMHO a solution with 2 MCUs is useless because you'd have to a) implement some kind of bootloader on the master anyway and b) you have to implement a protocol between the master and slave which is more trouble than it is worth.

[mikeselectricstuff]

IMHO a solution with 2 MCUs is useless because you'd have to a) implement some kind of bootloader on the master anyway and b) you have to implement a protocol between the master and slave which is more trouble than it is worth.

With a second MCU, the idea is it would use the main MCU's hardware programming interface.
A long time ago, before AVRs had self-programming, I did this in a product, using a PIC to program the main AVR

[alank2]

I've got a product that can remotely update AVR's over ISP.  It is a small, self contained programmer that you configure to handle one or more AVR's.  You can tell it to look for a specific AVR signature along with specific fuses, and you can configure it to only update X number of units before it will fail to update more.  I send it out to a customer with a prepaid return envelope so I get them back.  Here is my page at avrfreaks on it:

http://www.avrfreaks.net/forum/wts-standalone-pdiisp-auto-programmers

[e100]

The mbed platform does what you expect:  It downloads the new image over USB and stores it.  The new code will be loaded into the main processor when the board is reset.
To accomplish this, there is a separate MCU that deals with USB, downloading and flashing the main MCU.
The board itself looks like a mass storage device to the PC that downloads the image.

I think the Pyboard does a similar sort of thing emulating a mass storage device.
USB has a 25m length limitation (5 jumps of 5 meters) max due to timing issues.
I dream of having some kind of transparent bridge that can make a remote micro such as an mbed appear as a local device even though it is far away at the end of a long serial cable or on the either side of the world linked by TCPIP.

[Jeroen3]

Get an Segger J-Link PRO.
https://www.segger.com/jlink-pro.html

[technix]

I had an idea for AVR based on a Bluetooth module, an iPhone, and Optiboot. The Bluetooth module allowed iPhone to talk with the target AVR, preloaded with Optiboot, through BLE. Firmware updates can be packaged along with the mobile app, updated along with the app, and pushed into the user device upon connection.

[Psi]

Have a look at the nRF52 series. It's new and has BLE/RF radio on board.
But ignoring the RF, even as a generic MCU it has some pretty nice things going for it.
Really nice datasheets
Arm core
Any peripheral can be mapped to any of the gpio pins
A lot can be done in hardware. you can setup lists of i2C command in the easydma controller and have it talk to the I2C device to load stuff into memory all while the cpu is sleeping.
Plus it has remote firmware update build it, even over BLE. (Although it might be part of the softdevice binary)

[Bruce Abbott]

it mainly falls to the developer to add firmware download and upgrade code to their application which seems like a burden they could do without.

Developers having to write their own code? What a burden!

something like a co-processor that has the sole job of downloading and managing the firmware for the main processor.

A waste of hardware that does nothing most the time, when software can do the same job.

But if the 'burden' is too much for you, choose a platform that has in-circuit programming and interface to that. For example the Arduino Mini is programmed through a 'generic' TTL serial port. Interface that to a modem and boom - instant remote firmware updates!   

Use XBee for Wireless Arduino Programming

[diyaudio]

Hi, I was about to start a thread around the question (FOA) might as well join the conversation here.

I'm doing research around remote firmware update using a Arduino 328P (nano) platform, this is what I have thus far.

1) External Micron M25P16 Serial SPI Flash, I've allocated two firmware images a tested working recovery image and an area for firmware updates..
My idea's came from Mike's video which is really interesting from a theoretical point of view.
 

The issue I'm having is understanding how running "program code" copies firmware update code into memory code for the bootloader the programming the device.

Thanks to mike for the idea.

[mikeselectricstuff]

The issue I'm having is understanding how running "program code" copies firmware update code into memory code for the bootloader the programming the device.

Not sure what you don't understand - your main application code simply has some means to get data from whatever comms or storage interface it already uses and program it into the flash.

e.g. if your application already has a SD card filesystem, you add a routine that copies a file from the SD card to the flash. Then all teh bootloader has to do is  update the firmware from the flash, which is easy to access.

Intermediate memory bootloaders do not generally make sense for the simple case of updating from a simple serial port etc., though they might be useful in situations where the link may be unreliable - e.g radio - as you can ensure you have a complete and validated image before you start programming it. 

The whole point is that where the means of getting the new firmware image is complex ( filesystem, tcp/ip or whatever), you're greatly simplifying the job that the bootloader has to do, minimising its size and complexity.

The bootloader is typically difficult or impossible to update, so you want it to do as little as possible to reduce the likelihood of show-stopping bugs, and leave as much space as possible for your main application code.

[donotdespisethesnake]

@diyaudio I'm not sure I understand your question.

The application will write the firmware update to SPI Flash, then reset. After reset, the bootloader will read the SPI Flash to look for a new firmware update.

Using intermediate storage with a simple bootloader works quite well. There is an example here https://github.com/triffid/LPC17xx-DFU-Bootloader. The application presents a USB mass storage device and the firmware update is written to an SD card, the bootloader is still a little complex but the principle is similar.

[diyaudio]

The issue I'm having is understanding how running "program code" copies firmware update code into memory code for the bootloader the programming the device.

Intermediate memory bootloaders do not generally make sense for the simple case of updating from a simple serial port etc., though they might be useful in situations where the link may be unreliable - e.g radio - as you can ensure you have a complete and validated image before you start programming it. 

Hopefully I can modify Optiboot https://github.com/Optiboot/optiboot
to function as a 'Intermediate memory bootloader" using the following flow.

1) Program runs as per normal, go into update mode after a valid update firmware is found "after performing house keeping routines" i.e (CRC, Uncompres, Decrypt).
2) Go into DFU mode. Reset the device and run the boot loader.
3) Use a modified version of Optiboot bootloader to read the new firmware from external SPI flash (if a pending update is available) and perform the programming operation and exit.

I hope this is the correct technique to use.

[mikeselectricstuff]

The issue I'm having is understanding how running "program code" copies firmware update code into memory code for the bootloader the programming the device.

Intermediate memory bootloaders do not generally make sense for the simple case of updating from a simple serial port etc., though they might be useful in situations where the link may be unreliable - e.g radio - as you can ensure you have a complete and validated image before you start programming it. 

Hopefully I can modify Optiboot https://github.com/Optiboot/optiboot
to function as a 'Intermediate memory bootloader" using the following flow.

1) Program runs as per normal, go into update mode after a valid update firmware is found "after performing house keeping routines" i.e (CRC, Uncompres, Decrypt).
2) Go into DFU mode. Reset the device and run the boot loader.
3) Use a modified version of Optiboot bootloader to read the new firmware from external SPI flash (if a pending update is available) and perform the programming operation and exit.

I hope this is the correct technique to use.

Why would you mess about modifying something intended for a completely different scenario - most of thr code will be for comms which won;t be relevant - flash programming isn't hard - just read the  datasheet & write it from scratch.

[diyaudio]

The issue I'm having is understanding how running "program code" copies firmware update code into memory code for the bootloader the programming the device.

Intermediate memory bootloaders do not generally make sense for the simple case of updating from a simple serial port etc., though they might be useful in situations where the link may be unreliable - e.g radio - as you can ensure you have a complete and validated image before you start programming it. 

Hopefully I can modify Optiboot https://github.com/Optiboot/optiboot
to function as a 'Intermediate memory bootloader" using the following flow.

1) Program runs as per normal, go into update mode after a valid update firmware is found "after performing house keeping routines" i.e (CRC, Uncompres, Decrypt).
2) Go into DFU mode. Reset the device and run the boot loader.
3) Use a modified version of Optiboot bootloader to read the new firmware from external SPI flash (if a pending update is available) and perform the programming operation and exit.

I hope this is the correct technique to use.

Why would you mess about modifying something intended for a completely different scenario - most of thr code will be for comms which won;t be relevant - flash programming isn't hard - just read the  datasheet & write it from scratch.

Found a reference document.

Atmel devices have a feature called Self programming Program
memory. This feature enables an AVR device to reprogram the Flash
memory while executing the program. Such a feature is helpful for
applications that must self-update firmware or store parameters in Flash.
This application note provides the details about the C functions for accessing
the Flash memory.

http://www.atmel.com/images/atmel-2575-c-functions-for-reading-and-writing-to-flash-memory_applicationnote_avr106.pdf

[diyaudio]

The issue I'm having is understanding how running "program code" copies firmware update code into memory code for the bootloader the programming the device.

Not sure what you don't understand - your main application code simply has some means to get data from whatever comms or storage interface it already uses and program it into the flash.

e.g. if your application already has a SD card filesystem, you add a routine that copies a file from the SD card to the flash. Then all teh bootloader has to do is  update the firmware from the flash, which is easy to access.

Intermediate memory bootloaders do not generally make sense for the simple case of updating from a simple serial port etc., though they might be useful in situations where the link may be unreliable - e.g radio - as you can ensure you have a complete and validated image before you start programming it. 

The whole point is that where the means of getting the new firmware image is complex ( filesystem, tcp/ip or whatever), you're greatly simplifying the job that the bootloader has to do, minimising its size and complexity.

The bootloader is typically difficult or impossible to update, so you want it to do as little as possible to reduce the likelihood of show-stopping bugs, and leave as much space as possible for your main application code.

After a few hours of reading I have a better understanding.

Here is the catch, you cannot program flash memory from the main application code, The Store Program Memory (SPM) instruction performs self programming only from the Boot Flash section, what this means is and what I was trying to understand is application code will only be responsible for obtaining the firmware from the radio, then a highly stripped down version of optiboot will run read from SPI comms and perform the software upgrade.


I'm slowly going through the application notes and thus far it looks easy.

http://www.atmel.com/images/doc1644.pdf
http://www.atmel.com/Images/Atmel-0943-In-System-Programming_ApplicationNote_AVR910.pdf

Using SPM Instructions

The Flash memory may be programmed using the Store Program Memory (SPM) instruction. On devices
containing the Self Programming feature the program memory is divided into two main sections: (1)
Application Flash Section and (2) Boot Flash Section.

On devices with boot block, the SPM instruction has the ability to write to the entire Flash memory, but
can only be executed from the Boot section. Executing SPM from the Application section will have no
effect. On the smaller devices that do not have a boot block, the SPM instruction can be executed from
the entire flash memory.


 
 

[voltsandjolts]

You're making this hard for yourself.
The application code writes the new firmware file to a flash memory device which is NOT currently executing code.
Maybe an external flash chip that is solely dedicated to store firmware updates - they are cheap after all.

[diyaudio]

You're making this hard for yourself.
The application code writes the new firmware file to a flash memory device which is NOT currently executing code.
Maybe an external flash chip that is solely dedicated to store firmware updates - they are cheap after all.

I think there was a misunderstanding here.

1) The main application code obtains the software updates and stores it temporary inside external SPI flash.
2) The bootloader software loads the software from external flash, and programs "application flash" from "boot memory".

what do you mean I'm making it hard ?
     

[voltsandjolts]

Ahh, yes, I misunderstood, sorry.
I thought you were talking about programming the external flash.
You are a step ahead of me, you were actually talking about the bootloader programming the internal flash.
I'll get my coat on the way out....