What is "self-programming" and what is it used for?

[Moriambar]

Hey guys,
simple questions. I noticed that many mcus, e.g. Attiny 85 have a "self programming" feature.
I don't understand how it is used and what it is used for, can anyone help plz?
cheers!

[igendel]

Easier to think of this as "able to burn bytes in its own FLASH memory", as opposed to an external programming device doing it.
An extremely common use is a bootloader - code that's running on the microcontroller, accepts the new firmware data from some outside source and burns it into place.

[Edit: How it's used? Depends on the MCU, it usualy involves some dedicated registers you need to write to (addresses, safety locks etc.) ]

[CountChocula]

It's a feature that allows the program stored in the MPU to update itself (typically using a bootloader) without requiring an external programmer. You can download the new code using any of the serial communication features available (SPI, I2C, etc.), which can themselves be connected to external channels, like an Ethernet or Wi-Fi interface. Theoretically, you could also create self-changing code, but that's of limited use since you already have on-board EEPROM.

Microchip has an [application note](https://ww1.microchip.com/downloads/en/Appnotes/doc1644.pdf) on this if you want more details.


—CC

[Moriambar]

It's a feature that allows the program stored in the MPU to update itself (typically using a bootloader) without requiring an external programmer. You can download the new code using any of the serial communication features available (SPI, I2C, etc.), which can themselves be connected to external channels, like an Ethernet or Wi-Fi interface. Theoretically, you could also create self-changing code, but that's of limited use since you already have on-board EEPROM.

Microchip has an [application note](https://ww1.microchip.com/downloads/en/Appnotes/doc1644.pdf) on this if you want more details.


—CC

Thanks, so basically I could write some code that checks something and downloads the new firmware from an eeprom or something? this sounds so cool.
Also the appnote which is good and I just browsed through, is missing the c code samples that would've made this a bit more clear.

Also thanks to @igendel

[coppice]

Thanks, so basically I could write some code that checks something and downloads the new firmware from an eeprom or something? this sounds so cool.

The two main uses for a self-programming capability for the non-volatile memory are self updating the code in the MCU, and writing extra data, like configuration information or a log, which is specific to that sample of the product. Its something that needs to be used with care, as if you are in the middle of an update and the power dies you could leave the product in an unrecoverable state. There are quite a few gotchas you need to take care of for reliable operation. For example, an MCU might be able to run as low as 1.8V, but can't program its own flash reliably unless the supply is at least 2.8V. Good MCUs have ways to check things like this before you start writing to the non-volatile memory.

For flash devices self-programming has a cost. You need a charge pump on the die, to produce the higher voltage needed to program the flash. Those pumps can take quite a lot of die area in a small MCU, and add to its cost. Most modern flash MCUs include that pump, as self-programming has such broad appeal.

[Moriambar]

First of all thanks for your replies.
I get that it’s smth to use with care, but I find the idea fascinating and I’d like to learn it. It’s like learning to use a powertool: powerful, sometimes dangerous, definitely worth it!
Apart from the aforementioned (and a bit incomplete from their part) Microchip note, are there any suggestions on where to look?

Cheers!

[radiolistener]

note that program flash in MCU often has limited cycles to fail. Usually it has about 10000 cycles or even 1000 cycles for old MCU. So if you use it intensively, your MCU will be broken soon due to failure of it's program flash memory.

[Psi]

There's two reasons you might want to do it.

The first reason and the most common is when you partition flash memory into different blocks for different things and one block updates another block. Like the bootloader block updates the application block. Or the application block updates the block used for NVM storage.

The second reason is much much less common and is 'self-modifying code'. Where the application may decide to change its own code based on the code its running.  I can't think of many examples for this because I don't think it's used much any more now that MCUs use flash memory. (Due to the problem of having to erase an entire flash page making it more compilated to change just one bit of code within a program). Also MCUs are much faster now and have much more storage so the advantages of self-modifying code kind of evaporated.

But an example might be a command causing the MCU to stop processing data and spend some time crunching hard-core math functions to update all its hard-coded lookup tables for a new mode of operation. Though if you wanted to do this today you'd probably just store the lookup table in a different block to the main application. Which would move this idea into the first category.

Or, another example is if the application code is extracting every little bit of performance out of the processor, or has very timing critical tasks to do, it might not be possible to add extra code to support many selectable operating modes. So it might internally reprogram one of its own functions between a few internally stored versions and cause a reboot to change operating modes. But this was a long time ago, when hardware was very slow and people were trying to gain speed any way they could

There's probably better examples of it though, it's not something i've ever had to do.

[radiolistener]

for better performance you're needs to copy flash memory to RAM at boot time and run it from RAM.

Self update usually used for self update during upload firmware to MCU for firmware update, or just to store some config data in the program flash if there is not enough or missing data flash memory.

[magic]

for better performance you're needs to copy flash memory to RAM at boot time and run it from RAM.

You can't even do that on many MCUs, including one the OP asked about.

And speaking of AVR, it has more bandwidth from flash than from RAM.

[mikeselectricstuff]

note that program flash in MCU often has limited cycles to fail. Usually it has about 10000 cycles or even 1000 cycles for old MCU. So if you use it intensively, your MCU will be broken soon due to failure of it's program flash memory.

Some MCUs have a high-endurance section of flash designed for a much higher number of writes

[Psi]

note that program flash in MCU often has limited cycles to fail. Usually it has about 10000 cycles or even 1000 cycles for old MCU. So if you use it intensively, your MCU will be broken soon due to failure of it's program flash memory.

Some MCUs have a high-endurance section of flash designed for a much higher number of writes

I wonder if they actually have different quality flash in that area. Probably more likely it's just got more bits assigned to error correction.

[Moriambar]

wow, any ideas on where to start?

Cheers

[igendel]

Apart from the aforementioned (and a bit incomplete from their part) Microchip note, are there any suggestions on where to look?

I didn't try it myself, but I assume there's Bootloader source code available on the web for many AVR models. You can try to download a few and study them and their notes.
Be warned, many bootloaders are written in Assembly to save FLASH space.

[Moriambar]

Thanks. Although I’m not a wizard with asm, it doesn’t scare me; on the contrary it’s just a great opportunity to learn better something new (and grow new reasoning paths)

I’ll try and see random bootloaders online then. Cheers

[coppice]

note that program flash in MCU often has limited cycles to fail. Usually it has about 10000 cycles or even 1000 cycles for old MCU. So if you use it intensively, your MCU will be broken soon due to failure of it's program flash memory.

Some MCUs have a high-endurance section of flash designed for a much higher number of writes

Are you referring to the MCUs which have a big block of 10k endurance flash for the code, and some higher endurance EEPROM for data that will change often? Those are two distinct technologies - one erasing page wise, and one working byte wise - with very different characteristics.

[JustMeHere]

Check out this.  Fairly good explanation of how the ESP32 can "self update" via Wifi. 

https://lastminuteengineers.com/esp32-ota-updates-arduino-ide/

[Moriambar]

note that program flash in MCU often has limited cycles to fail. Usually it has about 10000 cycles or even 1000 cycles for old MCU. So if you use it intensively, your MCU will be broken soon due to failure of it's program flash memory.

Some MCUs have a high-endurance section of flash designed for a much higher number of writes

Are you referring to the MCUs which have a big block of 10k endurance flash for the code, and some higher endurance EEPROM for data that will change often? Those are two distinct technologies - one erasing page wise, and one working byte wise - with very different characteristics.

Sorry, Talking like a noob here. Let's just say I'm fascinating and I'd like to know more. MAybe never use it, but unless I try I think I won't fully understand the technology and its use cases.

Check out this.  Fairly good explanation of how the ESP32 can "self update" via Wifi. 

https://lastminuteengineers.com/esp32-ota-updates-arduino-ide/

Thanks, I'll definitely check it

[mikeselectricstuff]

note that program flash in MCU often has limited cycles to fail. Usually it has about 10000 cycles or even 1000 cycles for old MCU. So if you use it intensively, your MCU will be broken soon due to failure of it's program flash memory.

Some MCUs have a high-endurance section of flash designed for a much higher number of writes

Are you referring to the MCUs which have a big block of 10k endurance flash for the code, and some higher endurance EEPROM for data that will change often? Those are two distinct technologies - one erasing page wise, and one working byte wise - with very different characteristics.

No - some PICs ( e.g.10F322) have a high-endurance flash area instead of EEPROM, typically 100K writes  vs. 10K for normal flash, which is accessed  the same way as normal flash, i.e. erase able in blocks of typically 16 bytes - it is byte-writeable where previously erased, but for full arbitary byte access you need to do a read to RAM/update/erase/write sequence.

[Electro Fan]

This is Arduino specific but might be useful.

https://www.electronicshub.org/types-of-memory-on-arduino/

If you use both an Arduino IDE and a programmer such as the TL866 with MiniPro software it can help you examine the code and the architecture of the MCU with respect to where the code resides and how the different types of memory can be managed (eg, Byte or block level).

[liaifat85]

The microcontroller can write to its own program memory when
running code from the boot sector.

https://ww1.microchip.com/downloads/en/DeviceDoc/doc2464.pdf

[coppice]

The microcontroller can write to its own program memory when
running code from the boot sector.

https://ww1.microchip.com/downloads/en/DeviceDoc/doc2464.pdf

I think you mean one very specific MCU can write to its own program memory while running code from the boot sector. Look around the market. There are many varied ways in which the conflict between memory block erasure and the need to maintain running code is handled. Flash split into numerous pages, each of which is eraseable and rewriteable, while the other pages continue as normal to run code is pretty common.