Are there too many different bootloaders in the Arduino world?

[elecdonia]

I’m doing several projects which require a small MCU with true 5V I/O. Atmel AVR devices are a perfect match for my needs. And because the Chinese offer a vast quantity of “Arduino compatible” boards for <$10 USD each, I stocked up. This speeds up breadboarding and debugging. My end products will likely contain “bare metal” AVR devices, but I’m not starting out that way.

I created this topic about bootloaders because some of these Chinese boards I received in Jan. 2023 came with really strange bootloaders:

1) A so-called “revised/upgraded Arduino-compatible 3V/8MHz pro-mini” board labeled “the simple.” By selecting board: “Arduino pro mini” in the Arduino IDE I successfully loaded one sketch into it. The sketch works fine, but afterwards the bootloader on this board totally lost its capability to connect to the Arduino IDE or to AVRDUDE. I’m using an adafruit FTDI friend which functions perfectly with 10 other (older) pro-mini boards, both 3V/8MHz and 5V/16MHz. Only this new “the simple” pro-mini board is acting strange. Fortunately it is still fully accessible with ICSP and a hardware programmer. Looking at it with ICSP I observed its bootloader is still there in flash. It didn’t get overwritten or erased. But it certainly doesn’t work now. This bootloader is 2k bytes, which seems kinda large to me. I’m about ready to load a different bootloader onto it. I use the 512 byte Optiboot along with “minicore” on several of my older pro-mini boards. I think I will try that with “the simple.”

2) I got a Chinese mega2560pro board which works nicely but has the largest bootloader I’ve ever seen on an Arduino-compatible board: 8k bytes. Compare that to most renditions of Optiboot which are 512 bytes. A bit of searching revealed the 256k AVR requires a whole different bootloader known as “STK500v2” in order to handle its gigantic address space. The person who originally developed the STK500v2 bootloader equipped it with a nifty self-contained debugging monitor which could be accessed by opening a serial terminal and typing “!!!” within 3 seconds of resettting the board. Unfortunately any sketch containing !!! would crash the boot loading process. Therefore the STK500v2 bootloader was soon revised to totally disable access to its internal debug monitor. But the debug monitor code is still in there today, occupying most of the 8k bytes. Ghost code? I realize that wasting 8k of flash isn’t a big deal for a 256k MCU. But this whole deal just seems really strange to me.

3) Next I started reading discussions about Arduino bootloaders on several different forums and observed that fully 50% of the threads are about being unable to use the bootloader. The most common fault is “error message says STK500 protocol won’t sync between board and AVRDUDE.”  Well, that’s exactly the message I’m getting from my “the simple” pro-mini clone board. But it worked perfectly… …one time. I wonder if the bootloader on it is some ancient version from the earliest days of Arduino?

If you are still with me I would like to learn which Arduino-compatible bootloaders you love and which ones you hate. Cheers!

[westfw]

You an put Optiboot on everything, including the 2560 boards.  Use one of MCUDude's or SpenceKonde's "cores", and/or refactor your thinking to decide that all ATmegaxx8 boards are "Unos."
Here's some history:

• Original Arduino bootloader is ATmegaBoot, which takes 2k and runs at a relatively low bitrate.  It uses stk500v1 protocol.
• It also had some bugs, and modified versions were distributed by Adafruit and Sparkfun.
• The Arduino Mega ships, and it has so much memory that that a fancier bootloader (including a "monitor" capability) is used - stk500v2 boot.  It also permits more than 64kwords to be uploaded, which stk500v1 didn't (at least, theoretically.)
• Optiboot shipped on the Uno.  At 512bytes, it frees up 1.5k for user programs (which no one really cares about, since the m328 has a whopping 32k!)  Inexplicably, upgrading old ATmega8 and ATmega168 boards does not catch on.
• Somewhere between (2) and (6), billions of "clone" vendors spring up and start shipping boards that are basically the same as one of the existing Arduino boards, but have ... who knows what bootloader on them.
• Optiboot is modified to support many more chips.  A cooperative hack between Optiboot and avrdude permits it to support more than 64kwords.
• Arduino adds (pretty good) 3rd-part board capability.  MCUDude implements most of the ATmega chips, SpenceKonde implements most of the ATtiny chips.  All using Optiboot.  (MCUDude adds some features, like a callable vector to write flash.  Optiboot is now at v8.1, and has Optiboot-x for mega0/xtiny and OptibootDX for AVR-Dx.  Arduino is still shipping version 4.4  Sigh.)
• Arduino starts shipping the Nano with Optiboot instead of ATmegaBoot.  If you have old Nanos, you have to set the board for "(old bootloader.)"
• Arduino ships mega0-based boards (Nano Every and Uno WiFi 2) that don't use a bootloader at all.
• "Native USB" platforms need an entirely different bootloader to talk USB.
• Non-AVR platforms need a different bootloader.

STK500 protocol won’t sync between board and AVRDUDE.

AVRDude error messages are not always the best.  "Can't sync" is what you're likely to get for a whole host of problems like: "you set wrong port", "the chip has a different bootloader", "your cable is bad", "Your USB/Serial Driver chip is bad", "Your AVR is bad", "You have something interfering with the serial port on the AVR", "There is no bootloader on the chip", "The bootloader is OK but the fuses are wrong", etc.

[NorthGuy]

My end products will likely contain “bare metal” AVR devices, but I’m not starting out that way.

Then your project will not have an Arduino bootloader and you will have to re-do all your Arduino work. If you want to go bare metal, start bare metal. This will save you lots of time and effort.

[elecdonia]

You can put Optiboot on everything, including the 2560 boards.  Use one of MCUDude's or SpenceKonde's "cores", and/or refactor your thinking to decide that all ATmegaxx8 boards are "Unos."
Here's some history: <snip>

Thanks for your suggestions and for providing both the history and context of Arduino bootloaders.

I’m thinking I will standardize by installing Optiboot on all of my Arduino-compatible boards which contain an AVR MCU, and especially on the Chinese clones which come with bootloaders.

I’ve used Optiboot and “minicore” on pro-mini boards and I give both an excellent rating. I may go this way with my larger boards too, since there is also a “mega core” variety available from MCUDude.

[elecdonia]

My end products will likely contain “bare metal” AVR devices, but I’m not starting out that way.

Then your project will not have an Arduino bootloader and you will have to re-do all your Arduino work. If you want to go bare metal, start bare metal. This will save you lots of time and effort.

By “bare metal” I am just referring to not using a factory-built pro-mini in my project (although using a pro-mini this way is a reasonable choice for one-off projects). In the past I’ve used the MCUDude cores, typically “minicore” along with Optiboot, by installing these directly onto AVR parts. I also put headers on my boards for connecting a USB to logic-level serial cable. This worked well for me. It provided bootloader access for firmware upgrades without needing a factory-built Arduino board.

[Buriedcode]

My end products will likely contain “bare metal” AVR devices, but I’m not starting out that way.

Then your project will not have an Arduino bootloader and you will have to re-do all your Arduino work. If you want to go bare metal, start bare metal. This will save you lots of time and effort.

By “bare metal” I am just referring to not having a factory-built pro-mini on my board (although using a pro-mini this way is a reasonable choice for one-off projects). In the past I’ve used the MCUDude cores, typically “minicore” along with Optiboot, by installing these directly onto AVR parts. I also put headers on my boards for connecting a USB to logic-level serial cable. This worked well for me. It provided bootloader access for firmware upgrades without needing a factory-built Arduino board.

I've started this for smaller projects, as a sort of half-measure. The convenience of a well supported bootloader system, without being completely reliant on specific hardware. I obviously don't trust this for "mission critical" work, but for my own projects and smaller commerical ones, it works fine.  Speeds up development a lot, and then it isn't much of a leap to drop the "Arduino" thing altogether.

[westfw]

Then your project will not have an Arduino bootloader and you will have to re-do all your Arduino work.

This is completely false.  The Arduino bootloaders are all completely separate and non-interacting with the user applications, whether they are "Arduino sketches" or programs written in some other development environment.  You can write Arduino sketches without using the bootloader (saves 0.5 to 8k of memory) (there's even an "upload using programmer" command), or you can use the Arduino bootloaders with non-Arduino applications.

The worst that is likely to happen is that for newer chips (ARM in particular), you may have to rebuild the application with a new segment start address for the vectors.  There are no run-time dependencies on the bootloader, and the bootloaders do their best to put the chip in a "just reset" state before starting the application.

[Peabody]

In your first example, in addition to selecting the Pro Mini in the IDE, did you also select the 8MHz version?  It just seems to me that if you were able to upload a sketch via the FTDI once, there's no reason why you couldn't do it again unless your IDE settings are different, or there's a bad connection.  Nothing would have changed the bootloader.

[NorthGuy]

You can write Arduino sketches without using the bootloader (saves 0.5 to 8k of memory) (there's even an "upload using programmer" command)

Really? I didn't know that.

[HwAoRrDk]

The only way an Arduino sketch could be dependent on the bootloader is if it called routines that were present in the bootloader's code. But AFAIK, the Arduino framework does not do that.

I seem to recall that there is some bootloader for AVRs that provides some of its routines as callable helpers for in-app flash programming, but I don't remember where I saw that or which one it was. But use of such would certainly be an explicit choice by the application developer.

[westfw]

I seem to recall that there is some bootloader for AVRs that provides some of its routines as callable helpers for in-app flash programming

The latest Optiboot has this feature, added by MCUDude..

And yes, if you want to write flash from your application, you have to have a function in the "bootloader section", because only the bootloader section is allowed to write to flash.

[elecdonia]

In your first example, in addition to selecting the Pro Mini in the IDE, did you also select the 8MHz version?  It just seems to me that if you were able to upload a sketch via the FTDI once, there's no reason why you couldn't do it again unless your IDE settings are different, or there's a bad connection.  Nothing would have changed the bootloader.

Yes, I'm familiar with this. I have plenty of pro-mini boards, both 8MHz and 16MHz. They use different baud rates depending on both the MHz and the type of bootloader installed on them. At this point I have MCUDdude "MiniCore" on all of them. There have been recent updates to both MiniCore and the Arduino IDE itself, so I'm in the process of confirming I have the most recent updates installed. I do need to install the latest Arduino IDE package (1.8.19) because I'm still using 1.8.9, which has worked perfectly until I got these new clone pro-mini boards. The PC is Windows 10.

What I find strange about these pro-mini clones is what happens immediately after using ICSP to erase flash and install a bootloader (this leaves low flash filled with 0xFF). At this point I can always load one sketch to the board with the bootloader. The sketch works fine, but afterwards the board fails to upload another sketch through its bootloader. Curiously the Arduino IDE “serial monitor” connection always works. Only failure is the bootloader stops working. I've been using the same FTDI USB-to-serial interface for everything. Perhaps it’s “reset out” pin has failed? I have several other USB-to-serial devices which I will try next. Sooner or later I will identify the "weak link."

One interesting discovery from today is that some of my 8MHz pro-mini boards (all Chinese clones) function perfectly when I use their 8MHz internal oscillator, but they fail to connect to the Arduino IDE when using their 8MHz resonator. I plan to activate the oscillator-out pin to measure actual clock frequency with my frequency counter. I'll use my ICSP programmer to change the clock fuses. I suspect the Chinese clone makers use the very cheapest resonators they can get. Most of my “Uno size” boards have actual crystals. But small boards (nano, pro-mini) have tiny resonators which may not be as accurate in frequency as crystals. I do have several 16MHz nano clones which always work perfectly even though their 16MHz comes from a resonator. I also have several SparkFun RedBoards (same form factor as UNO) with 16MHz resonators. They are totally reliable at all times, but then they aren't made out of the world's cheapest parts. Conclusion: Resonators aren't always "worse than crystals." 

[coromonadalix]

i had a sparkfun pro mini with a different bootloader they say,  you push the button and you have around 6 seconds to do something ?? 

witch i never got it to work,  put back an standard boot loader ...
and found out  about  some mega 328B vs 328P    they dont work the same, got refunded

[westfw]

What I find strange about these pro-mini clones is that if I use ICSP to erase flash and then install a bootloader, this leaves low flash is all empty (0xFF). At this point I can always load one sketch to the board through the bootloader.

There are two main reasons that this happens:

• No auto-reset circuit (or it's broken.)  A erased device with just the bootloader runs the bootloader and jumps to the "application", which is a string of near-nop 0xFFFF instructions.  Eventually (about 32k * 125ns), the FFFFs run out and the program counter loops around to the bootloader again.  So the bootloader runs over and over, ready to accept an upload.
• Fuses set incorrectly. If you install the bootloader manually (instead of using the "burn bootloader" Arduino tool), and don't set the fuses so that the chip knows it should start at the bootloader address in high memory, the chip with start running at 0x0, where there will be near-nop FFFFs that execute until the PC reaches the bootloader.  You get the same "bootloader runs over and over again" behavior, upside down.

In both cases, once you load a sketch, it's sitting there in what used to be a path of nops, and so it runs instead of eventually hitting the bootloader...

I don't recall whether the "pro mini" design(s) include the auto-reset feature.  I think the original pre-dated it, and the new "real" versions DO have it.  Clones ?   Who knows!

i had a sparkfun pro mini with a different bootloader they say,  you push the button and you have around 6 seconds to do something ??

That was the original atmegaboot behavior, back before they invented the auto-reset circuit.
One of the historical details I left out is that ATmegaBoot had the long timeout since it was designed to work with the manual reset, whereas Optiboot has an ~1s timeout, since it expects to operate with the auto-reset circuit.

[Peabody]

All the Pro Minis I've used have that auto reset circuit.  It has DTR tied to a series 0.1uF cap going to the reset pin of the processor, which also has a pullup resistor - 10K I believe.  That resets the processor, but only for a little while.  I would think the mostly likely cause of that not working is a bad DTR connection.

[elecdonia]

What I find strange about these pro-mini clones is that if I use ICSP to erase flash and then install a bootloader, this leaves low flash is all empty (0xFF). At this point I can always load one sketch to the board through the bootloader.

There are two main reasons that this happens:

• No auto-reset circuit (or it's broken.)  A erased device with just the bootloader runs the bootloader and jumps to the "application", which is a string of near-nop 0xFFFF instructions.  Eventually (about 32k * 125ns), the FFFFs run out and the program counter loops around to the bootloader again.  So the bootloader runs over and over, ready to accept an upload.
• Fuses set incorrectly. If you install the bootloader manually (instead of using the "burn bootloader" Arduino tool), and don't set the fuses so that the chip knows it should start at the bootloader address in high memory, the chip with start running at 0x0, where there will be near-nop FFFFs that execute until the PC reaches the bootloader.  You get the same "bootloader runs over and over again" behavior, upside down.

In both cases, once you load a sketch, it's sitting there in what used to be a path of nops, and so it runs instead of eventually hitting the bootloader...

OK. I will check both items: Auto reset circuit and fuse settings.

I am using the “burn bootloader” button in Arduino IDE to install the MiniCore bootloaders onto my pro-mini boards

I’m curious about bootloader baud rates: For boards operating at 8MHz the two “closest to ideal” baud rates appear to be 38.4k and 57.6k. When reading through the boards.txt file for MiniCore with ATmega328p, I discovered that it specifies 38.4k for “8MHz internal oscillator” but 57.6k for “8MHz crystal.”  I have some pro-mini boards where the bootloader is fully functional when using the 8MHz internal oscillator but fails with 8MHz crystal oscillator. Should 38.4k be used for both cases?

[elecdonia]

All the Pro Minis I've used have that auto reset circuit.  It has DTR tied to a series 0.1uF cap going to the reset pin of the processor, which also has a pullup resistor - 10K I believe.  That resets the processor, but only for a little while.  I would think the mostly likely cause of that not working is a bad DTR connection.

I suspect there is a potential issue with auto-reset when powering the target board with 5V.
The logic levels coming from most USB —> serial devices are 3V3.
However most USB-serial devices supply 5V power to the target board.

Here’s the issue: Their logic-high DTR output signal is still only 3V3.

If the MCU on the target board is receiving 5V power and the 10K pullup on the target board is also tied to 5V, then the reset pulse reaching the mega328P reset pin won’t drop all the way to 0V. It will drop only to about 1V7. This is too high to be considered an “officially valid” logic 0 level. The result could be that the MCU on the target board sometimes fails to reset.

This fault could be diagnosed by rewiring the USB-serial device so that the 5V power it provides connects to the “raw V” power input pin of the target board. Most target boards (pro-mini for example) have an onboard 3V3 regulator IC. Therefore running the 5V from the USB-serial device through the 3V3 regulator will ensure that the reset pulse drops all the way to 0V. All logic levels will be 3V3 in this case, and the MCU itself will also be operating at 3V3. This is OK for pro-mini boards operating at 8MHz.

I’ll test this.

[Peabody]

The FTDI USB to UART module that I use has a jumper to select 5V or 3.3V.  And that selection applies to the power going to the target device as well as Tx and DTR.  They're all 5V or all 3.3V.  I don't think the OP's Adafruit FTDI friend is as easily switchable, but there are solder blob selections on the bottom to accomplish the same thing.

But the levels issue wouldn't explain why flashing works once, but not again.

On my "3.3V" Pro Minis, I almost always select 5V on the module if the FTDI will be powering the Pro Mini during flashing.  The 328P works just fine at 5V, and flashing seems to go better at 5V (although there really should be no difference).

[elecdonia]

Problem solved: My USB-to-serial converters were not resetting my pro-mini boards.

This “failure-to-reset” is caused by insufficient voltage swing from the USB-serial converter DTR output pin: Its logic-high output voltage is <3.6V. It isn’t possible to raise this with a pull-up resistor because it is “driven” at both its high and low output voltage.

When the target device (pro-mini) is operating at 5V then the voltage reaching the MCU /reset pin fails to drop low enough to actually reset the MCU. I measure about 1.4V minimum at the lowest portion of the /reset pulse. However this needs to be <1V, preferably <500mV.

I will check my other USB-serial cables and breakout boards, but I don’t recall having any which actually generate full 5V logic-high levels at their outputs. All AVR devices can accommodate logic-high levels of 3.3V when powered at 5V. However, they do require the “logic-low” level to be <1V regardless of whether the power rail is 5V or 3.3V.

A quick fix for my pro-mini boards powered at 5V was to connect a 22k resistor from the /reset pin of the pro-mini board to ground. This reduces the logic-high voltage on the /reset pin from 5V to 3.5V. Therefore the existing series capacitor from DTR to /reset is able to generate a negative-going pulse which takes /reset all the way down to 0V.

This mod is not needed when the MCU on the pro-mini board is actually powered at 3.3V. However, the connectors on my USB-serial adaptors apply 5V directly to the MCU on my pro-mini boards. To use the internal 3.3V regulator on the pro-mini requires moving the 5V power source provided by the USB-serial device to the “V-ext” pin of the pro-mini. Doing this will change the power supply voltage for the MCU to 3.3V.

[elecdonia]

Is Optiboot the only popular Arduino IDE bootloader which “double-blinks” the onboard LED when it starts up after a reset?

[elecdonia]

The FTDI USB to UART module that I use has a jumper to select 5V or 3.3V.  And that selection applies to the power going to the target device as well as Tx and DTR.  They're all 5V or all 3.3V.

Could you please do us a favor and post the model# and photos of this USB-serial converter with switchable output voltages?
I want to get one.

[Peabody]

The FTDI USB to UART module that I use has a jumper to select 5V or 3.3V.  And that selection applies to the power going to the target device as well as Tx and DTR.  They're all 5V or all 3.3V.

Could you please do us a favor and post the model# and photos of this USB-serial converter with switchable output voltages?
I want to get one.

Sure.  Here's the one I use:

https://www.ebay.com/itm/295528212187

The yellow jumper switches everything between 3.3V and 5V.

But reading the Adafruit description of your FTDI "friend", it appears to have jumpers on the bottom that set the supply voltage and the I/O voltage.  You should set both the same voltage.  I would pick 5V.

Alternatively, you could not connect the Vcc line from the FTDI, and power the Pro Mini separately from a 3.3V supply.  Then the 3.3V DTR should work.

So the bootloader wasn't the problem at all.

[westfw]

Some of the Nano (and perhaps pro mini) clones/designs ended up with rather low-valued pull-up resistors on RESET (1k, rather than the 10k on an Uno.)  That might help explain your problem.

Is Optiboot the only popular Arduino IDE bootloader which “double-blinks” the onboard LED when it starts up after a reset?

I think they all blink the on-board LED.  If there is one.  Optiboot does a triple-blink, actually.  Number of blinks is a compile-time option.  There are LOTS of options:

Code: [Select]


optiboot>make help


 Option AVR_FREQ=<n>          - Clock rate of AVR CPU
 Option CUSTOM_VERSION=nn     - set a customer version number
 Option BAUD_RATE=nnnn        - set the bit rate for communications
 Option LED=B3                - set LED pin to particular port/bit
 Option LED_START_FLASHES=n   - set number of LED flashes when bootloader starts
 Option LED_DATA_FLASH=1      - flash the LED each time data is received.
 Option LED_START_ON=1        - Turn the LED on at bootload start
 Option BIGBOOT=1             - enable extra features up to 1kbytes
 Option SUPPORT_EEPROM=1      - Include code to read/write EEPROM
 Option SOFT_UART=1           - use a software (bit-banged) UART
 Option SOFTTX=B5             - pin for software UART xmit
 Option SOFTRX=B6             - pin for software UART receive
 Option SINGLESPEED=1         - do not use U2X mode on UART
 Option RS485=B0              - Pin for optional rs485 tx enable
 Option NO_APP_SPM=1          - disallow application call of do_spm
 Option OSCCAL_VALUE=nnn      - set set OSCCAL_VALUE in bootloader
 Option BOOT_ON_POR           - Run bootloader on power-on
 Option APP_ON_EXTR           - Run App on External Reset
 Option UART=n                - use UARTn for communications
 Option TIMEOUT=n             - set WDT to 1, 2, 4, or 8 seconds


[Peabody]

Problem solved: My USB-to-serial converters were not resetting my pro-mini boards.

This “failure-to-reset” is caused by insufficient voltage swing from the USB-serial converter DTR output pin: Its logic-high output voltage is <3.6V. It isn’t possible to raise this with a pull-up resistor because it is “driven” at both its high and low output voltage.

When the target device (pro-mini) is operating at 5V then the voltage reaching the MCU /reset pin fails to drop low enough to actually reset the MCU. I measure about 1.4V minimum at the lowest portion of the /reset pulse. However this needs to be <1V, preferably <500mV.

It's the series capacitor that gives this result.  When idle, the DTR side is at 3.6V, and the reset side is at 5V.  When DTR goes low, both sides of the cap instantaneously drop by 3.6V - that's the voltage drop on the DTR side.  So the reset side also drops by 3.6V, bringing the reset pin down to 1.4V, which isn't low enough to trigger the needed reset.   Both sides need to be at the same voltage, either 3.3V or 5V, so when DTR goes to ground, so does reset.  If the USB-to-UART adapter is providing 5V to Vcc, but only 3.3V on DTR, you could power the Arduino from some other 3.3V source.  But ideally the adapter should provide power and I/O at the same voltage.

[elecdonia]

Some of the Nano (and perhaps pro mini) clones/designs ended up with rather low-valued pull-up resistors on RESET (1k, rather than the 10k on an Uno.)  That might help explain your problem.

I’ll check that. I have plenty of boards ranging from brand new to 5 years old. So far I’ve seen only 10k pullup resisters on \reset.

I think they all blink the on-board LED.

This clue helped me identify my “target board fails to reset” issue. I saw no blinks at all, except after power-up or hitting reset button. However, the LED should blink each time Arduino IDE opens the USB interface port, for example when Arduino IDE starts it’s “upload sketch to target board” process. LED also blinks when Arduino IDE “serial monitor” starts up.

Optiboot does a triple-blink, actually.  Number of blinks is a compile-time option.  There are LOTS of options:

Yep, plenty of items can be customized. I’m using MCUDude MiniCore (in my opinion quite well documented and easy for me to understand). Now that I’ve beaten my hardware into submission I’ll be moving forward with re-learning how all of this works. I did a bunch of projects with “embedded Arduino” structure 4 years ago but my skills got a bit rusty. I’m rapidly getting back on track.