Newbie questions about writing apps for microcontrollers

[Winfried]

Hello,

I only have a shallow experience with embedded software, and have a couple of questions, especially regarding writing software for ESP and Arduino processors:

1; What are the alternatives to run code on those processors?

• Arduino
• Adafruit CircuitPython
• esp-idf (= CircuitPython?)
• other?

2. What is the advantage of using the Arduino IDE (ie. C) instead of eg. CircuitPython? Smaller + faster code?

Thank you.

[Ian.M]

ESP and Arduino aren't processors, they are names of classes of stuff, in the case of 'Arduino', of 'maker' friendly MCU boards with many many different MCUs ('processors') - I count ten in Wikipedia's list of official boards [here], and in the case of 'ESP' its the prefix Espressif use for all their MCU part numbers, but ESP-IDF only applies to their ESP32, ESP32-S and ESP32-C families of SoCs (also 'processors').

The IDE you use has nothing to do with the programming language you use, except that the IDE has to support the language.

[snark]
Most of us would say the Arduino IDE is a disadvantage, not an advantage!  The only things to be said in its favor is it is a convenient way for wannabee programmers who are 'hard of thinking' to get a GCC cross-compiler toolchain + bootloader utility + USB serial device drivers installed on a Windows PC or Mac, (i.e. where you cant just do: e.g.  sudo apt-get install -y gcc-avr to install the AVR GCC cross-compiler for Atmel AVR MCUs), and its a convenient source of low-quality peripheral libraries, for when the effort of 'doing the job right' cant be justified for a one-off, you aren't pushing the performance envelope, and you *REALLY* don't care about efficiency or future maintainability. 
[/snark]

[pcprogrammer]

Well the title of the thread gives it away. "Writing apps for microcontrollers" show that you have to do some more studying.

Writing software for microcontrollers is a very different ballgame then writing apps for Android or whatever OS based system. But that is the trend nowadays, we call all software "apps".

To be a bit pedantic here (as if I was not already  ), the Arduino IDE works with C++ which is also different from writing plain C.

As Ian.M already pointed out, the Arduino IDE is nice for some rapid testing of an idea or some dev board you bought on Aliexpress or Ebay, but not for real embedded programming.

Also the Phyton based systems, sure they are fun to play with and get something working quick, but if you need some performance the way to go is C. Others will come out of the woodworks and start yelling C++ is what you need, and others might come up with other "high" level languages, but trust me, everything you want can be done with plain old C.

Now onto your question. There are basically three targets to consider here.
a) AVR based MCU's
b) ARM based MCU's
c) ESP32 based MCU's from Espressif. (Based on different architectures)

Sure there are more, but these are the main ones.

For AVR Ian.M already pointed to the GNU gcc-avr tools.
For ARM there are the GNU gcc-arm tools. (There are some varieties, google for them)
The ESP32 tools for the Espressif line of MCU's  I'm not familiar with yet but I'm sure they are out there.

Edit: to reflect the comments made by brucehoult, I changed the bits about ESP32.

[westfw]

the Arduino IDE is nice for some rapid testing of an idea or some dev board you bought on Aliexpress or Ebay, but not for real embedded programming.

Nonsense.  Or rather, only true for a narrow definition of "real embedded programming" that many applications don't require.

What is the advantage of using the Arduino IDE (ie. C) instead of eg. CircuitPython?

Much wider choice of hardware.  Using Arduino will require you to be become somewhat conversant with C, which can be used on an even broader range of hardware.  (whereas CircuitPyton or MicroPython will only run on pretty "substantial" hardware.  It may not matter, either.  "More substantial hardware" isn't necessarily more expensive or difficult to use than simpler hardware, given the help of the tools available.

The main thing you left out is Vendor-supplied IDEs and toolkits.  AVRs and SAMD chips have "Microchip Studio and ASF or Atmel Start", or "MPLabX and associated tools."  ESP has a C-based SDK that can be used without either Arduino or Python.  This way you get a whole different (and less useful?) set of poorly-written peripheral libraries.
("Less useful" in the sense that I can probably go out and find an Arduino library for an XYZ123 9dof inertial navigation chip (and half-a-dozen tutorials on how to use it), while the "vendor libraries" probably don't go much beyond "using the I2C peripheral in the micro." - Poorly-written library code that does what you want is generally better than non-existent code...

[brucehoult]

To be a bit pedantic here (as if I was not already  ), the Arduino IDE works with C++ which is also different from writing plain C.

If you want to be pedantic, the Arduino IDE works with a language called "Wiring" that is a compatible subset of both C++ and Java.

If you are working with a board that is programmed in C++ then the IDE prefixes your code with some #include and also generates prototypes for all your functions so that (unlike in C++, but like in Java) the order in which you write functions in the source code does not matter.

If you are working with a board that is programmed in Java then the IDE wraps your code with some Java class bollocks and puts some imports at the top.

As embedded Java has lost popularity since the mid-2000s a lot of Arduino code has started using C++-specific features, but you won't find that in the standard example code.

As Ian.M already pointed out, the Arduino IDE is nice for some rapid testing of an idea or some dev board you bought on Aliexpress or Ebay, but not for real embedded programming.

Rubbish. A professional can do "real programming" using whatever tools are available. Arduino IDE wraps exactly the same compilers and download tools that you might otherwise use "by hand". It provides libraries, but you don't have to use them and you can do chip-specific register programming if you want.

Now onto your question. There are basically three targets to consider here.
a) AVR based MCU's
b) ARM based MCU's
c) ESP32 based MCU's

Sure there are more, but these are the main ones.

There is no such thing as "ESP32 based MCUs". Older ESP chips used Cadence's Xtensa ISA. Newer ones use (and Espressif have said all future products will use) the RISC-V ISA.

Being snarky and pedantic is a bad look at the best of times, but it's even worse when you don't actually know what you're talking about.

[ledtester]

2. What is the advantage of using the Arduino IDE (ie. C) instead of eg. CircuitPython? Smaller + faster code?

The main advantage is faster development time -- if you can accomplish your task in those environments. You code will, in general, not be smaller nor faster, but it might be fast enough for your use case.

There is nothing wrong with using the Arduino platform or Circuit Python. However, modern microcontrollers have a lot of powerful functionality that these platforms do not give you access to. To use this functionality you'll need to use a C development environment tailored for the device.

Another aspect to consider is that some development environments have versatile debugging capabilities which can be very helpful when developing complex applications. The videos in this playlist, for instance, can give you an idea of what progamming and debugging is like using the STM32CubeIDE:

https://www.youtube.com/playlist?list=PL7tUZeMaichqrlJN4PGu3-n6DbYrvoG-s

(ignore the raccoon video at the end)

[brucehoult]

2. What is the advantage of using the Arduino IDE (ie. C) instead of eg. CircuitPython? Smaller + faster code?

The main advantage is faster development time -- if you can accomplish your task in those environments. You code will, in general, not be smaller nor faster, but it might be fast enough for your use case.

There are many many things you might want to do with a microcontroller that only require interacting with the outside world -- taking an ADC reading, or toggling an output pin -- a thousand times a second. Or 100, or 10, or once a minute, for that matter.

For anything working at the millisecond level or slower, a 16 MHz AVR using the Arduino libraries is JUST FINE. A millisecond is long enough to run 16000 instructions. It's long enough to do 200 floating point operations (at least add, subtract, multiply) using the software FP library.

If you need to work on shorter timescales than that, you can splash out $20 for a Teensy 4.0 with a 600 MHz Cortex M7 CPU on which you can still use the Arduino libraries -- just much much faster. Hell, it's fast enough that you could use Python if you wanted.


If you're designing a product for sale and it's going to be produced in the millions or billions of units then you'll want to use the slowest cheapest microcontroller you can get away with, and program it as efficiently as possible.

If you're doing a personal project and only building one copy (or ten, for that matter), then there is absolutely nothing wrong with spending a bit more on the hardware (like $20, see above) and using the development environment that will minimise how much of your -- far more valuable -- time is needed to create the thing you want.

[westfw]

the Arduino IDE works with a language called "Wiring" that is a compatible subset of both C++ and Java.

If you are working with a board that is programmed in Java then the IDE wraps your code with some Java class bollocks and puts some imports at the top.

I'm pretty sure that the Arduino IDE has never supported Java, nor has any "Arduino" board ever supported embedded Java.  The original thesis project investigated a Java based board, but discarded it very early because "no open source", and it never made it into the Arduino IDE.

"Processing" was an IDE for simplified Java programming (as you describe), and the Arduino IDE essentially used most of that IDE code, but it's target was always the PC-side, not embedded.  (Open Source can make for convoluted code histories.)

https://arduinohistory.github.io/

[pcprogrammer]

There is no such thing as "ESP32 based MCUs". Older ESP chips used Cadence's Xtensa ISA. Newer ones use (and Espressif have said all future products will use) the RISC-V ISA.

Being snarky and pedantic is a bad look at the best of times, but it's even worse when you don't actually know what you're talking about.

Being picky on specific parts of someones post that also contains good parts is just the same.

About the ESP32 I realize you are right. In hindsight I remember that Espressif used different architectures. As I pointed out at the end of my post I do not yet know about the ESP(32) parts yet. Just recently acquired some dev boards with ESP32 modules and have not played with them yet.

The main advantage is faster development time -- if you can accomplish your task in those environments. You code will, in general, not be smaller nor faster, but it might be fast enough for your use case.

There is nothing wrong with using the Arduino platform or Circuit Python. However, modern microcontrollers have a lot of powerful functionality that these platforms do not give you access to. To use this functionality you'll need to use a C development environment tailored for the device.

Another aspect to consider is that some development environments have versatile debugging capabilities which can be very helpful when developing complex applications. The videos in this playlist, for instance, can give you an idea of what programming and debugging is like using the STM32CubeIDE:

There are IDE's that allow you to use whatever compiler you need and make it possible to work on different architectures from the same trusted IDE. A bit like the Arduino IDE, but with much more nice features to make programming easier. Like visual studio, netbeans or eclipse. They allow debugging with a JTAG or SWD debugger.

This debugging is also available in the new Arduino IDE which shows they are working hard to make it better, but I personaly still would only use it for rapid testing.

Having all the best tools in the world does not make you a good or even fast programmer. Understanding what you are doing and the ability to solve problems when they occur are the tools that can make you a good programmer.

[MarkR42]

Answering your subject:

Microcontrollers do not run "apps" - they run firmware. Microcontrollers run a single application which is generally made by the hardware manufacturer (excluding things like dev boards) - and interfaces with various hardware.

Microcontrollers either do not have an operating system, or have a very specific one (RTOS etc) which only runs a single application, and is part of the firmware.

---

Some advantages of using C instead of Circuitpython (etc)
* You are more likely to find vendor examples in C (e.g. for interfacing with I2C, SPI, sensors, other chips)
* Embedded developers will usually be more familiar with C
* C programs generally use less memory (RAM / Flash) - which might be helpful as microcontrollers often have very little
* It is a simpler environment to set up on most microcontrollers - e.g. if it doesn't already have a "prebuilt" Circuitpython (like Pi Pico)
* It is likely to be easier to use third party libraries in C as most will be C

But the advantages of using Python
* Some things are much easier and less error-prone e.g. string handling
* Your source code is likely to be smaller and may be easier to understand (=fewer bugs?)
* You can transfer skills if you already know "real" Python

Runtime performance might be an issue but it is impossible to know in advance.

[SiliconWizard]

the Arduino IDE works with a language called "Wiring" that is a compatible subset of both C++ and Java.

If you are working with a board that is programmed in Java then the IDE wraps your code with some Java class bollocks and puts some imports at the top.

I'm pretty sure that the Arduino IDE has never supported Java, nor has any "Arduino" board ever supported embedded Java.

The history can be found here: https://en.wikipedia.org/wiki/Arduino
The very early history comes with "Wiring" and there was some Java compatibility there, although I don't know the full details.
But certainly the Arduino framework hasn't dealt with Java in ages now. (Other than the fact the IDE itself was written in Java.)

The "language" is described on the official Arduino web site. I have to say that this language "report" is very, uh, unimpressive for a language report.

The language itself uses a subset of C++ (avoiding any construct that would use heap memory allocation AFAIK) supporting almost none of the standard C++ libraries. It also uses some specific preprocessing as Bruce said, to "dumb down" some aspects such as the need for prototypes. But the description of the "language" looks very "shallow" to me, and good luck figuring out exactly what parts of C++ are usable or not. To add to this, these days, Arduino is supported by many different targets, and thus different compilers (most commonly either avr-gcc for the AVR targets or gcc for ARM targets), themselves supporting different revisions of C++. A nice mess.

But contrary to xxPython, which is interpreted, Arduino stuff is compiled, again using a small subset of C++, so execution-wise, that'll be uh... comparing an interpreted language with a compiled one.

[Winfried]

Thanks much everyone for the infos.

[westfw]

The language itself uses a subset of C++ (avoiding any construct that would use heap memory allocation AFAIK)

There is some disagreement as to whether "Arduino" is a "language" or not.  Arduino (the company) says "Yes, it's the Arduino Language."  Most experienced programmers say "the pre-processed program goes into a gcc C++ compiler, so you have the language features of C++, and can/should refer to C++ (and C) documentation for details (or if you want to understand things that the arduino reference doesn't document, like the ternary or comma operators.)  The degree to which C++ libraries are supported depends on when CPU you're using (and how much memory there is.  And there isn't a multi-user OS involved, so some things will never be like they are in a desktop C++ environment."

Even the AVR Arduinos use the heap to support their "String" type (which is not the same as C++ strings, and doesn't work very well, largely due to heap fragmentation.)  A bunch of Arduino libraries use the heap.

The very early history comes with "Wiring" and there was some Java compatibility there

Do you have any reference for that "Wiring had some Java compatibility" part?  It doesn't say so on the WP page you linked, and it's not mentioned at https://wiring.org.co or the  "Hernando Barragán" documet I linked...

[eugene]

Hello,

I only have a shallow experience with embedded software, and have a couple of questions, especially regarding writing software for ESP and Arduino processors:

1; What are the alternatives to run code on those processors?

• Arduino
• Adafruit CircuitPython
• esp-idf (= CircuitPython?)
• other?

2. What is the advantage of using the Arduino IDE (ie. C) instead of eg. CircuitPython? Smaller + faster code?

Thank you.

Most of those choices come down to either personal preference or suitability to a certain application. Being a newbie you have not developed a sense of personal preference. For that matter, you probably don't really know what applications will become interesting to you in the future.

So my advice is to try both (Arduino and CircuitPython.) Go to Adafruit and pick out one of their Feathers that will run CircuitPython or be programmed with the Arduino IDE. It doesn't need to be the fanciest model available. In fact, I suggest that you start out with the attitude that you're going to buy more hardware anyway, so expect that whatever you choose today will be replaced/augmented with something else in a month or two. That's all part of the journey.

When it comes to grey haired professional programmers grousing about Arduino or Python, it's safe to ignore them, at least for now. The Arduino IDE is insanely popular for a reason. It has made it possible for an uncountable number of newbies to get up and running with a minimum of pain and a maximum of fun. That describes you right now. When you have 20 years of professional full-time embedded coding under your belt, then you can complain about the Arduino IDE. Right now, you want it to be easy and fun. You can switch over to VS Code and Platformio when you're ready. Then thumb your nose at the people still using Arduino.

I have zero experience with CircuitPython, but it seems to be well supported by Adafruit. Again, you'll hear the grumpy professionals complain about them, but they're not the customer base that Adafruit is trying to please anyway. It's an extremely supportive community of makers. And if you buy one of their Feathers, you can try CircuitPython, Arduino IDE, and who knows what else. See for yourself what makes sense to you.

[tooki]

I think it’s really important to do something nobody here has done so far, and that’s to separate the Arduino IDE (the desktop software) from the Arduino framework (the language and libraries), because they aren’t the indivisible monolith many people think they are.

In fact, one of the most common IDEs used for programming with the Arduino framework is PlatformIO in Visual Studio Code. This is particularly popular among advanced users, because the two Arduino IDEs (the original Java-based “Arduino 1.x” and the new, currently in beta “Arduino IDE” 2.0) are both just too dumbed-down for advanced use. (2.0 adds a lot, but it’s still nowhere near PlatformIO in VS Code.) There are other alternatives, too, like VisualMicro (a commercial Arduino extension for the “big” Visual Studio), the Arduino extension for VS Code, and emCode, another extension for VS Code (and the spiritual successor to embedXcode, the same author’s now discontinued extension for Xcode).

In PlatformIO, when you create a new project using the Arduino framework, it literally creates a C++ file, with the Arduino.h header already included. The only change from regular .ino files is needing to create function prototypes yourself. But you can also rename your .cpp file to .ino, and then you don’t need to add the header or prototypes. (IIRC, code auto-completion works better when set to .cpp.)

Other than features to speed up text editing (autocomplete, etc) thanks to VS Code’s infinitely better editor and autocomplete engine, PlatformIO also supports hardware debugging on a wide array of MCUs, including boards not supported by Arduino IDE 2.0 (which itself only supports debugging on a small handful of boards). The official ESP32 debugger is just $11 or so, while there is a wide array of inexpensive STM32 boards (Nucleo and Discovery) with built-in ST-Link debuggers, which perform better than the ESP32 debugger. The last major advantage of PlatformIO is per-project, versioned library management, rather than Arduino’s global libraries. No more breaking one project when you update another project’s libraries!

PlatformIO also supports other frameworks, including Mbed, ESP-IDF, STM32 CMSIS, and STM32 Cube.

[peter-h]

The main decision is whether the application is a hobby or something commercial.

That will lead down mostly very different avenues. For a hobby job you can do anything that works. For a real product there are many considerations.

[DiTBho]

the Arduino IDE is nice for some rapid testing of an idea or some dev board you bought on Aliexpress or Ebay, but not for real embedded programming.

Nonsense.  Or rather, only true for a narrow definition of "real embedded programming" that many applications don't require.

Massimo Banzi was serious when he said "with Arduino you have the power at your fingertips ... you can even use it to create an interface for a spaceship"

It must be a very serious business, like Rat-man 

[SiliconWizard]

That's some serious shit!

[westfw]

What a lot of people tend to forget is that MANY "apps for microcontrollers" are NOT "complex projects."  You can write them "in Arduino", you can use circuit- or micro- Python; you could probably use a BASIC Stamp running interpreted bytecodes at ~1000 "instructions" per second, or Basic-52 entered on an ASR-33.  Or one of those weird languages like Forth, JAL, or even assembly language.

Stop giving me these "Arduino's IDE is unsuitable for advanced use" comments.  Especially if you can't demonstrate that your project is "advanced."  People have done more, with less, for 60 years...

(AND ... that's how you become an "advanced programmer."  By writing a lot of NOT advanced programs!)

[Ian.M]

A case in point is the Marlin firmware for 3D printers - v1.1.9 is a *very* large Arduino project, which can actually be run on an Arduino Mega2560.  However the .ino file of the sketch is essentially empty (just comments) as all the code is in the associated .cpp and .h source files to avoid the Arduino IDE's tendency to mung perfectly valid C++ code when it injects forward declarations, and its a real PITA to work on under the Arduino IDE, chasing down declarations and references in its multitude of source files, so many that the IDE cant display all the tabs in the usual way and you have to jump around from a massive scrolling dropdown list of files at the top right of the tab bar.

Its telling that Marlin 2.x uses PlatformIO as its preferred development environment/IDE.

[Bassman59]

A case in point is the Marlin firmware for 3D printers - v1.1.9 is a *very* large Arduino project, which can actually be run on an Arduino Mega2560.  However the .ino file of the sketch is essentially empty (just comments) as all the code is in the associated .cpp and .h source files ...

It is that exact .ino thing that makes me hate Arduino. Where is the actual fucking code?

[brucehoult]

I think it’s really important to do something nobody here has done so far, and that’s to separate the Arduino IDE (the desktop software) from the Arduino framework (the language and libraries), because they aren’t the indivisible monolith many people think they are.

I don't recall whether it is in this particular thread or not, but this is a point I and others have made many many times on this board.

In fact, one of the most common IDEs used for programming with the Arduino framework is PlatformIO in Visual Studio Code. This is particularly popular among advanced users, because the two Arduino IDEs (the original Java-based “Arduino 1.x” and the new, currently in beta “Arduino IDE” 2.0) are both just too dumbed-down for advanced use. (2.0 adds a lot, but it’s still nowhere near PlatformIO in VS Code.)

Whet few people seem to remember now is that PlatformIO used to cost $10/month (?) if you wanted to use the debugger and some other features, until in mid 2019 RISC-V companies SiFive and Western Digital stepped in and sponsored PlatformIO to make the product completely free to use.

https://www.westerndigital.com/en-au/company/newsroom/press-releases/2019/2019-06-18-western-digital-extends-openness-of-platformio-and-enhances-its-risc-v-portfolio

There was also the slightly bizarre (but welcome) sight a little earlier in December 2018 of WD CTO Martin Fink narrating a series of videos on how to get started programming SiFive's HiFive1 in assembly language using VSCode and PlatformIO:

https://www.youtube.com/playlist?list=PL6noQ0vZDAdh_aGvqKvxd0brXImHXMuLY

[tooki]

What a lot of people tend to forget is that MANY "apps for microcontrollers" are NOT "complex projects."  You can write them "in Arduino", you can use circuit- or micro- Python; you could probably use a BASIC Stamp running interpreted bytecodes at ~1000 "instructions" per second, or Basic-52 entered on an ASR-33.  Or one of those weird languages like Forth, JAL, or even assembly language.

Stop giving me these "Arduino's IDE is unsuitable for advanced use" comments.  Especially if you can't demonstrate that your project is "advanced."  People have done more, with less, for 60 years...

(AND ... that's how you become an "advanced programmer."  By writing a lot of NOT advanced programs!)

Again, one must distinguish between the people who think using the Arduino framework is bad (utter nonsense IMHO) and those who think the Arduino IDE (the desktop apps) is bad.

In the first case, I think that’s nonsense, since it’s just libraries that simplify some things. It doesn’t block you from digging deeper if (and when) you need to. And if anything, Arduino has become something quite rare in the MCU world: a fairly portable framework. The vast majority of Arduino code is portable across hardware architectures. This is proving to be a huge advantage right now, with the widespread component shortages.

In the second case: can you code complex things in it? Sure. But why torture yourself unnecessarily?

[brucehoult]

Again, one must distinguish between the people who think using the Arduino framework is bad (utter nonsense IMHO) and those who think the Arduino IDE (the desktop apps) is bad.

In the first case, I think that’s nonsense, since it’s just libraries that simplify some things.

Some people complain that things such as digitalWrite() take perhaps a µs or two longer than direct port writes because it has to translate a logical pin number to a physical pin number, and then convert that to a port address and mask.

Which is true enough, but totally irrelevant to the vast majority of users, who are doing things on a ms or longer time scale.

It might be interesting if there was a getPinPortAddress() and getPinMask() that could be called at startup and the values saved. But, having to load those those values from global variables each time instead of having them as hard-coded constants still wouldn't satisfy to moaners so ... whatever ...

It doesn’t block you from digging deeper if (and when) you need to. And if anything, Arduino has become something quite rare in the MCU world: a fairly portable framework. The vast majority of Arduino code is portable across hardware architectures.

Absolutely correct.

[pcprogrammer]

(AND ... that's how you become an "advanced programmer."  By writing a lot of NOT advanced programs!)

I beg to differ. You can write a lot of not advanced programs, but when these programs are basically stitching together bits and pieces of other peoples work and you never have to dig in deep to solve a bug you won't become an "advanced programmer".

To me an "advanced programmer" knows how the hardware works to some extend, and is able to come up with solutions to solve problems when they arise. And is also able to write something completely from scratch with just plain C or assembler. No libraries what so ever. Just your own code and understanding of what you are doing.

So to become an "advanced programmer" more is needed then just write a lot of code. And that is why I advise against using Arduino Framework if you really want to learn how to program. Start with the basics.

About the Arduino stuff, I did not know it was possible to use the framework in other IDE's, but then still I prefer to do completely bare metal development and write my own code. Sure, until you write up a solid code base, it is a lot of work, but when there is a bug it is your own and most of the time much easier to find then wading through other peoples code.

I sometimes use code from other people to save time, but end up formatting it to my own preference anyway.

And how did I learn programming. It started with basic on a TRS-80, but soon evolved to Z80 assembler and during my working live it went from 6502, 8051, 6800 etc assembler onto C/C++/C# and things like ASP, PHP, HTML, basic script, javascript, sql, etc. Programming is not in a language it is a skill.