Is my MCU enviroment big picture understanding right?

[[email protected]]

Hello,

Was not sure whether to post here or Begginers section, so sorry in advance, I am well well behind most people on this forum on most things 

My current understanding:
The MCU manufacture takes a CPU core (or several) and bundles it together with useful peripherals. the peripheral's controls and own registers are "mapped to memory" by the manufacturer and recorded in the device datasheet, they can be controlled by register access made easier with C code register access libraries, routine tasks are implemented in Hal libraries for further abstraction.  the registers appear all the same to the CPU which does not "Know" about the peripherals but these registers are not a physical extension of the memory that the peripherals also read, they are registers and states built into each peripheral, the CPU directly communicates with the peripherals over the Bus/Bridge (acting as a translation layer) obeying wait and confirmation commands (important for slow peripherals). their are placeholder error handling routines for these transactions and error flags that might get raised in the HAL libraries but generally they don’t do much or anything so for long term stability the application code should address error handling and recovery of the peripherals; the CPU core language is defined by the ISA by the CPU designer (A revelation to me that assembly is an umbrella term for all the different instruction sets (or languages) from 8051 to X86, and not a standard its self)  and the CPU does not need to know about the peripherals, so the Compiler is mostly the concern of the CPU IP holder such as ARM and not the MCU manufacturer. Because the compiler, like the CPU, does not know about the peripherals and and the FLASH, RAM ect.. are also peripherals, the compiler needs a "linker file" so it knows where to put stuff;  the language that the compiler understands is defined by the C programming language standard.  So a very determined person who knows C could write the code and program an MCU without any of the IDEs or libraries, as long as they had the compiler and the device datasheet.  It would just be slow inefficient
There is also the start up code that the IDE config GUIs help generate to create boot up routines and initialise clocks/ peripherals correctly depending on use case.

Is that an accurate picture of how these different bits fit together? This is obviously very zoomed out and there are a million and one rabbit holes within view but does it catch the lay of the land?

My angle if you are interested: I am machanical/mechatronics engineer so my formal knowledge of this stuff is thin on the ground but have tinkered a fair amount, plenty of projects using the arduino IDE. Finally making the move out of that sheltered garden for DPS reasons, namely to use TMS320F280034 after discovering one needs HRPWM for battery charging control loops, but that is not the topic of this post  . Anyway I started trying to understand the full picture of whats going on after floundering on the extra complexity when opening TI's CCS, namely the file structure that was previously obscured. I personally find it hard to work on something when I don't know how the bigger pieces fit together and why things are as they are.  So Being short on time and a slow reader I have not picked up a tome and started ploughing through, for better or worse I started searching online, and don't shoot me, chatting to GPT because a lot of this information is in an odd space that the average tinkerer does not come across it, or the information is for a specific MCU and the formally trained know it well enough they dont talk about it SO.  I was hoping I could attempt to paint the bigger picture as I think I understand it and someone on here could tear it to shreads and tell me to go read a book. Or not, just leave me to my AI Hallucinations.  Also a kind of interesting test for the state of AI

Thanks for anyone who takes the time to read this

Doug

[brucehoult]

Looks right other than that startup code doesn't have to be written or customised by an IDE. You don't need an IDE at all, and you can use generic hand-written (including modified by you) startup code. Or at most call a few defined functions to initialise the hardware you want to use, after some very brief assembly language code to get the chip running in the most basic way.

I'd suggest looking at a small and simple MCU such as the popular WCH CH32V003 or one of it's siblings. And look at Charles Lohr's CH32Fun (https://github.com/cnlohr/ch32fun) for startup code, header files for peripherals, and a ton of examples. It's even got the compiler and flashing tool right there, all in one non-IDE setup.

[[email protected]]

Thanks, that is good to know. I was planning on generating the start code with the GUI without any peripherals enabled,then disecting what it does, then adding in peripherals one by one to see the changes but as you say the F280034 is quite weighty so CH32 would be a better starting point.

From your experience is it better to dig into the start code/write your own or trust the IDE's outputs? Probably very case dependent

CH32Fun, Very cool project, setting it up now!

Thanks

[asmi]

Thanks, that is good to know. I was planning on generating the start code with the GUI without any peripherals enabled,then disecting what it does, then adding in peripherals one by one to see the changes but as you say the F280034 is quite weighty so CH32 would be a better starting point.

From your experience is it better to dig into the start code/write your own or trust the IDE's outputs? Probably very case dependent

CH32Fun, Very cool project, setting it up now!

Thanks

They very beginning of a startup code is typically a standard boilerplate for each MCU type, which sets up some very basic things like stack, and transfers control to a C function, which picks up a torch from there and does the rest of initialization (setting up PLL, configuring peripherals), and calls user code at some point.

[cfbsoftware]

My current understanding:
Because the compiler, like the CPU, does not know about the peripherals and and the FLASH, RAM ect.. are also peripherals, the compiler needs a "linker file" so it knows where to put stuff;  the language that the compiler understands is defined by the C programming language standard.  So a very determined person who knows C could write the code and program an MCU without any of the IDEs or libraries, as long as they had the compiler and the device datasheet.  It would just be slow inefficient
There is also the start up code that the IDE config GUIs help generate to create boot up routines and initialise clocks/ peripherals correctly depending on use case.
My angle if you are interested:I was hoping I could attempt to paint the bigger picture as I think I understand it and someone on here could tear it to shreads and tell me to go read a book. Or not, just leave me to my AI Hallucinations.  Also a kind of interesting test for the state of AI

We were in the position you were at when we started embedded software development 20 years ago. However, C was not an option having had 10 years of negative experience with it in the 1980s / 1990s. We decided to use Oberon as our programming language of choice. It evolved in the late 1990's from Modula-2 which has been around almost as long as C and is at a similar conceptual level as C. However, it is not as easy to shoot yourself in the foot. To cut a long story short, the Astrobe IDE is the end result. However, again having had negative experience with code-generating 'wizards' - great for creating code from scratch but a nightmare when you need to modify it, the IDE just looks after compilation and linking the soure files - that you have complete control over. The 'Build' command is the smartest part - it knows which files to recompile and link without resorting to separate 'Make' files - the design of the language makes them unnecessary. We currently support Arm Cortex-M0, M3, M4 and M7 microcontrollers from STM and FPGA devices from Cologne Chip, but our users have targeted microcontrollers from other manufacturers without needing any assistance from us. 

Switching to another MCU - even from the same manufacturer, has been a tedious task in the past - usually taking several days. However, now, this is the sort of mechanical translation that AI is good at. The end result is source code that looks just like the source code supplied with Astrobe. Just the details are different.

[Dave]

Yup, your understanding is correct. The CPU is just a dumb automaton that follows the sequence of instructions that you've written for it. Registers, flash, or RAM locations are pretty much all just another number to it that needs to be crunched one way or another.

Assembly instruction sets are different for different cores, but the basics are usually going to be largely the same. Once you learn to use one, it becomes fairly easy to expand to a different architecture.

So a very determined person who knows C could write the code and program an MCU without any of the IDEs or libraries, as long as they had the compiler and the device datasheet.  It would just be slow inefficient

This is not the case.
While getting off the start line might be quicker with code auto-generated by the configuration tools in the SDK, putting the peripherals to good use is usually going to involve thoroughly studying the reference manual. At that point it makes more sense (at least to me personally) to write your own hardware abstraction layer. I will support only the functionality that I will need in my own application and nothing more. Keeps things much more streamlined, without bloated code and awkward interface functions. You can be much faster when you're not fighting the code that is supposed to abstract the complexity of the underlying system away from you.

[julian1]

Many devs prefer a simplified dev setup that side-steps specific IDE requirements/ and vendor ecosystems.
It is also common to use a lightweight shim library to the hardware, to replace template code generation wizards.

A platform neutral environment with minimal dependencies helps when sharing projects - by giving other devs freedom to choose for themselves what dev-environment they want.
Also a minimal build configuration is good for automated build and test pipelines etc, that may run on remote/headless servers.

[pqass]

See this project for a simple cross-compiler toolchain setup.
Read the README - at least up to "Getting Started" section to understand its purpose.
Although it's for the 68K CPU, it can be a framework for any CPU given that you've got a C cross-compiler for it.
To boot a CPU/MCU into main() requires just four files: Makefile, platform.ld (linker script), crt0.S (C runtime startup), and your custom main.c

As a concrete example, I've provided a zip attachment of the four files plus two additional UART HAL files.  It's for a board with a 68008 CPU, 128K ROM, 32K RAM, and 16550 UART.

The 68008 looks (to software) like a 32bit CPU capable of 32bit integer operations and 32bit addressing except that it only has a (hardware) 8bit data bus and 20bit address bus (1MB).  Address decoding uses just one 74LS138; where each output pin (of 8 ) divides the 1MB address space into 128KB chunks where only 3 pins are used to enable ROM, RAM, or UART at a time. The memory map is: 0x00000:ROM, 0x20000:RAM, 0xA0000:UART    The 68008 doesn't have a separate IO address space like x86 (which uses IN/OUT instructions to access it).  Instead, all IO is memory-mapped.  In this case the UART appears as a memory at 0xA0000 to 0xA0007 (registers internal to the 16550).

By design, the 68K CPU has the vector table in the first 1KB of address space; there are 256 32bit addresses which can be invoked when events occur eg. on reset, hw interrupts, div by 0, etc.  Notice the linker script initializes the vector table at address 0 with the stack address (top of RAM usually) in the first table entry, the first code execution address (_start) as the second table entry, and defaults for the rest of vectors (although it doesn't do all 256 of them - lazy). It then appends all the compiled code following the vector table, then the read-only data (constants), and finally initialized (read-write vars) data.  This is the binary that's created to be burned into the ROM.  The first code that executes (_start in crt0.S) clears the bss area in RAM (for uninitialized vars), copies the initialized (read-write vars) data from the ROM binary into RAM, then jumps to main (in ROM).

The main.c that is provided is just a simple serial terminal echo program.  But it could be a monitor program or a full-blown OS.  The first order of business is to initialize the UART via init_tty() call.  See the two HAL files tty_16550.h and tty_16550.c where the UART driver specifics are kept; addesses of all the registers, code for the interrupt service routine (attached at vector IRQ2() including a systick; a 555 tickling the RI pin @50Hz), line buffer, and putchar()/getchar() primitives which will be called from main.c  Other IO drivers will have .h and .c files with their own init_xxx() and primitives too.

You don't need a bloated, obscuring IDE to do this for you.


To compile:
    make clean
    make all
Produces bmbinary.rom file to be burned onto a ROM.

Also included in the zip are outputs produced via:
    make althexdump
    make dumps
The first is a hex+ascii dump of bmbinary.rom.  Notice the various sections:
    vector table between 0x0000 and 0x03FF,
    code between 0x0400 and 0x06DF,
    read-only (constants) data between 0x06E0 and 0x071F, and
    initialized (read-write vars) data between 0x0720 and 0x072F.
The second is a dump of the various sections in the bmbinary elf file produced (incl. assembly and data).

[westfw]

Your understanding is basically correct.
But of course, we can come up with some nit-picking corrections!

the peripheral's controls and own registers are "mapped to memory" by the manufacturer and recorded in the device datasheet

Not all MCUs have "memory mapped" peripherals.  Sometimes there is a special address space for the "IO registers" and special instructions for accessing them.  8051 comes to mind.  (less and less common, BTW.  Peripherals have gotten too large and too numerous to fit in the (usually much smaller, eg 256Bytes on an 8080) special address spaces, while memory address spaces have gotten bigger.)

they can be controlled by register access made easier with C code register access libraries routine tasks are implemented in Hal libraries for further abstraction.

There are at least three levels where "C makes it easier."

• vendor-provided definitions of names and data structures for the peripherals.  (USART0->CTRLA = USART_DB8|...)
• low-level libraries for accessing them, provided by vendor or by 3rd parties. (ll_usart_init(USART0, usartInitStruct))
• high-level libraries for doing things (think "Arduino" or MBed.) (Serial.begin(115200))

the CPU directly communicates with the peripherals over the Bus/Bridge (acting as a translation layer) obeying wait and confirmation commands (important for slow peripherals).

Don't confuse "slow to access peripherals", where bus timing is likely to be controlled by the hardware, and "slow peripherals" (like a USART) where software will have to wait for various operations to complete.

their are placeholder error handling routines

Hmm.  Maybe.
This is probably more of a carryover from operating systems where some simple flag means "one of many possible errors has happened.")

the CPU core language is defined by the ISA by the CPU designer (A revelation to me that assembly is an umbrella term for all the different instruction sets (or languages) from 8051 to X86, and not a standard its self)

Yes.  Very important.  There are a lot of commonalities to assembly languages, so that your 2nd and 3rd will be much easier to learn than the first, but...

the Compiler is mostly the concern of the CPU IP holder such as ARM and not the MCU manufacturer.

Note that the "CPU Designer" and "MCU Manufacturer" are sometimes the same company.  (Used to be "frequently."  Multiple MCU Manufacturers sharing a instruction set and compiler is a relatively recent thing, as are "CPU designers" that are separate from manufacturers.  (Probably a good thing, since good compilers are so much work to create. (heh.  "If we use this CPU architecure, we can just use the already existing gcc!" is lazy, common, and probably a good thing.)

Because the compiler, like the CPU, does not know about the peripherals and and the FLASH, RAM ect.. are also peripherals, the compiler needs a "linker file" so it knows where to put stuff;

That's one way to do it.  Most (but not all) chips seem to specify the peripheral locations in .h files, rather than linker files.

the language that the compiler understands is defined by the C programming language standard.

It doesn't have to be C.
However, C is relatively easy to "port" to new CPUs, and ... gcc already exists for many CPU architectures.
Chips USED to come out with nothing but an assembler (and maybe not that), leading people to write their own languages.  ("JAL" for PIC is an example.  Assorted mutually-incompatible BASIC implementations.  The original Mac OS was written in a flavor of Pascal.)

So a very determined person who knows C could write the code and program an MCU without any of the IDEs or libraries, as long as they had the compiler and the device datasheet.

Certainly.  And there's always assembly language.  Or even machine code.
I wrote my own 8085 assembler (sort-of) and can remember hand-assembling 1802 code to hex...

It would just be slow inefficient

Slow and inefficient for the programmer, perhaps.  The code is likely to be faster and more efficient than most vendor-provided libraries.

There is also the start up code that the IDE config GUIs help generate to create boot up routines and initialise clocks/ peripherals correctly depending on use case.

This can also be written by hand, directly from the datasheet.  Some modern CPUs are designed so that you can write code entirely with a high level language (usually C), without needing any machine or assembly code.   I find it moderately amusing to note which ARM vendors provide asm vs C "startup code."
Alas, how much the startup code does can vary (whether or not clocks are set up there, or whether you need to do something else to configure them, seems particularly variable. (Perhaps because clock systems have become so complex.))


It's relatively interesting to have lived through the history of all of this.

• Here's a chip and a description of its instruction set.  (~1975)
• Here's a chip, and we'll sell you a cross-assembler written in fortran that will run on your mainframe.
• Oh cool!  Personal Computers!
• Here's a chip, and a cross assembler written in BASIC that might run on most PCs.
• Oh look!  MIT/others have a X compiler that will run on your mainframe!  Shareware!  Freeware!  OSSW!
• Someone ported a (C/BASIC/Pascal) compiler to CP/M!  Only $$$!
• "Portable C compiler."  "USCD P-System"
• Personal Computers with Hard Drives! (1984)  Myriad 68000-based "unix" systems. (About the same time, more $$)
• GCC C compiler (68k: 1987, x86:1988)
• First C compiler for 8051: also 1988.
• First electrically eraseable mcirocontrollers: 1993
• First microcontroller (Atmel AVR) "designed for C": 1997
• First ARM microcontroller (ARM7TDMI): 1998
• First MIPS microcontroller (PIC32): 2007
• First Cortex ARM: 2004

(wow.  It's been more years between the advent of "general purpose microcontrollers" than it was between the first microcprocessor and gcc!  (Counting the advent of gcc as the start of wide-spread use of compiled HLLs in embedded systems.))

[Alien Brother]

the Compiler is mostly the concern of the CPU IP holder such as ARM and not the MCU manufacturer.

From what I've seen, MCU vendors tend to provide a toolchain that they want you use. For example - https://www.st.com/en/development-tools/stm32cubeclt.html - even if it's GCC, it comes with pre-built libc, there's a ready-to-use Windows version, you can expect ST library code to work with this GCC version, etc.

[[email protected]]

Thanks for all the replies! (Sorry for my slow one, got caught up in life) I dont know where else I would get such good information. I have learnt alot just from the comments and have followed some of those rabbit holes which has been interesting, especially about build environments.  That + the general vibe not to become too hooked on the manufacturers eco system has made me switch from TMS320F280034 to STM32G474 to get out of TI's garden which seems to have slightly higher walls.  I prefer the idea of learning an ARM based system as it is more transferable. Back to KiCAD i guess 

Also @westfw that was a great read, and useful clarifying points. I appreciate the time it takes to type that out, the history summary is wild,  the "personal computers with hard drives" got me . the things one takes for granted!

[westfw]

the general vibe not to become too hooked on the manufacturers eco system

I think this especially applies to embedded operating systems.
I mean, most vendors are going to provide you with a IDE and compiler environment based on gcc, and some Open Source Software IDE with some amount of customization.  And most of those IDEs are largely similar, whether you're talking about Eclipse (used to be very popular), Visual Studio Code (up-and-coming), regular Visual Studio ("Atmel Studio"), Netbeans (older (?) MPLAB-X), Arduino, or Keil (to mention a bunch.)

Then you'll get libraries, which are somewhat more dangerous - use a vendor library, and you'll have a harder time porting to some other vendor's libraries (maybe, anyway.)

But start using TI-RTOS in addition to the TI libraries, and you'll really be locked in...(sadly, this can include things like TI's Arduino port ("Energia", at least originally), which used TI-RTOS to provide multitasking for TI ARM processors...)

[asmi]

But start using TI-RTOS in addition to the TI libraries, and you'll really be locked in...(sadly, this can include things like TI's Arduino port ("Energia", at least originally), which used TI-RTOS to provide multitasking for TI ARM processors...)

When it comes to RTOS, it's best to stick to CMSIS-RTOS as it abstracts away specifics of the underlying RTOS, which should make migration to another RTOS easier as long as the latter also has an adapter for CMSIS API.

[Doctorandus_P]

My first serious uC programming in C was with the ATMEGA8. It "just worked" and I did not have to think much about details. I just used the header files, for the address definitions of I/O stuff, and I could very much just start by putting some contents into main().

Later I wanted to look into STM32, and that was a whole other sort of adventure. The biggest problem is an extreme overload of low quality crap, that can barely be mentioned "information", for example the 66 pages of:

https://www.eevblog.com/forum/microcontrollers/is-st-cube-ide-a-piece-of-buggy-crap/

I really disliked eclipse with immediately phoning home (or elsewhere) each time I start the software, and the idea of ST for "helping" people is to make an ever bigger pile of crap and then say "push this button".

So I dove deeper, and learned a lot about what's needed for a "minimum" environment.

1). First: GCC. I use the arm-none-eabi-gcc variant.

2). You need "startup code". this is usually provided by the microcontroller vendor, and you do not write it yourself. It's also dependent on the compiler. For example, the commercial compiler vendors use another compiler (duh) and thus also different startup code. This startup code is usually written in assembly, but (most, or a part) can be written in C too. The startup code has a table with ISR adresses, and it does the initial initialisation of the environment. For example it clears memory locations that should be initialized as "zero" in C. And it loads other initialized variables to a place where they can be used in C. For example, if you have: "int Baudrate = 9600;" in your code, then the startup code assigns an address to "Baudrate", and loads the "9600" from flash and writes it to that variable. The last thing the startup code does is a call of main(), and that is where your own program starts. (I skipped some details).

3). You need a linker script. A linker script is a list of addressees and sizes of memory regions. Such as: What is the address range of FLASH in this uC? How much RAM does it have, and what are it's adresses. The linker script is also provided by the uC vendor.

4). Makefile. A makefile not strictly necessary, but without it, you have to type very long command lines to tell the compiler what to do. GCC has very many options (optimization, which warnings to create, etc.) The makefile automates this. Apart from controlling how the C (and C++) source code is compiled, it also links the parts together and it can call other programs such as objcopy to create intel hex, Srecord, or binary files that can be used to program the uC. The syntax of makefiles was devised by some devious minds in the '60-ies or early '70-ies. It's quite powerful, but horrible to read and write. There are also other build systems such as SCONS (used by platformio for example). In principle you could also write a simple python program to spit out the right command lines to compile and link your code. Some IDE's skip the makefile, and generate the build commands internally in some way.

5). Your source code. At least a text file that contains the "main(){}" function.

6). Header files. At a minimum, you want a header file that defines all the locations of important registers. You do not want to search through datasheets to figure out the address of an I/O register.

7). C standard headers and libraries. These are optional. Code to manipulate for example strings is very common. Therefore, a part of the C standard are pre-defined functions. These may be distributed as source code, but often are already compiled blobs that have to be linked into your code if you want to use them. https://en.wikipedia.org/wiki/C_standard_library

. "Framework". STM32 has the HAL and the LL stuff. These are also optional. It looks like the HAL is bloated obfuscation, while the LL is a very thin layer between your software and the hardware. I think I like the LL layer. I have for example a library that uses RS485 over a uart, and with the LL layer, I can make this code run on both AVR processors and STM32. LL code is also quite readable, and is at least a learning aid with examples of how to program the STM32 processors. (Unfortunately) these have a quite complex setup. You have to configure PLL's and routing / enabling of different clock domains to make use of the processor.

9). There are other "more elaborate" frameworks. such as "arduino" (ugly obfuscation bloatware) and mbed. (officially discontinued now). In general, I don't like such frameworks much. Such bigger frameworks often make a lot of assumptions. It's easy to get a blinking led going, but to do "useful stuff" you get stuck very quickly. For example, integrating my (already written and optimised) ISR driven uart/rs485 library going on such a bigger framework is difficult. As it hyjacks the serial port for it's own use. Yuch!

10). Third party library code. There are all sorts of libraries. From things like an RTOS, MP3 encoders, TCP/ip networking, LCD controller libraries etc.

11). IDE. For learning how all the parts fit together, a simple text editor is perfectly usable tool to write all code. I use Geany myself, and SciTe before, when I was still using windows. Both these editors also have syntax highlighting, and this helps with making code more readable, and finding simple typo bugs. But when your programs become more complex and you're using more "template" code, you want a more capable IDE, that can for example find win which files functions are defined, and then quickly jump to them. The "modern frameworks" can easily have 50 files with source code and manually loading and scrolling though those files to find a particular library function is quite horrible, but it's easy to manage in a "good" IDE. "Eclipse" (used by ST / CUBE) is a powerful IDE, but very much bloated (long startup times), and when "something breaks" in the build system, it's nearly impossible to repair. Some IDE from microsoft seems to be popular, but I don't use software from that brand. I liked QT Creator, but it complains too much about the "community edition". Used Monodevelop for a while, but that got bought and killed by microsoft. Anjuta never got to a "good usable" state. For now I'm using Codeblocks, which seems a reasonable (for me) compromise between functionality, features and simplicity. Maybe I'll look deeper into "Codelite" some day. I once did some experiments and it did seem usable. Main thing for me is that with a working makefile, the IDE is "optional". I can always fall back to the simplest text editor, and then use "make all", "make flash", etc. from a terminal window.