Heterogeneous multi-core MCUs

[Unixon]

Many top level "high performance" families of MCUs often have two or more cores, but the cores are different.
For the sake of this discussion let's consider STM32H7R3 and N32H765 as two examples of heterogeneous dual-core MCUs.

If you know a project using one of these or a similar MCU, what are the roles of low- and high-performance cores?

[Unixon]

p.s. NSING has a very nice product selection guide 

[westfw]

There are lots of examples for RP2040 and etc, where the "PIO" is essentially a second core, specialized for bit-banging stuff...
And then there are all the chips with "dedicated GPUs"
The chips where the multiple cores are "closer" (M7 plus M0, for instance) are tougher to figure use cases.  I think frequently, one core is designed for low-power, and the other ... isn't.

[fchk]

Look here:
https://www.ti.com/product/TDA4VM
There is a Linux cluster of 64 bit A72 cores for communication, a C7000 64 bit DSP, 2 C6000 32 bit DSPs, and some Cortex R5F realtime MCUs. Each type of cores has its own compiler and architecture and its own memory.

At work we use similar chips for motion control tasks where the DSPs and MCUs run real time tasks and the Linux cores do communication etc.

[Unixon]

Look here:
https://www.ti.com/product/TDA4VM
There is a Linux cluster of 64 bit A72 cores for communication, a C7000 64 bit DSP, 2 C6000 32 bit DSPs, and some Cortex R5F realtime MCUs. Each type of cores has its own compiler and architecture and its own memory.

This SoC is insane!
What normal person is even able to master it and not lose mind?

[Dave]

One of the more common uses is to have the core functionality running on the higher performance core and then use the lower performance core just to perform real-time fault checking for safety functions.

[fchk]

Look here:
https://www.ti.com/product/TDA4VM
There is a Linux cluster of 64 bit A72 cores for communication, a C7000 64 bit DSP, 2 C6000 32 bit DSPs, and some Cortex R5F realtime MCUs. Each type of cores has its own compiler and architecture and its own memory.

This SoC is insane!
What normal person is even able to master it and not lose mind?

Oh, I designed a PCB for it at work. And yes, I do take my pills.

[Unixon]

This SoC is insane!
What normal person is even able to master it and not lose mind?

Oh, I designed a PCB for it at work. And yes, I do take my pills.

I'd like to know the name of pills the firmware guys are taking

[tszaboo]

I'm using MCUs like these. An M33 or M3F is the main micro, and an M0 is one responsible for the radio.

[Siwastaja]

This SoC is insane!
What normal person is even able to master it and not lose mind?

Oh, I designed a PCB for it at work. And yes, I do take my pills.

I'd like to know the name of pills the firmware guys are taking

It's the "working in teams" pills. No one needs to master the whole stack. Like, one guy would be writing a FOC motor control thing on the microcontroller core. Another guy would be writing the part that gives the motor firmware some commands, and sends status to a cloud. Yet another guy would write a web browser user interface. And they would use linux on the application processor which gives them stuff like well-tested TCP/IP stack and USB for free.

That said, teamwork brings in the complexity of agreeing on interfaces between the parts. But that's inevitable at some point in complexity.

This kind of chip is still in the realm that one very bright engineer could build the whole stack alone, but it's a different question if that's a good idea. Buses are well known to chase such rockstar programmers on the streets.

[John Celo]

Look here:
https://www.ti.com/product/TDA4VM
There is a Linux cluster of 64 bit A72 cores for communication, a C7000 64 bit DSP, 2 C6000 32 bit DSPs, and some Cortex R5F realtime MCUs. Each type of cores has its own compiler and architecture and its own memory.

This SoC is insane!
What normal person is even able to master it and not lose mind?

You get used to it I think.
I remember when opening a datasheet or ref manual of simple cortex m0 seemed overwhelming.
Now I can open up manual of any MCU from any vendor, cortex m7 with whole load of peripherals and it doesn't even faze me.

Its all stuff I've seen before, and most peripherals poken and prodded a few times, chained together with timers and dmas and what not, and all the cryptic shorthands seen hundreds and thousands of times before.

Know what diagnostics tools and debuggers to use to figure out when something doesn't work.

MCUs are actually quite simple once you get comfortable with them.

I'm sure if you stare at that A72 datasheet long and frequently enough, along with some hands on work, the same would happen too. The scope here is much, much larger admittedly.

[Unixon]

MCUs are actually quite simple once you get comfortable with them.
I'm sure if you stare at that A72 datasheet long and frequently enough, along with some hands on work, the same would happen too. The scope here is much, much larger admittedly.

I'm more or less confident with regular single core MCUs like ARM Cortex-Mx, probably I won't have any problems with dual core ARM Cortex-Mx too.
More complex stuff like multi core ARM Cortex-Axx MCUs/CPUs or heterogeneous SoCs are too scary yet.

[jheissjr]

What is your project for the TDA4VM?

[SiliconWizard]

Actually, "heterogeneous" multi-core development is usually conceptually simpler than symmetric multi-core, not per se but because you will then assign a specific role to each core with ad-hoc interface and synchronization between them, and it's a simpler architecture than implementing "generic" SMP.

Of course, you can also use this approach with a multi-core CPU with identical cores, but it's less common to do so. The fact the cores are different lends itself more naturally to dedicating each to specific tasks and that's conceptually simpler.

As to the benefit of such systems, it's pretty simple. You can split execution of different tasks to completely separate cores which run entirely concurrently, and having simpler or less generic (such as DSP-specific)  additional core(s) cuts costs and power consumption.

[Unixon]

What is your project for the TDA4VM?

None so far. I just learned about its existence.
However, I may get my hands on STM32H7R3 or N32H765 someday.

[nctnico]

Actually, "heterogeneous" multi-core development is usually conceptually simpler than symmetric multi-core, not per se but because you will then assign a specific role to each core with ad-hoc interface and synchronization between them, and it's a simpler architecture than implementing "generic" SMP.

Of course, you can also use this approach with a multi-core CPU with identical cores, but it's less common to do so. The fact the cores are different lends itself more naturally to dedicating each to specific tasks and that's conceptually simpler.

As to the benefit of such systems, it's pretty simple. You can split execution of different tasks to completely separate cores which run entirely concurrently, and having simpler or less generic (such as DSP-specific)  additional core(s) cuts costs and power consumption.

But the downside is that you can't lift processing power from one core to another. Linux (for example) allows to allocate cores towards a single process. If you need more processing power, you simply parallellise the problem and assign more cores. With a mixed core system: If you run out of DSP resources, you need to do much more work to partition the task differently across tools and a different architecture.

[fchk]

What is your project for the TDA4VM?

I can't go into details, but in general it's some kind of control of multiple servo motors. The previous system (>10 years old) uses a Sitara AM3xxx, and they had to disable Ethernet in order to keep the realtime constrains. They used UART for communication and a TCP-UART device server for network connectivity. The new board uses the DSP cores for all the realtime tasks and uses the Linux cores for network communication in order to get rid of the device server and decrease the cycle time.

[Unixon]

I can't go into details, but in general it's some kind of control of multiple servo motors.

Are you building T-800 prototype?

[jheissjr]

Interesting and fun project. Thanks for sharing.

[John Celo]

I'm more or less confident with regular single core MCUs like ARM Cortex-Mx, probably I won't have any problems with dual core ARM Cortex-Mx too.
More complex stuff like multi core ARM Cortex-Axx MCUs/CPUs or heterogeneous SoCs are too scary yet.

I think the truly scary part is that unlike MCUs, where you kind of "own" the entire software... uh... "stack".
And HALs are rather thin wrappers over poking some registers. And are super easy to do that yourself if need be.
You can quickly wrap your head around the whole thing and entire 32bit memory space of any given MCU.

The SoCs that are supposed to be explicitly used with Linux kernel (A72 with MMU and protected mode, complicated bootloaders and startup sequences) is that suddenly you're swimming in 40 millions LoC codebase, all sorts of proprietary IP units with their kernel modules (some just binary blobs?)

And it's a completely new skillset to be able to efficiently swim in and navigate that sea.

If you're using a TI or NXP part I would expect that they have rather decent documentation for this, so it probably isn't THAT bad.

Another thing is just how easy it is to source any generic MCU and get all the documentation and tools. With SoCs you quickly enter into "sign the NDA" territory to get your hands on stuff. And if you're just learning, it's a barrier. And the dev/eval boards are in entirely different price category also. (that with DRAM shortages stacked on top).

[hans]

Isn't the STM32H7R3 a single-core? IIRC its their 600MHz series. The STM32H74x devices are a 480MHz M7+ 240MHz M4 variant..

In terms of CoreMark that brings them pretty close (H7R3 is 3174CM, H747 3224CM). But having 2 cores might be easier or harder to program depending on how you can segment a program. E.g. real-time number crunching on M7 and comms on the M4 (240MHz is still plenty fast compared to a classic F407 or F429).

I think chips like the RP2040 are also interesting, especially because they have 2 identical cores. There you could opt for a RTOS that schedules the same task list on two cores. Whereas if you want to do that on a STM32H747, then both programs need to run on the M4 level instruction set. So for example, you can't use the DP-FPU..

Oh and yes I agree that you get used to complexity. Some people literally fall of their chair when I tell them this 5$ chip has a 3000+ page reference manual with over 50 chapters.
On the other hand, setting up a timer+DMA+SPI+RCC+interrupt orchestration is not that complex anymore. Actually getting a bit boring if you have done it a dozen times on various ARM chips. The workings is all fairly similar, just a question of which bit to set where..
I also remember the first weeks of programming a PIC16, where me and a pupil got stuck on a UART Rx interrupt for full session(s) because we still had no clue what we're doing. That last part is of course the lesson.. not to get the actual program to work.
In my experience, I first learn conventions and specifics of a particular device/protocol, and then with experience you can abstract and generalize. Then things can look pretty similar, just a case of how much baggage a committee have decided to attach to a protocol spec. This is why I don't mind dealing with USB if that means I don't have to deal with e.g. Bluetooth.

[Unixon]

Isn't the STM32H7R3 a single-core? IIRC its their 600MHz series. The STM32H74x devices are a 480MHz M7+ 240MHz M4 variant..

Oh, it's really only a single core. I guess I put it alongside NSING MCU as the only equivalent to NSING part with same frequency and double precision FPU and last one in non-BGA package.
The only one that's even faster is STM32N6 series, but those are also single core MCUs.

In terms of CoreMark that brings them pretty close (H7R3 is 3174CM, H747 3224CM). But having 2 cores might be easier or harder to program depending on how you can segment a program. E.g. real-time number crunching on M7 and comms on the M4 (240MHz is still plenty fast compared to a classic F407 or F429).

That would be interesting to work on a part like this. Even after reading datasheets it still isn't clear to me how these two cores access memory and if they share it in some way.

I think chips like the RP2040 are also interesting, especially because they have 2 identical cores. There you could opt for a RTOS that schedules the same task list on two cores. Whereas if you want to do that on a STM32H747, then both programs need to run on the M4 level instruction set. So for example, you can't use the DP-FPU..

I still have a lot of [stupid] questions about dual core MCUs. Like... how do I compile and upload firmware to such an MCU?
From what I've read (referencing STM32 HAL based examples) there are three parts to source code: a common part (?) and two different libraries (HAL) for each core and that FLASH for each core is mapped to different addresses.
How should I compile this combination? Should I produce two sets of object files for two architectures?
Should I link it into one binary (containing all code for both cores) or two binaries (for each of cores separately)?
Should I upload it to each target separately or to a single combined(?) target? This is likely depends on MCU architecture as there may be one or more DAPs.
What if I need to run same firmware on both cores? What if I want each core to run a different firmware?
How do I debug both cores simultaneously? For single core it's just ST-Link/SWD + OpenOCD + GDB + some IDE.
Etc, etc.

[NorthGuy]

I still have a lot of [stupid] questions about dual core MCUs. Like... how do I compile and upload firmware to such an MCU?

Read ST's AN5557 ...

There was a lengthy thread a few years back with a person experimenting with dual cores.  I just cannot find it at the moment, but you may be luckier than I am

[John Celo]

For RP2040 you don't have to do "anything"... it just *works*. Both are connected to the same "bus fabric".

See "2.1. Bus Fabric" section of the rp2040 datasheet.

Try it, it's probably the cheapest and most accessible (and intuitive) multi-core MCU on the market IMO.
It also has one of the better written, human readable documentations that I have run across.

multicore_launch_core1(core2_loop);

And there you have it. And the second core is executing "core2_loop()".

I suspect STM32H747/757 dual core parts would act similarly - ie they are both connected to the same bus.
The difference here is that "core2_loop" would have to be specifically marked to have only m4 instruction set,
and everything within it could only call functions that use no more than m4 instructions.
So both of the cores would probably execute from different address ranges in flash.

See attached image for stm32h757. Three seperate (interconnected) bus domains...

[Unixon]

See attached image for stm32h757. Three seperate (interconnected) bus domains...

In STM32H757 bus matrices are connected in circular fashion, as one bus is represented as a device on another bus.
This makes me think that some memory and peripherals are shared and accessible from both cores but each core has faster access to devices in its domain via nearest bus matrix.
Should the source code for each core also be different? This depends on how everything is mapped onto address space on both cores.