Cortex M55

[ali_asadzadeh]

Hi,
Have you seen the news?
https://developer.arm.com/ip-products/processors/cortex-m/cortex-m55

Any Ideas who would make the first chips? and when they became available? I prefer to have something like M75 too, a Cortex M7 with a lot of improvements over it and added DSP and AI capabilities, and some math accelerators for Cordic  and Filter algorithms.

[SiliconWizard]

Have you seen the news?
https://developer.arm.com/ip-products/processors/cortex-m/cortex-m55

Yes.

Any Ideas who would make the first chips? and when they became available?

Nope. This is all very recent.

[lucazader]

Interesting that they have implemented half-precision float calcs in hardware (fp16). As well as fix point acceleration in their dsp.

Also interesting that the FPU and the Vector unit share their registers to save silicon area.

Although I dont personally use AI inferencing in my line of work so this announcement overall doesn't impact me, I could definitely make use of the the fp16 support.

[ataradov]

I could definitely make use of the the fp16 support.

For what? This stuff puzzles me.

I can see if you use FP16 to save resources on massive SIMD instructions, but otherwise why limit yourself to 16 bits?

[SiliconWizard]

I could definitely make use of the the fp16 support.

For what? This stuff puzzles me.

I can see if you use FP16 to save resources on massive SIMD instructions, but otherwise why limit yourself to 16 bits?

Yeah, I personally don't see much use for this either. What's the typical use case for FP16?

Accelerated fixed point, OTOH, yep!

[lucazader]

My use case is in some of my bldc motor control applications where i dont need the full precision of fp32, and the speed increase of moving to an fp16 calc would be beneficial to allow higher max commutation rate of the motor.
That is one example that i have, I'm sure i could find others.

This is based on the assumption that the fpu doing an fp16 calc will return a result in fewer cycles than fp32. If I am wrong, and there is no speed-up then of course there is no benefit.

[Yansi]

Saturating arithmetics and SIMD and FLOAT FPU is present since Cortex M4.

Since some newer Cortex M7 revision, there is even a hard double FPU.

So what improvements are we talking about here?

[SiliconWizard]

My use case is in some of my bldc motor control applications where i dont need the full precision of fp32, and the speed increase of moving to an fp16 calc would be beneficial to allow higher max commutation rate of the motor.
That is one example that i have, I'm sure i could find others.

OK, I see. I think this is a typical use case where using fixed point would make sense, but of course FP is usually easier (less to worry about with overflows and underflows, although you'd quickly reach those with FP16 too).

Now I'm not sure how critical your calculations could be for motor control. Given the inertia of a typical motor, how could you ever be limited by computation speed unless you do really HEAVY computation on a really slow MCU?

This is based on the assumption that the fpu doing an fp16 calc will return a result in fewer cycles than fp32. If I am wrong, and there is no speed-up then of course there is no benefit.

I would suspect this would be the case for divides, maybe multiplies (not sure, depends on the FPU) and probably not additions and subtracts.

[mach665]

The vector register size in all previous Cortex-M is only 32bit. The vector register size in Helium extension in ARMv8.1-m is 128 bit. Plus some other instructions to speedup some math calculations. Think about the increase of the throughput.
As a comparison, intel's Advanced Vector Extensions (AVX-512) has a vector size of 512bits and includes preparations to extend 1024 bits.

Copied from Wikipedia for ARMV8
"The Scalable Vector Extension (SVE) is a new extension for ARMv8 allowing "implementation choices for vector lengths that scale from 128 to 2048 bits";[126] a complementary extension that does not replace NEON. A 512-bit variant has already been implemented."

[brucehoult]

The vector register size in all previous Cortex-M is only 32bit. The vector register size in Helium extension in ARMv8.1-m is 128 bit. Plus some other instructions to speedup some math calculations. Think about the increase of the throughput.
As a comparison, intel's Advanced Vector Extensions (AVX-512) has a vector size of 512bits and includes preparations to extend 1024 bits.

So far, almost all shipping consumer Intel CPUs have either been limited to AVX-256 or else they slow the clock rate for AVX-512, which negates much of any theoretical performance increase.

Copied from Wikipedia for ARMV8
"The Scalable Vector Extension (SVE) is a new extension for ARMv8 allowing "implementation choices for vector lengths that scale from 128 to 2048 bits";[126] a complementary extension that does not replace NEON. A 512-bit variant has already been implemented."

SVE and MVE are different things.

SVE has 32 vector registers and 16 predicate registers (which contain one bit for each data element in a vector register). The vector registers are, as stated above, anywhere from 128 bits to 2048 bits in size, varying from CPU to CPU. SVE was announced 3.5 years ago in August 2016. ARM has, as far as I know, not yet announced let alone shipped any CPU core implementing SVE. Fujitsu has prototype chips of their own CPU core design implementing SVE for an uncoming supercomputer.

SVE is quite similar to the RISC-V "V" extension which is nearing final specification status (I'm on the Working Group and implemented the first Spike simulator support for it, and first assembly-language kernels). RVV has 32 vector registers of length anywhere from 32 bits to 2^32 bits (i.e. much more scalable in both directions than SVE). If your code doesn't require a full 32 register variables you can gang the registers together into 16, 8, or 4 registers of 2, 4, or 8 times the hardware length for greater efficiency. Predicates are stored in normal vector registers and always take exactly one register even when the registers are ganged for data elements. SVE and RVV both support using only the initial part of the vector register for short vectors or for the last elements of a long application vector. RVV has a "current vector length" register while SVE does this purely by setting the predicate bits for the tail of the vector to false.

MVE (aka Helium) is a completely different and simplified but again similar vector instruction set announced by ARM out of the blue a year ago in February 2019. It has a fixed set of 8 registers of 128 bits each -- the same as the smallest RVV in ganged x4 mode.

The advantage of SVE and RVV is that you can write one assembly-language (or compiled) program that will run at maximum efficiency over a wide range of different CPU vector sizes. You don't have to rewrite your program to go from MMX to SSE to SSE2 to AVX to AVX-512. And new CPUs don't need entirely new instructions every time alongside all the older instuctions.

[mach665]

I understood that MVE is not SVE. It is not even NEON, but after all we are talking about micro-controllers. With 128bit vector registers, we can do 4 times as many calculations as Cortex-M7 in an ideal world. It's a lot faster for DSP or ML jobs. And I would assume the right way to use MVE is using CMSIS libraries, or we may have to embed assembly.

[Yansi]

Assembler?  That is what the intrinsics are for. Perfectly fine to be used within the C environment. Even the CMSIS DSP library is written completely in C, using the intrinsics.

(I would not call that an assembly language).

[tszaboo]

I could definitely make use of the the fp16 support.

For what? This stuff puzzles me.

I can see if you use FP16 to save resources on massive SIMD instructions, but otherwise why limit yourself to 16 bits?

Yeah, I personally don't see much use for this either. What's the typical use case for FP16?

Accelerated fixed point, OTOH, yep!

FP16 is used for AI, especially for image related stuff.

[brucehoult]

Assembler?  That is what the intrinsics are for. Perfectly fine to be used within the C environment. Even the CMSIS DSP library is written completely in C, using the intrinsics.

(I would not call that an assembly language).

Not sure what you are replying to, but I seem to be the only one who has mentioned assembler.

You can't use intrinsics in C until you add them to the compiler. I helped with that work in Oct/Nov/Dec. In the case of vector instructions you also need to first add to the compiler the new register file, a register allocator, C types representing the vector registers -- which are tricky for both SVE and RVV because previously gcc and llvm had no concept of a register that you don't know the size of at compile time, which complicates a lot of things from stack frame allocation and spilling to argument passing.

You can start trying new instructions in assembly language as soon as you have added them to the assembler (which the C compiler also needs), and the emulator. That was Jan/Feb/Mar last year.

[mach665]

Hi brucehoult,

What is the current compiler status? For example, does GCC optimize C source code into those SIMD instruction with wide vector registers? If it can, how efficient is it. What is the recommended way to do it now? Using library like NEON lib, or relying on GCC is good enough?

[brucehoult]

Hi brucehoult,

What is the current compiler status? For example, does GCC optimize C source code into those SIMD instruction with wide vector registers? If it can, how efficient is it. What is the recommended way to do it now? Using library like NEON lib, or relying on GCC is good enough?

Hell no.

GCC and LLVM can auto-vectorize only the most basic and trivial things, on *any* ISA. Support for SVE and RVV is not there at all yet, at least in released versions. It'll be years before there is good support. I think SVE may have intrinsics support released. RVV has intrinsics support in company internal testing and that's the best you can hope for on initial hardware releases. Basically assembly-language programming but the compiler handles the register allocation for you.

[SiliconWizard]

GCC and LLVM can auto-vectorize only the most basic and trivial things, on *any* ISA.

Any? A bit much here. any that supports vector operations probably. Surely obvious, but "any" sounded weird. And I don't think they would (at least for GCC) unless you enable the corresponding optimizations (either automatically through some -O level or specifically.)

I had done some experiments with GCC for x86-64 targets.
In order to guide GCC to make the most of it, it's often useful to use specific qualifiers/builtins, otherwise your chances of getting something remotely efficient are much lower, since the compiler can't make the required assumptions.

To illustrate, you can compare the compiled assembly of the two versions (it would be interesting to try this for various targets at least supporting some SIMD instructions, you may need to adjust the alignment hint in __builtin_assume_aligned()):

Code: [Select]

void MAC1(double *Y, double *X1, double *X2, unsigned int n)
{
for (unsigned int i = 0; i < n; i++)
Y[i] += X1[i] * X2[i];
}

void MAC2(double * restrict Y, double * restrict X1, double * restrict X2, unsigned int n)
{
Y = __builtin_assume_aligned(Y, 32);
X1 = __builtin_assume_aligned(X1, 32);
X2 = __builtin_assume_aligned(X2, 32);

for (unsigned int i = 0; i < n; i++)
Y[i] += X1[i] * X2[i];
}

The second one will have less overhead.

[SiliconWizard]

I could definitely make use of the the fp16 support.

For what? This stuff puzzles me.

I can see if you use FP16 to save resources on massive SIMD instructions, but otherwise why limit yourself to 16 bits?

Yeah, I personally don't see much use for this either. What's the typical use case for FP16?

Accelerated fixed point, OTOH, yep!

FP16 is used for AI, especially for image related stuff.

Are there really benefits compared to fixed point for this application? Or is it just because people don't want/don't know how to use fixed point anymore?

[tszaboo]

Are there really benefits compared to fixed point for this application? Or is it just because people don't want/don't know how to use fixed point anymore?

Most AI calculations involve multiply-add, where the operand is between 0 and 1. You cannot represent that number with INT16. Also, you most likely would need to take care about overflows and that would mean that instead of 1 operation, you have to do more than 1. There is something called neural network quantification, which  moves the model from floating point to fixed, but apparently, it is very computing intensive.
There is also bfloat16, which apparently takes much less gates than fp16, and porting models to it is simple. Look at the Edge TPU. I really think high performance microcontollers will have AI coprocessor built in with FP arithmetic.

[SiliconWizard]

Are there really benefits compared to fixed point for this application? Or is it just because people don't want/don't know how to use fixed point anymore?

Most AI calculations involve multiply-add, where the operand is between 0 and 1.

Exactly like "most" DSP calculations. It's been done on DSPs with fixed point for ages and still is. What's the problem?

You cannot represent that number with INT16.

Oh really? I'm talking about fixed point. You can put the virtual point wherever you like. If you absolutely need to represent "1" exactly (which I'm not even sure matters in most cases, as all calculations are approximate anyway, especially with FP!), you can just use a 1.15 fixed point number. Done, you can get your exact '1'. You'll actually get better accuracy than FP16 (except for the very low values) which has a mantissa of only 10 bits and an exponent of... 5 bits! Now that's very limited.

Also, you most likely would need to take care about overflows and that would mean that instead of 1 operation, you have to do more than 1.

This is again very typical in many (most) DSP applications using fixed point. Nothing new here. Oh and FP16 will overflow/underflow pretty quickly too. You don't have a lot of leeway.

I guess the real point is as I hinted earlier, relative ease of use, and most of all, that FPUs are now a commodity, so they are used everywhere without really giving that a second thought.
I'm just saying that fixed point can have definite benefits especially in terms of accuracy, and especially in applications dealing with this kind of numbers in the [0, 1] interval. Low-precision FP, like FP16, will quickly lose a LOT of accuracy after just a few operations.

I get the idea of using FP instead of fixed point in a processor that already contains one or more FPU(s) and nothing specific for fixed point, but here we were talking about a new core that apparently implements accelerated fixed point as well, so to me using FP16 on this instead of fixed point makes a lot less sense from a technical and computational POV. IMO. Again fixed point is still heavily used on conventional DPSs for a large range of applications, and if used exclusively, would cost significantly less silicon area (even with nice accelerated MAC operations). Now sure if an FPU is required for other uses, then reusing it for this and not bother with fixed point makes sense, but when you implement both, I'm not so sure. And OTOH, if you're going to implement a dedicated "AI" coprocessor, I would personally at least consider fixed point seriously.

Thing is, as I said above, FPUs are now so ubiquitous that just getting FPU IPs to implement accelerators is likely a lot simpler - I would guess that there are a lot fewer (and good) accelerated fixed-point units available as IPs.

[mach665]

Are there really benefits compared to fixed point for this application? Or is it just because people don't want/don't know how to use fixed point anymore?

Most AI calculations involve multiply-add, where the operand is between 0 and 1.

...........

From what I read, an important usage of FP16 is for applications that needs high dynamic range, but not very high SNR. Speech recognization as an example,  the audio samples can have huge dynamic range, but recognizing the sound may only need a moderate SNR. Either you speak right next to the microphone or 10 feet away, algorithms implemented with FP16 works well in both cases.

Someone may argue that a good AGC plus VGA for the microphone can get the work done too. But obviously that's harder to get right and much less appealing in this digital world.

[SiliconWizard]

From what I read, an important usage of FP16 is for applications that needs high dynamic range, but not very high SNR. Speech recognization as an example,  the audio samples can have huge dynamic range, but recognizing the sound may only need a moderate SNR. Either you speak right next to the microphone or 10 feet away, algorithms implemented with FP16 works well in both cases.

I see, but FP16 has only a 5-bit exponent... the dynamic range will still not be very "high".
As to dynamic range vs. SNR, that would be an interesting discussion.

Someone may argue that a good AGC plus VGA for the microphone can get the work done too. But obviously that's harder to get right and much less appealing in this digital world.

Yes that too. Well if you're relying on FP operations, you're still in the digital domain. So if it can be done with FP, it can be done with a purely digital AGC with fixed point (and get you better SNR).
But yeah it again all comes down to keeping things very simple, and likely relying on ready-made FPUs as I hinted above.

[langwadt]

Are there really benefits compared to fixed point for this application? Or is it just because people don't want/don't know how to use fixed point anymore?

Most AI calculations involve multiply-add, where the operand is between 0 and 1.

Exactly like "most" DSP calculations. It's been done on DSPs with fixed point for ages and still is. What's the problem?

You cannot represent that number with INT16.

Oh really? I'm talking about fixed point. You can put the virtual point wherever you like. If you absolutely need to represent "1" exactly (which I'm not even sure matters in most cases, as all calculations are approximate anyway, especially with FP!), you can just use a 1.15 fixed point number. Done, you can get your exact '1'.

with signed Q1.15 you can't quite get 1, you can get exactly -1, so -1*-1 overflows

[SiliconWizard]

with signed Q1.15 you can't quite get 1, you can get exactly -1, so -1*-1 overflows

I was implying unsigned fixed point here (which covers many of the algorithms in which only values in the [0, 1] range are used and combined through positive weighted sums).

The Q notation (that I didn't use on purpose) is always a bit confusing. Strictly speaking, from the usual definition, Q1.15 would be signed indeed and wouldn't fit within 16 bits actually, as it means one integer bit, 15 fractional bits, and an implied sign bit. So the correct notation would be Q1.14... just a small detail.

[langwadt]

with signed Q1.15 you can't quite get 1, you can get exactly -1, so -1*-1 overflows

I was implying unsigned fixed point here (which covers many of the algorithms in which only values in the [0, 1] range are used and combined through positive weighted sums).

The Q notation (that I didn't use on purpose) is always a bit confusing. Strictly speaking, from the usual definition, Q1.15 would be signed indeed and wouldn't fit within 16 bits actually, as it means one integer bit, 15 fractional bits, and an implied sign bit. So the correct notation would be Q1.14... just a small detail.

I've never seen Q1.15 used for anything but 16 bit with the range -1,0.999..