Apples new M1 microprocessor

[brucehoult]

It’s not an emulation. It is directly recompiling the x86 code as ARM at startup. It treats x86 as a compiler IL.

Ah, I missed that, so the "hit" is taken on startup?  - or is it perhaps only done once, and the resulting binary stored for later use?

In order to be faster than an x86 hardware implementation,  the target processor has to be faster at running its transpiled object code than the x86 processor is at running its native code... 

That just doesn't sound right, unless the hardware x86 implementation is really handicapped and unfixable... and nothing is unfixable, given clever enough engineers! 

Its proabobly the same as JIT compiling that has gotten so popular these days in the form of .Net and JavaScript. It crunches a block of code into native machine language and places a jump back into the compiler at the end of it, once the block of code gets to the end the JIT compiler takes over and compiles more of it. So code only gets compiled once its first executed. When it executes again its already native machine code, if it never executes then little/no work is wasted compiling it because it never gets there.

That's not what .NET does. It compiles the entire program from bytecode to native before running any of it.

This was already used back in 2000 in the form of so called "High level emulation" where a Nintendo 64 emulator called UltraHLE used this aproach to get a massive performance boost in emulating the console on x86. Back then PCs ware generaly not fast enugh to emulate consoles this powerful, especialy something like the N64 with its very different 64bit instruction set, weird float math, proprietary GPU etc..

There are plenty of examples before that.

- java Hotspot 1999

- VMware 1998. JITed x86 to x86 while replacing non-virtualizable code with safe code. This became unnecessary in 2005 once Intel added VT-x

- MAME 1996. Slow CPUs (mostly 8 bit) are emulated interpretively, but later CPUs such as PowerPC, MIPS, SH4 are JITed.

- Connectix Speed Doubler 1995 dynamically compiled m68k code to PowerPC. Apple's m68k emulator at the time was an interpreter.

- IBM System/38, AS/400 1979. Programs are compiled to "MI" (renamed "TIMI" in AS/400) then compiled on first execution to the actual instruction set of the machine.

[brucehoult]

I have almost zero experience with ARM, but off the top of my head there are several differences between the two which may affect performance:
ARM has three operand instructions - the result can be written to a third register which is neither of the source registers
X86 has two operand instructions - the result typically overwrites one of the inputs
X86 can encode memory load and arithmetic in one instruction, ARM needs two instructions for that
X86 has variable length instructions and some common ones are very short
ARM64 doesn't have variable length instructions and is easier to decode

A lot of these things cancel out. Two instructions vs two uops on a complex instruction makes very little difference. Complex addressing modes might as well be instructions as they need their own "opcode-like" instruction bytes. The only thing complex addressing modes gains you is not having to specifically find and name a temporary register for the thing you loaded from memory -- which admittedly can be important if you only have 8 programmer-visible registers, but doesn't matter at all when you have 32.

Let me just concentrate on x86's variable length instructions, which are not at all designed optimally for code size. Bear in mind there are a total of 256 possible 1-byte opcodes.

- "X86 has variable length instructions and some common ones are very short"

- yes, some of them. But x86_64 got rid of the very handy and commonly-used 1 byte 0x40-0x4f INC/DEC A/C/D/B/SP/BP/SI/DI and makes you use 2 byte add/sub instructions instead

- there are a ton of 1 byte instructions which are 1 byte because, apparently, they only need one opcode, and not because they are common. For example 6 Add-with-carry and 6 subtract-with-borrow instructions. Sure, you need them (kinda) sometimes, but they are very very rare, especially in 32 bit or 64 bit code. AAA, DAA, DAS, XLAT, AAM, AAD, IN, OUT, HLT, LES, LDS (at least LSS, LFS, LGS were hidden away in longer opcodes), CMC, CLC, CLD, CLI, STC, STD, STI.

- many common, or at least not unusual or weird, things are quite long. For example (omitting the return instruction in each case)...

int64_t foo(int64_t n){return n + 128;}

x86_64: 48 8d 87 80 00 00 00    lea    0x80(%rdi),%rax
Aarch64: 91020000    add   x0, x0, #0x80
RV64: 08050513             addi   a0,a0,128
PowerPC64: 38 63 00 80    addi    r3,r3,128

Everything is 4 bytes except x86_64 which is 7 bytes. And it doesn't even use an ADD instruction, but uses an addressing mode instead! If it used an add instruction it would be:

48 89 f8                mov    %rdi,%rax
48 05 80 00 00 00       add    $0x80,%rax

The actual add is a little shorter at 6 bytes, but in total 9 bytes are needed.

As a bonus, 32 bit ARM (Thumb2):

Thumb2:
3080         adds   r0, #128
f141 0100    adc.w   r1, r1, #0

Two instructions and 6 bytes, but it's still shorter than x86_64. If it was a 32 bit value then only the first, 2 byte, instruction would be needed.

I don't know how real world code density (bytes per actual work done) compares between these architectures, and this too can affect performance because of more or less "work" fitting in instruction cache.

Aarch64 and x86_64 tend to overall work out to very very similar overall code size and overall uops executed.

In 64-bit land, RISC-V 64 is the code size winner by quite a long way -- and the number of uops is again very very similar.

Here's a four year old presentation on this subject by Chris Celio who did the OOO RISC-V "BOOM" CPU core and now works designing x86 CPUs at Intel:

The ultimate proof is in the pudding. Which laptop will be the fastest to compile a million lines of C++ code and to do so without catching on fire?

Definitely.

My AS Mac Mini has cleared customs in Auckland and has another 300 km to come. I'll have it in 2-3 days maximum and let everyone know my results building binutils, gcc, llvm firefox and other things like that.

[Berni]

It’s not an emulation. It is directly recompiling the x86 code as ARM at startup. It treats x86 as a compiler IL.

Ah, I missed that, so the "hit" is taken on startup?  - or is it perhaps only done once, and the resulting binary stored for later use?

In order to be faster than an x86 hardware implementation,  the target processor has to be faster at running its transpiled object code than the x86 processor is at running its native code... 

That just doesn't sound right, unless the hardware x86 implementation is really handicapped and unfixable... and nothing is unfixable, given clever enough engineers! 

Its proabobly the same as JIT compiling that has gotten so popular these days in the form of .Net and JavaScript. It crunches a block of code into native machine language and places a jump back into the compiler at the end of it, once the block of code gets to the end the JIT compiler takes over and compiles more of it. So code only gets compiled once its first executed. When it executes again its already native machine code, if it never executes then little/no work is wasted compiling it because it never gets there.

That's not what .NET does. It compiles the entire program from bytecode to native before running any of it.

https://www.geeksforgeeks.org/what-is-just-in-time-jit-compiler-in-dot-net/

The whole thing gets compiled from C#/VB.net/F#...etc text source files into byte code all at once at compile time. Then when the exe is run it pulls in the .net libraries that start JITing the contained bytecode into machine code for the particular platform. Hence there is no long pause at startup of large multi megabyte .net apps.

But there are tools to turn .net code into machine code up front in a additional recompilation step, this is mostly used to get .net code running on platforms that don't support .net framework, mono ..etc

This was already used back in 2000 in the form of so called "High level emulation" where a Nintendo 64 emulator called UltraHLE used this aproach to get a massive performance boost in emulating the console on x86. Back then PCs ware generaly not fast enugh to emulate consoles this powerful, especialy something like the N64 with its very different 64bit instruction set, weird float math, proprietary GPU etc..

There are plenty of examples before that.

- java Hotspot 1999

- VMware 1998. JITed x86 to x86 while replacing non-virtualizable code with safe code. This became unnecessary in 2005 once Intel added VT-x

- MAME 1996. Slow CPUs (mostly 8 bit) are emulated interpretively, but later CPUs such as PowerPC, MIPS, SH4 are JITed.

- Connectix Speed Doubler 1995 dynamically compiled m68k code to PowerPC. Apple's m68k emulator at the time was an interpreter.

- IBM System/38, AS/400 1979. Programs are compiled to "MI" (renamed "TIMI" in AS/400) then compiled on first execution to the actual instruction set of the machine.

Yep i was just using early N64 emulators one example that i could remember off the top of my head, showing a widespread use of JITing a foreign weird instruction set machine code into native machine code with impressive speed. The MAME dev history also shows the bigger 32bit CPUs getting supported around the same time.

Point was about how that this approach to emulation is very much viable and can be impressively fast.

[andersm]

Just a sidenote, but UltraHLE's claim to fame wasn't JIT, but rather detecting and intercepting library calls, and executing native reimplementations instead, like eg. Rosetta.

[DiTBho]

[..]Seymour Cray's CDC 6600 [..] RISC-V is also bringing back some other of Cray's brilliant 1970s ideas in a modern setting.

Which ideas, in details? That's super interesting 

[brucehoult]

[..]Seymour Cray's CDC 6600 [..] RISC-V is also bringing back some other of Cray's brilliant 1970s ideas in a modern setting.

Which ideas, in details? That's super interesting 

For example, two ideas from the Cray 1:

- shortening vector registers for small vectors or the last part of vectors longer than vector registers, avoiding the need for a scalar loop to process the "tail" as well as the main vector loop.

- chaining of operations. RISC-V vector operations don't require a chaining implementation, but they permit one

http://homepages.inf.ed.ac.uk/cgi/rni/comp-arch.pl?Vect/cray1-ch.html,Vect/cray1-ch-f.html,Vect/menu-cr1.html


In addition, while the Cray 1 had vector registers with 64 elements and the program had to know that, with RISC-V vectors the vector registers can be any length, and the program doesn't have to know the length.

For example with the following C code:

Code: [Select]

void saxpy(float s, float *x, float *y, float *res, int len){
  for (int i=0; i<len; ++i)
    res[i] = s * x[i] + y[i];
}

Cray 1 code will look like (each of the functions starting with vec_ is actually an intrinsic function that translates to a single machine code instruction):

Code: [Select]

void saxpy(float s, float *x, float *y, float *res, int len){
  vector v1, v2, v3, v4;
  while (len > 0){
    int vl = len > 64 ? 64 : len;
    vec_set_len(vl);
    vec_load(v1, x);
    vec_load(v2, y);
    vec_mul_scalar(v3, v1, s);
    vec_add(v4, v2, v3);
    vec_store(v4, res);
    x += vl;
    y += vl;
    res += vl;
    len -= vl;
}

RISC-V vector code will look like:

Code: [Select]

void saxpy(float s, float *x, float *y, float *res, int len){
  vector v1, v2, v3, v4;
  while (len > 0){
    int vl = vec_set_len(len);
    vec_load(v1, x);
    vec_load(v2, y);
    vec_mul_scalar(v3, v1, s);
    vec_add(v4, v2, v3);
    vec_store(v4, res);
    x += vl;
    y += vl;
    res += vl;
    len -= vl;
}

The difference is the code for the Cray has to know there are exactly 64 elements in a vector register, but the RISC-V code automatically adjusts to hardware with vector registers holding anywhere from 1 element to 2^31 elements.

[DiTBho]

What is the "Neural Engine" core ? 

[tszaboo]

What is the "Neural Engine" core ? 

Probably a special part of the CPU, doing parallelized FP32 computations. Similar to Google TPU for example.

Third party tests are coming in. https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested
Anandtech sad it's impressive, and they know what they are doing. Particularly impressive this Rosetta2, performance is 70-80% of native. And these are first generation chips, with low power design goals.

[SilverSolder]

"What’s really important for the general public and Apple’s success is the fact that the performance of the M1 doesn’t feel any different than if  you were using a very high-end Intel or AMD CPU. "

The only remaining question is - what are the downsides?  Are there any?

Remember, amateurs worry about how things work...  professionals worry about how they fail! 

[bd139]

"What’s really important for the general public and Apple’s success is the fact that the performance of the M1 doesn’t feel any different than if  you were using a very high-end Intel or AMD CPU. "

The only remaining question is - what are the downsides?  Are there any?

Remember, amateurs worry about how things work...  professionals worry about how they fail! 

Very true and exactly why I'm not buying one.

OCSP validation of binaries meaning rights to run software you purchased can be arbitrarily revoked, Linux probably will never work on the hardware apparently due to signed boot and no documentation so it's landfill immediately when macOS is dumped, half the dev tools out there aren't even compiled for ARM, docker doesn't work, firewall can be entirely bypassed trivially from unprivileged applications and finally the gaping hole I'd leave in my wallet. It might be a decent bit of hardware but the stewardship is border-line incompetent imperialism.

I'll just buy a 5900X and an RTX 3070 for my PC and still be up £100 

Edit: the bit I really can't get over is they're charging £200 here for 8Gb more RAM. I paid £240 for the entire 64Gb of stuff in my desktop...

[borjam]

The only remaining question is - what are the downsides?  Are there any?

Many Mac users work on technical fields, especially on ISPs, and they rely heavily on virtualization of Intel based Linux and FreeBSD systems. Those users also need a lot of memory, more than 16 GB.

So these machines won't work for them.

Remember, amateurs worry about how things work...  professionals worry about how they fail! 

And what do worshippers in denial worry about?

[SilverSolder]

[...]
Many Mac users work on technical fields, especially on ISPs, and they rely heavily on virtualization of Intel based Linux and FreeBSD systems. Those users also need a lot of memory, more than 16 GB.

So these machines won't work for them.

[...]

So, can we think of these things as decently performing but closed appliances, suitable for not-too-challenging applications by "aspirational" users that are not too price sensitive?   

If so, it sounds like Apple's strategy over the last decade or more is still alive and well - and there is no reason why they won't sell like hot cakes?

[Cerebus]

OCSP validation of binaries meaning rights to run software you purchased can be arbitrarily revoked,

It looks like Apple have already quietly indirectly acknowledged that they've fucked up there.

The Silicon Valley titan also said it plans to implement an encrypted protocol for developer ID certificate revocation checks, to take steps to make its servers more resilient, and to provide users with an opt-out mechanism.

The certificate checks are already in place in earlier versions of MacOS. The difference with BigSur, as I understand it, is that there aren't the same options as exist in earlier versions to say, "I know it's not signed to Apple's satisfaction, run it anyway" or those options are harder to access. Although Apple are one of the most arrogant vendors out there, to their credit they've done the turn around bloody quickly.

Linux probably will never work on the hardware apparently due to signed boot and no documentation so it's landfill immediately when macOS is dumped,

Rather early to say that, many versions of Apple hardware have been poorly supported or even unsupported  by Linux when they first came out, and became perfectly tractable a few weeks/months down the line.

[Cerebus]

The only remaining question is - what are the downsides?  Are there any?

Many Mac users work on technical fields, especially on ISPs, and they rely heavily on virtualization of Intel based Linux and FreeBSD systems. Those users also need a lot of memory, more than 16 GB.

So these machines won't work for them.

VMWare have had a preview of Fusion for ARM on the Mac available for a while, they are also running one of their public previews of ESXi for ARM based servers. So it cuts the other way too, an ARM based Mac may become the only option for developers who want to play with virtualised ARM on the desktop to support playing with ARM server farms.

Much of what's being said along the "it won't work" lines about the switch to ARM was being said back in 2006 when Apple did the PPC->Intel switch. I'd wait a bit for the world to catch up before I draw any final conclusions. Apple have done this before with the 2006 PPC->Intel shift, if any vendor knows how to do this without shooting themselves in the foot then it's Apple (assuming that they have a functioning corporate memory).

Don't forget that Apple still have Intel processor based machines in their line up, so for anyone who can't play "wait and see" and needs an Intel processor there's still the 16" laptop with up to 64Gb of memory and all the desktop and deskside systems (up to 28 cores, up to 1.5TB memory).

[Marco]

It looks like Apple have already quietly indirectly acknowledged that they've fucked up there.

OCSP was a giant mistake for the web before stapling, but stapling obviously doesn't work for apps. It will never stop being stupid for apps. There aren't that many app certificates, there's no good reason to pull the revocations. Just push them.

I shouldn't need to opt out of the revocation mechanism to prevent leaking private data, it should just not leak private data.

[bd139]

OCSP validation of binaries meaning rights to run software you purchased can be arbitrarily revoked,

It looks like Apple have already quietly indirectly acknowledged that they've fucked up there.

The Silicon Valley titan also said it plans to implement an encrypted protocol for developer ID certificate revocation checks, to take steps to make its servers more resilient, and to provide users with an opt-out mechanism.

The certificate checks are already in place in earlier versions of MacOS. The difference with BigSur, as I understand it, is that there aren't the same options as exist in earlier versions to say, "I know it's not signed to Apple's satisfaction, run it anyway" or those options are harder to access. Although Apple are one of the most arrogant vendors out there, to their credit they've done the turn around bloody quickly.

Linux probably will never work on the hardware apparently due to signed boot and no documentation so it's landfill immediately when macOS is dumped,

Rather early to say that, many versions of Apple hardware have been poorly supported or even unsupported  by Linux when they first came out, and became perfectly tractable a few weeks/months down the line.

On the last point it used the same signing infrastructure as iOS and we’ve seen how many iOS devices are running Linux kernels. It has got to “why bother” territory now.

It looks like Apple have already quietly indirectly acknowledged that they've fucked up there.

OCSP was a giant mistake for the web before stapling, but stapling obviously doesn't work for apps.

It will never stop being a giant mistake for apps. There aren't that many app certificates, delta CRL would work perfectly fine and not leak your usage data to Apple.

It’s about controlling the App Store so they can pull Epic games type customers off etc. Nothing to do with signing. This is about authorisation not authenticity. If it was about authenticity then a delta CRL would scale fine particularly at the pace and size of macOS updates or interim ones.

[SilverSolder]

It looks like Apple have already quietly indirectly acknowledged that they've fucked up there.

OCSP was a giant mistake for the web before stapling, but stapling obviously doesn't work for apps. It will never stop being stupid for apps. There aren't that many app certificates, there's no good reason to pull the revocations. Just push them.

I shouldn't need to opt out of the revocation mechanism to prevent leaking private data, it should just not leak private data.

What is "stapling" in this context?

[Marco]

It’s about controlling the App Store so they can pull Epic games type customers off etc. Nothing to do with signing. This is about authorisation not authenticity. If it was about authenticity then a delta CRL would scale fine particularly at the pace and size of macOS updates or interim ones.

If they push a revocation for Epic's certificate OSX can still just refuse to run it, push vs pull is irrelevant in that respect.

[Marco]

What is "stapling" in this context?

The website request proof that the certificate is still valid on your behalf and then staples it to the reply of your request to the website, so you're not leaking private usage data all over the place.

[borjam]

So, can we think of these things as decently performing but closed appliances, suitable for not-too-challenging applications by "aspirational" users that are not too price sensitive?   

If so, it sounds like Apple's strategy over the last decade or more is still alive and well - and there is no reason why they won't sell like hot cakes?

The Apple obsession thinking that all Apple products are iPhones is really too much.

No, it's an outstanding portable Unix workstation with the development toolchain and what you would expect on a Unix system (unless you are a lobotomized systemd worshipper) that happens not to run amd64 software. So in a way it's coming back in time to the old days when the workstationn processors were not Intel nor they were capable of running Windows.

[borjam]

VMWare have had a preview of Fusion for ARM on the Mac available for a while, they are also running one of their public previews of ESXi for ARM based servers. So it cuts the other way too, an ARM based Mac may become the only option for developers who want to play with virtualised ARM on the desktop to support playing with ARM server farms.

Much of what's being said along the "it won't work" lines about the switch to ARM was being said back in 2006 when Apple did the PPC->Intel switch. I'd wait a bit for the world to catch up before I draw any final conclusions. Apple have done this before with the 2006 PPC->Intel shift, if any vendor knows how to do this without shooting themselves in the foot then it's Apple (assuming that they have a functioning corporate memory).

Don't forget that Apple still have Intel processor based machines in their line up, so for anyone who can't play "wait and see" and needs an Intel processor there's still the 16" laptop with up to 64Gb of memory and all the desktop and deskside systems (up to 28 cores, up to 1.5TB memory).

Don't point fingers at me, I am likely to get one!

I very seldom use virtualization, running stuff on my servers instead. So the lack of Intel virtualization is not an issue for me.

I was just pointing out that many of my friends are skeptical because of that single issue. Maybe in a month or two we will see a workable
Intel virtualization solution based on emulation? Certainly it won't run games but it will be enough when testing server software, the purpose for which lots of developers run FreeBSD or Linux virtual machines.

[Cerebus]

It looks like Apple have already quietly indirectly acknowledged that they've fucked up there.

OCSP was a giant mistake for the web before stapling, but stapling obviously doesn't work for apps. It will never stop being stupid for apps. There aren't that many app certificates, there's no good reason to pull the revocations. Just push them.

I shouldn't need to opt out of the revocation mechanism to prevent leaking private data, it should just not leak private data.

Yeah, OCSP was created to solve the CRL problem - which is only really a problem when there are many different roots of trust you need to get CRLs from. In Apple's ecosystem Apple are a single source of trust for signing certificates, so a CRL, delivered incrementally, would have worked fine.

[Cerebus]

VMWare have had a preview of Fusion for ARM on the Mac available for a while, they are also running one of their public previews of ESXi for ARM based servers. So it cuts the other way too, an ARM based Mac may become the only option for developers who want to play with virtualised ARM on the desktop to support playing with ARM server farms.

Much of what's being said along the "it won't work" lines about the switch to ARM was being said back in 2006 when Apple did the PPC->Intel switch. I'd wait a bit for the world to catch up before I draw any final conclusions. Apple have done this before with the 2006 PPC->Intel shift, if any vendor knows how to do this without shooting themselves in the foot then it's Apple (assuming that they have a functioning corporate memory).

Don't forget that Apple still have Intel processor based machines in their line up, so for anyone who can't play "wait and see" and needs an Intel processor there's still the 16" laptop with up to 64Gb of memory and all the desktop and deskside systems (up to 28 cores, up to 1.5TB memory).

Don't point fingers at me, I am likely to get one!

Why do you think fingers are being pointed? It's just continuing the discussion.

[borjam]

Why do you think fingers are being pointed? It's just continuing the discussion.

Sorry, poorly worded I wanted to point out that I wasn't claiming "it doesn't work, won't work", but for now it is not ideal for certain users.

[SilverSolder]

[...]
I very seldom use virtualization, running stuff on my servers instead. So the lack of Intel virtualization is not an issue for me.
[...]

I wouldn't be able to fit all the computers in the space available if I had to go physical with all my VMs -  They really are God's gift to computing kind, in my view!