Memory management bug in Intel CPUs threatens massive performance hits.

[IanMacdonald]

Probably the best answer to this is to do all browsing in a virtual machine.

That said, the real issue here is that this has existed since 1995 and no security expert noticed it until now.  During that time it may well have been exploited by bad guys. There is no way of telling if it has.

When you think about it, if you forget to lock your house or car, the security issue doesn't arise when you realise you did so.  It arises when you walk away without locking it. Likewise, assuming this vuln hasn't been exploited in over 20 years of the computer being 'left unlocked' is naive. This is the fallacy behind the idea that patching and updating makes a computer secure. It is only marginally better than a placebo.

Especially as there are thousands of similar vulns in all operating systems, still unpatched.

Linux and MacOS are in principle no better then Windows in this respect, since they are all based on C, whose buffer overflow risk is the No1 cause of vulns. What is really needed is a completely new OS that ditches all of this bad code. 

The decision to use C instead of Pascal for the IBM PC, must rank as the single worst decision in the entire history of computing. That said, the really inexcusable thing is that C was allowed to become entrenched in the IT industry even after its security flaws became apparent. So deeply entrenched that it's now going to take the IT equivalent of D-Day to oust it. 

[bd139]

I agree with C being a big problem but it's not the root of all evil here. We also don't use C for anything at all these days. It's python, C#, Java and Go. Go is looking like a good replacement for C in a lot of areas.

However none of the above languages prevent this issue. Also a VM won't help you at all here. And people did notice problems in this space as far back as 2005/7.

There are three travesties here:

1. We've allowed the glorified 8080 to progress as long as it has, tacking more bits to it, shovelling more turds onto the pile and hiding a RISC processor and all sorts of "go faster" hacks underneath it.
2. We've blindly used bits of computer science from the 1960's without formal verification on modern architectures. Turns out with all the hacks above, the output is not as deterministic as people were hoping.
3. There is actually no need for processors as fast as we have to get work done. We've been burning the "it's cheaper to make it fast later" paradigm candle.

[paulca]

3. There is actually no need for processors as fast as we have to get work done. We've been burning the "it's cheaper to make it fast later" paradigm candle.

But you favor virtual machine based languages over C?

I'm a bit confused by the aggression against C.  Buffer overrun is not a C issue, it's a CPU issue.  If you ask the CPU to read memory and that memory is mapped it will.  Having languages/VM boundary check all your array accesses would have significant impact on performance and trash various coding styles with dynamic unbound arrays.

Bad code is bad code, moving it between languages is just like using public toilet paper, it just moves the problem around.

Pascal?  Seriously?  Look up what pascal was intended for.  Teaching students.  Yet oddly it's one of the weirder of the languages.  Only slightly better than BASIC in a lot of regards. Thankfully I haven't touched Pascal since the Amiga in the 90s.

[bd139]

I favour virtual machine based languages over C because it's down to the implementation of the language to do the optimisations rather than the end user and it doesn't require recompilation then to add and remove instrumentation or optimisations.

Buffer overrun is a C issue because C has a virtual machine model as well (stack frames + heap) and offers no compile or execution time guarantees at all.

Pascal was rather nice, particularly in a p-code virtual machine environment. I have seen programs written on one architecture running on another before without modification and this was in the late 1980s. Then there was Turbo Pascal which paired with DESQview turned your little 386 into a stupidly reliable multitasking workstation.

Ada is another good one. Did a bit of that on PPC in the late 1990s. Absolutely bomb proof, apart from when it was actually used in a bomb and blew up.

[GeorgeOfTheJungle]

I'll update my browsers, but on that front I'm angry.  I have warned people for years to keep the damn script kiddies at bay.  Now we even have Javascript on servers (NodeJS) FFS and Javascript with memory access.  WTF?  Idiots.  One of the worst, most annoying languages ever written.  It should have remained a junk interrupted, sandboxed, mickey mouse script tool for making text flash and modifying HTML.

Sorry, I like C as much as JS, both have their things, but which language hasn't?

Code: [Select]

((a,b)=>{[22,24,25].forEach((i)=>b+=a[i]+' '),alert(b+=a[30].split('').reverse().join(''))})("I have warned people for years to keep the damn script kiddies at bay.  Now we even have Javascript on servers (NodeJS) FFS and Javascript with memory access.  WTF?".replace(/[(.?.)]/g,'').split(' '),'')


[bd139]

Sorry, I like C as much as JS, both have their things, but which language hasn't?

Common LISP

[GeorgeOfTheJungle]

Dear applications programmers, systems programming is none of your business, so stop disparaging C you silly script kiddies. 

--
Linus T.

[paulca]

Buffer overrun is a C issue because C has a virtual machine model as well (stack frames + heap) and offers no compile or execution time guarantees at all.

Done much assembler?  C does this for a reason.  If you write any complex "functions" in asm you will find yourself using self rolled stack frames.  Though I agree that is some ways you are forced to do things how the compilers wants and have little to no control over that.

Heap is a C++ thing.  C you are on your own with malloc et al.  C++ has the whole botch local, reference, pointer, heap (new/delete construct/destruct) bollox that causes endless confusion and bugs, 16 different ways to swing the cat so nobody understands each others code.

All this said when you move to higher level languages you don't lose the baggage, you just add more on top and lose performance and control.

[Marco]

systems programming is none of your business, so stop disparaging C you silly script kiddies. 

Once you learn to completely avoid buffer overflows and use after free, so never.

[bd139]

Buffer overrun is a C issue because C has a virtual machine model as well (stack frames + heap) and offers no compile or execution time guarantees at all.

Done much assembler?  C does this for a reason.  If you write any complex "functions" in asm you will find yourself using self rolled stack frames.  Though I agree that is some ways you are forced to do things how the compilers wants and have little to no control over that.

Yes. I've written a couple of non trivial compilers as well. The problem is that the hardware provides no isolation guarantees between stack frames. There are some admittedly clever things like stack canaries but these aren't comprehensive

Heap is a C++ thing.  C you are on your own with malloc et al.  C++ has the whole botch local, reference, pointer, heap (new/delete construct/destruct) bollox that causes endless confusion and bugs, 16 different ways to swing the cat so nobody understands each others code.

All this said when you move to higher level languages you don't lose the baggage, you just add more on top and lose performance and control.

malloc/free is just an abstraction over a heap. I have written allocators. Agree with C++. The problem with C++ is complexity.

Not quite true with higher level languages. Your job is to solve something in the problem domain, not incur problems outside of the problem domain. 99.9% of all problems don't need a system programming language and 80% of the problems in the system programming domain don't need low level languages. Literally the only two things that need direct memory access not via an abstraction are talking to hardware and any IO buffering etc.

Programmers have lost the right to manage their own memory at this point.

All of the above is solved easily (common lisp etc) but there's a lot of investment in a shitty way to do stuff that people don't want to burn or admit was wrong.

[paulca]

Programmers have lost the right to manage their own memory at this point.

That's a slippery slope though.  Just look at Enterprise Java.  I get angry everyday about over engineering.  Object models of data structures and ORM frameworks, abstract, parallel builder patterns and the whole works.  When you cancel it all out against each other the net is 0.  0 fucking point.  They argue that it abstracts complexity into frameworks, but those frameworks don't get included in their complexity calculations...  Again public toilet toilet roll, it just moves the turd around.  Solving complexity with complexity is the Emperors new clothes of Object Orientation which has gone way too far IMHO that it's getting to ridiculous extremes.

The relevance I want to draw out, is that while you might be right that most programmers should stay in higher levels, they should be absolutely FORCED to spend some time doing low level C, Asm etc.  So they at least know what the computer is doing underneath.  Giving juniors high level languages from the get go and not providing them with any low level experience leads to extremely inefficient code.

I have seen people importing a mulit-megabyte jar in Java, rewriting a bunch of classes to support the framework in that jar all just to use a single string formatting function for dates!  Utter madness.... and they still got the fecking timezones wrong and it made it to production because nobody ever tested it in another timezone!

More related to electronics, I watched a video the other night that measured the "digitalWrite()" function in the Arduino libs taking approximately 179 instruction cycles!  Using it is fine, as long as you understand the cost.

[bd139]

No one said enterprise Java. That's it's own special turd

Agree with forcing people to do low level stuff as well. I think you should start at the bottom and work your way up. Step 1: here's a resistor ... Step 50: here's an AbstractBeanFactory.

Don't talk to me about time zones. We can't get anyone who knows their shit on that front. It's impossible. It's quite a difficult subject. I dealt with an event tracking system a while ago which used date spans. Turns out some clever fucker stored the start date in UTC and end date in local time. Bring on DST, all the events dropped an hour in length. CHAOS this caused. That's the sort of shit I get paid to fix.

Arduino is horrid. I use neat AVR-GCC for that banana and use the arduino as a dev board.

[dmills]

I would add to the low level direct memory pile anything needing DMA, anything with short deadline real time constraints, anything running without an OS where the peripherals are registers mapped into the memory map, and anything that needs to be deterministic (To the point of running from non cachable memory sometimes!).

C & C++ undoubtedly get used way outside the appropriate application domains, with C++ adding the fun of a leaky, fragile and complex set of abstractions, but actually there is a reason people still write in those languages that goes way past inertia. And some of us just like a language that (like Latin) stays still for years at a time.

If you are doing systems on small cores (Maybe a dozen kB of flash and a few kB of RAM), as I see it your choices are C and assembler, with maybe a very stripped down C++ as a third contender, what else is there that will actually let me deal with the memory mapped peripherals in a sane way?

Go and Rust will be taken seriously when there is a defined language that does not change every other month, and when there are a few different compilers implementing that defined language (Also a defined ABI for the common platforms would be nice).

Incidentally, GCs are right out! Reference count if you must, but garbage collection does bad things to realtime code (Yes I know bounded GCs exist, comment still usually stands with real implementations).
Personally, I favour the old embedded guys approach, figure out how many of what size at compile time and statically allocate the lot! You can still run off the end, but there will be no 'use after free' if you never free anything!

For date and time stuff, "Calendrical Calculations" is still my goto reference, but mixing UTC and well **Anything else** is just always going to be a source of pain.

Regards, Dan.

[bd139]

Can't disagree with anything there

[paulca]

Incidentally, GCs are right out! Reference count if you must, but garbage collection does bad things to realtime code (Yes I know bounded GCs exist, comment still usually stands with real implementations).
Personally, I favour the old embedded guys approach, figure out how many of what size at compile time and statically allocate the lot! You can still run off the end, but there will be no 'use after free' if you never free anything!

In the stock exchange (order entry gateways where customers would complain if we breached 100uS wire to wire) we would pre-allocate everything.  We are talking about 60Gb of pre-allocation.  C++ abstractions where banned in most places.  Even function calls where frown upon in some places.  Loops where a matter of "find someone elses and add you code there".

There was a legacy Java version and while we measured our latency in microseconds (which I achieved a sub micro-second "New Order" message I was proud of), the Java guys measured in milliseconds.  We were averaging 50-80 micros they were averaging 10-20 millis.  But we had few outlyers and those caused by TCP reorders and stuff like Nagles not being configured.  Java had them all over the place measuring up to 1000 miliseconds while it resized a hash map or did some large scale garbage collection.

[bd139]

Why the hell you'd do that with Java I don't know. That's definitely real time territory.

Our guarantees are merely 100ms RTT 95th percentile. Than again most of our messages are fucking XML so that has it's own can of worms 

[paulca]

Why the hell you'd do that with Java I don't know. That's definitely real time territory.

Our guarantees are merely 100ms RTT 95th percentile. Than again most of our messages are fucking XML so that has it's own can of worms 

We had a guy show up to an interview from the Java space.  He told us a story about his previous project and said it was an XML message gateway.  We asked how many messages per second.  He looked at us uncertainly and said, about 40 a minute with a questioning inclination.

We smiled affectionately and said, "We do 20,000 per second per session and we support 20 sessions per gateway." 

His mouth dropped.

[bd139]

We interview them occasionally as well. They're the sort of people who you ask what a profiler is and they think it's an attachment for their clippers.

Our front office stuff handles about 20,000 requests a second. And that's all shitty code written by the lowest bidder with layers and layers of accumulated crap built over 20 years. Fun fun fun. Thank god I only do messaging. We had one guy almost crying because he found latency on one page was due to 22 SQL joins with an aggregate of about 50 gig of data on one endpoint 

[paulca]

SQL

Standard Quota of Latency

[cdev]

How do you optimize performance on servers, especially when it comes to non-uniform memory access? (NUMA)

[bd139]

SQL

Standard Quota of Latency

Yep that’s about it.

Or Shit Queries Lock which is fixed by Dickheads Bearing Attitudes.

Today’s exploration is Erlang and Riak which is fun. That goes like the clappers. Learning me some erlang for great good.

How do you optimize performance on servers, especially when it comes to non-uniform memory access? (NUMA)

One avoids the fuck out of NUMA architectures if you can.

[dmills]

And failing that, profile the hell out of it **ON THE EXACT HARDWARE AND OS AND OTHER CODE YOU WILL BE USING!**

Regards, Dan.

[Old Don]

Warning Will Robertson, your self driving car is equipped with Takata air bags and Intel microprocessors. What could go wrong!   

[Bicurico]

I have not read this whole thread, so excuse me if this thought has come up already, I think it hasn't.

I have a major question:

How much is this really a "bug" and Intel's fault?

I have read the technical explanation on the Raspberry Pi website (very well done) and to me it seems that the problem is not a "fault" but a "consequence" of trying to implement predicted branching, out of order processing, etc.

One could decide for security and switch it off, at the cost of less performance.

But it seems to me like with a car: you can turn off all driving assistance features and the car drives faster, doesn't reduce power on drifts, etc., but at the expense of less security.

How is Intel to blame for this?

I am not arguing in favour of Intel (or other chip manufacturers), I am just asking.

It would be a bug, if after a predictive branching you could issue 3 consecutive NOP commands that would dump memory pages to user space. That would be a bug.

But programming a routine that deliberatly will fool the predictive branching, making it wrong to then allocate the previously used memory blocks of discarded operations and read them out, does not necessarily seem like a bug to me?

Also, how probable is it to actually get to a memory page that contains usefull data? How would you then analyse binary data? Like getting the page that contains half the cryto string? How would you know which part it is? This is something I cannot visualize in my mind. Would be nice if someone could explain how these new "bugs" could in PRACTICE be expoited.

Mind me, I am not a low level programmer, nor am I too familiar with CPU architecture, hence why I have this question.

Also, how could you avoid this kind of attacks and still have all these speed optimizations?

Thanks,
Vitor

[paulca]

How much is this really a "bug" and Intel's fault?

Simply because address space isolation is the bastion of multi-program, multi-user systems for the past 30 years.  It is meant to be absolutely impossibly for one program to read another's private memory.  The processor's architecture is meant to prevent this at a hardware level.  The bug means there is a way past this.