Future Software Should Be Memory Safe, says the White House

[SiliconWizard]

https://www.whitehouse.gov/oncd/briefing-room/2024/02/26/press-release-technical-report/

[DiTBho]

these days, Linus is discussing two things, well three things ...

• more Rust support for more drivers written in Rust
• micro kernel approach for linux, with a userspace scheduler already written as proof of concept
• development languages for future kernels must be memory safe, because with multi core and modern features we have already reached the point of no return in term of kernel complexity

(
oh, and from how some have responded - my speculation - it also seems that many Linux developers ...
... have now reached retirement age, so they would like to make way for younger people
mumble ... 
)

[tggzzz]

"Science Engineering progresses one funeral at a time".
https://en.wikipedia.org/wiki/Planck%27s_principle

[newbrain]

• micro kernel approach for linux, with a userspace scheduler already written as proof of concept

Wow, I did not see this coming.
So Tanenbaum had a point, in the end (others were proven wrong by history).

It won't be an easy transition, quite the radical architectural change.

DiTBho, do you have some good sources? DDG only return stuff about the original T-T debate...

[DiTBho]

February 2024, Andrea Righi has started a blog "Writing a Scheduler for Linux in Rust that runs in user-space" series on writing a user-space CPU scheduler with the BPF-based extensible scheduler class:

".select_cpu() implements the logic to assign a target CPU to a task that wants to run, typically you have to decide if you want to keep the task on the same CPU or if it needs to be migrated to a different one, for example if the current CPU is busy; if you can find an idle CPU at this stage there's no reason to call the scheduler, the task can be immediately dispatched here.

see here 

[coppice]

I wish people wouldn't bullshit and call these things memory safe languages. Memory less dangerous is all you get. As usual names are not chosen to illuminate, but to obscure.

[dferyance]

Memory safety is very important and is good to have efforts in that direction. I can write memory safe code in C++. It isn't terribly difficult in most cases. However, I usually work in a team and anyone (or myself) can easily break that. Everyone starts somewhere and not everyone understands how to write safe code. I like having tooling check things for me. I like static typing for that reason and memory usage checking is helpful.

I wish the solution wasn't rust though. Rust isn't a standardized language unlike C++, ECMAScript, C# and others. Rust has made breaking changes. While there is work on gcc-rust, practical rust usage is limited to clang. I wish there was more of rust encouraging other languages borrowing from it and less of, let's all use rust.

[ejeffrey]

I'm most excited about Herb Sutter's cppfront/cpp2.  In some ways it's a long way from the sort of "production readiness" as languages like rust -- I think classes were added just last year -- but in other ways it's decades ahead  because it essentially is C++, 100% compatible warts and all with existing C/C++. You can even  mix cpp1 and cpp2 syntax in a single source file, and it compiles down to regular C++ that any standards compliant C++ compiler can compile so it has the same platform support as gcc+clang+msvc.

Instead of making a completely new language it tries to make all the behaviors that we teach people "this is how to write C++ safely" the default.  In doing so it makes the syntax a lot more concise because you don't have to specify the non default behaviors you almost always should want.

It also compiles to C++ pretty much in the way you would expect -- it doesn't introduce it's own runtime or standard libraries and it doesn't wrap things in opaque containers.  This means it's also a relatively safe project to play with.  You can always just abandon the cppfront compiler and use the generated code with minimal to no cleanup.

Cpp2 doesn't have anywhere near the adoption of even rust but I hope people look at it when considering language options especially if they have a body of existing C++ to keep.

[ejeffrey]

I wish the solution wasn't rust though. Rust isn't a standardized language unlike C++, ECMAScript, C# and others. Rust has made breaking changes.

It's not even just that.  C++ has made breaking changes as well.  But it's done through proposals that are published, vetted, tested against existing code bases to see how widespread actual breakage is, and in the end you can give your compiler of choice the --std=c++11 flag or whatever you need.  This is a huge development cost on both the C++ language development and the compiler / standard library developers, and it's a cost rust doesn't want to incurr which is understandable.  But it is a real issue and people pretending like it's no big deal are super annoying.

I actually like rust.  I think it's a pretty interesting language, but the "my way or the highway attitude" that seems to emanate both from their development strategy and a lot of the promoters is pretty off-putting. 

[DiTBho]

alias(memory safe, less dangerous): technically and politically incorrect, but encouraging, and that's  what we need

to give an example, it's like claiming that "homo sapiens comes from monkeys" no one shouts "bullshit", even if scientifically speaking it's bullshit.

[coppice]

alias(memory safe, less dangerous): technically and politically incorrect, but encouraging, and that's  what we need

to give an example, it's like claiming that "homo sapiens comes from monkeys" no one shouts "bullshit", even if scientifically speaking it's bullshit.

"Memory safe language" makes people think its a solution to a problem. These languages may be a part of a solution, but claiming they are a solution is serious bullshit, that should get people thrown out of decent society,

[tggzzz]

alias(memory safe, less dangerous): technically and politically incorrect, but encouraging, and that's  what we need

to give an example, it's like claiming that "homo sapiens comes from monkeys" no one shouts "bullshit", even if scientifically speaking it's bullshit.

"Memory safe language" makes people think its a solution to a problem. These languages may be a part of a solution, but claiming they are a solution is serious bullshit, that should get people thrown out of decent society,

They are a solution to part of the problem. An important part of the problem.

The same is true of seatbelts and road safety. And speed limits and road safety.

Putting a big spike in the steering wheel is not a practical solution to road safety, any more than requiring "correct" usage of many of C's features is a practical solution for safe memory use.

[Marco]

It won't be an easy transition, quite the radical architectural change.

There's a lot of work being done lately to do it transparently. See :
Intra-Unikernel Isolation with Intel Memory Protection Keys
Preventing Kernel Hacks with HAKC
etc.

By limiting what parts of the kernel is callable/accessible per kernel module, you are essentially turning it into a microkernel ... but the existing code of the kernel modules stays the same.

PS. I think AMD supports memory protection keys for zen3 and up.

[ejeffrey]

"Memory safe language" makes people think its a solution to a problem. These languages may be a part of a solution, but claiming they are a solution is serious bullshit, that should get people thrown out of decent society,

It's much less BS that what you are saying.  Memory safety (which is well described and explained, even if it comes in several levels of strictness) is absolutely the solution to several problems.  It's not the solution to every problem, but that is an insane standard and not really an argument worth discussing.

Every year, a large number of exploited security problems in the wild are traced to use after free and out of bounds access.  And it's not like it's only "old" code, it happens in brand new code as well.  Both of those can 100% be eliminated by memory safe languages, although bounds checking adds runtime cost.  Yes, it sometimes makes sense to opt-out of those protections, and every memory safe language provides some way to over-ride the behavior, but it's almost never needed.  Use before initialization is another issue that can be eliminated as well. "Initialized" obviously doesn't mean "to a useful value", but in the context of security, avoiding allowing uninitialized pointers or capturing potentially sensitive data from previously deallocated objects solve a lot of real world security problems even when the program is going to malfunction either way.

I think a big part of the reaction against memory safety came from Java and how it was promoted, especially back in the 90s.  Java was heavily promoted as using managed memory and garbage collection to "prevent memory leaks."  It turns out that with nearly 30 years of hindsight memory leaks are among the least bad forms of memory errors.  It made sense at the time, programs and datasets were rapidly growing larger and more complex.  Getting things to work at all was a big deal.  A memory leak could make a program crash on a large data set.  Focusing on memory leaks was focusing on making it easier to get a program to work properly, but the focus on security now is much more on preventing a program from being made to work improperly.  The most important and useful properties of Java's memory management are not actually garbage collection per-se, but initialization safety, bounds checking, and use-after-free safety.

The other problem with Java was that people didn't really know how to use it properly.  While gargbage collection was well established in languages like LISP, the tools and practices to use it properly were not well known among Java's target audience.  If you tell 1990s C/C++ developers they can write something that looks basically like C++ and just not bother deleting things, you are going to get memory leaks galore. People had to be taught, and libraries had to be developed that made correct use of weak references.  In this way, more modern alternatives like C++ unique pointers and the rust borrow checker are IMO much better default options than garbage collection with much better semantics with respect to object destruction.  They don't prevent memory leaks -- a circular path of unique pointers will never be destroyed, but that's much less of a problem than use after free.  On the other hand, unique pointers guarantee timely destruction for the normal case rather than "whenever the GC gets run".

[coppice]

alias(memory safe, less dangerous): technically and politically incorrect, but encouraging, and that's  what we need

to give an example, it's like claiming that "homo sapiens comes from monkeys" no one shouts "bullshit", even if scientifically speaking it's bullshit.

"Memory safe language" makes people think its a solution to a problem. These languages may be a part of a solution, but claiming they are a solution is serious bullshit, that should get people thrown out of decent society,

They are a solution to part of the problem. An important part of the problem.

The same is true of seatbelts and road safety. And speed limits and road safety.

Putting a big spike in the steering wheel is not a practical solution to road safety, any more than requiring "correct" usage of many of C's features is a practical solution for safe memory use.

Since people like Michael Jackson in the 1970s until today hardly anyone has pushed their latest new software thing as something which helps a bit in putting solid applications together. Its always pushed as a magic bullet. What wrong with names that reflect reality, and set reasonable expectations? Maybe people would get less disillusioned when they find the real strengths and weaknesses of the new thing.

[ejeffrey]

Since people like Michael Jackson in the 1970s until today hardly anyone has pushed their latest new software thing as something which helps a bit in putting solid applications together. Its always pushed as a magic bullet. What wrong with names that reflect reality, and set reasonable expectations? Maybe people would get less disillusioned when they find the real strengths and weaknesses of the new thing.

You can read the linked article.  And you can read the 20 page report linked from it.  And you can read the couple dozen references at the end of that report.  Or you can read academic literature on the subject or you can read programing guides on languages.  But it's actually you who is over-simplifying things and complaining without providing any nuance or context.

Memory safety is a well established and broad term of art.  It's well understood what it means, and it includes not just features of current languages, but current and future research to improve safety further.  Your complaint is just stupid.

The only people saying "XYZ language claims to fix all errors" are people arguing for attaching spikes to their steering wheel.  By the way, the same thing happened with car safety.  People railed against seat belts and airbags because they couldn't "make driving safe."  But by and large, everyone except a few whiners understood what was going on, the nomenclature was not actually confusing.

[tggzzz]

alias(memory safe, less dangerous): technically and politically incorrect, but encouraging, and that's  what we need

to give an example, it's like claiming that "homo sapiens comes from monkeys" no one shouts "bullshit", even if scientifically speaking it's bullshit.

"Memory safe language" makes people think its a solution to a problem. These languages may be a part of a solution, but claiming they are a solution is serious bullshit, that should get people thrown out of decent society,

They are a solution to part of the problem. An important part of the problem.

The same is true of seatbelts and road safety. And speed limits and road safety.

Putting a big spike in the steering wheel is not a practical solution to road safety, any more than requiring "correct" usage of many of C's features is a practical solution for safe memory use.

Since people like Michael Jackson in the 1970s until today hardly anyone has pushed their latest new software thing as something which helps a bit in putting solid applications together. Its always pushed as a magic bullet. What wrong with names that reflect reality, and set reasonable expectations? Maybe people would get less disillusioned when they find the real strengths and weaknesses of the new thing.

A large part of my professional career, from the late 70s onwards, involved assessing and - where appropriate - using and creating all sorts of new technologies.

If you look for and understand the fundamental properties and possibilities (i.e. not just the surface glitz), then it becomes relatively easy to spot when "new" is merely "different variation" rather than "better". Thus Delphi was the same as C, C# is the same as Java, 6800 is the same as Z80, xCORE is the same as Transputer, xC is the same as Occam, Objective-C is Smalltalk without GC and reflection, C++ is a mess. As an example of that lack of change, I was horrified at how few changes there had been between 2015 (when I re-started playing with embedded systems) and the early 80s (when I started). It was still C on 8-bit processors cross-compiled on UNIX and downloaded for debugging. The only difference was smaller, faster, cheaper.

Another thing to watch out for is significantly different technologies that aren't sufficiently practical outside a laboratory/academia. Examples have been formal methods, Erlang, the various ML/Haskell/etc languages. Interesting, but not worth spending too much time on.

Nonetheless, I've kept looking out for technologies that offer significant advantageous changes  - and jumped on them when they occurred. Major examples have been Smalltalk=>Objective C=>Java, and discrete logic=>semi-custom ICs=>PLAs=>FPGAs. And, of course, for hard realtime multicore embedded, C/Transputer/Occam=>xCore/xC.

I've had my eye on a few new languages for the last decade, with Rust and Go at the top of the list. They are both clean and tasteful,  and - within separate areas - potentially better than their alternatives. It is wryly pleasing to see that Rust has gathered a significant following and that it will probably, over the coming decades, relegate C/C++ to the level of COBOL. Maybe I will get the chance to use Rust, maybe not.

TL;DR: be very suspicious of the technology-du-jour, but be receptive to clean technologies that remove fundamental limitations of existing mainstream technologies.

[SiliconWizard]

these days, Linus is discussing two things, well three things ...

• more Rust support for more drivers written in Rust
• micro kernel approach for linux, with a userspace scheduler already written as proof of concept
• development languages for future kernels must be memory safe, because with multi core and modern features we have already reached the point of no return in term of kernel complexity

For the micro-kernel approach, that would make it a completely different kernel. At this point, I fail to see how and why it could still be called Linux, and how this transition could be done while being able to keep most of existing code (otherwise, what's even the point of calling it Linux and doing this under the Linux umbrella? Probably just to take advantage of the traction and the money the Linux foundation makes? Around $260 millions a year.)

No doubt a micro-kernel approach would be better on many levels, but also with the downsides we all know and that Linus himself has advocated against for several decades.

As to Rust, I consider it a trojan horse here, and I'm being serious.

(
oh, and from how some have responded - my speculation - it also seems that many Linux developers ...
... have now reached retirement age, so they would like to make way for younger people
mumble ... 
)

Yes, that is certainly a problem. And most of these younger developers, with few exceptions, don't want or just plain can't write proper C. So, there is that.
I said that a little while ago - when I heard one of Linus's latest talks, it definitely sounded very different from the usual Linus and it looked as though he was preparing his retirement, for sure.

Speaking of the LInux foundation, I'm starting to have a problem with it, but that's just like with almost all of these big foundations around large open-source projects.
Just look at who are the main contributors, and how its expenditures are distributed. The Linux kernel only accounts for about 2%.
Anyway.

Maybe time to think about starting alternatives.

[artag]

Future laws should be just, fair and free from loopholes too.
But I'm not holding my breath.

[JPortici]

And most of these younger developers, with few exceptions, don't want or just plain can't write proper C. So, there is that.

starting next week we're having a new intern for a couple of months. still in high scool, part of the curricula
he has an interest in embedded so apparently has been using C. All programmers looking for work i had the misfortune to interact with were adamant in thinking that frontend was the only true programming (as if the world needed more of them)
My father knows next nothing about programming (he's a chemist) besides what colleagues of him use and he's always fascinated by the fact that i write C for a living, he always thinks we would have moved on ages ago. And yet..

I don't know if rust is going to be "it", but it certainly looks like it has an appeal to the younger crowd (though i'm almost 33, i don't think myself as an old developer) and i also picked that vibe you mention from the recent linus talks.
Though i looked at the language a while ago and honestly, if i still have to go the unsafe route to do any peripheral access might as well keep writing C...

[ejeffrey]

• micro kernel approach for linux, with a userspace scheduler already written as proof of concept

Wow, I did not see this coming.
So Tanenbaum had a point, in the end (others were proven wrong by history).

Sort of, but also computers have changed a lot in 30 years.  SMP went from the domain of high end servers and supercomputers to universal, memory hierarchies got deeper and less uniform.  Even when Linux first supported SMP with the big kernel lock, lock contention in the kernel wasn't a huge deal, as long as kernel overhead overall was low.  Concurrency was really more important in userspace.  At the same time, pipelines have gotten deeper, and reorder buffers larger.  A few extra branch free instructions can literally be free (at least in time if not power) if the CPU would just be waiting for a load anyway.

One thing that hasn't changed is system call overhead.  A context switch or privilege level change still takes about 1 microsecond, only now it's two orders of magnitude more foregone computation.  A microkernel that increased context switches would be even worse than when Linus and Tanenbaum argued about it.

However, developments in lock free and wait free data structures, better locking algorithms, and the ability to have multiple cores running at the same time make it possible to have message passing between tasks with no context switches -- something that was impossible on the uniprocessor systems of the 90s.  Sandboxing, virtualization, and run-time compilation has dramatically changed both in hardware and software.  This allows isolation of code similar to independent tasks without the cost of context switches at all

And it doesn't need to be a radical change or re-architecture.  It's honestly been going on for years.  My desktop has 400+ kernel threads running right now.  Pulling individual components partly into user space doesn't necessarily make a huge radical change in the kernel itself, even if that is something as core as the scheduler.  The Linux kernel has *always* strived to be modular, and core components like schedulers, memory allocation, and IO have been refactored or rewritten multiple times.

[ejeffrey]

Though i looked at the language a while ago and honestly, if i still have to go the unsafe route to do any peripheral access might as well keep writing C...

That's a pretty silly argument even for embedded development.  On all but the most trivial programs (which: who cares what you use), only a tiny fraction of the code is actually performing the register read/writes.  I'm not completely thrilled with the way rust works on embedded, or the way unsafe works in rust, but most code even in embedded could be safer.

[coppice]

Though i looked at the language a while ago and honestly, if i still have to go the unsafe route to do any peripheral access might as well keep writing C...

That's a pretty silly argument even for embedded development.  On all but the most trivial programs (which: who cares what you use), only a tiny fraction of the code is actually performing the register read/writes.  I'm not completely thrilled with the way rust works on embedded, or the way unsafe works in rust, but most code even in embedded could be safer.

It depends what you are programming. A lot of small embedded processors are doing endless manipulation of peripherals. However, what you say is true for most larger embedded programs. You really ought to make life as easy as possible on the bulk of the code, so you have more time to put more focus of the gritty stuff.

[tggzzz]

Though i looked at the language a while ago and honestly, if i still have to go the unsafe route to do any peripheral access might as well keep writing C...

That's a pretty silly argument even for embedded development.  On all but the most trivial programs (which: who cares what you use), only a tiny fraction of the code is actually performing the register read/writes.  I'm not completely thrilled with the way rust works on embedded, or the way unsafe works in rust, but most code even in embedded could be safer.

It depends what you are programming. A lot of small embedded processors are doing endless manipulation of peripherals. However, what you say is true for most larger embedded programs. You really ought to make life as easy as possible on the bulk of the code, so you have more time to put more focus of the gritty stuff.

I prefer to get the computer to prevent me from making as many avoidable errors as possible. Attitudes like "I can safely handle loaded guns pointed at my foot" is macho libertarian nonsense.

Twiddling bits in a register is not a major issue. Ensuring the correct bits are twiddled at the correct time is far more complex and error prone.The

Thus I don't care if peek and poke are implemented in C or assembler. Correctly calculating the invocations of peek and poke in a multimode processor is far more challenging. Any tool that helps automatically predicting those values aren't incorrect is valuable.

[coppice]

Though i looked at the language a while ago and honestly, if i still have to go the unsafe route to do any peripheral access might as well keep writing C...

That's a pretty silly argument even for embedded development.  On all but the most trivial programs (which: who cares what you use), only a tiny fraction of the code is actually performing the register read/writes.  I'm not completely thrilled with the way rust works on embedded, or the way unsafe works in rust, but most code even in embedded could be safer.

It depends what you are programming. A lot of small embedded processors are doing endless manipulation of peripherals. However, what you say is true for most larger embedded programs. You really ought to make life as easy as possible on the bulk of the code, so you have more time to put more focus of the gritty stuff.

I prefer to get the computer to prevent me from making as many avoidable errors as possible. Attitudes like "I can safely handle loaded guns pointed at my foot" is macho libertarian nonsense.

Twiddling bits in a register is not a major issue. Ensuring the correct bits are twiddled at the correct time is far more complex and error prone.The

Thus I don't care if peek and poke are implemented in C or assembler. Correctly calculating the invocations of peek and poke in a multimode processor is far more challenging. Any tool that helps automatically predicting those values aren't incorrect is valuable.

You just aren't going to make small embedded machines super safe to program, unless, perhaps, AI assistants get so good they pick up the errors. We need to take the wins where they present themselves. There are many weird and wonderful multi-core devices for communications applications, with very heterogeneous layouts of memory and peripherals. Good luck trying to build moderately safe programming environments for those They are a big security issue, as its often very hard to figure out from their documentation just what it is you are supposed to do in all cases.