A (runtime) memory-safe C, Fil-C

[DiTBho]

Maybe in your world.  In the real world

well, "myC" is a researching object in many ways, a toy that as such makes no sense in development environments.

True, but if "myC" is hobby, when you say "real world" ... I write and test avionics software for a living, working only with C and Ada assembly.

"my world" is weird and different from the typical C application, and our "fly applications" are usually written in Ada and rarely written in C, while the bootloader, the BSP, the iBSP, all the drivers and the kernel all written in C and compiled by GreenHills C/99 cc.

I think you thought about the "use case" in which Python calls things written in C.
Never don, but I think you need a wrapper between them, so it would be the same for Fil-C, I guess.

Nerver thought about that, I don't have much experience in "mixing" various languages.
I have a rough answer when I compile a piece of application in Erlang and link it with modules written in C because I had to debug it from the bottom, and to write a wrapper.

The same between ghdl and C, but it worked better than ErLang and C.

[DiTBho]

I can't imagine choosing Fil-C over rust for technical rather than political reasons.  The big advantage Fil-C has is the ability to keep using existing battle tested code with a tolerable overhead.

The goal I had in mind is to take a minimal GNU/Linux rootfs (all written in C) and recompile it from scratch with Fil-C!

It's something I do regularly for routers.
The result is a slightly slower rootfs, but one that doesn't suffer from memory leaks (Fil-C with the garbage collector enabled) or off-by-one bugs.

[5U4GB]

"my world" is weird and different from the typical C application, and our "fly applications" are usually written in Ada and rarely written in C, while the bootloader, the BSP, the iBSP, all the drivers and the kernel all written in C and compiled by GreenHills C/99 cc.

How good is GreenHills in terms of preserving source code semantics?  For example if you compile 'for(i=0; i<10; i++)' will it rewrite it to 'for(i=0; i!=10; i++)' like gcc will?

[ejeffrey]

The Fil_C also implements a garbage collector.
This may be a problem for things like "Kernels" and "embedded stuff".
But you can disable it.

I don't think you can.  At least not if you want to free memory.  The way it works is that when you call free() the objects capability block marked "freed".  That causes any future access to panic.  But the memory can't be reclaimed yet.  When doing a mark and sweep, any pointers found that reference that capability are redirected to the global "freed object".  Only after a full sweep guarantees all such ponters at cleaned can the memory be reclaimed.  Some without GC, even explicitly freed memory will not be reclaimed.

I can't find in the documentation how this works with stack allocations, but I think it means that all local variables and alloca() allocations that potentially escape the scope must get promoted to dynamic allocations, and they are automatically freed (but not yet reclaimed) when the function returns.

[ledtester]

I can't imagine choosing Fil-C over rust for technical rather than political reasons.  The big advantage Fil-C has is the ability to keep using existing battle tested code with a tolerable overhead.

The goal I had in mind is to take a minimal GNU/Linux rootfs (all written in C) and recompile it from scratch with Fil-C!

DJB (Daniel J Bernstein) is attempting to recompile all of Debian with Fil-C:

https://cr.yp.to/2025/fil-c.html

Scroll down to "Debian using Fil-C" to see his notes on that effort.

Other links:

https://news.ycombinator.com/item?id=45788040

https://www.theregister.com/2025/11/05/djb_tries_filc_and_approves/

[ejeffrey]

That makes a lot of sense to me.  I'd much rather use a fil-C compiled coreutils than a rust re-implementation for instance even if the latter is faster.  Much of that code was already fast enough for me on a 486 and I'd rather not have weird incompatibilities and bugs from a new implementation.  Stable and widely used C code pretty much by definition has no bugs that I care about, but it could have security issues in cases way outside normal operation.  Compiling with Fil-C would protect against some of those while preserving the functionality.

I just don't think anyone writing new code is going to want to pay the performance price when they have alternatives that are faster.

I do hope this leads to more reconsidering about whether memory safety is even possible in C/C++.  I think a lot of people would have said that it's not, or that the overhead would be a python like-50x, or that it would be at best statistical like pointer tagging.

[JPortici]

Qt can be built with fil-c

https://git.qt.io/cradam/fil-qt

[MarginallyStable]

"my world" is weird and different from the typical C application, and our "fly applications" are usually written in Ada and rarely written in C, while the bootloader, the BSP, the iBSP, all the drivers and the kernel all written in C and compiled by GreenHills C/99 cc.

How good is GreenHills in terms of preserving source code semantics?  For example if you compile 'for(i=0; i<10; i++)' will it rewrite it to 'for(i=0; i!=10; i++)' like gcc will?

Not sure how this can be true, what if 'i' were modified >= 10 within the loop?

[Siwastaja]

"my world" is weird and different from the typical C application, and our "fly applications" are usually written in Ada and rarely written in C, while the bootloader, the BSP, the iBSP, all the drivers and the kernel all written in C and compiled by GreenHills C/99 cc.

How good is GreenHills in terms of preserving source code semantics?  For example if you compile 'for(i=0; i<10; i++)' will it rewrite it to 'for(i=0; i!=10; i++)' like gcc will?

Not sure how this can be true, what if 'i' were modified >= 10 within the loop?

Well quite obviously in that case GCC won't implement it with non-equality comparison.

This only matters if you want to harden the application against memory corruption, which (e.g. rad-hard software) is actually pretty complicated thing to do and I'm not sure if micromanaging that through C + non-optimizing compiler is a good idea at all.

[SiliconWizard]

"my world" is weird and different from the typical C application, and our "fly applications" are usually written in Ada and rarely written in C, while the bootloader, the BSP, the iBSP, all the drivers and the kernel all written in C and compiled by GreenHills C/99 cc.

How good is GreenHills in terms of preserving source code semantics?  For example if you compile 'for(i=0; i<10; i++)' will it rewrite it to 'for(i=0; i!=10; i++)' like gcc will?

Not sure how this can be true, what if 'i' were modified >= 10 within the loop?

Well quite obviously in that case GCC won't implement it with non-equality comparison.

This only matters if you want to harden the application against memory corruption, which (e.g. rad-hard software) is actually pretty complicated thing to do and I'm not sure if micromanaging that through C + non-optimizing compiler is a good idea at all.

It clearly isn't and it looks off-topic as Fil-C is only about "memory safety" as far as I've understood, not about general "hardening" of compiled code.

Fil-C being a modification of Clang (if I got it right), there's nothing it can do on the optimization level as optimization is not handled in the front-end when using LLVM, and I think the same applies to GCC. So I'm not sure any amount of massaging in the front-end would generate an IR that guarantees that the optimizer won't replace a "greater or equal" with just an "equal" test if they are functionally identical.

That question of hardening code on the assembly level is interesting though but I don't know of many compilers that actually bother with that, at least not general-purpose, non-commercial ones.

[5U4GB]

"my world" is weird and different from the typical C application, and our "fly applications" are usually written in Ada and rarely written in C, while the bootloader, the BSP, the iBSP, all the drivers and the kernel all written in C and compiled by GreenHills C/99 cc.

How good is GreenHills in terms of preserving source code semantics?  For example if you compile 'for(i=0; i<10; i++)' will it rewrite it to 'for(i=0; i!=10; i++)' like gcc will?

Not sure how this can be true, what if 'i' were modified >= 10 within the loop?

Then the loop will run forever, and off into the weeds doing who knows what sort of damage, since it's been rewritten from "make sure i stays within bounds" to "allow i to be any value except this one out of four billion".  So if you've got any situation where data corruption or faults can occur, which in practice is all instances where a computer is used, then gcc will transform code with safety checks into unsafe code.  It's not just that case, it will remove all manner of safety and bounds checks if it decides they're unnecessary, in the worst case doing things like outputting code that segfaults when run because it's decided that a null pointer check is redundant.  No warning that it's done it, just an instant crash when run.

Thus my question about how good GreenHills is for preserving code semantics.

[brucehoult]

"my world" is weird and different from the typical C application, and our "fly applications" are usually written in Ada and rarely written in C, while the bootloader, the BSP, the iBSP, all the drivers and the kernel all written in C and compiled by GreenHills C/99 cc.

How good is GreenHills in terms of preserving source code semantics?  For example if you compile 'for(i=0; i<10; i++)' will it rewrite it to 'for(i=0; i!=10; i++)' like gcc will?

Not sure how this can be true, what if 'i' were modified >= 10 within the loop?

Then the loop will run forever, and off into the weeds doing who knows what sort of damage, since it's been rewritten from "make sure i stays within bounds" to "allow i to be any value except this one out of four billion". So if you've got any situation where data corruption or faults can occur, which in practice is all instances where a computer is used, then gcc will transform code with safety checks into unsafe code.

No, that's not true at all.

If there is a statement within the loop that potentially makes i >= 10 then either the main loop control will not be modified, or the interior assignment will have it's own exit test.

It is the prime job of compilers to maintain the meaning of the code, as written according to the rules of C, and gcc and llvm are very good at doing that.

It's not just that case, it will remove all manner of safety and bounds checks if it decides they're unnecessary, in the worst case doing things like outputting code that segfaults when run because it's decided that a null pointer check is redundant.  No warning that it's done it, just an instant crash when run.

That can't happen unless you change the value of a pointer behind the compiler's back e.g. by an asynchronous process such as hardware or another CPU changing a memory location that you have incorrectly not marked as "volatile".

[ejeffrey]

Thus my question about how good GreenHills is for preserving code semantics.

No, it's not.  All compilers preserve code semantics.

You are asking about something different: code that is resistant to external corruption.  Might be a useful tool, although I share the intuition that doing so with C is nonsensical.  And I meant that literally.  I don't mean a "dumb" or "bad" idea, but one that is logically inconsistent and cannot be made to correspond to anything sensible.

[5U4GB]

It's not just that case, it will remove all manner of safety and bounds checks if it decides they're unnecessary, in the worst case doing things like outputting code that segfaults when run because it's decided that a null pointer check is redundant.  No warning that it's done it, just an instant crash when run.

That can't happen unless you change the value of a pointer behind the compiler's back e.g. by an asynchronous process such as hardware or another CPU changing a memory location that you have incorrectly not marked as "volatile".

Try compiling something like the following with gcc and see what happens:

int foo( struct thing *ptr ) {
  const int value = thing->value;

  if(ptr==NULL) return -1;
  if(is_inverse(thing->data_in)) [...]
  if(value == 0) [...]
}


No compiler warning, it just segfaults when run because the NULL pointer check has been removed because gcc sees the deref at the start, long before the value is used (which follows the NULL pointer check), and decides that since you've deref'd it there it can remove the NULL pointer check and emit code that it knows will crash when run.

In this case the behaviour is extremely obvious because it's been written to be so, but in a lot of code it's not obvious at all, or the segfault is extremely rare and hard to diagnose.  In once case several folks spent some hours trying to figure out how gcc had decided to remove a NULL pointer check and could never figure it out.  There was no obvious cause for it, it simply output code that had the NULL pointer check removed.  The code eventually got rewritten with some volatile hack to make sure the pointer check wasn't quietly deleted.

And then there's the nonnnull attribute, which in gcc does the opposite of what you'd expect: It doesn't warn if you pass in a NULL value for a nonnull parameter, but it will remove NULL pointer checks in your code.

Expected behaviour: Either see that the value isn't used until after the pointer check and emit code as expected, or warn that there's a reference/check conflict in the code.

[5U4GB]

Thus my question about how good GreenHills is for preserving code semantics.

No, it's not.  All compilers preserve code semantics.

How is a compiler that rewrites for(i=0;i<10;i++) into for(i=0;i!=10;i++) preserving code semantics?

[ejeffrey]

Try compiling something like the following with gcc and see what happens:

int foo( struct thing *ptr ) {
  const int value = thing->value;

  if(ptr==NULL) return -1;
  if(is_inverse(thing->data_in)) [...]
  if(value == 0) [...]
}


No compiler warning, it just segfaults when run because the NULL pointer check has been removed because gcc sees the deref at the start, long before the value is used (which follows the NULL pointer check), and decides that since you've deref'd it there it can remove the NULL pointer check and emit code that it knows will crash when run.

OK, but an imaginary "literal compiler that does exactly what is written with no 'optimization'" (which again is a poorly formed concept) would _also_ segfault.  Only a very weird optimizing compiler which defers evaluation of `const int value = thing->value;` to after the null check but still performs the null pointer check would return successfully here.

The code you wrote is dangerous, but not for the reason you say.  The problem is that even if "thing" is NULL, "thing->value" can result in a valid memory address.  And yes, this sort of thing _would_ be caught by Fil-C.  Not that it would prevent a trap, since the code is defective, but if the supplied pointer is NULL, it would have empty capabilities.  There would be no danger that it would inadvertently (due to pointer arithmetic) turn into a valid memory address and cause corruption to an unrelated object.  Instead the code will either return -1 or trap without allowing memory corruption.

Expected behaviour: Either see that the value isn't used until after the pointer check and emit code as expected, or warn that there's a reference/check conflict in the code.

That doesn't make sense.  The value *is* used before the pointer check. 

In once case several folks spent some hours trying to figure out how gcc had decided to remove a NULL pointer check and could never figure it out.  There was no obvious cause for it, it simply output code that had the NULL pointer check removed.  The code eventually got rewritten with some volatile hack to make sure the pointer check wasn't quietly deleted.

Deciding to leave your code defective is not a good choice.  The obvious reason for gcc to remove a check is that previous code already used the value, but did so silently.  By plastering volatile around until it doesn't crash, you have simply decided to leave the real bug in place.  There are a ton of tools available that will help diagnose the real problem, for instance ubsan would likely find it immediately.  Or use a different language!  Most languages don't have C/C++ style undefined behavior which is the problem here, not the compiler optimziing.

And then there's the nonnnull attribute, which in gcc does the opposite of what you'd expect: It doesn't warn if you pass in a NULL value for a nonnull parameter, but it will remove NULL pointer checks in your code.

That's funny because I that's exactly what I expect.  It's an optimization flag to tell the compiler to optimize assuming the value passed is not NULL.  And it _does_ emit a warning if you explicitly pass NULL to a non-null parameter  if you use -Wnonnull or -Wall.

I could certainly see an argument for an attribute that requested the warnings but didn't add the optimization.  But it is not what I would expect that attribute to do.  In any case, nonnull is not part of C, it's a controlled non-conformity for gcc.  A compiler running in a conformant mode will only remove the check when it's redundant based on the program text.

How is a compiler that rewrites for(i=0;i<10;i++) into for(i=0;i!=10;i++) preserving code semantics?

Well, to be annoyingly precise, it doesn't do that.  It chooses a possible implementation of the former in the target machine code, and it does so within the constraints of the language (you aren't allowed to modify memory from an independent context).  It may pick an implementation that "looks" more like the latter, but technically that's only in your imagination.  And that in a nutshell is my point about people who complain about C optimizer "changing" their program.  Compilers _have_ to change your program for it to be able to run. 

[brucehoult]

Try compiling something like the following with gcc and see what happens:

int foo( struct thing *ptr ) {
  const int value = thing->value;

  if(ptr==NULL) return -1;
  if(is_inverse(thing->data_in)) [...]
  if(value == 0) [...]
}


No compiler warning, it just segfaults when run because the NULL pointer check has been removed because gcc sees the deref at the start, long before the value is used (which follows the NULL pointer check), and decides that since you've deref'd it there it can remove the NULL pointer check and emit code that it knows will crash when run.

That is not even close to being valid compilable code. I'm a compiler engineer and I have no idea what you think you're demonstrating there.

In this case the behaviour is extremely obvious because it's been written to be so, but in a lot of code it's not obvious at all, or the segfault is extremely rare and hard to diagnose.  In once case several folks spent some hours trying to figure out how gcc had decided to remove a NULL pointer check and could never figure it out.  There was no obvious cause for it, it simply output code that had the NULL pointer check removed.  The code eventually got rewritten with some volatile hack to make sure the pointer check wasn't quietly deleted.

Volatile is not a hack, it is essential information about your code.

And then there's the nonnnull attribute, which in gcc does the opposite of what you'd expect: It doesn't warn if you pass in a NULL value for a nonnull parameter, but it will remove NULL pointer checks in your code.

It is a promise from you to the compiler that the passed value will not be NULL. On the caller side gcc will warn you if it knows that the value is null, but it is frequently not possible to know, so it comes down to being the programmer's responsibility. On the called side, naturally it can assume that the pointer is not null, because you told it so.

[Siwastaja]

Then the loop will run forever, and off into the weeds doing who knows what sort of damage, since it's been rewritten from "make sure i stays within bounds" to "allow i to be any value except this one out of four billion".

Yes, the design of the C language is as such that random memory corruption can lead to more random memory corruption. The primary way to deal with this is avoidance of the original random memory corruption, in which case Fil-C is actually very relevant.

Rad-hard (or memory corruption hardened) defensive programming is a valid field of study (and practice) on its own, but I'm also sure that just throwing C at a non-optimizing compiler isn't the right way to do it at all - your amateurism is showing. For starters, non-optimizing compiler generates larger code, and as such, more surface area for corruption. Optimizations, like any other implementation choice, increase or decrease safety against memory corruption depending on what they exactly do.

Defensive programming against memory corruption is hard, with a lot of case-by-case thinking, and proving paths of program execution. Expecting a more literal and less optimized compiler to automatically achieve any of that is a textbook example of false sense of security.

Compilers preferred in the safety-critical fields likely optimize less only because for safety-critical, complexity is enemy in general and keeping things unchanged is positive because then you don't introduce new bugs. Therefore, they avoid trying to squeeze last bits of performance, not because there is something fundamentally wrong in that, but because the risk of any new feature having bugs is real.

[MarginallyStable]

I'm not aware of any programming language that is designed to deal with hardware data corruption. That is what ECC, Lock-step processing, redundant voting are designed to mitigate. If you want C to assume that hardware corruption may be an issue, then all variables must be considered volatile, but this doesn't buy you much from a functional stand point, the program is still going to act incorrectly with corruption.

[MarginallyStable]

Try compiling something like the following with gcc and see what happens:

int foo( struct thing *ptr ) {
  const int value = thing->value;

  if(ptr==NULL) return -1;
  if(is_inverse(thing->data_in)) [...]
  if(value == 0) [...]
}


No compiler warning, it just segfaults when run because the NULL pointer check has been removed because gcc sees the deref at the start, long before the value is used (which follows the NULL pointer check), and decides that since you've deref'd it there it can remove the NULL pointer check and emit code that it knows will crash when run.

In my mind it is crazy to think that this code would not segfault given a null pointer passed in. You are literally de-referencing it before the null check (assume thing and ptr are supposed to be the same)

[SiliconWizard]

I don't know what  5U4GB was expecting. (Bruce: let's assume the code is fixed and compilable.)

The pointer is dereferenced first before checking it against NULL, so of course the code will crash when executed (at least on a platform where derefencing a NULL pointer causes a crash).

So what were you expecting? A warning about the check against NULL happening after dereferencing? That could be nice, but way above what GCC (and most compilers) are designed to warn about.

That's where using an external static analyzer comes in handy. Even just the open-source CppCheck catches it with no issue:

Code: [Select]

ohmynull.c:18:21: warning: Either the condition 'ptr==NULL' is redundant or there is possible null pointer dereference: ptr. [nullPointerRedundantCheck]
  const int value = ptr->value;
                    ^
ohmynull.c:20:9: note: Assuming that condition 'ptr==NULL' is not redundant
  if(ptr==NULL) return -1;
        ^
ohmynull.c:18:21: note: Null pointer dereference
  const int value = ptr->value;

And of course I also highly recommend to validate function arguments and validate them first thing before doing anything with the arguments. But admittedly, even if the above example is simplified, this is the kind of things that happen all the time on non-trivial code if one doesn't strictly follow this rule of validating all inputs FIRST THING upon function entry rather than on an "ad-hoc" manner, which is bound to cause overlooks like the above. It definitely happens.

[TheCalligrapher]

Expected behaviour: Either see that the value isn't used until after the pointer check and emit code as expected, or warn that there's a reference/check conflict in the code.

Yes, it has been discussed recently. Indeed, every optimization option of this kind is also accompanied by a matching diagnostic option. It is just that in the current implementation the optimization code is out-of-sync with the diagnostic-issuing code. The latter is somewhat behind. But once it catches up, the compiler should issue a diagnostic when it performs this optimization.

[brucehoult]

Expected behaviour: Either see that the value isn't used until after the pointer check and emit code as expected, or warn that there's a reference/check conflict in the code.

Yes, it has been discussed recently. Indeed, every optimization option of this kind is also accompanied by a matching diagnostic option. It is just that in the current implementation the optimization code is out-of-sync with the diagnostic-issuing code. The latter is somewhat behind. But once it catches up, the compiler should issue a diagnostic when it performs this optimization.

The "expected behaviour" is just plain wrong.

The definition of C says that when things are separated by semicolons then they must be performed "as if" in the order written. The "const int value = ptr->value;" must be done before the null pointer check, and either crash or else if it doesn't crash then the pointer can henceforth be assumed to not have been NULL.

Deferring that access until after the NULL check would be a violation of the semantics of C.

[ejeffrey]

The "expected behaviour" is just plain wrong.

Agreed, it's not just wrong, it's nonsensical.

The definition of C says that when things are separated by semicolons then they must be performed "as if" in the order written. The "const int value = ptr->value;" must be done before the null pointer check, and either crash or else if it doesn't crash then the pointer can henceforth be assumed to not have been NULL.

Well, "crashing" isn't observable behavior so it's perfectly acceptable for the compiler to optimize by defer the read access.  But it has to be clear that it's an as-if optimization: if the pointer is valid it's OK to to defer, if it's invalid it's not defined anyway.

[SiliconWizard]

Well, "crashing" isn't observable behavior so it's perfectly acceptable for the compiler to optimize by defer the read access.  But it has to be clear that it's an as-if optimization: if the pointer is valid it's OK to to defer, if it's invalid it's not defined anyway.

It's not really about crashing being an observable behavior or not, "crashing" is just UB.

But yes, since 'value' is a local variable, the compiler would be free to optimize accesss until just before value is actually used in the code, but this would of course be optimization, not *guaranteed* behavior by any means. Expecting the compiler to generate code that derefences the pointer *just right* before the dereferenced value is used is the unfortunately nonsensical part.

Only if the pointer was volatile-qualified would we have a *guarantee* that it's dereferenced right where 'value' is declared. Otherwise, you just can't know.

It's interesting to see that quite a few people tend to get annoyed with C compilers when these do not generate compiled code that strictly matches the source code (for instance by reordering statements when it can), and that there are others (maybe even the same ones) that will get annoyed with C compilers when these actually do respect the original order, if this isn't convenient for the developer.

IMO the only reasonable thing to do here for a compiler is to emit a warning, and as I said, any decent static analyzer will, and I highly recommend using that. The reason both GCC and Clang do not emit any warning for such a piece of code is that optimizations happen at a later stage after code transformation to IR, and at this point, it's too late to emit "static analysis" warnings. It's a problem that I have also raised a ticket for in the past and just got answered that it wasn't possible to emit warning about the code at the optimization stage. Now of course, nothing prevents a compiler to have better static analysis before optimization, and as I showed, devent static analysis will catch this kind of issue, but that would also 1/ take a lot more compile time and 2/ probably annoy a lot of users, since more thorough static analysis almost invariably means a lot more warnings that they have to go through. So the question becomes more like: "why doesn't GCC behave exactly like I would like it to behave in my particular use cases?".

Anyway, for those who haven't, I really suggest trying CppCheck on their code, just to see what it comes up with. You'll probably be surprised.