Poll: Freeing NULL Pointers

[hamster_nz]

I've been made aware of this in the man page for free(3):

The free() function frees the memory space pointed to by ptr, which must have been returned by a previous call to malloc(), calloc() or realloc(). Otherwise, or if free(ptr) has already been called before, undefined behavior occurs. If ptr is NULL, no operation is performed.

Currently I deliberately avoid freeing NULL pointers. So if memory is free()ed in a different function to where they are malloc()ed I usually use this pattern:

Code: [Select]

   if(mystruct->ptr) {
      free(mystruct->ptr);
      mystruct->ptr = NULL;
   }

Now I see I could just:

Code: [Select]

   free(mystruct->ptr);
   mystruct->ptr = NULL;

I think I might move to the dark side...

[andersm]

If it's normal that nullptrs are freed, there's no point in checking. If they should not be nullptrs, it's a good place for a runtime assert. But a check that does nothing if the pointer is NULL serves no purpose.

[SiliconWizard]

I don't. Mostly because in many cases, freeing an allocated "object" may require further operations to properly free it aside from freeing the object itself, so I make it a habit to test for NULL pointers before proceeding. But if the freeing is associated with no other steps, then it's useless to test for NULL indeed. It may still be an opportunity to check that all your free'ing operations have a matching allocation. YMMV.

But you're right, provided that the implementation of free() on your particular target is compliant, it's perfectly valid.

[PlainName]

a check that does nothing

Strictly speaking, the check prevents a spurious write.

If you're using strncpy or snprintf or any of those, do you automatically shove a nul at the end of the destination as a matter of course, or do you check the returned value and only write the nul if necessary? I generally write it anyway just to be sure, and I generally check for NULL on the same basis. Doing that at the point of freeing, as opposed to after the free call, isn't that much of a biggie. Forgetting to do it when something extra needs to be done can be pretty huge.

[Rick Law]

Do you really mean "freeing NULL pointers" or do you actually mean "setting the pointer to NULL when an object is no longer used?"

Freeing NULL pointers make no sense.  What is it freeing when the poitner is not pointing to anything.

On the other hand, if you mean "setting point to NULL when the object is no longer in use":  I don't recall where I read it, but I do remember reading that for JAVA it is a good practice to set the pointer of unused object to NULL - that made it clear to the garbage collection utilities that memory for the object can be freed.

[hamster_nz]

Do you really mean "freeing NULL pointers" or do you actually mean "setting the pointer to NULL when an object is no longer used?"

Freeing NULL pointers make no sense.  What is it freeing when the poitner is not pointing to anything.

Yes, exactly the second. Some feel it makes sense because it saves a few lines of code, and believes it avoids bugs.

[andersm]

a check that does nothing

Strictly speaking, the check prevents a spurious write.

I meant that there is no else case. If freeing a nullptr is an unexpected error (otherwise, why bother checking?) it should be handled in some way.

Setting the value of a free'd pointer works best if you can set it to a trapping value. Unfortunately, on most microcontrollers it's either not possible to generate access traps, or the region around address zero contains registers or valid memory. Both double-frees and use-after-free are serious, memory-corrupting bugs, and frequently the root cause of security vulnerabilities, and it's certainly worth protecting yourself against them. But silently ignoring a potential cases isn't right either.

[golden_labels]

No, it needlessly duplicates operation that is an inherent part of free itself. It makes as much sense as:

Code: [Select]

if (NULL != ptr) {
    if (NULL != ptr) {
        doSomething(ptr);
    }
}
Skipping the NULL check may seem weird if you first missed the fact that free has well-defined and expected behaviour for NULL. Just to provide more general source, C11 §7.22.3.3:

The free function causes the space pointed to by ptr to be deallocated, that is, made
available for further allocation. If ptr is a null pointer, no action occurs.

Another function that can be used that way is realloc (§7.22.3.5):

If ptr is a null pointer, the realloc function behaves like the malloc function for the
specified size.

This also shows why this situation is natural and not merely some “weird”, unexpected operation which I usually opose. realloc is literally designed to be used with NULL and free fits with realloc nicely.

The question should rather be about assigning NULL. Some suggest it prevents dangling pointers and in some contexts it does. If freeing something in a structure, that will live beyond the current function or a pointer that may later be accessed in the current function, it is a good idea that will save you a ton of pain. If you want to have a habit of doing so indiscriminately, it is still fine — good habits are good unless proven harmful, even if in some cases they do nothing. However, if you are consciously assigning a NULL to something that is never to be used again, do not waste your time. It will not do what you think it does, because to a compiler it’s a no-op and most likely it will be skipped in the output binary.

Strictly speaking, the check prevents a spurious write.

Strictly speaking it introduces a spurious test and a branch, and prevents either nothing or a call in situation where a call is expected. Good that most compilers reserve some kind of a zero-register, because that test alone would introduce a spurious write itself.

[hamster_nz]

So I went hunting... it seems that older C libraries (from my time learning C in the 80s) did indeed crash if you tried to free a NULL pointer. For example the original K+R C book has an implementation of free() that crashed with NULL.

As mentioned, since K+R ANSI C, a standards-complaint free(NULL) does nothing.

And my second result for the search "example C free" (https://www.guru99.com/free-in-c-example.html) showed exactly the sort of thing that made me flinch - it calls free() even when it is known that ptr is NULL:

Code: [Select]

#include <stdio.h>
int main() {
  int* ptr = malloc(10 * sizeof(*ptr));
  if (ptr != NULL){
    *(ptr + 2) = 50;
    printf("Value of the 2nd integer is %d",*(ptr + 2));
  }
  free(ptr);
}

However, armed with my newfound knowledge this is fine. Perhaps a little odd, but causes no harm.

(The first example I found - https://www.tutorialspoint.com/c_standard_library/c_function_free.htm - made me flinch is so many ways... don't look!)

[chriva]

If anything it's important to make sure pointers are set to 0 before they're allocated and after they've been freed if you have any intention of looking for a 0 pointer before freeing.

Reason:
In larger apps where you don't have full control over the program flow, it's entirely possible for gui elements or other external triggers to enter that part of the code before it has even been allocated (example: you want to make sure another object is freed before allocating a new one).

Basically I've ran over myself enough times to find it worthwhile to do that as a precaution. Especially if I have to check for 0 pointers

[golden_labels]

hamster_nz:
Some comments on that last code, as it contains some traps.

While malloc may return NULL, on some platforms it may return a non-NULL value that will still be invalid. That’s the case with Linux systems with overcommit (which is the default). This is a trap for new programmers in many ways. This is also why sometimes you may see Linux-specific code, which doesn’t care about NULL returned directly from malloc or calls abort: the test is nearly meaningless and it is more likely that a write to that memory will fail than seeing a NULL there, so it is fine to just let it crash that way.

That brings another question: what if you need to allocate memory in a manner that, if it crashes, it does that in a predictable place? The answer is: write the memory immedietely after allocating it. And, as it happens, C alredy has a function to do that: calloc. It will allocate objects and fill their bytes with 0, so if a crash is to happen, it will happen at that call. And if you receive a non-NULL value, it reliably indicates that the memory is valid. There are some situations in which you want to allocate memory but not obtain it from the system at that particular moment and then this method is not fine. But those are rare and, if you ever need them, you will likely already know about that pecularity or even use mmap directly.

calloc has one more advantage: it handles multiplication overflow properly. 10 * sizeof(int) usually can’t overflow, but that is not true for arbitary expression. In particular if the value is based on user input.

The disadvantage is that tools like Valgrind will not be able to detect invalid reads from calloc-ed memory. There is also no standard realloc counterpart to calloc.

[PlainName]

I meant that there is no else case.

There isn't in that example, but it's easy to put one in if you have the framework I think the implied 'else' would be problem-specific and not relevant to an example.

Of course, there may not be an else, but just like it's good practice to still have the {} in an if which has only one statement, it might be good practice to have this test anyway for future hooking into. And as demonstration (to a maintainer) that you knew about the possibility of a NULL there.

[PlainName]

Just to provide more general source, C11 §7.22.3.3:

Ouch! maybe you're a C11 buff and get tasked to maintain an older codebase and don't realise this free() doesn't handle NULL well... That alone is a good reason to shove it in unless there's a better reason not to.

[PlainName]

While malloc may return NULL, on some platforms it may return a non-NULL value that will still be invalid.

I would expect to manually set the pointer to NULL in that case. malloc() should have some mechanism for returning status - it isn't acceptable to not say a thing when it's failed and, instead, rely on the code crashing. In fact, I can't believe such a malloc() doesn't have some other indicator of success or failure.

The convention is that NULL is invalid. Doesn't matter how that got there - if malloc() does it then fine, use the return value. If not, it's up to you as the programmer to sort it out. That's why we are programmers and not script kiddies! The exception is if your system has a different convention. That's fine. Whatever it is, you need to be consistent.

[SiliconWizard]

So I went hunting... it seems that older C libraries (from my time learning C in the 80s) did indeed crash if you tried to free a NULL pointer. For example the original K+R C book has an implementation of free() that crashed with NULL.
As mentioned, since K+R ANSI C, a standards-complaint free(NULL) does nothing.

Yes, hence why I said a compliant implementation. I would even expect a number of implementations with not so old libraries to possibly crash as well, or otherwise fuck something up.

Code: [Select]

#include <stdio.h>
int main() {
  int* ptr = malloc(10 * sizeof(*ptr));
  if (ptr != NULL){
    *(ptr + 2) = 50;
    printf("Value of the 2nd integer is %d",*(ptr + 2));
  }
  free(ptr);
}


That is one example of ptr being checked against NULL anyway, to proceed with other steps before freeing it. So in this example, I don't really see a reason not to put the 'free(ptr)' call inside the if block.

But maybe some people would consider a matching free() to any malloc(), whether it succeeds or not, good practice/good style.

[Nominal Animal]

I have never used an ISO C library that handled free(NULL); incorrectly.  It is safe to do.  (That is, anything that claims to implement C89 or later.  The older buggy C libraries have multitudes of other quirks you need to cater for anyway.)

When using POSIX C, the following pattern is often used for reading input line-by-line:

Code: [Select]

#define _POSIX_C_SOURCE 200809L
#include <stdlib.h>
#include <stdio.h>

int main(void)
{
    char  *line = NULL;
    size_t  size = 0;
    ssize_t  len;

    while (1) {
        len = getline(&line, &size, stdin);
        if (len == -1)
            break;

        /* Do something with line.
             line has len characters, including the newline (if any). */
    }

    /* Discard unneeded line buffer */
    free(line);
    line = NULL;
    size = 0;

    /* Do something else, possibly read another file the same way */

    return EXIT_SUCCESS;
}
At the Do something with line point, it is completely safe and acceptable to steal the buffer, or even free it, if one also sets line = NULL, size = 0 .  Just like for the initial line, getline() will then dynamically allocate a buffer large enough for the next line.

I regularly apply the topic at hand, in the point with comment Discard unneeded line buffer.  If there was no data to read from standard input, then at that point line could well be NULL.  However, it is completely safe to just free it, and ensure the pointer and the allocated size get zeroed.

But why ensure line = NULL, size = 0?  Because this is a pattern that avoids a common error case: use-after-free.  It costs basically nothing, and it helps with debugging when something goes b0rkb0rkb0rk.

I have seen many new C programmers, and even tutorials, that suggest using malloc() to allocate an initial buffer.  That is complete waste: extra code (with bug risks) with zero benefits.

The above pattern has proven its worth to me, in that it yields code that tends to have less bugs, and is easier to debug.  So, nowadays I apply this pattern extensively.

As an example, consider a highly dynamic (entries and their number change often) hash table implementation based on

Code: [Select]

struct hash_entry {
    struct hash_entry *next;  /* Next entry in the same table slot */
    size_t  hash;  /* Actual hash value for this entry */
    /* Payload */
};

struct hash_table {
    size_t  size;
    size_t  entries;
    struct hash_entry **slot;
};
#define  HASH_TABLE_INITIALIZER  { 0, 0, NULL }

static inline void hash_table_init(struct hash_table *ht)
{
    ht->size = 0;
    ht->entries = 0;
    ht->slot = NULL;
}
The idea is that the user creates a hash table using either
    struct hash_table  my_table = HASH_TABLE_INITIALIZER;
or
    struct hash_table  my_table;
    hash_table_init(&my_table);

The actual slot pointer array is allocated or reallocated whenever the size of the hash table is changed; and typically that happens when the first entry is added, last entry is removed, or the ratio entries/size exceeds (so the table is grown) or drops below (so the table is shrunk) some heuristic limit.

In this scheme, each hash table slot is a pointer, and an unused slot is just a NULL pointer.  (Obviously, an item with hash h is stored in the chain hanging off .slot[h % .size].)  Because each hash table entry has the original hash in it, the hash table can be resized (and entries moved to their corresponding new slots) when needed.  The shown one is strictly a single-theaded structure; multithreaded access requires a careful locking scheme anyway.

If you implement the operations for this (type of) hash table, you'll see how useful it is to explicitly keep unused pointers marked NULL.  Here, the fact that even the slot array itself will not exist when .size == 0 does mean an "extra" check (an "extra" conditional) in the function implementations, but it also means they become much simpler.  For example, you can provide a function that allows the caller to specify exactly how many slots they want, or how many new hashes they intend to add, without adding complexity to the other functions.

(This is not the most efficient hash table implementation, obviously, but it has proven to be quite acceptable and robust in real life.)

So yeah, it is definitely an useful pattern.  While it is not very common, the cost (NULLifying the pointer after free()) is utterly neglible in real life, and the pattern itself helps write more robust and easier to maintain code, which to me means it is worth it.

Of course, even I avoid doing that just before returning from main or exit()ing.  I fully trust the OS to clean up after the process anyway, so freeing any dynamic memory allocations (except for deleting any shared memory segments!) just before the process exits isn't part of this pattern.

[SiliconWizard]

While malloc may return NULL, on some platforms it may return a non-NULL value that will still be invalid.

I would expect to manually set the pointer to NULL in that case. malloc() should have some mechanism for returning status - it isn't acceptable to not say a thing when it's failed and, instead, rely on the code crashing. In fact, I can't believe such a malloc() doesn't have some other indicator of success or failure.

I have personally never run into cases of malloc() that would NOT return NULL in case of failure to allocate the requested memory.

I've read about this "overcommit" issue, but have never run into it personally. Yes that would certain suck to have to deal with this.

According to this: https://pubs.opengroup.org/onlinepubs/009695399/functions/malloc.html
which refers to ISO C (I guess C90):

Upon successful completion with size not equal to 0, malloc() shall return a pointer to the allocated space. If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned. Otherwise, it shall return a null pointer [CX] [Option Start]  and set errno to indicate the error.

So not returning a null pointer if not successful doesn't seem to be compliant. I'd have to check C99 and C11, but I doubt this has really changed?

Now of course - that may subtly depend on what a given implementation calls "successful completion". Little buggers.

[golden_labels]

Ouch! maybe you're a C11 buff and get tasked to maintain an older codebase and don't realise this free() doesn't handle NULL well... That alone is a good reason to shove it in unless there's a better reason not to.

It’s the same in 9899:1999, §7.20.3.2. The draft for 9899:1990 §7.10.3.2 also states the same. Sure, one may be maintaining some non-conforming or really old software. But if someone works with tools that fails to support 30 years old version of the language, I am sure this situation very different from writing new software or even working with relatively recent code..

I would expect to manually set the pointer to NULL in that case. malloc() should have some mechanism for returning status - it isn't acceptable to not say a thing when it's failed and, instead, rely on the code crashing. In fact, I can't believe such a malloc() doesn't have some other indicator of success or failure.

There is no way to detect that condition at the point of malloc call. As far as I know there is no way to detect it at all in a manner that doesn’t kill the process. Sorry, sometimes reality doesn’t match language’s abstraction.

I have personally never run into cases of malloc() that would NOT return NULL in case of failure to allocate the requested memory.

Make sure you have swap disabled (unless you want 20 minutes of thrashing ), ensure you have important data saved (OOM killer may eat any of your processes) and run that, setting n to more thatn you have memory GiB:

Code: [Select]

// WARNING: may kill other apps, may cause swap thrashing
#include <stdlib.h>
#include <stdio.h>
#include <stdint.h>
#include <string.h>

int main(void) {
    enum {n = 16}; // more than you have GiB of memory
    void* gigmems[n] = {0};
   
    for (int i = 0; i < n; ++i) {
        puts("Allocating 1GiB.");
        fflush(stdout);
        gigmems[i] = malloc(UINT32_C(1073741824));
        printf("Result (total: %dGiB): %p\n", i + 1, gigmems[i]);
        fflush(stdout);
    }
   
    printf("Successfully allocated %d x 1GiB\n", n);
    fflush(stdout);
   
    for (int i = 0; i < n; ++i) {
        printf("Writing 1GiB to %p\n", gigmems[i]);
        fflush(stdout);
        memset(gigmems[i], 0x5A, UINT32_C(1073741824));
    }
   
    // Not even caring about freeing, as the program will not reach this
   
    return EXIT_SUCCESS;
}

hamster_nz:
Extending a bit what I have earlier written about ptr = NULL not neccesserily doing what you think it does, see examples below. Note that I am not claiming that assigning NULL is wrong! Merely showing that there are situations in which the final code will not do exactly the thing you expected it to do, if the compiler notices that this assignment can’t have further effects. That code:

Code: [Select]

#include <stdlib.h>

int fooize(int* ptr) {
    int const value = *ptr;
   
    free(ptr);
    ptr = NULL;
   
    return value;
}
… produces:

Code: [Select]

=== gcc 9.3.0, x86_64/Linux ====================================================
0:    41 54                    push   %r12             | prologue
2:    44 8b 27                 mov    (%rdi),%r12d     | value = *ptr
5:    e8 00 00 00 00           callq  a <fooize+0xa>   | free(ptr) [1]
a:    44 89 e0                 mov    %r12d,%eax       | return via eax
d:    41 5c                    pop    %r12             | \_ epilogue
f:    c3                       retq                    | /

[1] The argument is already in the register, since it was passed to `fooize`


=== clang 10.0.0, x86_64/Linux =================================================
0:    53                       push   %rbx             | prologue
1:    8b 1f                    mov    (%rdi),%ebx      | value = *ptr
3:    e8 00 00 00 00           callq  8 <fooize+0x8>   | free(ptr) [1]
8:    89 d8                    mov    %ebx,%eax        | return via eax
a:    5b                       pop    %rbx             | \_ epilogue
b:    c3                       retq                    | /

[1] The argument is already in the register, since it was passed to `fooize`


=== gcc 10.1.0, Atmega16 =======================================================
 0:    cf 93           push    r28                     | \_ prologue
 2:    df 93           push    r29                     | /
 4:    fc 01           movw    r30, r24                | \_ value = *ptr
 6:    c0 81           ld      r28, Z                  | |
 8:    d1 81           ldd     r29, Z+1                | /
 a:    0e 94 00 00     call    0    ; 0x0 <fooize>     | free(ptr) [1]
 e:    ce 01           movw    r24, r28                | return via r24:r25
10:    df 91           pop     r29                     | \_ epilogue
12:    cf 91           pop     r28                     | |
14:    08 95           ret                             | /

[1] The argument is already in the register, since it was passed to `fooize`
As you may see, that assignment is removed. If you expect to see any NULL  during debugging, you may be surprised.

[SiliconWizard]

I have personally never run into cases of malloc() that would NOT return NULL in case of failure to allocate the requested memory.

Make sure you have swap disabled (unless you want 20 minutes of thrashing ), ensure you have important data saved (OOM killer may eat any of your processes) and run that, setting n to more thatn you have memory GiB:

Alright. Now how are you supposed to properly deal with out-of-memory conditions on such a system then? And how do you detect them?
Finally, do you think this makes malloc() really compliant with the standard?

From C99: talking about malloc, calloc and realloc:

If the space cannot be allocated, a null pointer is returned.

Yeah.

hamster_nz:
Extending a bit what I have earlier written about ptr = NULL not neccesserily doing what you think it does, see examples below. Note that I am not claiming that assigning NULL is wrong! Merely showing that there are situations in which the final code will not do exactly the thing you expected it to do, if the compiler notices that this assignment can’t have further effects. That code:

Code: [Select]

#include <stdlib.h>

int fooize(int* ptr) {
    int const value = *ptr;
   
    free(ptr);
    ptr = NULL;
   
    return value;
}

As you may see, that assignment is removed. If you expect to see any NULL  during debugging, you may be surprised.

Well, of course. This OTOH doesn't really have anything to do with assigning NULL to a pointer, but just to assigning values to a variable that are never used.
In your above example, obviously the "ptr = NULL' statement has no effect anyway. But, if you were further using 'ptr' in the rest of the function before returning, it could have, and then the assignment wouldn't get pruned.
I'm sure hamster_nz knows this, and from what I saw, the example he gave was for instance for pointers that were members of structures that would potentially live AFTER the freeing operation.
Different story.

[PlainName]

This wouldn't be a question at all if free() set the ptr to NULL (or whatever passes for invalid) when you call it. Of course, you'd have to go free(&ptr) but that's hardly difficult. There is no doubt some hangover compatibility thing with K&R, but it's not like there's half a dozen ways of allocating the stuff yet only one way to get rid of it.

[golden_labels]

Alright. Now how are you supposed to properly deal with out-of-memory conditions on such a system then? And how do you detect them?

AFAIK you can’t.

(doesn’t work)I guess you could manually mmap an anonymous page and then mlock it, but there  are some disadvantages. The memory region will remain in RAM, which is not what one wants in a normal application. You need to run it as a root or with CAP_IPC_LOCK capabilities, which is not desirable for a normal process and, effectively, you may accidentally kill some other apps. mlock may be expensive.

Finally, do you think this makes malloc() really compliant with the standard?

This situation is outside of C scope, just like a microcontroller losing power or RAM getting corruped. From the point of view of malloc and the program the memory is allocated. It’s the operating system that fails to deliver it and kills the whole process, completely outside of the C abstraction.

Well, of course. This OTOH doesn't really have anything to do with assigning NULL to a pointer, but just to assigning values to a variable that are never used. (…)

Yes, but it affects assigning NULL. I have explicitly said that I do not deliver this example to deprecate setting NULL in that manner — my goal was to show that it may produce outputs different than one expects if seen from the world outside C. For example while using a debugger.

[SiliconWizard]

Alright. Now how are you supposed to properly deal with out-of-memory conditions on such a system then? And how do you detect them?

AFAIK you can’t. I guess you could manually mmap an anonymous page and then mlock it, but there  are some disadvantages. The memory region will remain in RAM, which is not what one wants in a normal application. You need to run it as a root or with CAP_IPC_LOCK capabilities, which is not desirable for a normal process and, effectively, you may accidentally kill some other apps. mlock may be expensive.

That bites. I suppose you can disable overcommitting if you're not happy with this?

Finally, do you think this makes malloc() really compliant with the standard?

This situation is outside of C scope, just like a microcontroller losing power or RAM getting corruped. From the point of view of malloc and the program the memory is allocated. It’s the operating system that fails to deliver it and kills the whole process, completely outside of the C abstraction.

Yeah, I see the point, but as I said earlier, I think this is a twisted approach from an implementation POV. Allocated memory that can't be used is not allocated memory IMHO. I understand the rationale of the implementation, but I do not agree with it entirely, and think this is a twisted definition of being allocated. I do not completely agree with that being outside of C scope. From the standard, any allocated memory with the above std functions is supposed to be usable if the allocation succeeds. I do think there's a slight problem here, however you see it.

Well, of course. This OTOH doesn't really have anything to do with assigning NULL to a pointer, but just to assigning values to a variable that are never used. (…)

Yes, but it affects assigning NULL. I have explicitly said that I do not deliver this example to deprecate setting NULL in that manner — my goal was to show that it may produce outputs different than one expects if seen from the world outside C. For example while using a debugger.

Sure - but as I said, this is nothing specific to this topic whatsoever. It's just a general thought about statements that get pruned at compile time because the compiler considers they have no effect. Normally, any moderately experienced developer should be aware.

You can really take a much simpler example of this even:

int func(int n)
{
    int a = 1;
    return n;
}

Likewise, there will usually be absolutely no trace of a in the compiled code. Something that should only surprise beginners IMHO.

[andersm]

Extending a bit what I have earlier written about ptr = NULL not neccesserily doing what you think it does, see examples below. Note that I am not claiming that assigning NULL is wrong! Merely showing that there are situations in which the final code will not do exactly the thing you expected it to do, if the compiler notices that this assignment can’t have further effects.

The issue is a bit more insidious than your example shows. This type of code has been used to clear out sensitive data (passwords, crypto keys, etc.) before returning the memory:

Code: [Select]

void safe_free(unsigned char *buf, size_t len)
{
    memset(buf, 0, len);
    free(buf);
}Looking at the disassembly, the memset has been optimized away:

Code: [Select]

0000000000000000 <safe_free>:                           
   0:   48 83 ec 08             sub    $0x8,%rsp       
   4:   e8 00 00 00 00          callq  9 <safe_free+0x9>
   9:   48 83 c4 08             add    $0x8,%rsp       
   d:   c3                      retq                   

[SiliconWizard]

Extending a bit what I have earlier written about ptr = NULL not neccesserily doing what you think it does, see examples below. Note that I am not claiming that assigning NULL is wrong! Merely showing that there are situations in which the final code will not do exactly the thing you expected it to do, if the compiler notices that this assignment can’t have further effects.

The issue is a bit more insidious than your example shows. This type of code has been used to clear out sensitive data (passwords, crypto keys, etc.) before returning the memory:

Code: [Select]

void safe_free(unsigned char *buf, size_t len)
{
    memset(buf, 0, len);
    free(buf);
}Looking at the disassembly, the memset has been optimized away:

Code: [Select]

0000000000000000 <safe_free>:                           
   0:   48 83 ec 08             sub    $0x8,%rsp       
   4:   e8 00 00 00 00          callq  9 <safe_free+0x9>
   9:   48 83 c4 08             add    $0x8,%rsp       
   d:   c3                      retq                   

Now this is indeed a more interesting, and less obvious case of pruning. But the idea is still the same - to the compiler, anything assigned without being used afterwards is considered having no effect.

Even more interesting - you'd think using a volatile qualifier could work around this. Unfortunately, it doesn't here: since memset() itself has no volatile qualifier for its first parameter, whatever you do, the compiler will consider memset() itself to have no useful effect in this context.

The only simple workaround I can think of right now is to implement that yourself: (of course I didn't bother to optimize the loop depending on the pointer alignment, so this would not be as efficient as memset)

Code: [Select]

void safe_free2(unsigned char *buf, size_t len)
{
volatile unsigned char *buf2 = buf;
size_t i;

for (i = 0; i < len; i++)
buf2[i] = 0;
   
    free(buf);
}


[golden_labels]

That bites. I suppose you can disable overcommitting if you're not happy with this?

I’ve checked the mmap+mlock method and it doesn’t work. So the answer is: I have no idea, completely.

Yes, you can disable overcommiting or put limits on how much memory can be allocated (googling reveals a lot of results, relevant for the particular situation and configuration). But then you lose benefits of using overcommit. And the problem of your application being killed due to memory issues is still not completely solved, because during memory exhaustion the OOM killer may still decide to eliminate your process. Having huge swap may seem like a relief, but that comes at the cost of performance.

In other words: sometimes reality trumps abstraction. Otherwise software development would be ten times easier.

andersm:
Though it is a bit off-topic, this is a good point. And, unfortunately, there is still no portable solution. What is employed now is a bunch of platform-specific solutions (like FreeBSD’s explicit_bzero, hoping that volatile pointers will in fact cause overwrite etc.). Even those are still not fully effective.