Products > Programming

How to change a buffer from main RAM (DATA or BSS) to the heap?

<< < (3/5) > >>

Nominal Animal:
I recommend the following pattern:

--- Code: ---TYPE   *buffer_data = NULL;
size_t  buffer_size = 0;

size_t  buffer_need(size_t  size)
{
    if (size <= buffer_size)
        return buffer_size;

    free(buffer_data);
    buffer_data = malloc(size * sizeof buffer_data[0]);
    if (!buffer_data) {
        buffer_size = 0;
        return 0;
    }

    buffer_size = size;
    return size;
}

void buffer_free(void)
{
    free(buffer_data);
    buffer_data = NULL;
    buffer_size = 0;
}

--- End code ---
Before any operation that starts using the buffer, you call buffer_need(number of elements).  It will return the number of elements available in the buffer (which is at least as many as requested), or zero if the dynamic allocation fails; in which case the operation must fail also.

To conserve memory, when you know you won't need the buffer for a long time, you can call buffer_free().  It is not necessary to call it, ever.

Different operations that cannot be in progress at the same time can use the same buffer.  If you have multiple buffers, just use malloc()/calloc()/realloc()/free() for each operation; that's why those functions exist in the first place, and whatever allocator you build on top will usually be less efficient anyway.

The trick to avoiding dynamic memory allocation "costs" is not to avoid calling malloc()/calloc()/realloc()/free(), but to do so when beneficial.

Before anyone asks: free(NULL) is safe to do, and does nothing.  There is no reason to check for a NULL pointer before freeing it.

DavidAlfa:
I think I'm not dereferencing anything. The compiler knows that ucHeap pointers to a ucHeap_t type.
So it knows that *ucHeap[0] is ucHeap address+0, while *ucHeap[n] is ucHeap address+n.
I've seen this a lot of times. Malloc a typedef, then use the pointer to access its elements.

But I'm not a professional programmer, I'm willing to hear more! :D

Nominal Animal:

--- Quote from: DavidAlfa on May 31, 2021, 07:12:03 pm ---I think I'm not dereferencing anything.
--- End quote ---
Yes, you are, the variable ucHeap.  It just happens to point to an array.  Only in this case (*ucHeap)[n] points to the n'th element of that array; not so for non-array types.

Thing is, array indexing has precedence over dereferencing a pointer.  That is, *ucHeap[n] is equivalent to *(ucHeap[n]).  Which means that *ucHeap[1] does NOT refer to the value of the second element in the array; it refers to the value of the first element in the SECOND ucHeap buffer, at offset sizeof *ucHeap .

Run the following program:

--- Code: ---#include <stdlib.h>
#include <stdio.h>

typedef  char  block[1024];

int main(void)
{
    block *b = malloc(2 * sizeof *b);
    if (!b) {
        fprintf(stderr, "Not enough memory.\n");
        return EXIT_FAILURE;
    }

    printf("Address of *b[0] is %p.\n",   &( *b[0]   ));
    printf("Address of *b[1] is %p.\n",   &( *b[1]   ));
    printf("Address of (*b)[1] is %p.\n", &( (*b)[1] ));
    printf("Address of *(b[1]) is %p.\n", &( *(b[1]) ));

    return EXIT_SUCCESS;
}

--- End code ---
On my machine, it prints something like

--- Code: ---Address of *b[0] is 0x559eeb17c260.
Address of *b[1] is 0x559eeb17c660.
Address of (*b)[1] is 0x559eeb17c261.
Address of *(b[1]) is 0x559eeb17c660.

--- End code ---
See the difference of 1024 (0x400) between *b[0] and *b[1]? They are NOT CONSECUTIVE IN MEMORY.

You'd need to use (*b)[0], (*b)[1], and so on, to refer to consecutive elements in the buffer.


--- Quote from: DavidAlfa on May 31, 2021, 07:12:03 pm ---I've seen this a lot of times. Malloc a typedef, then use the pointer to access its elements.
--- End quote ---
The problem here is that the typedef'd type is an array type.  That throws things off.

A very common pattern is to use C99 flexible array members in a structure:

--- Code: ---typedef struct {
    size_t  size;
    char  data[];
} my_char_array;

--- End code ---
To allocate N data elements, you do

--- Code: ---    my_char_array *ma = malloc(N * sizeof ma->data[0] + sizeof (my_char_array));
    if (!ma) {
        fprintf(stderr, "Out of memory.\n");
        exit(EXIT_FAILURE);
    }
    ma->size = N;

--- End code ---
and then you can refer to the elements in the buffer via ma->data[i], for i between 0 and ma->size - 1, inclusive.

newbrain:

--- Quote from: DavidAlfa on May 31, 2021, 07:12:03 pm ---I think I'm not dereferencing anything.

--- End quote ---
You definitely are: an expression such as *ucHeap[n] contains not one, but two dereferences.

The first one is due to the [] postfix operator (C11, 6.5.2.1  Array subscripting) and allow us to rewrite the expression as:

--- Code: ---*(*((ucHeap)+(n)))
--- End code ---
given the definition of [] postfix operator and the fact that it binds stronger than * (see Note)


--- Quote from: DavidAlfa ---So it knows that *ucHeap[0] is ucHeap address+0, while *ucHeap[n] is ucHeap address+n.
--- End quote ---
No, this is patently not true, you are forgetting the two * indirection operators (C11, 6.5.3.2  Address and indirection operators), as described above.

Now let's analyze the rewritten expression for the cases of 0 and configTOTAL_HEAP_SIZE-1 (I set the size to 8192, as an example, my fault not making ti explicit in the answer!).

The variable ucHeap is a pointer to an array of configTOTAL_HEAP_SIZE elements of type uint8_t.
Just to make it absolutely clear: it is not a pointer to uint8_t.

For the case of a 0 offset, the expression is equivalent to:

--- Code: ---*(*(ucHeap+0))
--- End code ---
Obviously, ucHeap + 0 is equal to ucHeap.
The indirection of ucHeap (a pointer to an array) gives us an array of type ucHeap_t.
As arrays decay to a pointer their first element (C11, 6.3.2.1  Lvalues, arrays, and function designators, §3), we obtain a pointer to the first uint_8 in the malloc'ed memory area.
All is good.

Now let's take the case of configTOTAL_HEAP_SIZE-1, for brevity, allow me to use a literal number for configTOTAL_HEAP_SIZE, e.g. 16.

--- Code: ---*(*(ucHeap+15))
--- End code ---
Where will ucHeap + 15 point?
We know that ucHeap is a pointer.
C11, 6.5.6  Additive  operators, §7 tell us that ucHeap is treated as the pointer to the first element of an array of length one of type ucHeap_t: remember, we have defined it as a pointer to an array of 16 uint8_t, not as a pointer to an element of the array (uint8_t).
C11, 6.5.6  Additive  operators, §8 tells us that the result of the addition will point to the 15th element of an array of ucHeap_t.
Translating it in term of address, 15*16 will be added to ucHeap, as each element is an array of 16 bytes (uint8_t).
The behavior is undefined already - as we have a created a pointer exceeding the size of the array - even before applying the two indirection oprators.


--- Quote from: DavidAlfa ---I've seen this a lot of times. Malloc a typedef, then use the pointer to access its elements.
--- End quote ---
Of course. But not in this way: did you check the godbolt link in my answer and what your code was actually doing?


Note: for convenience, I'm using the cppreference site instead of the standard, as the operator precedence rules are not explicitly spelled out in the standard, rather they are derived from the grammar definitions.
Note 2: I've used C11, but there are no relevant changes in these topics since C89.

EtA:

--- Quote from: DavidAlfa ---I'm willing to hear more!
--- End quote ---
You now have both the practical answer from Nominal Animal  :-+ and the (boring) language lawyer answer from me, choose your poison!

Nominal Animal:

--- Quote from: newbrain on May 31, 2021, 09:58:13 pm ---
--- Quote from: DavidAlfa on May 31, 2021, 07:12:03 pm ---I think I'm not dereferencing anything.

--- End quote ---
You definitely are: an expression such as *ucHeap[n] contains not one, but two dereferences.

--- End quote ---
Yup, newbrain is definitely correct here, and using the language used in the C standard.  Me, I was just trying to show the practical side, as reality wins over theory.  Showing exactly why, based on the C standard, shows that the practical observed behaviour is according to the standard, and therefore as expected.
Our two answers are in complete agreement, just look at it from different perspectives.  (Assume any differences are my wording errors; me often fail English.)

All that said, I wish I could recall or understand the problems some C programmers have with pointers and pointer dereferencing.  I honestly just don't perceive those problems at all anymore, and don't remember if I ever did (nor how); the kinds of off-by-one errors I occasionally make with C programs I make in other programming languages as well.  For example, in Python array slicing operations.  So, the cause of those is my personal failure of logic or math, and not a property of the programming language used.  If I was able to perceive or remember how I myself struggled with the more common pointer stuff problems, I'm sure I could be of better help.  Sorry.  :(

Navigation

[0] Message Index

[#] Next page

[*] Previous page

There was an error while thanking
Thanking...
Go to full version