GCC ARM32 compiler too clever, or not clever enough?

[ataradov]

C11 standard quote:

If there are fewer initializers in a brace-enclosed list than there are elements or members of an aggregate, or fewer characters in a string literal used to initialize an array of known size than there are elements in the array, the remainder of the aggregate shall be initialized implicitly the same as objects that have static storage duration.

So, this initialization would set the remaining elements to 0, even for the automatic variables.

Note that technically 0 is not required here, but the default initializer for static objects is 0 (section 6.7.9 of the same standard).

[Siwastaja]

That's just horrible, because whether the whole array or just the first element get zeroed is compiler dependent, no?
...
That's even if N has been correctly evaluated at compile-time... I didn't think that worked at all anyway.

May I suggest you do yourself a favor, and instead of assuming every time, just... look it up. Either directly from the standard, or if you for some reason don't like the language of standard, just google it.

Checking how it actually works usually takes 1-5 minutes; much much less than working under incorrect assumptions and misunderstanding other's code, or worse, writing incorrect code for years. (In this particular case, you wouldn't be writing incorrect code per se, because you would be just avoiding a perfectly valid construct. But OTOH, you could misunderstand other's code and waste time looking for a bug somewhere where it cannot be. Or, you could waste time writing explicit loop doing an initialization which would have worked with much less typing, and smaller risk of creating bugs.)

I have developed the habit of looking it up every time I'm even a bit unsure. The great thing about C is, it's quite well standardized, and those parts that are "implementation defined" or "undefined", are clearly said so.

[SiliconWizard]

Code: [Select]

int a[N] = { 0 };
That's just horrible, because whether the whole array or just the first element get zeroed is compiler dependent, no?

Absolutely not, but others already replied. =)

[gf]

Code: [Select]

int a[N] = { 0 };
That's just horrible, because whether the whole array or just the first element get zeroed is compiler dependent, no?

Absolutely not, but others already replied. =)

Of course it is sufficient if it calculates a result as if it were zeroed, e.g. https://godbolt.org/z/T5KboPcYr (the zero return value comes from the last zero-initialized byte in s[N+1]).

[TheCalligrapher]

Code: [Select]

int a[N] = { 0 };
That's just horrible, because whether the whole array or just the first element get zeroed is compiler dependent, no?

No.

1. All intialization in C follows the "all-or-nothing" principle. It is not possible to initialize only a part of an aggregate (array of structure) in C. When you supply an explicit initalizer for a just a little part of an aggregate, you can rest assured that the remaining parts of the aggregate will be initialized to zero.

2. In C `= { 0 }` is an idiomatic universal zero-initializer. It works with everything. It sets everything to zero. (Except one can't use with VLAs. Which is an oversight.)

[TheCalligrapher]

Of course it is sufficient if it calculates a result as if it were zeroed, e.g. https://godbolt.org/z/T5KboPcYr

The example is actually illegal in both standard C and standard C++. C++ does not allow `char s[N+1]` for such an `N`. C does not allow `= { 0 }` for such an `s`.

[brucehoult]

Of course it is sufficient if it calculates a result as if it were zeroed, e.g. https://godbolt.org/z/T5KboPcYr

The example is actually illegal in both standard C and standard C++. C++ does not allow `char s[N+1]` for such an `N`. C does not allow `= { 0 }` for such an `s`.

Clang diagnoses this correctly. Maybe gcc regards it as an extension.

It's even funnier if you make it N instead of N+1. Then gcc returns a random uninitialised byte from the stack -- which in the case of x86 is a byte from the return address.

[Nominal Animal]

Since C17/C18 (both refer to exactly the same C standard revision), a standards-compliant C compiler does not need to support variable-length arrays anyway.
(Those that do not support them, should pre-define the __STDC_NO_VLA__ preprocessor macro.)

I don't mind.  In practice, VLAs tend to have OS-dependent practical limits (specifically, stack size or stack size growth mechanism) that make them not very useful anyway.

It is, however, important to differentiate between variable-length arrays/variably modified types, and flexible array members.  Sometimes new C programmers do confuse the two (variably modified types and flexible array members), by trying to declare a variable of a type containing a flexible array member, which is not allowed.  (You can only have pointers to such types.)

Flexible array members are supported in all C standard revisions since C99, but not in any C++ standard revisions.  For example,

Code: [Select]

struct bst_node {
    struct bst_node *le;
    struct bst_node *gt;
    char data[];  /* Flexible array member */
};

struct bst_node *bst_node_new_mem(const char *src, size_t len)
{
    struct bst_node *new_node;

    new_node = malloc(sizeof (struct bst_node) + len + 1);
    if (!new_node)
        return NULL;

    new_node->le = NULL;
    new_node->gt = NULL;
    if (len > 0) {
        memcpy(new_node->data, src, len);
    }
    new_node->data[len] = '\0';

    return new_node;
}

struct bst_node *bst_node_new(const char *src)
{
    return bst_node_new_mem(src, (src) ? strlen(src) : 0);
}
defines a bst_node type with a flexible array member data, so that instances of that type have no limitations on the data string length other than what the implementation (hardware architecture, operating system, etc.) sets anyway.  The bst_node_new() and bst_node_new_mem() show how to properly create variables of such a type.

[gf]

Of course it is sufficient if it calculates a result as if it were zeroed, e.g. https://godbolt.org/z/T5KboPcYr

The example is actually illegal in both standard C and standard C++. C++ does not allow `char s[N+1]` for such an `N`. C does not allow `= { 0 }` for such an `s`.

Yes, you are right, I was thoughtless, it's just a GCC extension. Still s[N] is zero - from the initialization - and the function returns zero. Without initialization, it returns a random value from the stack, which is OK as well, when accessing an uninitialized variable.
I just wanted to demonstrate that the compiler does not need to generate any code which actually allocates and initializes the array variable. It just has to calculate the same result as if it were allocated and initialized (and it must preserve all visible side effects, like volatile access, for instance).

It's even funnier if you make it N instead of N+1. Then gcc returns a random uninitialised byte from the stack -- which in the case of x86 is a byte from the return address.

That's UB and therefore OK, since "return s[N]" references a value beyond the end of the array, if sizeof(s) == N.

[SiliconWizard]

Of course it is sufficient if it calculates a result as if it were zeroed, e.g. https://godbolt.org/z/T5KboPcYr

The example is actually illegal in both standard C and standard C++. C++ does not allow `char s[N+1]` for such an `N`. C does not allow `= { 0 }` for such an `s`.

I can't tell for C++ (don't know it enough), but I'll trust you on that.

For C, as I said before, it certainly is illegal.
Using GCC with the same code (just changed the include for C), GCC gives not just a warning, but an error, as it should:

Code: [Select]

#include <string.h>

static int f(const char *p, size_t N)
{
    char s[N+1] = {0};
    memcpy(s, p, N);
    return s[N];
}

int g()
{
    static const char s[] = "abcd";
    return f(s, sizeof(s));
}


VLA1.c: In function 'f':
VLA1.c:5:5: error: variable-sized object may not be initialized
    5 |     char s[N+1] = {0};
      |     ^~~~

[TheCalligrapher]

Clang diagnoses this correctly. Maybe gcc regards it as an extension.

Neither Clang nor GCC is even trying to diagnose anything correctly until one supplies `-std=...` and `-pedantic` in the command line. Basically, C or C++ begin with these command-line switches. Without them the aforementioned implementations have only superficial connection to C or C++.

[TheCalligrapher]

I don't mind.  In practice, VLAs tend to have OS-dependent practical limits (specifically, stack size or stack size growth mechanism) that make them not very useful anyway.

^^^ The same strange fallacy based on complete lack of understanding what VLA is and what its purpose is. VLA have absolutely no issues with "practical limits" and no connection to "stack size or stack size growth mechanism" whatsoever.

VLA is and has always been nothing more than a piece of `size_t` sidecar data accompanying each variable modified type. A single `size_t` per type is too negligible to bump into any practical limits.

VLA is an absolutely necessary feature of the language since it covers a very critical omission in C array support model. This

Code: [Select]

void foo(size_t n, size_t m, int a[n][m])
{
  /* access A[i][j] using natural syntax */
}

Without VLA we are forced to employ low-grade cludges or ugly workarounds or to hack up a replacement.

[gf]

Yet another use case are still local variables having a size which is not known at compile time. Using malloc() + free() instead were even more expensive.
Even the old Unixes from the 1970-ies or 1980-ies already did "invent" alloca(), for allocating blocks of variable size on the stack, with function-local lifetime. Obviously there was a need for it (although there was of course no direct language support yet in K&R C).

[TheCalligrapher]

Yet another use case are still local variables having a size which is not known at compile time.

Full example:

Code: [Select]

#include <stdio.h>

void foo(size_t n, size_t m, int a[n][m])
{
  for (size_t i = 0; i < n; ++i)
  {
    for (size_t j = 0; j < m; ++j)
      printf("%d ", a[i][j]);
    printf("\n");
  }
}

int main()
{
  int a[3][3] = { { 1, 2, 3 }, { 4, 5, 6 }, { 7, 8, 9 } };
  foo(3, 3, a);
 
  int b[2][5] = { { 0, 1, 2, 3, 4 }, { 5, 6, 7, 8, 9 } };
  foo(2, 5, b);
}
http://coliru.stacked-crooked.com/a/186e848798907c99

This example is critically dependent on VLA functionality, yet it does not have any "local variables having a size which is not known at compile time".

This is what VLA is for. First and foremost it is an extension of type system, not a feature for creating run-time-sized arrays on the stack.

[Nominal Animal]

I don't mind.  In practice, VLAs tend to have OS-dependent practical limits (specifically, stack size or stack size growth mechanism) that make them not very useful anyway.

^^^ The same strange fallacy based on complete lack of understanding what VLA is and what its purpose is. VLA have absolutely no issues with "practical limits" and no connection to "stack size or stack size growth mechanism" whatsoever.

Fallacy my ass.  Even the C standard itself makes a distinction between "variable length array" and "variably modified types", even though they are controlled by the exact same mechanism, and closely related.  You took a shortcut, and claim the two are the exact same thing.

When you declare an array size in a function prototype, for example
    double  determinant(size_t rows, size_t cols, double matrix[rows][cols]);
the third parameter has a variably modified type.  While you claim this is essential, it is extremely rarely used in existing C or POSIX C code, because explicitly specifying the data stride is just much more versatile:
    #define  DET(m)  (determinant(sizeof (m) / sizeof *(m), sizeof *(m) / sizeof **(m), sizeof *(m) / sizeof **(m), 1, (const double *)(m)))
    double  determinant(size_t rows, size_t cols, ssize_t rowstride, ssize_t colstride, double *origin);
since the latter supports both row-major and column-major data orders (and trivial transpose), submatrices, and so on.  The DET() preprocessor macro calls the function with the correct parameters, when a two-dimensional double array variable is specified; this avoids problems with the array dimensions (a typical problem, when the programmer changes the array dimensions in one place, but forgets to update them everywhere).

For what it is worth, I have no problems with either of the above, but the former does fall under the same optional support depending on __STDC_NO_VLA__.  For linear algebra, I have a very nice library approach that hides the complexity in two structures – one describing the matrices, and the other containing the matrix data in refcounted form – with views and submatrices indistinguishable from primary matrices; the outline of which I've described in e.g. here.  I am also very familiar with BLAS and LAPACK (and the Intel and AMD math libraries that implement them) and GSL, and have contributed to dozens of projects from the GNU C library to the Linux kernel, and have examined the sources of hundreds if not thousands of open source C projects, so I do claim I know pretty well what kind of code is actually used in practice, and why.  Theory is one thing; practical matters are much more important in real life.

What has become much more common in existing C or POSIX C code, is declaring variables of variably modified types, "local variable length arrays":
    void foo(size_t rows, size_t cols) {
        double  cache[rows][cols];
exactly because people think "malloc() is expensive and prone to bugs".  This is the problematic case, and is not supported by the C compiler if it defines __STDC_NO_VLA__.

For multi-threaded POSIX C programs, the stack often has a relatively small fixed size limit; which means that single-threaded code (due to the automatically growing stack) has completely different stack size limit than multithreaded code, making silly developers think "oh, multithreaded code is just too hard and unpredictable".

The stack access scheme for these has the same issue as large local variable arrays on stack on many OSes, including Linux.  When the stack has no defined upper size limit, in virtual memory systems it is implemented using guard pages.  (When there is a strict upper limit, the entire stack area is reserved in virtual memory, just not populated with RAM yet.  Because virtual memory is not "free", this scheme actually uses much more total RAM than the guard page approach.)  Essentially, after the stack, you have an area of memory that is allocated but inaccessible.  The first access to this area causes the OS kernel to populate the guard pages (backed by RAM, so that they can be made accessible), and set up new guard pages just after.  The guard page mechanism avoids the need to set up page tables for example for a gigabyte of stack; if the new guard pages cannot be set up, the kernel simply notifies the process about a segmentation violation, which typically causes the process to abort.
The problem appears when a function has more than the guard pages' worth of local variables.  If the existing stack is also nearly full at the time the function is called, the first access to stack (local variables) can be beyong the guard pages, and lead to immediate segmentation violation.
I have seen this in practical code, when buffer sizes are increased, and the programmer or the code assumes that local variables do not have any practical size limits – but they do.

I do not care what the C standard says about this, because this is practical, and as a system admin, I needed this – be able to limit stack and heap sizes separately – to catch leaky code early enough.  (Consider simulations running for days, in a distributed HPC cluster.)
For me, the C standards since C99 are like Nobel Peace Prize: a fantasy of what the world is, and an attempt to manipulate the world toward that view.  It is useful, but not always practical; and we live in the practical world, not in that fantasyland.

[peter-h]

Gosh I've been away travelling and look at the trouble I have started again

Are you experts telling me that

uint8_t fred[1000] = {0x22};

will set all 1000 bytes to 0x22, and this is not compiler dependent?

I wonder how this is implemented. In a small case

uint_t fred[] = {"this is a few bytes"};

the string, with an 0x00 at the end, is stored in the initialised data section (DATA), and there is a loop in startupxxx.s which copies it over (FLASH -> RAM).

But a case like [1000] above, that would be dumb, and the compiler should need to use a loop. But maybe it doesn't.

[ataradov]

will set all 1000 bytes to 0x22, and this is not compiler dependent?

No, it will set the first byte to 0x22 and the rest to 0

How it is done depends on the type of the variable. Global variables (static storage) will just go into a data section as one big array, the same way as if you explicitly specified all those extra 0s.

For an automatic variable there may be different approaches the compiler can take, including memset() to 0 and initializing just the first byte. Same will happen with your string example for a local variable - it will not just go into the data section. The variable will be located on the stack and will be initialized (memcpy) at the time of declaration (provided there are no possible optimizations).

And again, compiler explorer is your friend. Here https://godbolt.org/z/rTjrsTjxr you can see that Clang does call a memset() followed by an initialization of the first member.

[peter-h]

If the 1000 byte array is static and uninitialised then it is in COMMON which is not set to anything at startup.
If the 1000 byte array is static and initialised then it is in DATA which at startup is initialised from values in FLASH.
If any variable is initialised to 0 then it goes into BSS which is zeroed at startup.

I've never tried the case of a 1000 byte array set with { 0 } - I guess it would go into BSS, but would it, given the { 0 } sets only the first element?

The above is what I have found with ST ARM32 GCC.

[ataradov]

There is no COMMON. There is only data and BSS. All static variables that are initialized go into the data segment, everything else goes into BSS.

I already  quoted a part of the C standard that says that all static variables that are not explicitly initialized would be set to 0 (or NULL). It is impossible to have a static variable that has an undefined value.

[SiliconWizard]

As far as C is concerned, global data is either initialized with defined values if provided, or filled with zeros otherwise.

But what peter is seeing is implementation-specific stuff.

GCC puts data in a pretty large number of different sections actually. 'common' is one of them, but there are literally tens of them.

That's the linker script job to aggregate those sections into fewer sections. In my linker scripts (as with almost all default scripts I've seen), the 'common' subsection (and a number of others) is put within the .bss section, so that becomes zeroed out at startup. But specific linker scripts can do otherwise for a number of reasons. Note that in this case, the result is not compliant with the standard anymore.

Not placing the 'common' section emitted by GCC into the BSS section in the linker script allows to leave non-explicitely initialized global variables untouched, which can save some time at startup.
Again, it will give a non-conforming behavior though.

[newbrain]

uint8_t fred[1000] = {0x22};

will set all 1000 bytes to 0x22, and this is not compiler dependent?

If the 1000 byte array is static and uninitialised then it is in COMMON which is not set to anything at startup.

peter-h, this is not the first time I say it, but at the risk of becoming unpleasant, I'll repeat it one more time:
Please, read and study the language.
Find a good book, the standard, an online course or a tutor* and clean up all the misconceptions you seem to have.
(these are not the first, you often seem surprised by not so obscure features of the language).
Then, how  a compiler implements something mostly becomes a moot point (but I love Godbolt too), unless you're going for the utmost optimization or you find a bug (rare, but it happened to me recently).

I'm sorry if I seem patronizing or insulting, please understand this is absolutely not my intention.


*to each their own preferred method of learning.
After a bad experience with a Shildt book, I relearned C directly from the C99 standard - but I understand it's not everyone's cup of tea.

[peter-h]

I don't get offended if someone just tells me I am wrong But I am not clever enough / have enough time, to read 1000 page compiler standards. Especially when the behaviour is so system dependent.

I looked at the setup of my project, which is based on the ST Cube IDE setup and code examples.

The linker script does indeed lump together BSS and COMMON (which may be something I did) and this is all zeroed at startup (in the asm code):

Code: [Select]

/* Uninitialized data section */
  . = ALIGN(4);
  .bss :
  {
    /* This is used by the startup in order to initialize the .bss section */
    _sbss = .;         /* define a global symbol at bss start */
    __bss_start__ = _sbss;
    *(.bss)
    *(.bss*)
    *(COMMON)
    *(.common)
    *(.common*)
    . = ALIGN(4);
    _ebss = .;         /* define a global symbol at bss end */
    __bss_end__ = _ebss;
  } >RAM


Code: [Select]

  ldr  r2, =_sbss
  b  LoopFillZerobss
/* Zero fill the bss segment. */ 
FillZerobss:
  movs  r3, #0
  str  r3, [r2], #4
LoopFillZerobss:
  ldr  r3, = _ebss
  cmp  r2, r3
  bcc  FillZerobss
(yes I know the above asm is strange but that's how ST did it for their 32F407/417).

Now, the Q is what ends up in BSS and what ends up in COMMON. So I did these statics:

Code: [Select]

int fred111;
int fred222=0;
int fred333=1;

Looking at the .map file, I see

fred111 is in BSS
fred222 is in BSS
fred333 is in DATA (which is in FLASH, and gets copied to RAM at init)

The above is probably correct as per what people above are saying, but is different to what I determined a year or two ago when I was doing this testing, but I don't know what has changed. Nothing in my project, but with Cube 1.9 GCC has gone from v9 to v10. The previous behaviour was documented by me as follows, based on exactly these tests:

With ARM GCC, statics are categorised thus:
int fred; goes into COMMON
int fred=0; goes into BSS (which by definition is zeroed by startup code)
int fred=1; goes into DATA (statics initialised to nonzero)


So something has changed from GCC v9 to v10 in that v10 doesn't seem to be generating anything for COMMON. Or it could be Cube 1.9 running GCC with a different script; I never looked at that stuff.

[ataradov]

This behaviour has not changed in a log time, probably ever.

The reason for common to exist is some real legacy stuff. It allows multiple compilation units to declare an extern variable, and I think there is a reason why this is necessary in fortran.

For C there is a flag "-fno-common" to just place things into bss. And I think C++ always acts as if this flag is specified because having those commons may break name mangling or something like this.

And yes, for compatible behaviour you have to place all common symbols into BSS. Don't use this for uninitialized variables, it might break things badly, possibly producing duplicate allocations for the same variable.

[peter-h]

Don't use this for uninitialized variables,

Why not? You don't care what the value is anyway

Uninitialised statics do end up in BSS currently, so

int fred1;
int fred2=0;

are equivalent.

That said, the vast majority of C code and variable declarations are inside functions, where initialisation is different and fred1 will probably be genuinely undefined. I think statics are used mostly for globals which most people will initialise explicitly at declaration. In the old Z80 etc days, statics produced much faster code than stack based variables but I don't think that's true anymore.

[ataradov]

Why not? You don't care what the value is anyway

If you don't collect common objects, you risk running into strange situations with memory allocations. You can still have them uninitialized, just collect them into a separate section.

Also, it looks like at some point they enabled "-fno-common" by default, so commons are no longer issued by recent compilers unless you force them. And this is a good thing.

EDIT: According to their bugzilla, it is the default since GCC 10. So you can just forget that commons existed.