the c-semantics project is aimed against - C undefined behavior -

[tjaeger]

Yes, that's what I was trying to say, that when it's defined they leave it alone. I did read the paper further and changed my mind on kcc being a good idea and here is why:

When it's undefined they just stop execution, for example if those variables where signed then the result will be undefined by the standard so it will stop execution in that case and give a run time error. That itself implies programming overhead, I don't want that.

Guess what, they'll also give you all possible outcomes if your program is non-deterministic.  This is not made (or even possible to) be executed on an embedded processor.  Doesn't mean it's not a worthwhile endeavor.

The worse of it is when I found out that they are actually checking for things that are not even part of the C language, for example checking if strcpy is copying out of bounds. I don't consider strcpy part of C so the compiler shouldn't be even care about functions or their interpretation on how a function should behave.

Nobody gives a crap if you consider strcpy to be part of C or not.  The standard committee clearly did, that's why they included it in the standard.  These things are not hard to look up.

[miguelvp]

The worse of it is when I found out that they are actually checking for things that are not even part of the C language, for example checking if strcpy is copying out of bounds. I don't consider strcpy part of C so the compiler shouldn't be even care about functions or their interpretation on how a function should behave.

Nobody gives a crap if you consider strcpy to be part of C or not.  The standard committee clearly did, that's why they included it in the standard.  These things are not hard to look up.

I guess then all my code has to have the whole stdlib linked in order to run, brilliant!

[miguelvp]

The worse of it is when I found out that they are actually checking for things that are not even part of the C language, for example checking if strcpy is copying out of bounds. I don't consider strcpy part of C so the compiler shouldn't be even care about functions or their interpretation on how a function should behave.

Nobody gives a crap if you consider strcpy to be part of C or not.  The standard committee clearly did, that's why they included it in the standard.  These things are not hard to look up.

I guess then all my code has to have the whole stdlib linked in order to run, brilliant!

Let me rephrase that, some processors actually have an instruction to copy from source to destination until 0 is found on the source. That's a single instruction that allows overlaps. If I do want to use it that way, I guess the committee wont let me, right?

tempted to put Linus picture giving the finger at MS here

Edit: And say what you want, libraries are not part of the C language definition
Edit2: furthermore, I can link to stdlib with cobol if I wanted to, or visual basic, or fortran or pascal or ..... so I guess everything is the C language then. Read the books those are extensions to C by definition not part of it.

Edit3: just for kicks take this test: http://www.indiabix.com/c-programming/library-functions/114001

Edit4: (yeah on a roll here because I don't want to post so I rather edit)
riddle me this, can your recompile the C language itself? now what about the libraries?
And by that I don't meant recompile the compiler I mean the language.
The libraries are written in C, therefore they can't be C themselves!
The language is just definitions not written in a language but interpreted to make a compiler.
And this following comment is as nasty as I can get: Get a CLue!

Edit5: Multiple choice time, simple questions:

Code: [Select]

1) Associate a verb to a compiler:
    a) Run
    b) Link
2) Associate a verb to a library:
    a) Run
    b) Link
3) What generates assembly code from a source:
    a) Library
    b) Compiler
4) What aggregates assembly code to a program:
    a) Library
    b) Compiler
I could keep going but why?

[tjaeger]

I guess then all my code has to have the whole stdlib linked in order to run, brilliant!

Basically, yes, the compiler is free to assume that it can take advantage of the standard library.  Try it out for yourself if you don't believe me:

Code: [Select]

void foo(uint32_t *p) {
  for (int i = 0; i < 128; ++i)
    p[i] = 0;
}
Now run arm-none-eabi-gcc with -O3 and look at the output:

Code: [Select]

foo:
        @ Function supports interworking.
        @ args = 0, pretend = 0, frame = 0
        @ frame_needed = 0, uses_anonymous_args = 0
        stmfd   sp!, {r3, lr}
        mov     r1, #0
        mov     r2, #512
        bl      memset
        ldmfd   sp!, {r3, lr}
        bx      lr
It basically just calls memset.

[miguelvp]

I guess then all my code has to have the whole stdlib linked in order to run, brilliant!

Basically, yes, the compiler is free to assume that it can take advantage of the standard library.  Try it out for yourself if you don't believe me:

Code: [Select]

void foo(uint32_t *p) {
  for (int i = 0; i < 128; ++i)
    p[i] = 0;
}
Now run arm-none-eabi-gcc with -O3 and look at the output:

Code: [Select]

foo:
        @ Function supports interworking.
        @ args = 0, pretend = 0, frame = 0
        @ frame_needed = 0, uses_anonymous_args = 0
        stmfd   sp!, {r3, lr}
        mov     r1, #0
        mov     r2, #512
        bl      memset
        ldmfd   sp!, {r3, lr}
        bx      lr
It basically just calls memset.

That doesn't define C, it's an implementation, so you mean that when you divide numbers on a processor that doesn't have an instruction and the compiler brings _lldiv then _lldiv is part of C?

Please try again!

[tjaeger]

The C standard defines the behavior (not the implementation) of the C standard library.  If you want to split hairs and say that this means that the standard library is not a "part of C" then that's fine with me.  If you take that to mean that an analysis tool isn't allowed to check whether a program invokes a standard library function in an undefined way (as you clearly did above), we have a problem.

[miguelvp]

The C standard defines the behavior (not the implementation) of the C standard library.  If you want to split hairs and say that this means that the standard library is not a "part of C" then that's fine with me.  If you take that to mean that an analysis tool isn't allowed to check whether a program invokes a standard library function in an undefined way (as you clearly did above), we have a problem.

But strcpy is well defined (even if it's not part of the C language), it takes a source and a destination and copies from the source to the destination until the source reaches the value 0 and copies that to the end of the buffer and it's done.

I don't want some tool to tell me that I'm miss-using it because they overlap or I don't have enough space.
I might want to avoid bringing the strtok functions due to space limitations and use strcpy to tokenize strings just because I'm already using it somewhere and using strcpy by making it overrun buffers if I feel that's more optimal for my program.

I have a better brain that the compiler and if I know what I want to do, let me!

[tjaeger]

Let me rephrase that, some processors actually have an instruction to copy from source to destination until 0 is found on the source. That's a single instruction that allows overlaps. If I do want to use it that way, I guess the committee wont let me, right?

You really don't appear to understand undefined behavior.  If you have two overlapping ranges and you want to copy from one to the other, you are not allowed to use strcpy, period.  So if your compiler can statically prove that the ranges overlap, it is perfectly within its rights to "optimize out" the call to strcpy altogether or do any number of nasty things.  The only way to rely on your architecture's behavior is use inline asm.

[tjaeger]

The C standard defines the behavior (not the implementation) of the C standard library.  If you want to split hairs and say that this means that the standard library is not a "part of C" then that's fine with me.  If you take that to mean that an analysis tool isn't allowed to check whether a program invokes a standard library function in an undefined way (as you clearly did above), we have a problem.

But strcpy is well defined (even if it's not part of the C language), it takes a source and a destination and copies from the source to the destination until the source reaches the value 0 and copies that to the end of the buffer and it's done.

I don't want some tool to tell me that I'm miss-using it because they overlap or I don't have enough space.
I might want to avoid bringing the strtok functions due to space limitations and use strcpy to tokenize strings just because I'm already using it somewhere and using strcpy by making it overrun buffers if I feel that's more optimal for my program.

I have a better brain that the compiler and if I know what I want to do, let me!

Then you have to use a different language than C.  Sorry, them's the rules.

[miguelvp]

Let me rephrase that, some processors actually have an instruction to copy from source to destination until 0 is found on the source. That's a single instruction that allows overlaps. If I do want to use it that way, I guess the committee wont let me, right?

You really don't appear to understand undefined behavior.  If you have two overlapping ranges and you want to copy from one to the other, you are not allowed to use strcpy, period.  So if your compiler can statically prove that the ranges overlap, it is perfectly within its rights to "optimize out" the call to strcpy altogether or do any number of nasty things.  The only way to rely on your architecture's behavior is use inline asm.

I do understand undefined behavior, but I also understand historical implementations.
memcpy and strcpy have always been implemented from low memory to high memory
yeah, I get I can do inline assembly but why if I don't need to?

Then again, both of them are not part of C that's my point, I can rewrite them the way I want and link them instead of the ones someone else wrote (in C at that).

[miguelvp]

The C standard defines the behavior (not the implementation) of the C standard library.  If you want to split hairs and say that this means that the standard library is not a "part of C" then that's fine with me.  If you take that to mean that an analysis tool isn't allowed to check whether a program invokes a standard library function in an undefined way (as you clearly did above), we have a problem.

But strcpy is well defined (even if it's not part of the C language), it takes a source and a destination and copies from the source to the destination until the source reaches the value 0 and copies that to the end of the buffer and it's done.

I don't want some tool to tell me that I'm miss-using it because they overlap or I don't have enough space.
I might want to avoid bringing the strtok functions due to space limitations and use strcpy to tokenize strings just because I'm already using it somewhere and using strcpy by making it overrun buffers if I feel that's more optimal for my program.

I have a better brain that the compiler and if I know what I want to do, let me!

Then you have to use a different language than C.  Sorry, them's the rules.

Again, they have nothing to do with C, they are extensions i.e. object files that get linked. C doesn't even do I/O that's the beauty of C that it only cares about code generation and I don't want some compiler telling me about how to use libraries that I can write myself to begin with.

Take for example asctime, what if I don't want it in English and have day of week, month, hour, minute, second & year? Oh but the standard is English, so adapt right?

Well it's the standard! you'll say, but that's not even C, and that's my point.

Anyways, I'm done and spent too much time on this, I won't adopt kcc and stick to my tools. It's been a while since I needed a babysitter.

[tjaeger]

You really don't appear to understand undefined behavior.  If you have two overlapping ranges and you want to copy from one to the other, you are not allowed to use strcpy, period.  So if your compiler can statically prove that the ranges overlap, it is perfectly within its rights to "optimize out" the call to strcpy altogether or do any number of nasty things.  The only way to rely on your architecture's behavior is use inline asm.

I do understand undefined behavior, but I also understand historical implementations.
memcpy and strcpy have always been implemented from low memory to high memory
yeah, I get I can do inline assembly but why if I don't need to?

Then again, both of them are not part of C that's my point, I can rewrite them the way I want and link them instead of the ones someone else wrote (in C at that).

It doesn't work that way.  The compiler is allowed to assume that if it encounters a function named strcpy, this function's behavior conforms to the C standard.  And to do whatever it wants if it encounters undefined behavior.  Most compilers won't be that mean-spirited, but they could if they wanted to.  Try it out for yourself:

Code: [Select]

void foo(uint32_t *p) {
  memset(p, 0, 4);
}
compiles to

Code: [Select]

foo:
        .cfi_startproc
        movl    $0, (%rdi)
        ret
        .cfi_endproc
This happens before any linking takes place.

[andersm]

But the standard defines exactly what should happen in the code sample you posted.

No. In this example, sizeof(int) was supposed to be 2 and the behavior of the code fragment is undefined.

In some ways both of the above statements are actually true.

The promotion rules for the expression evaluation are well defined.

The value of what gets assigned to c is undefined purely because C does not define what happens when an integer expression overflows (normally it's just whatever the underlying CPU does).

The range of nonnegative values of a signed integer type is a subrange of the corresponding unsigned integer type, and the representation of the same value in each type is the same. A computation involving unsigned operands can never overflow, because a result that cannot be represented by the resulting unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the resulting type.

The fragment in question used unsigned integers, so there is nothing undefined about it.

[miguelvp]

here is my foo with memset included on a P12F609

Code: [Select]

_memset:
;__Lib_CString.c,77 ::
;__Lib_CString.c,80 ::
0x0009 0x1283      BCF        STATUS, 5
0x000A 0x084B      MOVF       FARG_memset_p1, 0
0x000B 0x00F2      MOVWF      R2
;__Lib_CString.c,81 ::
L_memset20:
0x000C 0x084D      MOVF       FARG_memset_n, 0
0x000D 0x00F0      MOVWF      R0
0x000E 0x084E      MOVF       FARG_memset_n+1, 0
0x000F 0x00F1      MOVWF      R0+1
0x0010 0x3001      MOVLW      1
0x0011 0x02CD      SUBWF      FARG_memset_n, 1
0x0012 0x1C03      BTFSS      STATUS, 0
0x0013 0x03CE      DECF       FARG_memset_n+1, 1
0x0014 0x0870      MOVF       R0, 0
0x0015 0x0471      IORWF      R0+1, 0
0x0016 0x1903      BTFSC      STATUS, 2
0x0017 0x281E      GOTO       L_memset21
;__Lib_CString.c,82 ::
0x0018 0x0872      MOVF       R2, 0
0x0019 0x0084      MOVWF      FSR
0x001A 0x084C      MOVF       FARG_memset_character, 0
0x001B 0x0080      MOVWF      INDF
0x001C 0x0AF2      INCF       R2, 1
0x001D 0x280C      GOTO       L_memset20
L_memset21:
;__Lib_CString.c,83 ::
0x001E 0x084B      MOVF       FARG_memset_p1, 0
0x001F 0x00F0      MOVWF      R0
;__Lib_CString.c,84 ::
L_end_memset:
0x0020 0x0008      RETURN
; end of _memset
_foo:
;MyProject.c,1 :: void foo(unsigned long *p) {
;MyProject.c,2 :: memset(p, 0, 4);
0x0021 0x1283      BCF        STATUS, 5
0x0022 0x084A      MOVF       FARG_foo_p, 0
0x0023 0x00CB      MOVWF      FARG_memset_p1
0x0024 0x01CC      CLRF       FARG_memset_character
0x0025 0x3004      MOVLW      4
0x0026 0x00CD      MOVWF      FARG_memset_n
0x0027 0x3000      MOVLW      0
0x0028 0x00CE      MOVWF      FARG_memset_n+1
0x0029 0x2009      CALL       _memset
;MyProject.c,3 :: }
L_end_foo:
0x002A 0x0008      RETURN
; end of _foo

Seems it has no trouble at all, and that's as primitive as a chip can be.

Edit: But yeah it can be optimized if you are setting a 32bit unsigned integer and your processor can clear it that way. But memcpy and strcpy can be optimized that way.

Still memcpy, memset, strcpy etc are not part of C, implementation aside.

[tjaeger]

But the standard defines exactly what should happen in the code sample you posted.

No. In this example, sizeof(int) was supposed to be 2 and the behavior of the code fragment is undefined.

In some ways both of the above statements are actually true.

The promotion rules for the expression evaluation are well defined.

The value of what gets assigned to c is undefined purely because C does not define what happens when an integer expression overflows (normally it's just whatever the underlying CPU does).

The range of nonnegative values of a signed integer type is a subrange of the corresponding unsigned integer type, and the representation of the same value in each type is the same. A computation involving unsigned operands can never overflow, because a result that cannot be represented by the resulting unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the resulting type.

The fragment in question used unsigned integers, so there is nothing undefined about it.

This was the fragment in question, as far as I can tell.

Code: [Select]

unsigned int a = 1000, b = 1000;
long int c = a * b;


[tjaeger]

here is my foo with memset included on a P12F609

Code: [Select]

_memset:
;__Lib_CString.c,77 ::
;__Lib_CString.c,80 ::
0x0009 0x1283      BCF        STATUS, 5
0x000A 0x084B      MOVF       FARG_memset_p1, 0
0x000B 0x00F2      MOVWF      R2
;__Lib_CString.c,81 ::
L_memset20:
0x000C 0x084D      MOVF       FARG_memset_n, 0
0x000D 0x00F0      MOVWF      R0
0x000E 0x084E      MOVF       FARG_memset_n+1, 0
0x000F 0x00F1      MOVWF      R0+1
0x0010 0x3001      MOVLW      1
0x0011 0x02CD      SUBWF      FARG_memset_n, 1
0x0012 0x1C03      BTFSS      STATUS, 0
0x0013 0x03CE      DECF       FARG_memset_n+1, 1
0x0014 0x0870      MOVF       R0, 0
0x0015 0x0471      IORWF      R0+1, 0
0x0016 0x1903      BTFSC      STATUS, 2
0x0017 0x281E      GOTO       L_memset21
;__Lib_CString.c,82 ::
0x0018 0x0872      MOVF       R2, 0
0x0019 0x0084      MOVWF      FSR
0x001A 0x084C      MOVF       FARG_memset_character, 0
0x001B 0x0080      MOVWF      INDF
0x001C 0x0AF2      INCF       R2, 1
0x001D 0x280C      GOTO       L_memset20
L_memset21:
;__Lib_CString.c,83 ::
0x001E 0x084B      MOVF       FARG_memset_p1, 0
0x001F 0x00F0      MOVWF      R0
;__Lib_CString.c,84 ::
L_end_memset:
0x0020 0x0008      RETURN
; end of _memset
_foo:
;MyProject.c,1 :: void foo(unsigned long *p) {
;MyProject.c,2 :: memset(p, 0, 4);
0x0021 0x1283      BCF        STATUS, 5
0x0022 0x084A      MOVF       FARG_foo_p, 0
0x0023 0x00CB      MOVWF      FARG_memset_p1
0x0024 0x01CC      CLRF       FARG_memset_character
0x0025 0x3004      MOVLW      4
0x0026 0x00CD      MOVWF      FARG_memset_n
0x0027 0x3000      MOVLW      0
0x0028 0x00CE      MOVWF      FARG_memset_n+1
0x0029 0x2009      CALL       _memset
;MyProject.c,3 :: }
L_end_foo:
0x002A 0x0008      RETURN
; end of _foo

Seems it has no trouble at all, and that's as primitive as a chip can be.

Congratulations! On this particular compiler, with the particular optimization settings you used, memset doesn't get "inlined".  So you'll probably be fine with an overlapping memcpy, too.  You'd still be relying on undefined behavior, though.  Just because you can get away with it doesn't make it right.

[IanB]

The range of nonnegative values of a signed integer type is a subrange of the corresponding unsigned integer type, and the representation of the same value in each type is the same. A computation involving unsigned operands can never overflow, because a result that cannot be represented by the resulting unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the resulting type.

The fragment in question used unsigned integers, so there is nothing undefined about it.

Thanks for that.

So the standard says that "1000 * 1000" with unsigned 16 bit integer types must produce 1000000 modulo 65536 = 16960? And that this is not implementation defined, but required. Now that I have access to my copy of K&R I see that this is indeed the case.

[miguelvp]

But the standard defines exactly what should happen in the code sample you posted.

No. In this example, sizeof(int) was supposed to be 2 and the behavior of the code fragment is undefined.

In some ways both of the above statements are actually true.

The promotion rules for the expression evaluation are well defined.

The value of what gets assigned to c is undefined purely because C does not define what happens when an integer expression overflows (normally it's just whatever the underlying CPU does).

The range of nonnegative values of a signed integer type is a subrange of the corresponding unsigned integer type, and the representation of the same value in each type is the same. A computation involving unsigned operands can never overflow, because a result that cannot be represented by the resulting unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the resulting type.

The fragment in question used unsigned integers, so there is nothing undefined about it.

This was the fragment in question, as far as I can tell.

Code: [Select]

unsigned int a = 1000, b = 1000;
long int c = a * b;


Even the OP paper mentions that the operation is totally defined

What if we make the types of a and b unsigned (0 to 65535)?

Code: [Select]

unsigned int a = 1000, b = 1000;
long int c = a * b;
Here, the arithmetic is again performed at the level of the operands,
but overflow on unsigned types is completely defined in C. The result
is computed by simply reducing the value modulo one more than
the max value [15, §6.3.1.3:2]. 1000000 mod 65536 gives us 16960

[tjaeger]

This was the fragment in question, as far as I can tell.

Code: [Select]

unsigned int a = 1000, b = 1000;
long int c = a * b;


Sorry about this, I finally see what you are saying. Yes, you're absolutely right, this is well-defined.  In my head, the fragment read as follows the whole time (which is actually the first example they give in the paper).

Code: [Select]

int a = 1000, b = 1000;
long int c = a * b;


[grumpydoc]

In my head, the fragment read as follows the whole time (which is actually the first example they give in the paper).

You're not the only one to have read it like that - I did as well.

[hamster_nz]

here is my foo with memset included on a P12F609

Code: [Select]

...

Seems it has no trouble at all, and that's as primitive as a chip can be.

Still memcpy, memset, strcpy etc are not part of C, implementation aside.

How about a recursive function?

Code: [Select]

   unsigned factorial(unsigned n)
   {
      return n > 1 ? n * factorial(n-1) : 1;
   }

IIRC that chip as 8 levels of call stack - do you think it factorial( 8 ) will work? How is the support for floats and doubles? also part of the standard C? How about using pointers to functions?

I guess a really smart compiler could implement jump tables and other tricks, but it is pretty clear  to me that C isn't a natural fit to these MCUs.

[tjaeger]

And to do whatever it wants if it encounters undefined behavior.  Most compilers won't be that mean-spirited, but they could if they wanted to.

Maybe I was a little hasty here.  Check out gcc (version 4.8.2):

Code: [Select]

#include <stdint.h>
#include <string.h>
#include <stdio.h>

uint64_t __attribute__ ((noinline)) foo(size_t i) {
  uint64_t a[3] = {1, 2, 3};
  memcpy(a+1, a, 2*sizeof(uint64_t));
  return a[i];
}

uint64_t bar(size_t i) {
  uint64_t a[3] = {1, 2, 3};
  memcpy(a+1, a, 2*sizeof(uint64_t));
  return a[i];
}

void *memcpy(void *dest, const void *src, size_t n) {
  printf("Why won't anyone pay attention to me?\n");
  return NULL;
}

int main() {
  printf("%lu, %lu\n", foo(2), bar(2));
}
If I compile with -O2 this outputs

Code: [Select]

1, 2
The only difference between foo() and bar() is that foo() is not allowed to be inlined.  For both clang and gcc, the output of the program changes with optimization settings.

[hamster_nz]

Code: [Select]

uint64_t __attribute__ ((noinline)) foo(size_t i) {
  uint64_t a[3] = {1, 2, 3};
  memcpy(a+1, a, 2*sizeof(uint64_t));
  return a[i];
}

For some reason, seeing non-static arrays declared within functions with it's elements being initialized makes me feel a little bit ill... I know it is a contrived example but EEEWWWW!

[legacy]

the issue seems related to uint64_t

[dannyf]

real men code;

everybody else debates about coding.