Author Topic: Cortex-M3+ GCC unaligned access quirks  (Read 6470 times)

0 Members and 1 Guest are viewing this topic.

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 3915
  • Country: gb
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #25 on: September 25, 2023, 04:38:52 am »
frankly, having the compiler replace ANY function call as part of optimization because it "knows" what that function does is annoying.  C used to be so "pure" in that sense - a whole language definition with no pre-written functions!  Sigh.  As someone who spent a long time programming in assembly language, I tend to think in terms of generic pointers - a memory address that can point to anything, and be treated appropriately.  And I don't understand why the language people seem so resistant to providing that service.  Clearly the memcpy-that-isn't shows that the compiler can defeat optimization issues when it wants to - why can't I have a type-punning mechanism that does that it a more intuitively efficient manner?

that's one reason i wrote my own c-like compiler: tired of that stuff, tired of asking for it not to be done and being ignored.
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Online gf

  • Super Contributor
  • ***
  • Posts: 1187
  • Country: de
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #26 on: September 25, 2023, 08:17:26 am »
and the following condition must be met in order that dereferencing ptr1 is not UB
Code: [Select]
ptr1 != NULL && (uintptr_t)ptr1 % _Alignof(S1) == 0

this is also what my-c does, and that condition must be met, otherwise (for UB avoidance) it refuses to compile.

The problem is, as long as reinterpret casts are allowed, you can only check that at compile time if you can track the data flow statically to the origin of the pointer value. In many (most?) cases you cannot.
 

Offline Jeroen3

  • Super Contributor
  • ***
  • Posts: 4078
  • Country: nl
  • Embedded Engineer
    • jeroen3.nl
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #27 on: September 25, 2023, 08:37:45 am »
This topic is the a nice summary of why C is both awesome and awful at the same time.
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4040
  • Country: nz
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #28 on: September 25, 2023, 08:48:12 am »
This topic is the a nice summary of why C is both awesome and awful at the same time.

C is not perfect by any means, but it has evolved and been battle-hardened for 50 years to be the best option for software that should be at the same time portable to everything and also high performance.
« Last Edit: September 25, 2023, 05:55:57 pm by brucehoult »
 
The following users thanked this post: newbrain

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 3915
  • Country: gb
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #29 on: September 25, 2023, 01:45:27 pm »
and the following condition must be met in order that dereferencing ptr1 is not UB
Code: [Select]
ptr1 != NULL && (uintptr_t)ptr1 % _Alignof(S1) == 0

this is also what my-c does, and that condition must be met, otherwise (for UB avoidance) it refuses to compile.

The problem is, as long as reinterpret casts are allowed, you can only check that at compile time if you can track the data flow statically to the origin of the pointer value. In many (most?) cases you cannot.

That's why casting is not allowed in my-c and you are always forced to define a datatype with its explicit alignment.

It may seem limiting, but in the long run it helps to make fewer messes.
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline mikerj

  • Super Contributor
  • ***
  • Posts: 3240
  • Country: gb
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #30 on: September 25, 2023, 04:10:30 pm »
by totally avoiding pointers (except when using the &variable when passing parms to functions, etc) I am saving myself a load of hassle :)

Totally avoid using any pointers except for when you use a pointer, got it  :D

Tbh you'd have to jump through a lot of hoops to not use pointers on any reasonably complex program, and doing so is likely to make it more complex and very probably slower.
 
The following users thanked this post: Siwastaja

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14490
  • Country: fr
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #31 on: September 25, 2023, 09:34:30 pm »
This topic is the a nice summary of why C is both awesome and awful at the same time.

Given that a significant chunk of this thread is about non-standard C (like using compiler-specific attributes to twist UB), everything goes.
So maybe one could change your statement to "the reason why C is both awesome and awful at the same time is that compilers add various extensions that make it an easily moving target unless one strictly sticks to the standard".
(In which case, people realize that C is more portable, but a lot less flexible than what they thought.)

As for me, I think the reason C is "awesome" was summed up by brucehoult above, and the reason it is "awful" is mainly that it's one of the least well taught programming languages of all times.
« Last Edit: September 25, 2023, 09:38:18 pm by SiliconWizard »
 
The following users thanked this post: hans, newbrain

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 8180
  • Country: fi
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #32 on: September 26, 2023, 07:55:17 am »
Totally avoid using any pointers except for when you use a pointer, got it  :D

This is a repeating discussion with peter-h.

"I don't use any pointers, at all, anywhere"
"How can you do anything at all then? Your code must be complete mess."
"Oh, I just use pointers to do this and that."

I agree with his idea of avoiding excess or "too clever" pointer use when not truly necessary, but I don't understand why it has to be represented this way. C programming without pointers at all is, if not impossible, painful enough. A pointer is truly useful feature in computers and in heavy use starting from assembly, and nearly all programming languages have pointers, or "pointers" under a different name (possibly with more advanced features providing safety, but still fundamentally pointers).
« Last Edit: September 26, 2023, 08:14:47 am by Siwastaja »
 
The following users thanked this post: mikerj

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 8180
  • Country: fi
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #33 on: September 26, 2023, 08:04:52 am »
and the reason it is "awful" is mainly that it's one of the least well taught programming languages of all times.

Bingo. I went to uni in 2005 and programming courses were basically this:

"Language does not matter at all. We are not teaching languages but concepts. We could use any synthetic language."
what utter bullshit! Of course it pays off to teach languages. We also had natural language (English, Swedish) courses. Math courses taught the math notation, not just "concepts", or not a synthetic separate "for teaching" math notation.

"C is obsolete, not worth teaching"
still bullshit, even more bullshit in 2005. Think about GNU and the linux kernel, what a huge ecosystem, mostly in C, not to mention embedded.

"C is difficult and error-prone"
Maybe this could be addressed by... drumroll... teaching? >90% of the "error-proniness" of C is following some weird practices copypasted from example code written in 1980's to early 1990's (e.g., random and unnecessary use of memcpy/memset, unnecessary type punning via pointers), which can be easily replaced with more robust and modern practices (e.g., use normal high level language features present in C since C89, like assignment operator).

"C++ is much easier and so much better in design, so we spend month after month teaching the weird intricacies of C++ instead of the general language-agnostic concepts we advertised at the beginning"
what utter bullshit.

Specifically I'm intrigued by the fact how teachers and other pedants give zero value to the fact that C is, in fact, a well standardized language, unlike some random hackjobs like Python where "anything goes" and every version is incompatible with other.

But likely the biggest disgrace is that C ain't object oriented. That was a big no-no in academy starting from late 1990's.
« Last Edit: September 26, 2023, 08:08:58 am by Siwastaja »
 
The following users thanked this post: mikerj, newbrain

Offline westfw

  • Super Contributor
  • ***
  • Posts: 4199
  • Country: us
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #34 on: September 26, 2023, 05:18:05 pm »
Quote
Given that a significant chunk of this thread is about non-standard C (like using compiler-specific attributes to twist UB)


One (Another!) of my complaints is the apparent race to specify certain "used for a long time" coding patterns as "undefined behavior."
It's not like people weren't using unions to accomplish type punning for decades before it was "outlawed" for reasons I don't really understand.

 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 26912
  • Country: nl
    • NCT Developments
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #35 on: September 26, 2023, 05:25:45 pm »
Quote
Given that a significant chunk of this thread is about non-standard C (like using compiler-specific attributes to twist UB)


One (Another!) of my complaints is the apparent race to specify certain "used for a long time" coding patterns as "undefined behavior."
It's not like people weren't using unions to accomplish type punning for decades before it was "outlawed" for reasons I don't really understand.
Maybe it has to do with the fact that most platforms C was running on where either 8 bit or x86 in the old days. In both cases you won't have alignment issues. Just a speed penalty. Nowadays the 16, 32 and 64 bit platforms are everywhere and many are less forgiving where it comes to alignment. So I kind of understand why type-punning is considered to be bad.

But there are also stupid things that get broken. Yesterday I spend a couple of hours to get gcc 4.9.x to compile on a recent Debian install ending up needing to patch a few files to change some attributes.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Online gf

  • Super Contributor
  • ***
  • Posts: 1187
  • Country: de
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #36 on: September 26, 2023, 05:43:58 pm »
One (Another!) of my complaints is the apparent race to specify certain "used for a long time" coding patterns as "undefined behavior."
It's not like people weren't using unions to accomplish type punning for decades before it was "outlawed" for reasons I don't really understand.

Some rules like e.g. strict aliasing were introduced to help compilers to do certain optimizations that are not possible with the traditional semantics. The compiler writers are happy, and the programmers are rather confused :scared:
 

Offline ejeffrey

  • Super Contributor
  • ***
  • Posts: 3727
  • Country: us
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #37 on: September 26, 2023, 06:25:26 pm »
Quote
Given that a significant chunk of this thread is about non-standard C (like using compiler-specific attributes to twist UB)


One (Another!) of my complaints is the apparent race to specify certain "used for a long time" coding patterns as "undefined behavior."
It's not like people weren't using unions to accomplish type punning for decades before it was "outlawed" for reasons I don't really understand.

Ok but that "race" was over two decades ago.

And it's an incredibly powerful optimization tool.  Without it, the compiler basically has to assume almost any two pointers could point to the same memory and must reload from memory into registers after every store.  It was probably the leading reason C was slower than Fortran which had much stricter aliasing rules.
« Last Edit: September 27, 2023, 01:41:38 am by ejeffrey »
 
The following users thanked this post: newbrain

Online peter-h

  • Super Contributor
  • ***
  • Posts: 3701
  • Country: gb
  • Doing electronics since the 1960s...
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #38 on: September 27, 2023, 04:37:46 pm »
Quote
"I don't use any pointers, at all, anywhere"
"How can you do anything at all then? Your code must be complete mess."
"Oh, I just use pointers to do this and that."

I never said the above and all here know exactly what I mean.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 
The following users thanked this post: nctnico

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 3915
  • Country: gb
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #39 on: September 28, 2023, 12:47:06 pm »
Maybe it has to do with the fact that most platforms C was running on where either 8 bit or x86 in the old days. In both cases you won't have alignment issues. Just a speed penalty.

Yes, in 1994, and specifically for my IDT MIPS board, I see from the notes on my paper manuals a few mentions of collaborations between Algorithmics and IDT to provide patches on Gcc to support their MIPS platforms.

It seems that many of these patches were for alignment and things that were problematic neither on x86 non or m68k, so "extra work" out of the ufficial gcc branch until they were finally in-tree merged.
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6266
  • Country: fi
    • My home page and email address
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #40 on: September 28, 2023, 01:44:24 pm »
It's not like people weren't using unions to accomplish type punning for decades before it was "outlawed" for reasons I don't really understand.
Type punning via unions is in standard C, and works as you expect it to.  It is type punning via pointers that is unreliable/"outlawed".

But yes.  In mid-to-late nineties the C standard codified the common ground among C compilers, up to C99.
Then it shifted into lets-dictate-what-compiler-writers-need-to-implement mode, including all the Annex K nonsense by MS, and MS' continuing refusal to acknowledge/support C99.

It is only in the latest few years (let's say since 2015 or so) that there has been a shift back to the nineties track that did work, with C17 being just corrections and clarifications to the weird misstep that was C11, but C23 will show whether we're on a better track now, back to codifying what has already been found to work in practice, instead of trying to dictate how compilers and users should use C.
« Last Edit: September 28, 2023, 01:50:45 pm by Nominal Animal »
 

Offline abyrvalgTopic starter

  • Frequent Contributor
  • **
  • Posts: 825
  • Country: es
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #41 on: September 28, 2023, 03:04:55 pm »
I found myself being afraid to increase -std=xx number w/o strong reasons - no hope to gain something significant, but fear to break something silently. In contrary, upgrading Python looks like fun :D
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 3915
  • Country: gb
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #42 on: September 28, 2023, 06:15:16 pm »
upgrading Python looks like fun

On Gentoo/Linux, the Portage system is based on Python, and upgrading Python, as well as Perl, can be catastrophic, meaning the entire system could stop working and explode into broken parts that cannot no longer compile anything, making your userland as useful up as a solid brick.

The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline abyrvalgTopic starter

  • Frequent Contributor
  • **
  • Posts: 825
  • Country: es
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #43 on: September 28, 2023, 06:50:15 pm »
No no, not the system one of course :D
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6266
  • Country: fi
    • My home page and email address
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #44 on: September 29, 2023, 09:02:30 am »
I found myself being afraid to increase -std=xx number w/o strong reasons - no hope to gain something significant, but fear to break something silently.
Yup, it is one of those things that are, and have to be, designed in from the get go; and not meddled with without a good reason or analysis/verification afterwards.

Most useful things the C compilers can provide tend to be provided via extensions anyway, before being codified as part of standard C anyway.

For example, most C compilers now support at least the two-parameter form of __builtin_assume_aligned(ptr_expr, minimum_alignment), which helps with the opposite situation compared to the one at hand.  It seems that the inverse, __builtin_assume_unaligned(ptr_expr), would be really useful here.  Unfortunately, none such exists, as far as I know.
 
The following users thanked this post: hans

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14490
  • Country: fr
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #45 on: September 30, 2023, 01:58:37 am »
I found myself being afraid to increase -std=xx number w/o strong reasons - no hope to gain something significant, but fear to break something silently.

While the successive C standard revisions have a rather good level of backward compatibility, at least compared to many other languages, it makes no sense per se to compile a given code base with a more recent revision than what it was coded against. This is not how standards work. As a general rule, restrict the std revision of your compiler to the revision the code was meant to be compliant with, nothing else.

There is usually and by nature nothing to gain by forcing a compiler to compile code that was written for an older std revision as a newer std revision. That doesn't make sense.

OTOH, if you really write code compliant with a newer revision and use a useful feature that wasn't present in older revisions, then it can definitely be useful and should certainly be considered.
Just do not throw a random C std rev at your code and see what sticks. :horse:

In contrary, upgrading Python looks like fun :D

Python is the last thing I would have cited as an example of painless transition for the same code base to a newer version of the language. Odd. But you didn't say painless, you said "fun". So, whatever's fun.
 

Offline bson

  • Supporter
  • ****
  • Posts: 2270
  • Country: us
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #46 on: October 02, 2023, 07:43:45 pm »
This is actually a real problem, thanks for pointing it out.  I would categorize it as a compiler bug or problem that could be solved with an alignment attribute.

memcpy/memmove is a total non-solution in many cases.  For example if you receive a frame from an ethernet controller it's going to be 16-bit aligned because the frame header is 14 bytes.  IP and TCP/UDP/ICMP headers then contain multiple 32-bit fields that are unaligned, but also smaller fields that might result in ldm/stm instructions. Needless to say, a memmove to align all ethernet traffic is a non-starter on such a small processor.  A workaround can be to 16-bit align DMA addresses for transfers, but this may not always be possible.  Some past MIPS32 SoC Ethernet controllers were notorious in this regard and could only do transfers to 256-byte aligned addresses, which misaligned protocol headers, resulting in millions of emulation alignment traps per second and abysmal network performance. (Disabling unaligned access emulation would cause the kernel to panic during boot when it tries to create the first interface.)  Adding memmoves to the driver was the workaround for those. But that fail was hardware-induced... there's no reason a compiler should be causing the same.

I think this might actually be the cause of some mysterious bugs I've seen in the past, that just randomly disappeared when I've debugged or instrumented code in vain attempts to hunt them down...
« Last Edit: October 02, 2023, 07:45:38 pm by bson »
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6266
  • Country: fi
    • My home page and email address
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #47 on: October 03, 2023, 07:25:10 pm »
One simple alternative is to use inline accessor functions.  Compiler Explorer example for Cortex-M3.

Because of the various issues with packed and/or unaligned structures, I do tend to use such accessors instead of structures, when transferring information.  (I do use packed structures, often with anonymous unions as in the above example, for example for type punning and to simplify access to memory-mapped peripherals.  In other words, I'm not saying one thing is bad and another is good, I'm only describing my preferred tool choices here.)

Because pointer arithmetic is not well defined for void pointers, I find it easiest to use an unsigned char pointer to the beginning of the buffer, even though the inline accessor functions do take a void pointer, i.e. their interface is basically type get_type(const void *const ptr), or occasionally type get_type(const void *const ptr, const unsigned char byte_order_change), where each bit i in byte_order_change corresponds to swapping 2i-byte groups, allowing run-time adaptation to different byte orders (in files or messages) based on known prototype multibyte values.  (For each type requiting 2k bytes of storage, there are also 2k possible byte orders, even though two –– in order of ascending or descending mathematical significance, i.e. little or big endian –– are normally encountered.)

(Note that in C function declarations, staticstatic inline.  I personally have the habit of marking internal/local functions as static, and accessor functions as static inline, that's all.  It helps in code maintenance, as it helps me structure the code better, and reduces the cognitive load with the ubiquitous "hey, what does this function do?" moments.)



I suspect, but have not verified, that
Code: [Select]
typedef struct {
    union {
        uint16_t  u16;
        int16_t   i16;
    };
} __attribute__((__packed__)) unaligned_16;

typedef struct {
    union {
        uint32_t  u32;
        int32_t   i32;
        float     f32;
    };
} __attribute__((__packed__)) unaligned_32;

typedef struct {
    union {
        uint32_t  u64;
        int32_t   i64;
        double     f64;
    };
} __attribute__((__packed__)) unaligned_64;
on current C compilers does yield 16-, 32-, and 64-bit number types one can use for unaligned accesses safely, with the extra "cost" of having to type (normal reference to structure member).i32 or whatever subtype one wants.

In other words, instead of using a packed structure with standard types, you can use a structure with unaligned members, defined as packed structures with effectively just one member.  (I need type punning between types of the same size so often that I prefer to bake that in to these 'single effective member packed structures', as they cost nothing.  Even if you used a single member, you'd still need to refer to that member; the anonymous union just lets you type-pun at that time if you wish.)

(EDITED to clarify: the above still requires global and static structures to be accessed via a temporary pointer.  Just declaring static or global structures with aforementioned unaligned_nn type members does not stop the compiler from combining accesses to their members; as the static or global declaration effectively "neutralizes" the packed attribute.)

With C11 _Generic, this could be extended to a macro that expands to an accessor call ensuring exactly one unaligned-safe access is done to the parameter object, i.e.
  typedef struct { int a, b; } MyStruct;
  MyStruct *src;
  MyFn(UNALIGNED_DEREF_ONCE(&(src->a)), UNALIGNED_DEREF_ONCE(&(src->b)));
although folding the address-of into the macro would shorten the code (but "violates" the normal passing-by-value logic in C):
  MyFn(UNALIGNED_ONCE(src->a), UNALIGNED_ONCE(src->b));
The idea is that _Generic is used to choose the size, byte order, and return type, by choosing a specific inline accessor function.  The inline accessor functions take a pointer to the object whose value they need to return.  For native byte order types, they cast that pointer to a (volatile) pointer to the suitable packed structure type, and obtain the value. 

As to why the "exactly one access", that (in conjuction with the C abstract machine rules) should ensure multiple unaligned accesses are not folded into a single access.  You ensure that (as much as one can in C, without resorting to assembly) by having the accessor function use a pointer with volatile type to the anonymous union structure type.  The reason for it working is that the parameter to the accessor function is not volatile itself.  This behaviour is not dictated by the C standard in any way, and is just a consequence of how current compilers implement the abstract machine described in the C standard, and specifically how they perform optimizations (especially at the abstract syntax tree level).

For non-native byte order types with "exactly one access", volatile is replaced by a memory copy to an array of unsigned char (or a structure with/or union that contains the desired type and an array of unsigned char); the unsigned char array is permuted according to the byte order change, and then the resulting type is returned via type-punned union access.

In any case, personally prefer a nested structure of single-member unaligned/packed structures instead, i.e.
  typedef struct { unaligned_32 a, b; } MyStruct;
  MyStruct *src;
  MyFn((src->a).i32, (src->b).i32);
noting that whether MyStruct is declared packed or not is irrelevant here; it is the members that are declared "unaligned".  This is as close to *(type)__builtin_unaligned(pointer) I can currently get.  (EDITED to add: the use of a pointer for the access is mandatory.  If you declare MyStruct foo; then calling MyFn(foo.a.i32, foo.b.i32) may lead to combined loads.  To fix, use const volatile MyStruct *src = &foo; and MyFn((src->a).i32, (src->b).i32);.)

Experimenting with this on Compiler Explorer suggests this works for all the compilers that it supports –– although I only tested a few examples I personally care about, and would thus appreciate a heads-up if anyone finds a counterexample (an architecture where the above, or the test example, fails to generate safe-unaligned-access code).  The only non-standard feature needed is the packed attribute support.
« Last Edit: October 04, 2023, 02:42:32 pm by Nominal Animal »
 

Offline bson

  • Supporter
  • ****
  • Posts: 2270
  • Country: us
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #48 on: October 03, 2023, 10:32:02 pm »
I don't see how this will help you at all.  An optimizing compiler gcc can still combine multiple 32-bit loads and stores into ldm/stm.
« Last Edit: October 03, 2023, 10:35:42 pm by bson »
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6266
  • Country: fi
    • My home page and email address
Re: Cortex-M3+ GCC unaligned access quirks
« Reply #49 on: October 04, 2023, 12:04:41 am »
I don't see how this will help you at all.  An optimizing compiler gcc can still combine multiple 32-bit loads and stores into ldm/stm.
Did you actually verify that?  I am claiming gcc (and gcc-compatible compilers like clang and sdcc) does not combine multiple 32-bit loads into ldm if you use the code patterns I showed, and generates 'ldr rn, [rm] @ unaligned' and 'ldr rn, [rm, offset] @ unaligned' (i.e., unaligned-safe loads on Cortex-M3) instead.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf