Memory model for Microcontrollers

[Nominal Animal]

How would one go about telling the compiler to put this lookup table in its own section in flash, and not in the RAM?

Generally, using
    __attribute__((section ("sectionname")))
    const type varname[] = {
        values...,
    };

Any decent compile/link ecosystem should by default put const data in flash unless explicitly told to do otherwise - as mentioned there are sometimes reasons to do otherwise, like harvard architectures that may have a speed penalty, or where flash and RAM have different access speeds, but these should be something that has to be explicitly requested.

For architectures with a single address space for Flash and RAM, yes.
In the above snippet, __attribute__((section ("sectionname"))) is only needed to put the variable in its own section in Flash.

That is,
    const type varname[] = {
        values...,
    };
suffices to put the variable in the normal Flash data section (whose name varies a bit depending on architecture).
This works with standard/default linker scripts for ARM Cortex-M targets using both GCC and Clang, for example.

Similarly, on older AVRs,
   __flash const type varname[] = {
        values...,
    };
suffices to put the variable in the normal Flash data section in (.progmem.data), and have instructions accessing it use the LPM instruction.
With GCC, this works only in C; with Clang, in both C and C++.  This is because GNU g++ does not support address spaces for C++, and is also the reason why the Arduino environment is so wonky.  Also note that the external definition, for use in other compilation units, is "extern __flash const type varname[size-if-known];", i.e. the __flash qualifier is required; otherwise the other compilation units would use data memory instructions for access.


To repeat why "its own section" is so useful with GCC/binutils/Clang toolchains, is that if you use a properly aligned and sized structures, you can do
    __attribute__((section ("sectionname"), used))
    static const struct-type varname_is_irrelevant = { definition };
in different compilation units, different source files, and the linker will provide you all of them (in a semi-random order) in a single array at runtime, via
    extern const struct-type __start_sectionname[];
    extern const struct-type __stop_sectionname[];
    #define  ELEMENTS  ((size_t)(__stop_sectionname - __start_sectionname))
    #define  ELEMENT(index)  (__start_sectionname[index])
For older AVRs, I recommend using the alias trick; you do also need to use extern __flash const ... above.

Also, knowing the section name lets you do all sorts of objdump and objcopy shenanigans.  Even relocation records are based on sections; that is, relocation records are typically offsets to a section, not offsets to a symbol value.  It is easy, almost trivial, to manipulate complete ELF sections as part of your build; manipulating individual symbols within sections containing other symbols is fraught with hidden dangers.  I don't manipulate individual symbols (other than adding them) outside the C compiler and linker script.  I often do manipulate (the contents of) entire sections outside the C compiler and linker script, knowing that it also affects any symbols defined within it.  (Which I often discard, by making them local via static const.)

[peter-h]

Yes indeed const does generate rodata. Or RODATA

[IanB]

Would the compiler possibly generate some initialization code that copies the data from flash to RAM on startup for ease of access?

Not really, but... The compiler just compiles code. There will be a startup file which does the system initialization, from initial boot to setting up memory (if necessary), copying initialized data, etc., then chaining to main().

Yes, this is exactly what I'm asking. When you compile C code for a desktop OS, the reference to startup code and startup entry point is automatically inserted by the compiler. You can use compiler options to make your own main() function the entry point, and avoid the initialization code being included, which can result in a much smaller executable. It follows that when you have an embedded target, the compiler could also insert some code that does some housekeeping on startup before reaching main(). I guess the only question would be if the startup/initialization code is standard, or if it is ever adjusted according to the contents of your program?

(If you are using C++, the compiler can automatically generate and include lots of code in your program that you didn't write.)

[Nominal Animal]

I guess the only question would be if the startup/initialization code is standard, or if it is ever adjusted according to the contents of your program?

If you use freestanding C, via -ffreestanding compiler option, then no such code is included.

Otherwise, the C runtime support (crt) is linked in; it is provided by one or more files named *crt*.o by the standard C library you use.
(This is separate from the actual standard C library itself, which can be dynamically or statically linked; and from compiler support library/objects like libgcc when using GCC on many architectures.)

The exact set linked in may depend on the compiler and the compiler options; especially whether generating position independent executables (PIE) or not, sometimes depending whether debug information is included or not.

Thus, it really depends on the standard C library you use, and compiler options; but not the contents of your program.

[nctnico]

Would the compiler possibly generate some initialization code that copies the data from flash to RAM on startup for ease of access?

Not really, but... The compiler just compiles code. There will be a startup file which does the system initialization, from initial boot to setting up memory (if necessary), copying initialized data, etc., then chaining to main().

Yes, this is exactly what I'm asking. When you compile C code for a desktop OS, the reference to startup code and startup entry point is automatically inserted by the compiler. You can use compiler options to make your own main() function the entry point, and avoid the initialization code being included, which can result in a much smaller executable. It follows that when you have an embedded target, the compiler could also insert some code that does some housekeeping on startup before reaching main(). I guess the only question would be if the startup/initialization code is standard, or if it is ever adjusted according to the contents of your program?

That depends on the target. When starting a regular C program, the very minimal requirement is to set the stack pointer and jump the main(). A Cortex-Mx microcontroller can do this in hardware so you won't need any startup code at all. When main is called, you'll need to clear the memory for the variables (call memset) and copy the initialised variables from flash to RAM (memcpy). The addresses (symbols) where the variables are located and the initialised data needs to go from / to are defined in the linker descriptor file.

On other CPUs some assembly code is needed to set things up before calling main.

[langwadt]

Would the compiler possibly generate some initialization code that copies the data from flash to RAM on startup for ease of access?

Not really, but... The compiler just compiles code. There will be a startup file which does the system initialization, from initial boot to setting up memory (if necessary), copying initialized data, etc., then chaining to main().

Yes, this is exactly what I'm asking. When you compile C code for a desktop OS, the reference to startup code and startup entry point is automatically inserted by the compiler. You can use compiler options to make your own main() function the entry point, and avoid the initialization code being included, which can result in a much smaller executable. It follows that when you have an embedded target, the compiler could also insert some code that does some housekeeping on startup before reaching main(). I guess the only question would be if the startup/initialization code is standard, or if it is ever adjusted according to the contents of your program?

That depends on the target. When starting a regular C program, the very minimal requirement is to set the stack pointer and jump the main(). A Cortex-Mx microcontroller can do this in hardware so you won't need any startup code at all. When main is called, you'll need to clear the memory for the variables (call memset) and copy the initialised variables from flash to RAM (memcpy). The addresses (symbols) where the variables are located and the initialised data needs to go from / to are defined in the linker descriptor file.

On other CPUs some assembly code is needed to set things up before calling main.

I think even for a cortex-m the memory initialization and such is normally done before calling main, but it can be written in C

random example from the interweb, https://github.com/westrup/stm32/blob/master/startup.c
 

[nctnico]

Probably ST wanted to hide the init as part of the HAL or avoid confusion and have everything initialised before calling main. In my own code these two lines do all the required initialisation:

Code: [Select]

memset(_data, 0, _stack_end - __bss_start__-32); //clear memory including stack
if (_edata!=_data) memcpy(_data, _datastart , _edata-_data); //copy initialised variables if present

I make it a point to clear the stack as well. In some rare cases old data on the stack can trigger very rare bugs in combination with (for example) an array / pointer going out of bounds even across resets. Clearing the stack results in consistent behaviour of the fault.

[peter-h]

Better still fill the stack with "s" or some such, and then dumping it afterwards will show the usage up to that point.

But yes it should deffo be filled with something consistent. I fill my general stack with "s", the RTOS area with something else, then the RTOS fills its task stacks with "S" (IIRC) etc.

[langwadt]

Probably ST wanted to hide the init as part of the HAL or avoid confusion and have everything initialised before calling main. In my own code these two lines do all the required initialisation:

Code: [Select]

memset(_data, 0, _stack_end - __bss_start__-32); //clear memory including stack
if (_edata!=_data) memcpy(_data, _datastart , _edata-_data); //copy initialised variables if present

I make it a point to clear the stack as well. In some rare cases old data on the stack can trigger very rare bugs in combination with (for example) an array / pointer going out of bounds even across resets. Clearing the stack results in consistent behaviour of the fault.

clearing the stack (or setting it to some magic number) can give you an hint on how much is being used by examining the stack area

If you use the ST IDE the startup is an assembly file added to the project, it also calls a systeminit function to setup clock/fpu/memory before calling main

https://github.com/STMicroelectronics/STM32CubeF4/blob/master/Projects/STM32F401RE-Nucleo/Templates/STM32CubeIDE/Example/Startup/startup_stm32f401ceux.s

[nctnico]

Probably ST wanted to hide the init as part of the HAL or avoid confusion and have everything initialised before calling main. In my own code these two lines do all the required initialisation:

Code: [Select]

memset(_data, 0, _stack_end - __bss_start__-32); //clear memory including stack
if (_edata!=_data) memcpy(_data, _datastart , _edata-_data); //copy initialised variables if present

I make it a point to clear the stack as well. In some rare cases old data on the stack can trigger very rare bugs in combination with (for example) an array / pointer going out of bounds even across resets. Clearing the stack results in consistent behaviour of the fault.

clearing the stack (or setting it to some magic number) can give you an hint on how much is being used by examining the stack area

I do that differently. I sample the stack pointer inside a timer interrupt and update the 'lowest stack address' variable when it is below the previous value. This gives a very quick idea of worst case stack usage without going through a lot memory. I do admit I 'stole' this idea from someone else though. Clearing memory to zero is a very safe thing to do as 0 or NULL is often regarded as the end of something or invalid. Setting the stack memory area to a different value, could cause more problems than is solves. Also, used stack space is likely to be unequal to zero.

[PlainName]

The stack could move between samples, so it will give you a good idea but risks missing peaks.

[tellurium]

When you compile C code for a desktop OS, the reference to startup code and startup entry point is automatically inserted by the compiler.

When you compile the code for desktop, you produce an ELF file. An ELF file has a "entry point" symbol. You can check the entry point by using objdump:

Code: [Select]

$ objdump -f ~/tmp/wizard/firmware.elf
...
start address: 0x30007a39

For the ARM firmware file, the entry point is basically ignored, because firmware binary is produced and flashed as I described earlier. The entry point is used by debuggers - by fetching an entry point from a firmware.elf, a debugger knows where to set a default breakpoint. An entry point is specified by a linker script. It is usually set to a reset handlier, e.g.: https://github.com/STMicroelectronics/cmsis_device_h7/blob/88f29645ec7e063bae3d5e58699fc541a81efac6/Source/Templates/gcc/linker/stm32h747xx_flash_CM7.ld#L35

On a desktop system, the situation is different. As far as I understand, there is a standard, called ABI (application binary interface). It says that on UNIX systems, an entry point of a binary should be a symbol called "_start".  When a unix kernel starts a process, it loads the first page of an executable file into memory, and loops over a list of so called "image activators". Those are functions that recognise file format and try to start an executable. There are image activators for ELF files, for shell scripts, etc etc.

So ELF image activator looks for and ELF signature at the beginning of the file, and if it finds it, it scans the ELF header for the _start symbol. Then it does a bunch of other things, and it jumps to _start. A function _start is part of a C runtime - it is located in a file crt*.o, which gets linked to your binary at the build stage, as Nominal Animal pointed out.  This part of C runtime implements an ABI - that's why it is separate from a C library itself. A function _start loads dynamic libraries, prepares argv, argc, environ, and then calls main().

So when you say "reference to startup code and startup entry point is automatically inserted by the compiler." - yeah, pretty much. By the linker. The startup code is that _start function.

[IanB]

So when you say "reference to startup code and startup entry point is automatically inserted by the compiler." - yeah, pretty much. By the linker. The startup code is that _start function.

I follow everything you are saying, thank you, especially the reference to the linker. But I think that in very simple compilations the distinction between the compiler and the linker is a bit blurred. The compiler invokes the linker and tells it what to do to make an executable, and it is the C compiler that tells the linker to include the C runtime with your code. You can tell the compiler not to include the C runtime, as illustrated in the video below, and that leaves you with a bare executable where you have no C runtime support.

What I was curious about is when you target an embedded platform, you might also have initialization code automatically added to your binary via the platform SDK, and that this code would do whatever is necessary on the platform you are targeting to set things up before entering your code. Other people have explained about this in the posts above (thank you). It also appears that if you know what you are doing, you can do some minimal initialization yourself, manually, and not ask the SDK to look after it for you.

Whatever the case, I understand that "it depends" and "read the documentation for your platform" are important memos, and these would not escape me if I were doing this for real.

[tellurium]

As it was described above, there is no hidden, or default, startup/init code added to the embedded firmware executable. A linker script defines an entry point symbol, and it is set so in the produced ELF header. However if you're creating a binary firmware from your .elf file, via "objdump -O binary firmware.elf firmware.bin" , then you don't even need to specify an entry point. It is NOT required to make a valid firmware binary. There is NOTHING hidden or secret, and you can't optimise an embedded code by those tricks described in the "desktop" videos. Because ELF format is NOT used by the firmware binary, and the startup code is explicit - taken from vendor's SDK or hand-written.

[IanB]

As it was described above, there is no hidden, or default, startup/init code added to the embedded firmware executable. A linker script defines an entry point symbol, and it is set so in the produced ELF header. However if you're creating a binary firmware from your .elf file, via "objdump -O binary firmware.elf firmware.bin" , then you don't even need to specify an entry point. It is NOT required to make a valid firmware binary. There is NOTHING hidden or secret, and you can't optimise an embedded code by those tricks described in the "desktop" videos. Because ELF format is NOT used by the firmware binary, and the startup code is explicit - taken from vendor's SDK or hand-written.

Yes, I've got it. Please don't think I am arguing with you 

It's not about hidden or secret, it's about automatic. As you wrote above, startup code may be "taken from the vendor's SDK". I don't think you mean I need to copy/paste the start up code from the SDK and include it in my project, I think you mean that the SDK will automatically include the necessary startup code for me when I create my project? I expect I can use the appropriate SDK from the vendor, say "create new empty project", build that project, and it will automatically include whatever startup code is needed for the platform? (I am assuming here that I would be using an IDE, rather than using bare bones command line tools.)

Beyond this point, I think there is nothing more for me to ask unless/until I actually get a development kit for a suitable microcontroller, install the SDK, and start writing code for it.

[nctnico]

It is never really automatic. There are two options to get the startup code linked into the binary automatically:
1) The SDK (like ST's cube) automatically adds a file with the startup code.
2) The SDK + C libraries automatically selects the right C startup library to be linked into the project

If you study the source files and list of libraries which are part of the project, you should be able to see what is what.

However, the SDK has to be tailored for the specific microcontroller. If you use vanilla VS Code or Eclipse CDT and a GCC / CLANG install, you'll need to do everything yourself (I'm not going into whether this is good or bad; I leave that to the readers to decide for themselves).

What doesn't help is that a lot of this is hidden under the hood. For example: GCC (as a collection of tools) has relative paths to the includes, linker descriptor files and libraries hard coded inside the executables so even without specifying paths for where to find specific inludes and libraries, it can find them. This makes GCC easy to use but it also hides some very important information which makes it look like a lot of things happen automagically.

[jnk0le]

My early indications, are that considerably earlier versions, of that 'bot', were much more clearly labeled.  So, that they could be traced to, what I would prefer to call the Google AI department.

But, they instead seem to call it the Google Deep Learning (entity/department).

Over a very long period of time, e.g. 2007 and perhaps longer.

Of course missed the most obvious explanation: collect unique "individual expertise" data to train their AI models.

That would explain weird questions on topics that can often be just googled up.



Also, there was another instance on r/riscv repeatedly asking the same 'questions' while gaslighting (deleting) previous ones.
@brucehoult has been dealing with it IIRC

[SiliconWizard]

I don't think the person itself is a bot - no bot can talk like this exactly. It sounds too "human" in some of the phrasing. But there's a high probability this is a real person working at testing & training AI.

[Kittu20]

Does my question not make sense? I see that many experts also have doubts, which they are clarifying in this thread by asking other members. Then, as an intern, why can't I have doubts?

[pcprogrammer]

As it was described above, there is no hidden, or default, startup/init code added to the embedded firmware executable. A linker script defines an entry point symbol, and it is set so in the produced ELF header. However if you're creating a binary firmware from your .elf file, via "objdump -O binary firmware.elf firmware.bin" , then you don't even need to specify an entry point. It is NOT required to make a valid firmware binary. There is NOTHING hidden or secret, and you can't optimise an embedded code by those tricks described in the "desktop" videos. Because ELF format is NOT used by the firmware binary, and the startup code is explicit - taken from vendor's SDK or hand-written.

Yes, I've got it. Please don't think I am arguing with you 

It's not about hidden or secret, it's about automatic. As you wrote above, startup code may be "taken from the vendor's SDK". I don't think you mean I need to copy/paste the start up code from the SDK and include it in my project, I think you mean that the SDK will automatically include the necessary startup code for me when I create my project? I expect I can use the appropriate SDK from the vendor, say "create new empty project", build that project, and it will automatically include whatever startup code is needed for the platform? (I am assuming here that I would be using an IDE, rather than using bare bones command line tools.)

Beyond this point, I think there is nothing more for me to ask unless/until I actually get a development kit for a suitable microcontroller, install the SDK, and start writing code for it.

Creating the project for par example a STM32F103C8 in the CubeIDE includes the process of generating the startup file, either by just copying the .s file or filling it in from a template. When you use another IDE it might not be the case and you have to add it yourself or write the needed code in C, or just copy a project from someone else and use that as a base.

The compiler(s) is(are) then instructed by the make process to compile the individual files that are in your project. I wrote compiler(s) because there can be different ones for the different file types, like C, C++ or assembler. For the latter it is not really a compiler but an assembler.

What a compiler does is translate the C, C++ code to assembler, and then assemble it into machine code. The latter being the binary data that is processed by the processor.

[pcprogrammer]

What doesn't help is that a lot of this is hidden under the hood. For example: GCC (as a collection of tools) has relative paths to the includes, linker descriptor files and libraries hard coded inside the executables so even without specifying paths for where to find specific inludes and libraries, it can find them. This makes GCC easy to use but it also hides some very important information which makes it look like a lot of things happen automagically.

Which is even worse on the Arduino platform.

[MK14]

I don't think the person itself is a bot - no bot can talk like this exactly. It sounds too "human" in some of the phrasing. But there's a high probability this is a real person working at testing & training AI.

Yes, to the last bit of your post.  Although, I think some of those 'human' elements, might themselves, be part of its AI, as well.

One example is here:
https://forums.freertos.org/t/practical-rtos-project-ideas-for-real-world-applications/17981

Where the responding poster, hasn't noticed that it might be an AI/deep-learning thing, but makes comments about its rather strange and weird 'learning' behavior.

RAc
Aug 2023

Hi kittu,

looking at your forum history, I can’t help but feel that you ask the same basic question every few months… could it be that you are searching for something to help you tie all those loose ends together?

Every real life project you may wish to tackle requires peripherals which is hardware, so you need to become clear about what hardware you are able to/ want to build your sample project on. Typical stand-alone projects that fit the bill would be sensor driven systems such as weather stations, smoke detectors or very rudimentary model train controllers. Those must also fit your personal preferences and intersts, of course.

Then again, very few embedded systems these days exist without host communication channels to concentrators, smartphones or PCs, typically network or UART based, so you should pick a target hardware also with that in mind.

[MK14]

Does my question not make sense? I see that many experts also have doubts, which they are clarifying in this thread by asking other members. Then, as an intern, why can't I have doubts?

Is it ok, if I ask you a question, as an intern.

As an intern, you must be a human adult.  A human adult, must know what a (paper) envelope or sleeve is?
Similarly, an intern, must know what a credit card is, what it looks like and what the different parts of it, basically are.

So, when we go to the following thread:
https://forum.allaboutcircuits.com/threads/biometric-system.192026/#post-1801070

How come, you don't seem to know about those basic facts.  Quotes of the relevant bits, follow:

In the link, a card is inserted under a box that looks like an envelope. is that box called a sleeve ?

An adult (intern), would reasonably be expected to know what a (paper?) sleeve looks like, and is.

I am trying to understand the process of the below type of cards.

Followed by a picture of a credit card, in that thread.

Surely as a (human) adult intern, you know what a credit card is?

[peter-h]

Beautiful

And so obvious that this is purely a response generation exercise.

[PlainName]

So, when we go to the following thread:
https://forum.allaboutcircuits.com/threads/biometric-system.192026/#post-1801070

Having followed that link, and the subsequent link, I don't find the questions indicate what you believe to be the case. The thing about the box vs sleeve is that it's not obvious what it is or should be called - I wouldn't call it a sleeve (it's cardboard, not paper). And the 'card below' is credit-card sized but isn't a credit card (since it has a fingerprint reader embedded, apparently).