-
The sprintf was looping - I was stepping through it. I will look more carefully when I get back to it.
This code has been running for years. What has changed is the linkfile, because I am now running this as a "user written overlay". I am suspecting uninitialised variables, and therefore one of theseCode: [Select]/* This stuff is copied from FLASH to RAM by C code in the main stub */
.all_nonboot_data :
{
. = ALIGN(4);
_s_nonboot_data = .; /* create a global symbol at data start */
*(.data .data*) /* .data sections */
. = ALIGN(4);
_e_nonboot_data = .; /* define a global symbol at data end */
} >RAM AT >FLASH_APP
/* used by the main stub C code to initialize data */
_si_nonboot_data = LOADADDR(.all_nonboot_data);
/* Uninitialized data section for rest of xxx */
/* This stuff is zeroed by C code in the main stub */
. = ALIGN(4);
.all_nonboot_bss :
{
_s_nonboot_bss = .; /* define a global symbol at bss start */
*(.bss .bss* .COMMON .common .common*)
. = ALIGN(4);
_e_nonboot_bss = .; /* define a global symbol at bss end */
} >RAM
Hence I wondered what section an initialised struct (a local one, thus on the function stack) is supposed to be in, but the answer must be None. It cannot possibly be anything like that, because the function would not be thread-safe.
This code sets up data and bssCode: [Select]
// Entry point for code called by the boot block - has to be located at the base of FLASH+32k
// This is done in linkfile
// Has to be "int" otherwise compiler complains :)
// Various measures to prevent it getting optimised away, which probably don't work. What does work
// is vector2 (which is in assembler and thus immune to optimisation) having a dword pointing to main().
__attribute__((optimize("O0")))
__attribute__((used))
int main()
{
// Initialise DATA
extern char _s_nonboot_data;
extern char _e_nonboot_data;
extern char _si_nonboot_data;
void * memcpy (void *__restrict, const void *__restrict, size_t);
memcpy(&_s_nonboot_data, &_si_nonboot_data, &_e_nonboot_data - &_s_nonboot_data);
// Zero BSS and COMMON
extern char _s_nonboot_bss;
extern char _e_nonboot_bss;
void * memset (void *, int, size_t);
memset(&_s_nonboot_bss, 0, &_e_nonboot_bss - &_s_nonboot_bss);
// Go to user program
main_real();
// We should never get here
for (;;);
}
// Vectab2.s ends up linked here, at base of FLASH + 32k + 512
Traditional ST-supplied code uses the famous startupxxx.s asm code to do the above, but that works only for the first part of the code (my boot block in this case). Any code loaded later (the above main()) has to be separately linked and has to set up its own sections.
Right now I have more fun running an STILNK V3 from one Cube on the PC and switching to another Cube in a VM
The Cube in the VM was a later version (for testing; I always check a new Cube generates the same binary byte for byte) and it decided to upgrade the STLINK, and due to a dodgy USB connection it bricked it. So now I am looking for an unbricking utility...
-
Quote
Hence I wondered what section an initialised struct (a local one, thus on the function stack) is supposed to be in, but the answer must be None. It cannot possibly be anything like that, because the function would not be thread-safe.
"sections" apply to variables allocated at compile/link time. Local non-static variables are placed to stack dynamically, at runtime. If they are explicitly initialized, the compiler generates the initialization code at the point of initialization (optimizer may decide to modify this in ways which are equivalent).Quote from: C99, 6.2.44 An object whose identifier is declared with no linkage and without the storage-class
I don't know and I don't want to know what's "thread safe".
specifier static has automatic storage duration.
5 For such an object that does not have a variable length array type, its lifetime extends
from entry into the block with which it is associated until execution of that block ends in
any way. (Entering an enclosed block or calling a function suspends, but does not end,
execution of the current block.) If the block is entered recursively, a new instance of the
object is created each time. The initial value of the object is indeterminate. If an
initialization is specified for the object, it is performed each time the declaration is
reached in the execution of the block; otherwise, the value becomes indeterminate each
time the declaration is reached.
JW -
Quote
I don't know and I don't want to know what's "thread safe".
Re-entrant code. Everything needs to either be on the stack, or, as was done in the old days of multiuser operating systems (and 24k of core on an ICL1904), the caller supplies an index and the called function keeps its variables offset by that index.
My "overlay" code is running but weird stuff is happening, suggesting uninitialised variables or some such.
The sprintf thing was a redherring. The format string is there and is null-terminated btwCode: [Select]sprintf(datetime, "%c%c%c %02d%02d20%02d %02d%02d%02d DOY=%03d",
Does anyone know if it is even possible to use an STLINK debugger to load code linked to run at 0x8008000 (instead of 0x8000000) and it programs the right cpu flash addresses? IOW, does not try program anything into 0x8000000-0x8007fff). -
Does anyone know if it is even possible to use an STLINK debugger to load code linked to run at 0x8008000 (instead of 0x8000000) and it programs the right cpu flash addresses? IOW, does not try program anything into 0x8000000-0x8007fff).
I use it in that way every day.
Not from CubeIDE, though - bare gdb+OpenOCD. But I don't see why that wouldn't work in CubeIDE either.
Maybe you could try to use simpler tools - STLink Utility or CubeProgrammer - just to see if it makes any difference.
[EDIT] CubeIDE may work out of the .elf, so you may want to have a look at what exactly is in the .elf you are generating (perhaps using objdump).
JW
-
Cube does use the ELF file, apparently.
But this problem is simpler:Code: [Select]
void xxx_get_date_time(char *datetime)
{
RTC_TimeTypeDef sTime = {0};
RTC_DateTypeDef sDate = {0};
HAL_RTC_GetTime(&hrtc, &sTime, RTC_FORMAT_BIN);
HAL_RTC_GetDate(&hrtc, &sDate, RTC_FORMAT_BIN);
int i = (sDate.WeekDay - 1) * 3;
sprintf(datetime, "%c%c%c %02d%02d20%02d %02d%02d%02d DOY=%03d",
d_o_w[i],d_o_w[i+1],d_o_w[i+2],sDate.Date, sDate.Month, sDate.Year,
sTime.Hours, sTime.Minutes, sTime.Seconds, day_of_year(sDate.Date, sDate.Month-1, sDate.Year+100));
}
In the above, the sprintf() is being passed a load of uninitialised data (0xffffff....) for the format string.Code: [Select]080582ee: str r6, [sp, #12]
080582f0: str r5, [sp, #8]
080582f2: str r4, [sp, #4]
080582f4: str.w r8, [sp]
080582f8: ldr r3, [sp, #32]
080582fa: mov r2, r7
080582fc: ldr r1, [pc, #20] ; (0x8058314 <xxx_get_date_time+168>)
080582fe: ldr r0, [sp, #36] ; 0x24
08058300: bl 0x8048a50 <sprintf_>
r1=0x8071870 (the format string address, fairly obviously) and contains a load of FFs
r0=0x2001ff20 and this is the right address on the stack for the sprintf output buffer
Looking in the .map file around 0x8071870 I seeCode: [Select].rodata.str1.4
0x000000000807157c 0x2c8 ../LIBxxx\libxxx.a(xxx_NTP.o)
0x2f0 (size before relaxing)
.rodata.str1.4
0x0000000008071844 0x58 ../LIBxxx\libxxx.a(xxx_rtc.o)
.rodata.str1.4
0x000000000807189c 0x1a ../LIBxxx\libxxx.a(xxx_sensors.o)
*fill* 0x00000000080718b6 0x2
so 8071870 is in the right module (RTC) but hey this stuff is coming out of a .a library! I wonder if that library is simply missing all DATA sections, and contains only TEXT?
The same code runs if rtc.o is from a .o file, but not if rtc.o is from a .a library < bang head again >
The library is made with
arm-none-eabi-ar rvs libxxx.a @libxxx_objs.txt
I wonder if "rvs" does just TEXT? I want to build that lib with everything in the .o files (listed in libxxx_objs.txt).
A lot on google but not relating to the AR library creator tool. Its options don't appear to include one which might strip stuff out of .o files. Or maybe the linker is stripping it out?
This is crazy. I could use .o files instead of the stupid library. But in Cube there is no evident linker option to fetch .o files from a path. This is available only for a library. Perhaps I could merge all .o files into one big .o file.
The string in question isCode: [Select]sprintf(datetime, "%c%c%c %02d%02d20%02d %02d%02d%02d DOY=%03d",
and the other possibility is that the linkfile is not pulling this string (which I presume is DATA) out of a library, but it obviously does pull it out of a .o file.
A hex dump shows the format string is in the library:
-
.rodata* is read-only data, as the name implies. Also, the placement is correct, inside the flash address space. Hence, nothing related to DATA or .data.
Next, place the breakpoint at sprintf in question, and when it is hit, inspect the registers. Or step into sprintf in disassembly mode, and inspect the registers. Anything suspicious? Where r1 points?
Next, inspect the memory at address in r1. Not the map file, not elf, not binary or any other file, but the real memory. What is there? Do you see your string? -
I need to find the section name in which that sprintf format string is being placed in the .a library.
I can't find a tool which can read a .a lib file and list all the stuff in detail.
If that format string was rodata, it should be collected hereCode: [Select]
/* This collects all other stuff, which gets loaded into FLASH after xxx_main.o above */
.code_constants_etc :
{
. = ALIGN(4);
*(.text) /* .text sections (code) */
*(.text*) /* .text* sections (code) */
*(.rodata) /* .rodata sections (constants, strings, etc.) */
*(.rodata*) /* .rodata* sections (constants, strings, etc.) */
*(.glue_7) /* glue arm to thumb code */
*(.glue_7t) /* glue thumb to arm code */
*(.eh_frame)
KEEP (*(.init))
KEEP (*(.fini))
. = ALIGN(4);
_e_code_constants_etc = .; /* define a global symbol at end of code */
} >FLASH_APP
I did inspect the contents of the addresses in r0 and r1. The format string is all 0xFF. So it isn't being extracted out of the library (but is extracted from a .o).
I need to dump the library.
Somebody posted somewhere that 7-zip can be used to dump a .a library. I tried that - it is BS.
But it gets better. that format string is found in the final binary file too. Yet the wrong address is being passed to sprintf. -
It should come with the toolchain, mine are named arm-none-eabi-nm (I've got several versions of them installed)
-
Yeah... much better now. The main problem was that the rodata stuff was at the very end of the overlay, and the last block (32k) of the overlay was not being programmed into the cpu flash because I was counting one block too few in the loader programming code
This took a bit of sorting out because even after an initial fix the loader code was still not programming the last few bytes, which happened to be DATA or RODATA or some such.
-
This is in case somebody comes this in the future.
Using libraries with a ST HAL project is a complete total waste of f*****g time, because as wek pointed out, a weak in a .o cannot be overriden by a non-weak in a .a
Some of this is easy to take care of but the ST code is full of "software purist" kind of total crap like thisCode: [Select]
/**
* @brief UART MSP Init.
* @param huart pointer to a UART_HandleTypeDef structure that contains
* the configuration information for the specified UART module.
* @retval None
*/
__weak void HAL_UART_MspInit(UART_HandleTypeDef *huart)
{
/* Prevent unused argument(s) compilation warning */
UNUSED(huart);
/* NOTE: This function Should not be modified, when the callback is needed,
the HAL_UART_MspInit could be implemented in the user file
*/
}
so I wasted a day finding why in a project using a xxx.a library the UARTs were not working. It was only when stepping through their config register config that I found this.
Why do people write this shit? You spend so much time typing up empty functions with __weak on the front.
The above weak function does exactly nothing. The project will compile but will obviously not run.
One approach is to not use libraries at all. That implies that all .c sources have to be present, which is in some variations of my project undesired, but just having the .o files is no good because the Cube build scripts do a del *.* in the project build directory. I suppose one could have the .o files (the ones without source) with a R/O attribute
Another one is to go through all __weak declarations and comment them out (135 of them in the ST code I have!). That is what I am doing. Well, just the ones affecting my project.
-
> Another one is to go through all __weak declarations and comment them out (135 of them in the ST code I have!). That is what I am doing. Well, just the ones affecting my project.
Do you intend to supply your version of CubeF4 (= the "library"), and resolve conflicts stemming from users wishing to use another version of that "library"?
I did some googling and the answer here appears to have a solution. I of course don't know what are the exact ramifications of using that switch, especially in your particular setting.QuoteI think it should be noted that the --whole-archive option also adds all the symbols within the given libraries, which will solve this problem but will increase the size of the executable and may produce additional linking errors.
JW
-
Some of this is easy to take care of but the ST code is full of "software purist" kind of total crap like this
...
Why do people write this shit? You spend so much time typing up empty functions with __weak on the front.
People write confusing bullshit boilerplate so that
A) they can be replaced by ChatGPT
B) those who got replaced can complain how ChatGPT took their jobs
C) those who replaced them can boast how excellent ChatGPT is as it can write human-level code.
Expect things to get worse in the future. Much worse. Amount of human time spent in counterproductive work was at least physically limited. Now counterproductive programmers can get much more "productive".QuoteOne approach is to not use libraries at all.
Excellent approach. Highly recommended. Libraries are always a pain; only use them when they solve non-trivial problems, so that the work spent understanding the library and getting it work is paid back as saved time. For example, if you need TLS, use a library. If you need UART on MCU, do not.
EDIT: almost forgot. I'm brilliant and you're stupid
-
Quote
I did some googling and the answer here appears to have a solution
Yes I saw that earlier and didn't want to go down another rabbit holeQuoteDo you intend to supply your version of CubeF4 (= the "library"), and resolve conflicts stemming from users wishing to use another version of that "library"?
My "library" is just code which I (well mostly I) have written.
The other sources for this project are mostly the ST ports of LWIP, FreeRTOS, etc. These do all work and have been well tested. All are in source form. It is actually quite old stuff e.g. LWIP goes back > 15 years. So there is no urgent operational need to update any of these. I mean, if your PC runs fine, are you going to update the BIOS?
If somebody wants to update any of these, they will be able to if they know their way around the directory structure and the port code (the glue code used to port say FreeRTOS to the 32F417). With me having hacked out various __weak functions, the new code will not just drop in. But it never just drops in anyway... whenever one is updating some chunk of code v1.3 with v1.4 one has to do a file compare and check that the stuff being changed isn't going to break something. For example I have MbedTLS v2.something which was modded to read a whole certificate chain from a file, so updating TLS to v3 will be a few days' work. But there is nothing wrong with v2 unless you are one of the "deprecated crypto BS experts". This is IOT; it is not a server for microsoft.com
If the new module is object-only then you are out of luck if it contains __weak functions which don't work. Well, the ST supplied libc.a had loads of weaks which were empty stubs... I had to unweaken the whole libc.a using objcopy and then they could be replaced. So there is a way forward.
Truth is... you can rarely just update a load of sourcecode by dropping in a new version. From what I have seen, it is always a ton of work. Most open source stuff has loads of issues, starting with zero support, so you spend days googling to find out what somebody else did in 2009.
Another way forward would be to use just .o files (no lib) and edit the Cube pre-build script which does the del *.* but that will break when a new version of Cube is installed. I looked at this briefly but it's another rabbit hole.QuoteFor example, if you need TLS, use a library. If you need UART on MCU, do not.
Actually TLS is supplied in source form I would have nothing to do with a non-source "library" unless the interface is well defined and "simple". MbedTLS is probably not a good example due to the complex interface and functionality.
What I now need is an editor macro which puts #if 0 and #endif around any function which is __weak
The worst part of this __weak template business is that there is easy potential for code to break in subtle ways. All you need to do is move a function from a .c/.o to a .a and it will break silently.QuotePeople write confusing bullshit boilerplate so that
A) they can be replaced by ChatGPT
B) those who got replaced can complain how ChatGPT took their jobs
C) those who replaced them can boast how excellent ChatGPT is as it can write human-level code.
I thought there was a real reason for __weak but I can't see one, apart from being able to build a project, which doesn't have to actually work.
Is there a tool which one can run on a whole project and which can report which functions are __weak but are actually being called by real code?
-
> Is there a tool which one can run on a whole project and which can report which functions are __weak but are actually being called by real code?
nm shows symbol types inside an ELF, but it can't tell if the function is really gets called AFAIK.
nm my_project.elf | grep -e " W " -e " w " -e " v " -e " V " should dump all weak symbols. -
There are tools for checking dodgy C syntax so I am amazed there aren't tools for checking if empty functions (optionally but not always __weak) are getting called.
-
Is there a tool which one can run on a whole project and which can report which functions are __weak but are actually being called by real code?
For 'are actually being called', no. For 'may be called by real code', sure.
The difference is this:
if (prng() == 42) foo();
Is foo() called or not?
If you want to know whether foo() really gets called, you have in your hands a variant of the classic halting problem. The only way to know is to instrument foo() and run it. For some specific cases, it is possible to instrument the internal state (variables, function result values) that affect whether the function gets called, and prove the conditions when the function gets called and when not.
However, if we want to know if the object file defines functions that are not called and whose addresses are never taken (noting that some function attributes like __attribute__((constructor)) and __attribute__((destructor)) actually cause the compiler to take the address of the function; it does not need to be explicitly taken), sure! ld itself does exactly that when you compile stuff with -ffunction-sections -Wl,--gc-sections, to discard unneeded functions.
The procedure differs a bit when one uses dynamic linking (for stuff running under a proper OS with virtual memory), so I'll specifically limit to statically linked stuff, since microcontroller stuff is almost always statically linked.
You just generate a list of all function type symbols in the object files the project generates, the intermediate ELF .o files. We then go through the list, and check that each defined function symbol is referenced, or it is unused; and that each referenced function symbol is defined at least once, or it is used but not implemented. Simples!
There are two ways you can do this. The simple way is to parse objdump -t output. The other way is to parse the ELF files yourself. In the latter case, you actually only need to find the symbol and string tables, since they contain all the information needed. (All string-form data is used by reference, so it is a bit annoying; but the format is utterly stable. On Linux, you can just use the already provided <elf.h>.)
In the objdump -t output, we are interested only in the SYMBOL TABLE, the lines that begin with a hexadecimal digit. On elf32-littlearm (Armv7e-m Cortex-M7), this has five fields per line, with the second field being fixed-width, containing flags. F denotes functions, and w denotes weak symbols; l (lowercase letter L) denotes local symbols, and g denotes global symbols. We only need the second, third, and fifth fields.
When the second field has F and g in it, it defines a function symbol (a weak one if there is also a w). We need these in one list.
When the second field is all spaces and the third field is *UNK* (exact string, not a pattern), it means that something in that object file references that symbol. We want these in a separate list.
Using your favourite scripting language, even Bash works fine for this if you use export LANG=C LC_ALL=C to explicitly set the default C locale, extract the two cases into separate lists or dictionaries. Then, it is just a matter of looking up which functions aren't referenced.
In C, a hash table on the function symbol name, with the data specifying whether it has been defined (and optionally in which object file or files and with which attributes), and whether it has been referenced (and optionally in which object file or files), should work very well and be extremely fast (bound by I/O bandwidth and latencies).
If you are asking for a tool that you can just install and run, I can't help you with those. They may or may not exist; I do not know. -
The other sources for this project are mostly the ST ports of LWIP, FreeRTOS, etc. These do all work and have been well tested.
Out of curiosity. When you say - well tested, what does that mean?
Do you test manually? Or is it an automatic unit test? Do you use a real hardware / jig to test? What is your test procedure? -
1) The stuff has been around a very long time, and been in fairly active development for years of that time.
2) The product has been in development for years and a number of boards have been running 24/7, running a fully loaded product.
-
Quote
The difference is this:
if (prng() == 42) foo();
Is foo() called or not?
What I had in mind is purely static analysis.
For example there is clearly a "decision point" during the link process where a __weak function is not overriden by a symbol which originates from a library, whereas it would be overriden by a symbol which originates from a .o file.
So the linker could in principle generate such a report.
Incidentally in my project there should not be any scenario where both a ex .o and a ex .a symbol are presented against a __weak function. But that scenario could also be detected by the linker.
As I say this is purely static analysis.
The danger I am trying to prevent is that in a project you may have 10k functions, of which 500 are __weak, and of the 500 there are 10 which are not defined by any real code, and this is not discovered because those 10 lie in some code which happens to not get tested. It's pretty nasty.
ST have generated __weak functions all over the place. I have now commented out everything I could find that I know is actually or potentially used in my project. But that's stupid.
-
Ok, so basically whether there is a reference to (a function) or not; and not whether that reference gets actually used at some point.QuoteThe difference is this:
if (prng() == 42) foo();
Is foo() called or not?
What I had in mind is purely static analysis.
It'd be no problem to write a script to do this. What's your development OS? Do you have Bash available? Or would you prefer Awk? (Any of Bash, Mawk, or Gawk would make this quick'n'easy.)
I've got a nasty cold right now, so I cannot concentrate too well, thus cannot promise I can right now write it as a standalone ANSI C program (+specific-width int types from stdint.h) without any external dependencies. It is, however, completely unlikely that the nm/objdump output format would change for Cortex-M7, so a script implementation should be perfectly fine for your needs.For example there is clearly a "decision point" during the link process where a __weak function is not overriden by a symbol which originates from a library, whereas it would be overriden by a symbol which originates from a .o file.
Quite. As wek mentioned in #25, symbol lookup in an archive differs from that of symbol lookup in an object file, by definition. (That is, when an object file contains a weak symbol, other object files are checked for a non-weak version of that symbol, but archives are not. It is a design decision, based on the use cases that weak symbols were invented to allow.)
When that is undesirable, you can tell the linker to treat the archive as a collection of object files instead (see --whole-archive in man 1 ld). So, at compile/link time, instead of -lfoo to link in libfoo.a, you use -Wl,--push-state,--whole-archive -lfoo -Wl,--pop-state (typically in LDFLAGS).So the linker could in principle generate such a report.
Certainly, yes.
It is easy to do it as a separate pass after all object files have been generated, but not linked into the final binary, too.
At that point, you can even use objcopy to move symbols/objects between object files, rename symbols, and delete symbols (for example, if you determine the archive/library version is superior than the one in any of the object files, and so on); but I'd restrict do deleting unneeded symbols if possible, or even better, use the whole-archive option above.ST have generated __weak functions all over the place.
It makes sense, if that code is to be compiled and linked into a library. Then, users can easily override any of such functionality by defining non-weak versions of those functions in their object code.
Your problem is that you put the user code in a library, and the ST code in a bunch of object files. Basically the inverse of what ST generates the code for. It is not an arbitrary choice one can make. -
Win7-64. However I have cygwin and so have bash and gawk, but not awk. 267 executables ...
I have a dir called awk but it contains grcat and pwcat.
AFAICT the ST code was never intended for a library. It is in source form and is meant to be loaded into Cube IDE in that form. I suppose somebody might have then made a .a lib out of it but I haven't come across that. Can't see the point of a lib if you have the sources, ever.
They supply newlib printf etc as a lib and w/o sources and that lib is not weak As previously posted, I had to weaken it to replace the printf code.
-
1) The stuff has been around a very long time, and been in fairly active development for years of that time.
2) The product has been in development for years and a number of boards have been running 24/7, running a fully loaded product.
that's - ironically - weak(1) safety
edit:
(1) necessary, but not sufficient -
Yes of course I agree that I should spend next 100 years of my life writing it all myself
A lot of corporate users buy in something commercial and get a nice warm feeling that way.
I have no illusions that a lot of open source stuff is crap. Probably 90%. But a lot of it is good, simply because of a large user base. The commercial products are often written by one guy anyway; the companies correspond as "we" but it is just one person doing it all The user base will only be according to how many they have sold, but there is plenty of competition in the non-free sector.
It's easy to write "weak safety" but do you want to work for me for a few years for free? Too many one-liners.
Fortunately, embedded products can have a relatively narrow application sphere, so can be adequately tested. With a PC etc you could not ever do that.
Writing a test harness for a TCP/IP stack is a huge job. By the time you have done one which explores all the areas where skeletons are buried, you can write it all yourself.
-
It's easy to write "weak safety" but do you want to work for me for a few years for free? Too many one-liners.
I meant, there are specific skills and specific tools in software testing. You learn the basics from courses like "software engineering" (engineering university courses, second, third and fourth year), you refine and master it later from seniors, and training on things like DO178B.