32F417 / 32F437 auto detect of extra 64k RAM

[abyrvalg]

Why do you need to move the stack at all? Just place it like this:
ram_start:
  .data
  .bss
  .stack
  .heap
and grow the heap into the additional 64K

[peter-h]

Does your end user end up including the correct stm32f417xx.h/stm32f437xx.h header?
I assume they have to, because how would they access the peripherals correctly otherwise?

No need to. The 437 is a superset of the 417 in every way except one: the Vbat measurement using ADC1, where the resistor divider is 1/4 versus 1/2.

So the only difference, if the extra 64k RAM of the 437 is desired, is moving the stack up 64k. And this is not just moving SP; in the interests of good coding one also has to fill that region with 'S' or some such.

I have now got a lot further. But I have a funny one, a real mystery, but something which must be very simple. The central issue is whether the Cube disassembly listing is a disassembly of the binary (with symbol names inserted where addresses match) or whether it is done with a copy/paste of bits of the assembler listing produced by the compiler. I suspect the latter.



The code disassembled at the yellow address bears no resemblance to memcpy().



On the face of it, there is something wrong with the values sent to memcpy and/or memset, but I can't debug them properly because I don't know the register conventions for parameter passing, and stepping into the functions is not possible because a) I don't have the C source and b) the code displayed is duff. Replacing those two functions with my own ones solves this problem and everything works!

Why do you need to move the stack at all? Just place it like this:

You are totally right - if the extra 64k was used only for heap.

But using the extra 64k RAM conventionally should work. I really cannot see why not. Something totally weird is going on. Could be just Cube doing silly buggers... but if I disable the checks for the extra 64k, it runs OK.

It also runs fine if the memcpy and memset at the start of the "overlay" are local functions and not the Newlib ones. And this is not some simple product which might work by luck; the amount of functionality is huge. As a 24/7 test I am running a TLS session every 60 seconds, and a constant traffic on serial over TCP, plus a GPS to ARINC429 converter, etc.

Code: [Select]


// Entry point for code called by the boot block - has to be located at the base of FLASH+32k
// This is done in linkfile
// Has to be "int" otherwise compiler complains :)
// Because it is reached via an asm jmp, various measures to prevent it getting optimised away, which don't work. What does work
// is vector2 (which is in assembler and thus immune to optimisation) having a dword pointing to main().

__attribute__((optimize("O0")))
__attribute__((used))   // does not do anything

int main()
{
// Initialise DATA

extern char _s_nonboot_data;
extern char _e_nonboot_data;
extern char _si_nonboot_data;
S_memcpy(&_s_nonboot_data, &_si_nonboot_data, &_e_nonboot_data - &_s_nonboot_data);

// Zero BSS and COMMON

extern char _s_nonboot_bss;
extern char _e_nonboot_bss;
S_memset(&_s_nonboot_bss, 0, &_e_nonboot_bss - &_s_nonboot_bss);

// Go to main program

main_real();

// We should never get here

for (;;);
}



// Local versions of functions used above. They should not be necessary but they avoid the use
// of the Newlib stdlib ones for which we have no sources.

__attribute__((optimize("O0"))) // prevent replacement with memcpy()
void S_memcpy (void *dest, const void *src, size_t len)
{
  char *d = dest;
  const char *s = src;
  while (len--)
    *d++ = *s++;
}

__attribute__((optimize("O0"))) // prevent replacement with memset()
void S_memset(void *s, char c,  size_t len)
{
    char * p=s;
    while(len--)
    {
    *p++ = c;
    }
}

// Vectab2.s ends up linked here, at base of FLASH + 32k + 512

[Nominal Animal]

Does your end user end up including the correct stm32f417xx.h/stm32f437xx.h header?
I assume they have to, because how would they access the peripherals correctly otherwise?

No need to. The 437 is a superset of the 417 in every way except one: the Vbat measurement using ADC1, where the resistor divider is 1/4 versus 1/2.

But the user's won't be able to exploit the additional features of the '437, like the 2D DMA engine, I2C FLTR register for controlling I2C analog and digital noise filter, Serial Audio Interface (SAI), RCC PLLSAICFGR or DCKCFGR for additional clocks (including PLLSAI oscillator), 1024k of additional Flash, SPI4, SPI5, SPI6, UART7, UART8, GPIOJ, GPIOK, AES_GCM and AES_CCM hash modes, additional HASH digest registers (8 instead of 5, so 256 bits instead of 160 bits), additional 8 Flash ACR wait latency options, voltage regulator over- and under-drive options (PWR_CSR_ODRDY, PWR_CSR_URDY), some SYSCFG_MEMRMP and SYSCFG_PMC flags (see AN4073), and increased maximum frequency from 168MHz to 180MHz?

Seems.. wasteful to me, especially because everything the user would need to do is to include the proper header file, depending on their MCU model, that can then include the corresponding stm32f?7xx.h and export the functions defined in the always-accessible boot Flash.


As discussed before, this would be much easier if split into parts, so that when the master boot loader is built, it also produces a skeleton project with correct linker script for the next level code, including user-accessible symbols for the exported functions and variables in the master boot loader.  These are simple assignments of form symbolname = runtimeaddress;. If used within an output section definition, then the address is relative to the start of that output section; if used outside the SECTIONS command, the address is absolute.

For example, if your master boot loader code exports a function named reboot(), which is executed at address 0x08000120 (offset 0x120 from the beginning of the FLASH region), then the header file you provide needs to declare it,
    extern void reboot(void) __attribute__((noreturn));
and in the linker script, for the user C code to execute it, you have
    reboot = 0x8000120;
outside any command blocks.  This does mean that each user project would need the particular headers and linker script for the currently installed master boot loader.

You can also do a jump table at a fixed address in Flash, each being exactly 4 bytes long and resolving to an unconditional branch instruction, b.w label.
Each entry in the jump table then corresponds to a jump to a specific function exported by the master boot loader, for example the first one could be the abovementioned reboot().  This costs one unconditional branch instruction, or four bytes, per function exported from the master boot loader to user programs, but this way their location in the Flash isn't fixed.
Then, the user can compile their code for any master boot loader version, because the actual function entry points are always the same in the jump table; the jump table itself may vary, as it is part of the master boot loader.

If you need some read-only configuration values (including the bootloader version), I recommend you put a structure at a fixed address in Flash, that contains the values, and export the structure symbol in the linker script.  Pad the structure with some zeroes, so that later versions of the master boot loader can export new values, and the user code detect if their master boot loader is too old to export that value.

[peter-h]

Seems.. wasteful to me

Those are all good points, but a finished product is likely constrained by GPIO pin allocations, so most extra features cannot be used anyway.

[Siwastaja]

Seems.. wasteful to me

Those are all good points, but a finished product is likely constrained by GPIO pin allocations, so most extra features cannot be used anyway.

I'm asking again, if the point of ONE FIRMWARE is for manufacturability with two BOM options, why would you need to take advantage of the "extra RAM", when you don't take advantage of any other extra feature (like peripherals) either?

On the other hand, if you plan doing different projects on different CPUs, then obviously it won't be one firmware anyway, so none of this crap is needed.

Or are you solving a nonexistent problem and trying to find a way which causes most trouble and maximizes the number of forum posts?

[peter-h]

why would you need to take advantage of the "extra RAM", when you don't take advantage of any other extra feature (like peripherals) either?

That's just the way it is. The extra RAM is all that is needed, and indeed most of the extra features cannot be used due to no GPIO left. The 180MHz could be used. What would be really handy would be hardware RSA Takes 3 seconds right now.

Or are you solving a nonexistent problem and trying to find a way which causes most trouble and maximizes the number of forum posts?

All of the above.

Just wondered... are you Russian, by any chance?

[nctnico]

If you need some read-only configuration values (including the bootloader version), I recommend you put a structure at a fixed address in Flash, that contains the values, and export the structure symbol in the linker script.  Pad the structure with some zeroes, so that later versions of the master boot loader can export new values, and the user code detect if their master boot loader is too old to export that value.

My default solution is to also include a version number in such a configuration struct so software can't (at least) use the wrong layout. Best case software can try to convert between configuration versions but I never implemented that so far.

[peter-h]

export the structure symbol in the linker script.

Or this
https://stackoverflow.com/questions/19252128/how-to-use-the-include-command-in-ld-linker-script

I think I've fixed it all. Several things, mostly silly. But a good learning experience.



but with 32F437, above is the same but:



Thank you all (those who helped) for all your huge help

[Siwastaja]

Just wondered... are you Russian, by any chance?

... nope?

[SiliconWizard]

  as well?

[peter-h]

Rudeness.

[Siwastaja]

You get somewhat more "rude" replies when you ask for extensive help with details while actively refusing to discuss the question what you are actually doing - i.e., playing the x-y question game. The reason is simple: such behavior itself appears rude. But I don't mean this is wrong; maybe your project is a commercial one with obvious limitations what you can discuss and what you can't. But then the discussion will not be as fruitful as it would with full disclosure available, you need to live with it.

Ignoring advice by others due to your stubborness also appears rude to others. You can't choose different rules for yourself and then expect others need to play extra nice with you. While I applaud your "I want to do it this way, help me with this, I don't want any help with the big picture" attitude as interesting and thought-provoking, you need to accept you are going to get honest replies (along the lines: "geez, what a stupid problem to waste time with") without sugar-coating, too.

As almost every message of yours start with blaming everything else except yourself (tools no one else has any problems with are now total crap, documentation is always crap even though others find absolutely no issue with GNU LD documentation), expect to be shown that there is blame to your actions, too. If you don't want that, stop the blame shifting game, you are the one who always starts it.

I have been reading your posts with interest for an extended time now and it is still quite unclear to me whether you complicate your already-solved problems on purpose because you enjoy doing things in esoteric ways, or if that is truly something you don't want, but still "somehow" things always end up that way, which you find very frustrating (no wonder), and then you get even more frustrated when others point to you that there are easier ways out of the whole minefield, as you fail to see those ways.

If the latter, what you would need is help with the big picture and priorization of things, and picking your fights better. After all, the problems you are solving are very common and solved gazillion of times, with no need to sort functions in linker scripts or compile code with -O0. For example, regarding bootloaders, you have got very good tips how to design a simple architecture, but it seems you ignore the advice.

[peter-h]

I've been on tech forums since about 1990. There is a tradeoff between forming a specific question and making a huge long post. The former should get intelligent reponses while latter is likely to be ignored.

The project is intended to sell eventually (although I also plan to use it for various hobby projects) and that constrains how much I can post, but I take care to post the relevant detail.

On this topic, auto RAM detect, there are great reasons for not building multiple projects, not building the boot block separately, etc, but I would have to post the whole RM which nobody would read. But let me give a little example: a while ago somebody set up a Cube kit with multiple projects, and I found some files were cross-linked across projects. Wasted god knows how much time and was discovered only by accident. Could have been a disaster. A bit like the (true) story about a company with 20 devs writing code for a year, backed up to some unix box every day, plus a tape drive, then somebody upgrades the HD, reformats it, and finds that the tape backup was dud the whole time. Luckily they had printouts so they collected all the secretaries and got 50% of them to type it all in, other 50% to proof read it. I am more careful than most, because it is all my business, and any shit lands on my lap. I have no help and use the internet to find answers, while hopefully contributing stuff which others may find helpful in years to come.

This

Or are you solving a nonexistent problem and trying to find a way which causes most trouble and maximizes the number of forum posts?

is purely gratuitous rudeness, which has no place in my "world". If you don't get that, nothing I can do about it. As a current tech forum mod/admin (343k total posts in 10 years) I am pretty familiar with that sort of thing, but actually I would just delete such a post because it is a borderline personal attack and thus contra the rules on that forum.

[Siwastaja]

I've been on tech forums since about 1990. There is a tradeoff between forming a specific question and making a huge long post. The former should get intelligent reponses while latter is likely to be ignored.

I'm not asking you to provide more details or make longer posts. I'm suggesting you should discuss more about the big picture, and less about weird corner cases in esoteric use pattern of tools, although I appreciate what you do because such esoteric details are interesting and I learn every time from your threads, but it is sad if you do not enjoy it or have a job to finish. Also, you should sometimes try what others suggest, instead of trying to find "just that one fix to this one line" which seemingly "solves" the problem you think you have.

Or are you solving a nonexistent problem and trying to find a way which causes most trouble and maximizes the number of forum posts?

is purely gratuitous rudeness, which has no place in my "world". If you don't get that, nothing I can do about it. As a current tech forum mod/admin (343k total posts in 10 years) I am pretty familiar with that sort of thing, but actually I would just delete such a post because it is a borderline personal attack and thus contra the rules on that forum.

OK, you are free to have that opinion but I don't agree. You are still allowing being rude and playing social games, and you only draw the line by honesty, such that more honest = worse. If I had formed the same sentence to be even more rude, but in a more subtle, hidden way, you'd have been OK with it. Fact is, you are rude all the time. This is why you receive rude replies. I'm just being directly rude, and the question I posed is honest one, and you should really pay attention to it. After all, if you are trying to make a product, trying to solve nonexistent problem after another is a total disaster time-wise and the project will never finish. I'm truly trying to be helpful here, as I have sometimes the same tendency to veer off to development time sinks. It truly seems to me you are just going in defensive coping pattern because I asked the right question. Do some introspection.

BTW, name-calling a Finn "a Russian" is quite some personal attack and definitely rude, and I think you know it (I don't believe you did it by accident). I don't mind name-calling if that eases your pain, but that's still hypocritical.

[nctnico]

If the latter, what you would need is help with the big picture and priorization of things, and picking your fights better. After all, the problems you are solving are very common and solved gazillion of times, with no need to sort functions in linker scripts or compile code with -O0. For example, regarding bootloaders, you have got very good tips how to design a simple architecture, but it seems you ignore the advice.

You have to define simple. AFAIK peter-h wants to have the entire project (bootloader + firmware) as a single project so it is easy to keep both together and thus makes it easier to distribute to people who want to work on the project. All while avoiding having to resort to command line tools / scripts. Sometimes making a truly simple solution is extremely hard to do.

[peter-h]

and less about weird corner cases in esoteric use pattern of tools

It is those where I get stuck. The rest I can just do...

You are still allowing being rude and playing social games

I don't get that... social games???

you are rude all the time

I think you have a problem. Sorry but I can't help.

I know why most forums allow this: it drives advert clicks way up.

AFAIK peter-h wants to have the entire project (bootloader + firmware) as a single project so it is easy to keep both together and thus makes it easier to distribute to people who want to work on the project. All while avoiding having to resort to command line tools / scripts. Sometimes making a truly simple solution is extremely hard to do.

Exactly.

I have just one Cube project and when this is built (with ctrl-b to build it only, or F11 to build + send to STLINK debugger) it runs a post build batch file which also generates a second version of the project for 3rd parties to use (without the boot block and various confidential parts). Same Cube config in every way but you just import this second project. That batch file is actually complicated (50 lines? - took me a whole day) but once done it is done for ever. It just works.

I actually run this 2nd dev kit in a VM, to ensure there is no cross contamination. That VM forms a totally standalone environment which will run tomorrow, or in 10 years' time. Taking suitable care to handle the windoze license inside it, it can be distributed, too. This is a major plus for a corporate environment; most companies lose sources (or can't build them anymore) and lose basically everything within a few years.

Like most things to do with today's chips (2000 page RMs etc) Cube itself is way too complicated to expect anybody to learn how to use it from scratch. The project config alone would be horrific.

There is a lot of other stuff e.g. security, inherent in remote firmware deployment.

And the various issues I've posted about have had very little to do with building it all together.

[Siwastaja]

I don't get that... social games???

Exactly - you don't get that - social games would pass your acceptance filter. You are not against being rude, you are against people saying the inconvenient truth out loud. Your bootloader saga has lasted for, what, more than a year with numerous threads and dozens of detailed posts. What is the reason you don't give a chance to any idea that was suggested to you by people like ataradov and others, like maintaining the bootloader as a separate "project". You can compile them together using makefiles or scripting language of your choice - the whole concept of "projects" is arbitrary, and only pushed to you because of CubeIDE, which was your choice to use.

Also, what is the reason you bang your head against the wall with CubeIDE - there are now 38 pages on your "buggy crap" thread - when you could simply automate the whole shebang using a simple <100-line makefile, so that anyone could just start working with the project (type make), without installing virtual machines running cubes and reading some documentation about what buttons to click?

This is what I'm after. If the reason is that you truly want to work this way, and letting the project grind to a halt for a year, two, maybe three is acceptable compromise - then this is totally fine. But I have this feeling you are not satisfied with the outcome and are projecting your anger against those who have guts to say this out loud. But go on, you can keep ignoring everyone who tries to truly help you. You can even put me in your ignore list, and you can continue name-calling me, which is what you have been doing all the time (sarcastic "genius" name calling, and annoyingly indirect "I'm being called an idiot by you" gaslighting, despite my numerous attempts to rectify this by clarifying I don't think you're an idiot. This is also a form of social game on your part, you know. But I'm sure you won't recognize it, or think as something which is against the rules.)

[peter-h]

Also, what is the reason you bang your head against the wall with CubeIDE - there are now 38 pages on your "buggy crap" thread - when you could simply automate the whole shebang using a simple <100-line makefile, so that anyone could just start working with the project (type make), without installing virtual machines running cubes and reading some documentation about what buttons to click?

Hmmm, debugging? OK, one can buy a Segger and use their 1970s user interface. I had a play with that. OK in 1980s...

I do know the difference, having come from Wordstar 4, then Brief (remember Underware, Inc?), Polymake, etc. I still sell a user programmable comms product which uses this stuff. For good measure the compiler needs DOS 6.2 so winXP command box, or a VMWARE equivalent. Must have sold 100+ of these kits at £450 each. Zero now.

But the acceptance of these 20th century tools today is roughly -273.16C.

Also the acceptance of a dev kit selling for £450 is roughly -273.16C.

[Siwastaja]

You can let users import the project in any IDE they like to debug with, without having to run all production or your official process through that IDE. This forces your project "project file free" and avoids mystical cross-dependencies or weird bugs. This brings the best out of both worlds. Fact is, for automation like code builds, command line tools are still relevant in 2023 because they can be easily automated. You can also integrate tests this way, while Cube does not provide any kind of test facilities whatsoever. (Unit and integration tests are way more important than single-stepping code in debugger and examining memory bits, but it seems many microcontroller folks are mentally stuck in 1980's, just with fancier graphical UIs. We should learn useful bits from application programming field, where tests and traceability are everything, and no one asks for a debugger.)

[peter-h]

Cube is free, and the project is just a tree, so they can import into any IDE which can import stuff.

Then spend days working out the compiler and linker options and libraries...

Good luck with that. Guess who will be supporting customers on it?

[Siwastaja]

Then spend days working out the compiler and linker options and libraries...

No no no, the idea is to put all that in a single, simple makefile. Once it's done, no one has to figure out anything; just git pull; make. If debugging or other specialties are needed, import to any IDE. But don't make the project depend on it, because such tooling needs are like farts, those by others stink more, so don't force people to one IDE.

And if you think you need some super special compiler and linker options, step back and try to imagine why other people do not need them. My projects have been with 99% the same options for over a decade.

[nctnico]

Then spend days working out the compiler and linker options and libraries...

No no no, the idea is to put all that in a single, simple makefile. Once it's done, no one has to figure out anything; just git pull; make. If debugging or other specialties are needed, import to any IDE. But don't make the project depend on it, because such tooling needs are like farts, those by others stink more, so don't force people to one IDE.

The problem is that at that point you'll have lost 100% of the people that just want to make a quick change to a project. I see that with all my customers that want to have a go with the source code themselves. You are definitely not doing people like that a favour with command line tools for which they can't look up 'howto' guides quickly or that need installing a whole bunch of other tools. It is much easier to say 'Install CubeIDE version X.Y', import the project and go from there. The whole reason CubeMX (and similar IDEs exist) is to get going quickly without much fuss either as the developer side and the customer side. Besides that, creating a good, universal makefile is an art in itself. Even up to the point where tools where created around it (autoconf) or alternatives (cmake for example).

[Nominal Animal]

Besides that, creating a good, universal makefile is an art in itself. Even up to the point where tools where created around it (autoconf) or alternatives (cmake for example).

That's a bit unfair, because autoconf was created to adjust Makefiles for different Unix systems without user intervention; not because of any intrinsic Makefile properties.  Stuff like whether certain features and functions were provided by the system or not; nothing to do with the Makefile syntax per se.  Autoconf wasn't and isn't a particularly good or useful way to adjust settings in Makefiles.  Other tools, like the Linux kernel configuration tools, don't actually edit the Makefiles directly, and instead work on flat text files that define macros/constants that define the configuration, and are included and used by the actual unmodified Makefiles.

While I much prefer Makefiles myself (and can craft robust ones), and do not like to use any specific IDE, I can see why an IDE would make a big difference when the intent is to provide a product for clients to develop further.

After all, at the point when Arduino came about, we already had full free open source development tools for AVR (avr-gcc, avr-binutils, avr-libc), while most vendors still charged for the development tools quite a lot.  The Arduino folks just wrapped it all within an IDE based on Java (with a weird preprocessor), but that made the field explode among hobbyists.

[wek]

No no no, the idea is to put all that in a single, simple makefile. Once it's done, no one has to figure out anything; just git pull; make.

Getting make to make on anybody's PC (and that's mostly flavours of Windows, however you don't like the idea)  is surprisingly hard. We've had a thread about it recently https://www.eevblog.com/forum/microcontrollers/lets-discuss-telluriums-tutorial/msg4437313/#msg4437313

I don't dig git either, and have arguments, but we are hijacking this thread already.

JW

[peter-h]

Yeah - not supporting Windoze, requiring the use of git, editing makefiles, will practically kill the market except to hard-core anoraks who are probably not going to buy anything like this anyway because they built their own during the previous night

Support is a huge angle and I strive to minimise this by design and clear docs. Auto detect of the RAM is a part of this.

I still don't have a good solution for the linkfile LENGTH=128K or 192K. I will live with this for now; it is easy to document.