STM Cube: distributing a project where not all modules are supplied in .c source

[eutectique]

Well, I have about 20 of these "no source provided" files. Making a library would be neater - especially as an automatic post build step in Cube.

Well, I would strongly advise on creating the library and using it for your own build. Not just a post-build step to produce a by-product. Eat your own dog food, you know

Yes, ar is the tool to create the library.

Funny you mention debug level. I am currently running with Max. But varying this makes no difference to the binary size. It sounds like it varies the symbol information loaded into the .o file - correct?

This is probably because sections debug.* are so big that the variance in the size of .text is at the noise level.

I am not actually that bothered about symbols in these files

Let me convince you with an example. That file with one-line version string.

Compile it without debug info, get the section headers, get the file size:

Code: [Select]

> arm-none-eabi-gcc -Os -c -o version-string.o version-string.c
> arm-none-eabi-objdump -h version-string.o

version-string.o:     file format elf32-littlearm

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .text         00000000  00000000  00000000  00000034  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 .data         00000000  00000000  00000000  00000034  2**0
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000000  00000000  00000000  00000034  2**0
                  ALLOC
  3 .rodata       00000044  00000000  00000000  00000034  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .comment      00000058  00000000  00000000  00000078  2**0
                  CONTENTS, READONLY
  5 .ARM.attributes 0000002a  00000000  00000000  000000d0  2**0
                  CONTENTS, READONLY

> ll version-string.o
-rw-rw-r-- 1  900 Mar  9 16:54 version-string.o


Now the same with debug info:

Code: [Select]

> arm-none-eabi-gcc -Os -ggdb3 -c -o version-string.o version-string.c
> arm-none-eabi-objdump -h version-string.o

version-string.o:     file format elf32-littlearm

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .group        0000000c  00000000  00000000  00000034  2**2
                  CONTENTS, READONLY, GROUP, LINK_ONCE_DISCARD
  1 .text         00000000  00000000  00000000  00000040  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  2 .data         00000000  00000000  00000000  00000040  2**0
                  CONTENTS, ALLOC, LOAD, DATA
  3 .bss          00000000  00000000  00000000  00000040  2**0
                  ALLOC
  4 .rodata       00000044  00000000  00000000  00000040  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  5 .debug_info   0000005d  00000000  00000000  00000084  2**0
                  CONTENTS, RELOC, READONLY, DEBUGGING, OCTETS
  6 .debug_abbrev 00000047  00000000  00000000  000000e1  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
  7 .debug_aranges 00000018  00000000  00000000  00000128  2**0
                  CONTENTS, RELOC, READONLY, DEBUGGING, OCTETS
  8 .debug_macro  00000011  00000000  00000000  00000140  2**0
                  CONTENTS, RELOC, READONLY, DEBUGGING, OCTETS
  9 .debug_macro  00000a78  00000000  00000000  00000151  2**0
                  CONTENTS, RELOC, READONLY, DEBUGGING, OCTETS
 10 .debug_line   00000031  00000000  00000000  00000bc9  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 11 .debug_str    00002c72  00000000  00000000  00000bfa  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 12 .comment      00000058  00000000  00000000  0000386c  2**0
                  CONTENTS, READONLY
 13 .ARM.attributes 0000002a  00000000  00000000  000038c4  2**0
                  CONTENTS, READONLY

> ll version-string.o
-rw-rw-r-- 1  20K Mar  9 16:52 version-string.o


So, it's 900 bytes vs 20KiB of file size.

The actual payload is the same though, 68 bytes:

Code: [Select]

> size version-string.o
   text    data     bss     dec     hex filename
     68       0       0      68      44 version-string.o


[peter-h]

This is getting interesting. Compiling a file with no debug makes it impossible to step through it with the debugger. Unless you debug in assembler (disassembly) mode, but that just shows mostly garbage.

ST built their libc.a with debug info because when I step through that (has to be in assembler mode; no source is provided) I see various function names.

[eutectique]

ST built their libc.a with debug info because when I step through that (has to be in assembler mode; no source is provided) I see various function names.

That was my point exactly. With debug info, you show much more than public functions and variables of your library.

[peter-h]

A bit of fun with the ar @file option... took a while to work this out, and not all ar versions support it, but the ST GCC seems to.

Done it.

The AR command processes about 30 .o files into a library in well under a second, so I have put this into my Cube post build batch file.

What I have not found a way to do with a batch file is how to set up an additional linker library path etc. I am sure it is possible but Cube has thousands of files.

The resulting .a lib is 60MB and I wonder how much of that must be symbols, which are not needed. I think that to some extent one could achieve a similar result with AR to compiling the .c sources with debug level zero, but a lot of symbols must be retained otherwise a linker won't be able to do anything with it.

[Jeroen3]

Go make a new workspace and play around with a project that compiles to a library that you can link in another project.
Learning this takes effort, and requires reading the docs, but it's worth it.

Or you just ask chatgpt how to do it. It has read all the docs for you.

btw eclipse does a make clean every time you change project settings.

An alternative, depending on how sensitive the code is, obfuscate the code.

[peter-h]

Thank you all.

I have pretty well done it.

One of the more fun bits was how to delete a list of files where the names are supplied in a file. Obviously there are loads of ways to do that, starting with various "unix" utilities that exist for windoze, but I wanted to do it just using built-in windoze cmd commands. It was done with DEL and the FOR... looping syntax.

The AR portion was actually easy.

I found some funny stuff e.g. if you config Cube Linker options to expect a library at a certain path (and you have to specify not just the path but also a lib name then, which is odd) then that library file has to be present, but if you don't want this lib to be used (which is legitimate because during a normal build of the non-distributable version you get all the .o files, and you get them before the linker is invoked, whereas the library generated in post-build is generated after the linker is invoked) the linker fails with an error. IOW, there seems to be no way to use libraries which may or may not be present. So I had to use AR to create an empty library, by creating one with a nonexistent-file.o). It is just 1k in size. What I am doing may not be obvious, without a lot more detail.

In the end the whole job was done using about 20 lines in a .bat file which runs as a post build in Cube.

That obfuscation is funny. About 30 years ago I paid someone to write a program to do that for Pascal source. It also usefully compressed the size of the program.

[paulca]

A few topics come to fore here.  Collaboration.  IP.  Evolutionary complexity.

None of them mix well.  More over, I expect you are about to find the hard way, as I've suggested at several times, once you start with multiple developers or worse, multiple teams of developers and you introduce "business stakeholders" it get exponentially more complicated extremely quickly.

I would echo those who have said, "DO NOT FIGHT ECLIPSE".  Your choices are to abandon it or learn to make it help you.  You can hybrid the projects, we tend to run hybrids in work with independant git, java, maven and eclipse (or inteliJ) build contexts.  When you set up and maintain so many projects in Java/Eclipse/Maven this doesnt even strike one as complex.  In context of your case, this would amount of adding your own custom "parts".  I note you already have a post build factory script.

On learning eclipse, cube mx, ST Cube IDE.  It breaks down to this:

Core Eclipse + CDT C/C++ Developer toolkit plugins <-- these are best downloaded as a pre-configured bundle which deletes most of the Java stuff for you too.
Then you add on top a handful of STM plugins.  These are embedded panels and tabs in various locations throughout Eclipse customized or entirely bespoke to STM. 
Cube itself is just a stand alone application which they have embeded into Eclipse now.
The VAST majority of those multi-gb files are source code, examples and documentation.  If you were to strip those folders back to what is the actual IDE it would be about 400Mb tops, probably half that is IDE cruft too.

Eclipse itself is not that hard to learn and a lot of what you need to solve is already configurable with it or by adding plugins.  If Eclipse won't help you, add a custom Make file or Sconscript or CMake and get eclipse to call it when you hit "Build".  Hybrid approaches can have side effects and work arounds, YMMV.

Try this for example.  Go and download a Vanila C/C++ CDT Eclipse IDE.  Follow a guide to add the toolchain to it yourself!  Get it to compile HelloWorld.a and link it into HelloWorld.exe   Without all the clutter from STM32 it should help you learn a little about what is Eclipse and what it will do and what are the STM complications.

[paulca]

STM32CubeIDE does have a "New static library" wizard if I'm not mistaken.

On brass tacs of how you solve having different audiences for different parts of the application and keep one group out of the others way or hiding IP and knowldge.

There are no short cuts.  You have to componentise.  It might be worth thinking it through and not just taking an evolutionary retro-active step but architect it a little now to make it more flexible to carry on getting more complex and remain managable.

In day to day, eclipse will happily "relate" projects in the work space, such that the dependencies comes from the locally built copies if they exist, OR come from the precanned imports.

You share code via git.  The other parties do no have access to all repos.  In the "public" repo you put the binary releases (.a/.la/.so et. al) and headers.  In the "private" repos you put the code for the libraries.

A core developer has full source code access to all modules.  In his/her eclipse they have each "project" open in the workspace and the relevant dependencies set between them.  If they built the top layer project and there are changes in the library project, Eclipse will rebuild and locally link them.

A 3rd party developer only has the code for the bit he is working on.  For him the "common" and "library" deps have to be downloaded and installed from the public releases repo. 

Both of these can be managed in a single set of Eclipse build configs.

On your diplomatic issues and political / techno-political aspects of how different people respond to different patterns, architectures and restrictions, ... you are on your own!

I will give you one bit of advise.  If you are going to hide the code from the 3rd party developers, one of two things will happen.  Either you have written extensive documentation or they will rewrite their way around your code and do  themselves, making everything twice as complex as it's need to be.