Makefile question

[westfw]

Off in another thread:

    I am pretty sure that I don't want to compile every library I've ever installed, for each sketch.

Actually, with a proper make facility or something like ccache, it could be pretty painless.

Ok, I'll bite.  I asked this question off on the avrfreaks forum:

Is there a standard mechanism used with Makefiles to somehow take into account DEFINES on the makefile command line when computing dependencies?

That is, if I say

   make F_CPU=8000000

I want it to rebuild any file that has uses $(F_CPU) as part of its build rule(s).  It would be really great if it only did the rebuild if the option was actually different than the last compile...

I didn't get any useful answers. :-(  So I'll ask here too!
It looks like ccache might do some of what I want?  Either by using it for the individual file compiles, or by using it to build an "options" file that is used as a traditional dependency for the rest of the compiles.  But it's complex enough that it's hard to tell for sure, and I was somewhat hoping for a lighter-weight solution...

[Brutte]

...
F... forum software "your session timed out" and my post went down the drain....

[nuno]

If you're running the same makefile, you set a default compilation rule that always passes the define.

If you're invoking make inside a makefile, you pass the define explicitely on the command line, or export an environment variable with it.

[legacy]

i think the problem is how to handle dependencies 

[c4757p]

...
F... forum software "your session timed out" and my post went down the drain....

protip: If you put a lot of time into typing anything into a field on any form on a web page, copy it to clipboard before you submit. Even better, write it in an external editor.

[westfw]

forum software "your session timed out" and my post went down the drain

BACK usually gets me back to my message. where it works the second time...  (YMMV.  I've had other instances where my message has just disappeared.)

======

As an example for the original problem.  Consider optiboot (https://code.google.com/p/optiboot/ ), which has a relatively conventional make file but allows options like:
    make atmeag1824p BAUD=38400 UART=1
A subsequent identical command need not compile everything, but a subsequent command that changes the baud rate or uart DOES need to recompile.

Or consider arduino "sketches" when the "board" type is changed (changes -mmcu=xxx option in compile commands.)

[nuno]

Ah, ok. Without thinking much, instead of passing those args in make's command line, put them in a config file and include the file as a dependency and ii the compilation (I think you even can tell GCC through a cmd-line switch to include a specific file in the compilation, but I'm not sure).

If you really don't want to let go command-line "configurationability", you can also check for cmd-line args in a "static" rule and re-write the config file if there are any of the relevant ones, then the mechanism above takes over.

[gmb42]

Put the definitions, as a make statement, in a separate make (.mk) include file.  Include that file in your main make before the definitions are required, then add it as a dependency for those other files that require the definitions.

[Rufus]

Is there a standard mechanism used with Makefiles to somehow take into account DEFINES on the makefile command line when computing dependencies?

That is, if I say

   make F_CPU=8000000

I want it to rebuild any file that has uses $(F_CPU) as part of its build rule(s).  It would be really great if it only did the rebuild if the option was actually different than the last compile...

Not sure I understand exactly what you are trying to do but it seems like not a good idea. If you build a target for say a particular CPU then the target file name should also include some indication of the CPU it was built for so you know what the hell it is.

Can't you just write rules which expand the option macros as part of the target file name?

Code: [Select]

Main_$(OPT).o : blah blah blah dependencies
$(CC) "Main.c" -D $(OPT) -o"Main_$(OPT).o"


[dgtl]

It is not a good idea to use command-line arguments directly. In case you start your compile with different arguments in different occasions, you may end up compiling different object files with different arguments and things get badly broken. Some months later, you'll change one source file and by mistake compile with different options. That object will now differ and it may not break the build. A decent project will use automatic dependancies, so in case a header file changes, then all objects which included that, will be re-compiled on next invocation of make. (see automatic depencency generation and dependency files (.d)). In case the config gets written to files, the dependency system of make can re-build everything that uses those files in case the files change.
It is a good practice to have configuration file. In case you really do not want to edit those files yourself, you may a) either generate that file dynamically with your makefile b) have a configure-like script, that performs configuration or c) have a bunch of pre-defined configuration files and make rules, that just copy that config to the to-be-used config file.
a) generate config file(s) dynamically: for exmple linux kernel. You may have simpler or more complex config sytem, the easiest way would just have separate make targets, that modify the config. You can have command line arguments that change some fields in those config files. It's a bit tricy to automate editing files, "sed" is your friend there. C projects will include the config file and no problems. If you need to configure make options, generate sub-makefiles and include them. By proper dependencies, if a setting is changed, the required objects and binaries will be re-built and it's bulletproof. Make can take variables from command line, so you may for example implement "make reconfigure F_CPU=8000000".
b) sometimes, it is easier to write a shell script. Do the same as a), just in shell script. The outputs of the script are header file(s) and/or sub-makefiles. Like the configure scripts are implemented, for example "./configure board=board1 f_cpu=8000000" etc.
c) for example, u-boot or linux kernel. You can have pre-defined config files, ie config/board1-config.h. Executing "make board1-config" will just copy the corresponding config file to /config.h. The project will use the toplevel config.h. All re-compiles will work.

[andyturk]

You can use the built-in variable MAKEFILE_LIST to recompile (or relink) things when any of the Makefiles referenced by make have changed. That might not be exactly what you want, but if you're passing #defines on the compiler command line from make, you'll definitely want to recompile things when you change the definitions. The tradeoff is that you'll basically end up rebuilding everything (which is safe, but takes a while) rather than rebuilding only the files that depend on the definitions you changed.

Code: [Select]

$(BUILD)/blink.elf : $(OBJECTS) $(MAKEFILE_LIST) | $(BUILD)
@echo Linking $(@)
@$(CC) \
-Xlinker -Map=$(patsubst %.elf,%.map,$(@)) \
$(LDFLAGS) \
-Tnrf51_softdevice.ld \
-o $(@) $(OBJECTS)

As a side note, the "| $(BUILD)" notation in this rule says that the products also depend on whatever is listed to the right of the pipe ("|"), but only in terms of time. In this case $(BUILD) the output directory that'll contain the binary and we want to make sure that directory exists before running the linker. There's a separate rule (not shown here) that creates the BUILD directory.

[westfw]

In case you start your compile with different arguments in different occasions, you may end up compiling different object files with different arguments and things get badly broken.

Yes, that is an accurate restatement of the problem I wish to solve.
Because I don't want 2^N "targets" for N options (well, more than that, since each option tends to have more than 2 possible values.)

Consider Arduino, which is appropriately criticized for recompiling too much, too often.  Without getting too complicated, Blink.ino will produce code (from the same source) for four different CPUs at two different F_CPU clock rates, both of which are passed to the compiler as command-line arguments.  (F_CPU could presumably be put in a hypothetical config.h file.  But the cpu type must be a compiler option, AFAIK...)  So ... how could I replace the current build algorithms with a Makefile-based approach?

[Rufus]

In case you start your compile with different arguments in different occasions, you may end up compiling different object files with different arguments and things get badly broken.

Yes, that is an accurate restatement of the problem I wish to solve.
Because I don't want 2^N "targets" for N options (well, more than that, since each option tends to have more than 2 possible values.)

2^n different results means 2^n different targets. You don't need to write 2^n rules you can customize one rule as in my previous post.

[westfw]

Main_$(OPT).o : blah blah blah dependencies

That actually looks somewhat promising.  I don't think I realized that you could put variables within targets...

[legacy]

how could I replace the current build algorithms with a Makefile-based approach?

which is the build algorithms ?
i am afraid you can't use makefile directly, i think you'd better implement the build algorithms into a C program that generates the makefile directly
something like

• configure options …. -> getting the old config, update the new, and produces Makefile
• make -> taking the generated Makefile and use it

in where the first step is able to analyze dependencies and whatever is required in order to say exactly what needs to be re-compiled

[djacobow]

I recognize my own comment from the original post.

ccache is interesting because it really doesn't even try to "fix" the problem of proper dependency tracking. What ccache does is impersonate gcc. It receives all the command line flags, all the input files, everything that gcc would see when called and creates a hash/digest of all of it. Then it looks to see if it has an object file associated with that digest in its cache. If so, it just produces that file from its cache, if it does not, it calls the real gcc, and adds that output to the cache.

The result is that you can have "dumb" makefiles that do in fact rebuild everything all the time, and still get reasonable performance.

Whether you find this solution attractive or not is one of personal philosophy and perhaps a bit of experience trying to get make to track all your dependencies correctly. My personal experience has been that in complex projects with sub-makefiles and lots of targets, auto generated code, etc, it gets pretty hard pretty fast -- or maybe I'm just too dumb to keep my makefiles in perfect order.

Anyway, ccache is an interesting tool, totally orthogonal to the make concept, but a friend to the person who just gives up.

[djacobow]

I believe that over the years there have been several projects to make a nice command line environment that works well with the Ard software. I used one years ago and it seemed OK, not great. But maybe they have progressed. Take a look at:

http://hardwarefun.com/tutorials/compiling-arduino-sketches-using-makefile

and

https://github.com/sudar/Arduino-Makefile

how could I replace the current build algorithms with a Makefile-based approach?

which is the build algorithms ?
i am afraid you can't use makefile directly, i think you'd better implement the build algorithms into a C program that generates the makefile directly
something like

• configure options …. -> getting the old config, update the new, and produces Makefile
• make -> taking the generated Makefile and use it

in where the first step is able to analyze dependencies and whatever is required in order to say exactly what needs to be re-compiled