C, inter modules dependencies

[legacy]

hi guys
I am in the need to have a tool able to show me all the dependencies(*) inter modules

a C-module is usually composed by two files

• header, defines types, global variables, constants and interfaces (C prototypes of public functions, methods than can be called by other modules), and dependencies (other include files)
• body, it's the implementation part, it also includes private variables, constants, and private functions (other modules can't see and call them, they are hidden to the world)

having a collection of these modules implies dependencies between them

two types of (*)dependencies at least

• type dependencies, a module in the need to use a typedef to define something (e.g. a struct field, or a function's parameter type) and the itypedef is defined in the header of an other module
• interface dependencies, a module which has a function call in its body, but the called function is defined in an other module

having a C module called lib_xxxx_vz, i'd like to see it composed by

• lib_xxxx_vz.h, including "lib_xxxx_vz.interface" and "lib_xxxx_vz.dep"
• lib_xxxx_vz.c, body, calling #include "lib_xxxx_vz.private"
• lib_xxxx_vz.interface public interfaces
• lib_xxxx_vz.private private interfaces
• lib_xxxx_vz.dep dependencies with other modules

is there anything able to do so ? able to built the lib_xxxx_vz.dep or something similar ? .


it makes sense for my hobby, but it makes more sense for my job
especially when it happens that I have to collaborate with the testing-squad
I mean I am often asked to fill "tables" of dependencies and manually filling them is a very boring activity,
nobody wants to do this job, so it's always my turn
and those damn tables are are absolutely required
by Microsoft Word and by a program called "Stood"   
so I also have to fill them twice
because it happens that Microsoft Word tables are not compatible with Stood tables
oh, well 


all the tricks&tips are the welcome, thank you in advance 

[nctnico]

Smells like some Doxygen could be useful for. But promise not to use Doxygen as a replacement for actual documentation. Source code never explains clearly why it is written that way so Doxygen output doesn't help understanding a program.

[legacy]

Understand by Scitools can help you to understand a project, I mean it creates different graphs and reports, e.g. tree-call graphs, and it can perform coverage (a plug in is required). It's useful for me, I have a few licenses for MacOSX/intel and Windows/XP, but it does not help me with the tables I have to create in order to do my job.

I will read the Doxygen documentation (never used before) in order to understand if it is able to create a dependencies-table. In case It will not, I can plan and allocate up to 2 weeks (and no more) of development in order to created the tool I need, but no more than 4 hours a day.

[Mechatrommer]

i also looking forward if any such app to make reverse engineering or understanding ossw code easier. i made my own very simple "inter-files dependency only" library manager (capture attached) to make things easier by searching #include keyword, to make inter-functions things can get too difficult to code, there will be too much to parse and generate. if i'm not mistaken, doxygen only generates comments (explanation) for each subroutines which is something i usually do myself and more effective for my need, no?

[westfw]

• lib_xxxx_vz.h, including "lib_xxxx_vz.interface" and "lib_xxxx_vz.dep"
• lib_xxxx_vz.c, body, calling #include "lib_xxxx_vz.private"
• lib_xxxx_vz.interface public interfaces
• lib_xxxx_vz.private private interfaces

You will probably make your life much easier, wrt "standard tools" if you name the files with a .h extension, even if that means
"lib_xxxx_vz_interface.h", "lib_xxxx_vz_private.h", etc.  (hopefully your compiler supports long filenames.)

lib_xxxx_vz.dep dependencies with other modules

Do you actually need something different than the dependency files that the compiler (most compilers) will generate for you automatically?  http://www.microhowto.info/howto/automatically_generate_makefile_dependencies.html
(make needs that list, anyway.)

There are a bunch of tools that build additional information about a project, that are very useful for understanding other peoples code (or your own, for that matter.)   The two that I use "a lot" are the "ID Database tools" (http://www.gnu.org/software/idutils/manual/idutils.html ), and EMACS "tags"  (there are equivalents for other editors.)   These are essentially old-fashioned CLI tools; a modern IDE like Eclipse or Visual Studio will build similar databases for you automatically and invisibly, letting you "browse" source pretty effectively.   I think.   Once you figure out how to use them.   (I guess one advantage of the old CLI tools is that they'll tend to work on a source tree regardless of whether it's set up to be built using some other IDE.)

[legacy]

You will probably make your life much easier, wrt "standard tools" if you name the files with a .h extension, even if that means
"lib_xxxx_vz_interface.h", "lib_xxxx_vz_private.h", etc.  (hopefully your compiler supports long filenames.)

It's not my choice, my boss wants project's files called that way, and I would be surprised if a modern OS could not manage long file names.

Do you actually need something different than the dependency files that the compiler (most compilers) will generate for you automatically?

I have to fill tables in Microsoft Word and Stood, with thousand and thousand of entry -> extremely boring! so if a tool is able to generate dependency files then it's the welcome because I will only have to parse its output in order to automatically build the tables I need for my purpose.

My ideal purpose: just to create 2 sub tools (one for Microsoft Word, one for Stood) making them able to produce what I need, so then I will just copy and paste their outputs

yo man, easy life, I am lazy 



ID Database tools looks very interesting to me

[tggzzz]

You will not, in general, be able to do this with C without (in one form or another) running the compiler. There are many reasons for that, of which the simplest and most obvious are conditional preprocessor directives, and macros.

If you have a very well disciplined codebase then you might be able to make simplifying assumptions.

To assess what will and won't be possible, consider importing your codebase into a decent IDE, and seeing what that can and cannot discover.

[legacy]

i also looking forward if any such app to make reverse engineering or understanding ossw code easier

try Understand by SciTools, shoot a PM to me for a few tricks

i made my own very simple "inter-files dependency only" library manager (capture attached)

it looks very good, do you have a tokeners ? a parser ? I have coded a C interpreter covering just a part of the C grammar, so I am only able to do a few checks, but I am able to check the McCabe Cyclomatic Complexity without the need to access the customer "code-validator" that  …. has a floating license manager, so just n=2 connects  are allowed (and I found it always refusing mine  ).

An other tool that I HATE is VectorCast, this tool needs to be used for the coverage analysis, but it's really a crap and it crashes too often 
It's too complex to be rewritten, or hacked (and I can't, in the principle, hack a customer tool), so I just have added it to my you-hate-but-you-must tolerance list.

to make things easier by searching #include keyword, to make inter-functions things can get too difficult to code, there will be too much to parse and generate. if i'm not mistaken, doxygen only generates comments (explanation) for each subroutines which is something i usually do myself and more effective for my need, no?

I guess yes.

[legacy]

obvious are conditional preprocessor directives, and macros.

why ?

• interfaces dependencies do not require the preprocessor
• typedef dependencies might require the preprocessor, but we (in the team) are not allowed by the boss to use any #define with typedef, and it makes him pissed off if he sees a an abuse of #define, and our code must also pass a lot of "code-validators" in order to make the commit accepted in the repository (our final customers want that way)

but in case of need, I have no trouble to "add" a "preprocessor" that expands macros and #defines, I have already coded a working one and I can integrate into my tools.

to tell you the truth, we have a special include file in our projects, it's called "types.h", it's located at the top of our libraries path and it's a mixture of #define and typedef, but mainly typedef is used to (re)define things like uint32_t, fp_IEEE_754 _t , fxp32_t (fixed point, used by our soft core, VHDL), etc, while #define is used to (re)define the language, e.g. we are NOT allowed to use "==" in the if-then-else statement, so we have to use "isEqualTo", and we can't use "&" to get the address of an entity, we must use "get_address()", which are both #defines

- "it's boring and pedantic, but it makes sense" - I was told by my boss, even if I am still too young to understand why? 

What I Know is that our sources need to pass a super-set of MISRA checks, I mean our customers give us their "code validators" and we have to apply them to our sources. These "code validators" are extremely pedantic and also apply special rules from DO178/A-B, plus other constraints, e.g. for each module the maximal McCabe Cyclomatic Complexity  must be < 7, which is not "C", and it's not MISRA, it's an additional rule imposed by our customers.

I am not in the development squad, I am the testing-boy, unfortunately, because of my junior-profile, against their senior-profile, so often I am asked to care about the full documentation, which I have to provide written in Microsoft Word (puuaahhh, I hate it, but I MUST use), plus an other document produced by Stood. A part of this documentation is related to the dependencies, so I have to fill a big matrix {1..n,1..n} in where I have to enumerate all the modules from 1---to--n and then, for each 1={1..n} I have to indicate which module is in dependency by which module, with both direct and inverse matrix.

Stood should be able to provide all of these informations, but ... it's an other story, like having the full version released only for SunOS, with the licence expired since a while, problems to get a new license (never understood the Why-on-the-Why(1)?), and a demo version for Windows installed on our desks, which is cool and free but with a lot of features disabled (including the dependency matrix  ) ....

(1) about the Why-on-the-Why
I guess, because a job in avionics must not be fun 

[free_electron]

Isn't it time for C to grow up and do away with all these stupid forward declarations and header files ?
Why is that needed anyway ? all the compiler has ot do is go over all your source files once , collect all globals and functions and build a map. and then start compiling.

All the misery with headers that don't match the actual routines , all the extra keyboard pounding .. it all goes away. Huge time and misery saver.

C is an archaic language full of restrictions and idiosyncrasies because the original code parser was severely limited in what it could handle.

I have seen IDE's for C where you can simply define a function and the IDE will create the forward declaration for you. i think it is called Cide ( short for C-IDE).
It maintains all dependencies for you. so no more problems with circular inclusions either. 

[tggzzz]

obvious are conditional preprocessor directives, and macros.

why ?

• interfaces dependencies do not require the preprocessor
• typedef dependencies might require the preprocessor, but we (in the team) are not allowed by the boss to use any #define with typedef, and it makes him pissed off if he sees a an abuse of #define, and our code must also pass a lot of "code-validators" in order to make the commit accepted in the repository (our final customers want that way)

but in case of need, I have no trouble to "add" a "preprocessor" that expands macros and #defines, I have already coded a working one and I can integrate into my tools.

to tell you the truth, we have a special include file in our projects, it's called "types.h", it's located at the top of our libraries path and it's a mixture of #define and typedef, but mainly typedef is used to (re)define things like uint32_t, fp_IEEE_754 _t , fxp32_t (fixed point, used by our soft core, VHDL), etc, while #define is used to (re)define the language, e.g. we are NOT allowed to use "==" in the if-then-else statement, so we have to use "isEqualTo", and we can't use "&" to get the address of an entity, we must use "get_address()", which are both #defines

- "it's boring and pedantic, but it makes sense" - I was told by my boss, even if I am still too young to understand why? 

What I Know is that our sources need to pass a super-set of MISRA checks, I mean our customers give us their "code validators" and we have to apply them to our sources. These "code validators" are extremely pedantic and also apply special rules from DO178/A-B, plus other constraints, e.g. for each module the maximal McCabe Cyclomatic Complexity  must be < 7, which is not "C", and it's not MISRA, it's an additional rule imposed by our customers.

I am not in the development squad, I am the testing-boy, unfortunately, because of my junior-profile, against their senior-profile, so often I am asked to care about the full documentation, which I have to provide written in Microsoft Word (puuaahhh, I hate it, but I MUST use), plus an other document produced by Stood. A part of this documentation is related to the dependencies, so I have to fill a big matrix {1..n,1..n} in where I have to enumerate all the modules from 1---to--n and then, for each 1={1..n} I have to indicate which module is in dependency by which module, with both direct and inverse matrix.

Stood should be able to provide all of these informations, but ... it's an other story, like having the full version released only for SunOS, with the licence expired since a while, problems to get a new license (never understood the Why-on-the-Why(1)?), and a demo version for Windows installed on our desks, which is cool and free but with a lot of features disabled (including the dependency matrix  ) ....

(1) about the Why-on-the-Why
I guess, because a job in avionics must not be fun 

Why? Because C allows and encourages poor practices, and claims it to be an advantage - not restricting the programmer etc. And there is a lot of strange code out there.

Another spanner in the works is pointers-to-functions, where the pointed to functions cannot be determined statically, but can only be determined dynamically i.e. at runtime. Such facilities are extremely useful in some cases.

However, since you have now told us that the code is MISRA compliant, most difficulties will be trapped out by the MISRA rules. Shame you didn't mention that earlier.

[Mechatrommer]

i also looking forward if any such app to make reverse engineering or understanding ossw code easier

try Understand by SciTools, shoot a PM to me for a few tricks

$1K tag? cough cough...

i made my own very simple "inter-files dependency only" library manager (capture attached)

it looks very good, do you have a tokeners ? a parser ?

no, nothing. just check for "#include" keyword and figure out what another file that the file tried to include, once collected, draw it in "tree-like" diagram as pictured (took me sometime to code all that), sort of like what Understand by SciTools does, but it surely got alot more features than mine. i wish i have "inter-subroutines or inter-functions" dependency as well, but that is too complex for me to code. i'm not a professional programmer, just for personal use, i dont even need any complexity analysis i prefer to do that or anything else by hand. so any tool with price tag greater than $50 is cough cough for me.

[legacy]

$1K tag?

quite shocking but $1K is a reasonable price for a tool like that, in the avionics budget. Apps&Tools by Green Hills and Windriver are mmmmmmmoooooorrrrrrreeeeeee expensive.

i wish i have "inter-subroutines or inter-functions" dependency as well, but that is too complex for me to code

yep, I can code these features, and I think they are extremely useful
I can modify and recycle the C interpreter, perhaps its AST tree will be useful to reduce the development time of the new tool.

so any tool with price tag greater than $50 is cough cough for me.

I know what you mean, the Scitools app is cute for my job tasks, my boss has paid it for me, but it's completely out of my hobby budget
(and … in the theory, I couldn't install anything on my personal laptop)

[donotdespisethesnake]

Isn't it time for C to grow up and do away with all these stupid forward declarations and header files ?
Why is that needed anyway ? all the compiler has got to do is go over all your source files once , collect all globals and functions and build a map. and then start compiling.

All the misery with headers that don't match the actual routines , all the extra keyboard pounding .. it all goes away. Huge time and misery saver.

C is an archaic language full of restrictions and idiosyncrasies because the original code parser was severely limited in what it could handle.

C is indeed a terrible language which should not be used for any serious programming. Unfortunately everyone does!

Way back when C started becoming popular, I thought "this is a terrible language, it will get superseded soon" but 30+ years later, there are better languages, but people don't use them.

C++ really doubled down on the whole mess, and unfortunately just prolonged the life of something that should have died.

We were having a quality exercise at work, and was asked what would really improve our software. I said "we should stop writing in C, for a start". I may as well have said "we should all go and live on Mars"!

And if anyone still thinks C is a good idea, ask them what "a[ i ] = i++;" should do, according to the C standard.

What is hilarious, is all those projects claiming to be "safety critical" but still writing in C. Even the book "Safety Critical Programming in C" starts off by saying, "don't use C"

[legacy]

book "Safety Critical Programming in C" starts off by saying, "don't use C"

don't use C, it's too hippy! and don't smoke too grass, buddy!
oh well, good theory, what about practical alternatives ?

lord Pascal and lady Modula2 seem died since a while
so here, in avionics, we have to deal with schoolmarm Ada (In Strong Typing We Trust)
but at home I can use python on my laptop, it's powerful and elegant
thousand light years away from all the crap about C, there is no need to invoke a "code validator", but ...
... I can't use it to compile for my DSP, oh shit 

in short, I guess there is a reason, a practical reason, if C has survived until now
I am too young to argue, I was born when C was already fixed in ANSI/C
and then into safeC (MISRA compliant), and all my embedded devices have a C toolchain

[legacy]

p.s.
here we have a professional tool called "AdaMULTI" made by Green Hills Software. It supports C/C++ and Ada, no pascal/modula2/... and my boss has showed me that Ada is used to code the high level, all the low level, including drivers and the operating system (VxWorks, Integrity, whatever_RTOS), is written in C, all my testing-activity is to care about drivers, and BSPs

In the open-source-world I found that gcc has no internal support for Ada, it requires an external core.

Code: [Select]

dev-lang/gnat-gcc

GNAT is a good choice, but it needs a bootstrap in order to be compiled, and it's hard to get Ada bootstrappers for MIPS, or for HPPA, while it's easy to get one for X86, AMD64, PPC, SPARC, and … may be ARM (I have seen bootstrap ARM/LE around)

Code: [Select]

SRC_URI="ftp://gcc.gnu.org/pub/gcc/releases/gcc-${PV}/gcc-core-${PV}.tar.bz2
        ftp://gcc.gnu.org/pub/gcc/releases/gcc-${PV}/gcc-ada-${PV}.tar.bz2
        amd64? ( https://dev.gentoo.org/~george/src/gnatboot-${BOOT_SLOT}-amd64.tar.bz2 )
        sparc? ( https://dev.gentoo.org/~george/src/gnatboot-${BOOT_SLOT}-sparc.tar.bz2 )
        x86?   ( https://dev.gentoo.org/~george/src/gnatboot-${BOOT_SLOT}-i686.tar.bz2 )"
        ppc?   ( mirror://gentoo/gnatboot-${BOOT_SLOT}-ppc.tar.bz2 )

so, i have emerged GNAT on my PPC laptop, but I am still struggling with practical things on my embedded toys: too effort is required, and often … it the toy is 8bit with a few resources … Ada needs to be reduced in its features (disabling exceptions, and other run-time stuff, like range-check)

in short, I guess it is a tradeoff

[amyk]

The way to make intermodule dependencies easier to understand is simply to create fewer of them, which means making the modules reasonably large and simplifying the whole project structure/design as much as possible. The worst are multi-file projects where each file contains a dozen or so lines of actual code at most.

[tggzzz]

The way to make intermodule dependencies easier to understand is simply to create fewer of them, which means making the modules reasonably large and simplifying the whole project structure/design as much as possible. The worst are multi-file projects where each file contains a dozen or so lines of actual code at most.

That is perilously close to saying make everything global

Minimising dependencies (i.e. coupling) is only half of it: you also need to maximise cohesion.

The number of source code files is a secondary concern. The objective is to have a module/library containing a suitable number of source code files whose overall operation can be specified by a small number of functions and data structures. Those (few) functions and data structures are declared in the single header file that is "exported" from the module/library an #included in other modules.

What's the best size for a module? Open to debate, but the neatest description I've seen is that "unit of modularity" = "unit of publication/distribution".

[ale500]

And those >1k $ tools are soo crappy inflexible and full of wtf didn't they think of this, that, why the hecks does it crash !, crappy reports and so on.

What I ask myself is if they want MISRA compatible code, why the heck the compiler does not enforce it !, it make no sense whatsoever to write the code twice, so to say. C has no strong types ? great, don't use C, use something else. Write a bloody compiler ! it is not rocket science after all...

One very nice example:

I don't remember which rule but one of them says that you have to cast the result of sizeof to something else because its size is implementation dependent and the code will not be portable.

This has two problems (at least)

1. What if the code is not gonna be ported to other compiler/architecture ? (I know about the exceptions...)
2. At least one of those "MISRA" checkers (costs like 3k per seat or so) allows:

(void)memcpy(dst, src, (size_t) sizeof(some_var)); // some people on our team think that this is a great solution... read below...

if you remove the cast to size_t is complains with the comment above, if you cast !! it doesn't. Yeah, poor implementation. It doesn't help that such "solutions" to a poor implementation of the checker pollute the way around such "problems" so to say

void *memcpy(void *, void * size_t)

[legacy]

don't use C, use something else.

exactly what? buddy. it's a rant until you do not provide practical alternatives: which language for my embedded DSP ? Have you ever tried to put Ada on 8bit MPU ? Python on Chip is a crap, and I can't use Ada or Python to write drivers or RTOSs.

(void)memcpy(dst, src, (size_t) sizeof(some_var)); // some people on our team think that this is a great solution... read below...

emm, all kind of casting is banned, buddy, and you can bypass only with the unchecked conditions method, but in this case you have to add exceptions to the code-checker and then justify by hands (it means writing documentation, and sign with your name); the code above will not pass up to 5 rules, including the comment line "//" is not allowed by MISRA,  while "/* … */" is allowed.

[legacy]

in short I hate C, but I think we have no practical alternatives, so we'd better improve the language (e.g. adding strong checks) instead of trashing it away.

about me, I have no alternatives with my Blackfin DSP, I can only use C, while if I am trying to integrate Ada/GNAT with my mips32-r2/32bit board. The low level is still written in C (because the RTOS is written in C), but … the high level (aka Applications) is going to be rewritten in Ada.

[Mechatrommer]

Isn't it time for C to grow up and do away with all these stupid forward declarations and header files ?

C is indeed a terrible language which should not be used for any serious programming. Unfortunately everyone does!

so everyone are not serious except you . btw, i dont think i'm expert or know it all enough to say the necessity of forward declarations can be thrown away in newer C+++ standard (if it ever to be made) without risking compilation integrity... i think they have thought of that its a miracle if they havent. though i'm also the one person who wish forward declarations to be thrown away. but i dont say C/C++ is a terrible language.

Way back when C started becoming popular, I thought "this is a terrible language, it will get superseded soon" but 30+ years later, there are better languages, but people don't use them.

there must be a reason for that! the problem with plethora of new languages you name it, is... they are not C/C++, thats why they dont survive, simple dont ask why, its just that, a fact.

ask them what "a[ i ] = i++;" should do, according to the C standard.

we all know what it does. if you dont like it, you may code it another way such as..

Code: [Select]

a[i] = i;
i = i + 1;
its perfectly valid

What is hilarious, is all those projects claiming to be "safety critical" but still writing in C

its hilarious that a language got blamed for the unsafetyness..

Even the book "Safety Critical Programming in C" starts off by saying, "don't use C"

thats an advice to the young players

[C]

Header files

In a great language if the header stays the same then the program using that header to link to code, data or other things does not need to change.

This allows many people to be working on different parts of the code.

A text hack preprocessor like C uses can cause all kinds of problems.
Is the code good when the C preprocessor changes a data type from 16 bits to 128 bits? What about the other direction 128 bits to 16 bits.

After all the text hacking of the preprocessor is all the different versions of code good?

C

[mikerj]

Do all these people that hate C routinely use higher level languages on PCs etc?  C is without doubt my favourite language though I would agree that it's not ideal of inexperienced programmers and that the learning curve is steeper than some higher level languages.

[retrolefty]

I have seen IDE's for C where you can simply define a function and the IDE will create the forward declaration for you. i think it is called Cide ( short for C-IDE).

 The Arduino IDE also performs creating any needed function prototypes in the user's 'sketch'.