Wanted - Source Code Analysis software

[NW27]

Good Morning All,
I have inherited quite a few legacy embedded products that were developed (>12 years ago) by another company and then purchased by my current employer.
It is now my job to maintain these products both from a firmware perspective and also from a hardware perspective.
The majority of the firmware that I'm looking at uses the IAR System compiler and uses the MSP430 microprocessor.
As an example of my problem, the current main.c file I'm currently looking at is 6168 lines in length and there are about 40 files (*.c,*.h and *.asm(hand written assembler)) that make up the project.
I'm trying to get my head around this code but when you can only see 30 odd lines of code on a screen at a time, it is hard to get the big picture of code flow etc.

I've been looking at source code analysis software to try and get a big picture of how all of this hangs together. As an example: https://www.imagix.com/products/source-code-analysis.html
However this software is in the $1000..$6000 area and there is no way I can get that approved.

Can anybody suggest alternatives?

Thanks,
Neil

[legacy]

ummm, it looks you need a good IDE and Understand C by SCI-Tools
(contact me in private for a demo)

[NivagSwerdna]

there are about 40 files

That's really not a very complex project.
You should put the whole thing into Sublime and then navigate symbols.

[andersm]

You can use eg. Doxygen to generate navigable call graphs.

[ejeffrey]

Good Morning All,
I have inherited quite a few legacy embedded products that were developed (>12 years ago) by another company and then purchased by my current employer.
It is now my job to maintain these products both from a firmware perspective and also from a hardware perspective.

The majority of the firmware that I'm looking at uses the IAR System compiler and uses the MSP430 microprocessor.
As an example of my problem, the current main.c file I'm currently looking at is 6168 lines in length and there are about 40 files (*.c,*.h and *.asm(hand written assembler)) that make up the project.
I'm trying to get my head around this code but when you can only see 30 odd lines of code on a screen at a time, it is hard to get the big picture of code flow etc.

That doesn't really sound that complex.  Do you have two monitors?  If so you should be able to comfortably look at (for example) 2 50 line views on one monitor, and a browser for documentation lookup and a document for taking notes on the other monitor.  I can understand your boss not wanting to spring multiple thousands of dollars for some software to parse some legacy code, but a second monitor should be an easy sell if you don't already have it.

I've been looking at source code analysis software to try and get a big picture of how all of this hangs together. As an example: https://www.imagix.com/products/source-code-analysis.html
However this software is in the $1000..$6000 area and there is no way I can get that approved.

There are several other good suggestions above, and I still think the main thing you need is an IDE that lets you quickly follow references.  But if you do this a lot, get the 15 day trial license, and get as much work out of it in that time period as possible.  Then divide the license cost by your hourly wage + estimated payroll taxes and benefits, and see if you can honestly make the case to your boss financially.  If maintaining these pieces of legacy software is really not a big part of your job and you don't expect to be spending a reasonable fraction of your time on it, then that license cost may legitimately not be worth it.  If you expect to be spending months of full time work on this (or your boss expects that), then a few thousand dollars for a software package that you have demonstrated makes it easier should be a reasonable sell.

[hans]

I think you've found the reason for 2 types of common behaviours:

It takes time investment to become familiar with legacy and code written by other people at a different point in time, which annoys people that want to design instead of supporting legacy products.
The adaption of code, or the active refusal thereof, is a common reason to rewrite firmware/software when a new guy takes over (or joins)...


From the sound of it, 6k lines in a single main file is not the best code organization. Refactoring code without knowing how it works and having good unit tests is hard, although splitting a file into multiple C files shouldn't cause that much pain (unless the organisation is really bad).

I think the only thing you can do with a situation like this is invest a lot of time in exploring how the code works. A tool may be able to create a pretty call graph or something, but it's usually of little use because you can generate thousands of those kind of graphs. A project that is a few 10k lines of code isn't that big, to be honest. But the key is organizing the code well.

The recommendations for a good code editor, that can properly navigate symbols, is IMO much more valuable.

[engineer_in_shorts]

Use Doxygen and create a graphical call graph (dotvis).

IAR EW430 is one of the better MSP430 compilers, even the older (2009) IDE has features to get you around the code base.

[nfmax]

A first-rate IDE including static code analysis may be the best choice. My personal recommendation is CLion from JetBrains - you can get a 30 day free trial.

https://www.jetbrains.com/clion/?fromMenu

[tggzzz]

Can anybody suggest alternatives?

You have an unpleasant job ahead of you.

Do you have a good and accurate specification of what they ought to do?
Do you have a good and accurate specification of what they actually do?
Do you have a good and accurate specification of what they now need to do?

Without those you are in danger of having a new job title: project scapegoat.

If you can't get authorisation for the necessary tools, then they aren't serious about the work. If  that's the case you should find another job.

[NW27]

Thanks All,

Some good feedback and links.

Now time to start ripping these projects apart.

Neil.

[rhb]

For most of my career I was handed 100K+ packages. often not under version control even.  But it was not embedded, so a bit easier to work on.  Though there was the 500K lines which had NO comments other than the author's name.  Those 10-12 programs were lots of fun.

Get a 2nd  monitor in portrait mode.

Use nm and awk to build caller-callee tables.  Focus on understanding the problem the code is supposed to solve.

Put it all under version control if it isn't already.  Then start reading and adding comments.

[westfw]

I'm trying to get my head around this code but when you can only see 30 odd lines of code on a screen at a time, it is hard to get the big picture of code flow etc.

I've been looking at source code analysis software to try and get a big picture of how all of this hangs together.

First, you need a bigger screen...  (I had a blind co-worker once, who used a braille terminal that had a (I think) one line output.  I ... can't even imagine!)

Do you have the IAR software?  (Um... "Embedded Workbench", I guess.)  IAR is pretty well respected (if expensive), and I'd expect their IDE/compiler to include some pretty good tools.  Learning to use them is probably a better career move than finding some obscure tool somewhere (even if the obscure tool does a better job.  ("Used IAR Embedded Workbench to analyze legacy software projects" looks better on a resume that "I've used OSSCAnalyze", at least to the keyword scanners...)
I have some "generic" tools that I like - they're more "code browsing tools" than actual analysis:

• tags - usable with EMACS and VIM (probably among others); gives you the "goto defintion" functionality from most IDEs, in perhaps a preferable editing envirionment.
• ID Utils ( https://www.gnu.org/software/idutils/ )similar to tags, I guess, but with a CLI.  Builds a database and provides quick tools for "show me all the places that this name is used", "find where this is defined", "give me the list of files that reference this."  Sort of like a CREF utility for one thing at a time.
• the standard elf tools like "nm" will show you what actually ends up in a binary, and which things are biggest.
• Most compilers will output some sort of cross-reference (CREF) listing (though I haven't often found CREF listings to be very useful; they tend to get very cluttered with relatively useless information - "putchar() calls putc()"

[Mechatrommer]

why dont you build your own software? i built my own "Library Manager" back in 2003. all it does is search for "#include" in each line and do some simple string stripping to get its dependency filename (header files), and then draw some graph. not even support inline comment which is pathetic (2nd picture), see "parser" code below to get the idea how pathetic it is, but in 5 or 10 minutes you can snap a code like that, 1 to 2 hours you can make something a lot better (support inline comment and token based parser) within 1 to 2 weeks you should be able to make something sellable. i think this is a good OSS project for anyone in real need... i wish instead of file level dependency/graph analysis like my manager does, i can also have down to functions/classes/structs level of dependency graph (like the above suggested doxygen and other paid sw) and some multi/tabbed view windows to view them all in one screen in some logical manner with just a single click on each graph element (when we click a function in graph, a text editor will pop up to show the function body/definition/code alone, no other adjacent functions/global variables declaration mess etc, just like the venerable VB6). logical tree view and all... well if just i can make a copy of myself, i will command him to spend entire life on this...

Code: [Select]

Private Sub CppTranslator(lst() As String)
  Dim str As String
  Dim i As Integer

  Do Until EOF(1)
    Line Input #1, str
    str = LCase(Trim(str))
    If Left(str, 8) = "#include" Then

      'dependency found, add to include list
      str = Trim(Right(str, Len(str) - 8))
      str = Mid(str, 2, Len(str) - 2)

      'adding include to file
      AddToList lst(), str
    End If
  Loop
End Sub


[Rasz]

30 odd lines of code on a screen at a time

start with 30" 4K monitor

However this software is in the $1000..$6000 area and there is no way I can get that approved.

you are from AU, and cant get 1K approved? change jobs

[legacy]

usable with EMACS and VIM (probably among others)

VIM, EMACS = poop, so holy shit, why do you suggest the shittest software ever made? it's ok for a hobby (if you like these ... sports), but ... jesus, it's like suggesting to someone asking how to sew a button that first he/she has to thread a dead big camel into the eye of the needle 

[legacy]

why dont you build your own software? i built my own "Library Manager" back in 2003. all it does is search for "#include" in each line and do some simple string stripping to get its dependency filename (header files), and then draw some graph. not even support inline comment which is pathetic (2nd picture), see "parser" code below to get the idea how pathetic it is, but in 5 or 10 minutes you can snap a code like that, 1 to 2 hours you can make something a lot better (support inline comment and token based parser) within 1 to 2 weeks you should be able to make something sellable

ummm, I did such a tool by myself 2-3 years ago, I remember a topic here on EEVBlog discussing the initial idea, and it took 2 months. I wanted to integrate media-gfx/graphviz (Open Source Graph Visualization Software)

This software is now used on HPPA/Linux and POWER9/Linux workstations, where you can't use x86 software, but it can't compete with Understand C or Imagix 4D.

Besides, there are customers that are not prone to let you develop your own tool during the working time. You have to develop it at home when you came back, or during weekends, if, and only if, your wife/girlfriend is prone not bitching about it, WHICH is THE real problem 

[knapik]

VIM, EMACS = poop, so holy shit, why do you suggest the shittest software ever made? it's ok for a hobby (if you like these ... sports), but ... jesus, it's like suggesting to someone asking how to sew a button that first he/she has to thread a dead big camel into the eye of the needle 

I would have to argue that Emacs is actually one of the best pieces of software written.

[bd139]

Emacs user at work

[onesixright]

VIM, EMACS = poop, so holy shit, why do you suggest the shittest software ever made? it's ok for a hobby (if you like these ... sports), but ... jesus, it's like suggesting to someone asking how to sew a button that first he/she has to thread a dead big camel into the eye of the needle 

I would have to argue that Emacs is actually one of the best pieces of software written.

VIM is awesome!  Poop. Now f-way!


Sent from my X using Tapatalk

[Mechatrommer]

Besides, there are customers that are not prone to let you develop your own tool during the working time.

well. since the boss cant afford to pay proper tools, you should can do it at work. when boss ask, you tell him you make a tool to do your job better, otherwise you cant do your job or just tapping feet all day long and bite pencil while reading the code bit by bit slowly. so its relevant to your "work" at work, efficiency wise at least in the long run. further why i suggested building our own tool, so we can do to our heart content, the way we want it to be. i tried the imagix 4d and sci-tool mentioned above they seem to not do what i want. imagix 4d just create some nice graph, when clicking on it it just direct me to the function location in a file, its just a nice shortcut, nothing more. the files are just "tabbed" like the other normal (free) IDE, we have to click the tab to view the next file, but the earlier file view will be gone! at most we can split into 2 view, crap. i cant test on real project since its trial version, but even with their sample project i can say it sucks. sci-tool is worst, opening a *.c file did nothing, just a normal text editor, not even map/collect the dependency files list, my "Library Manager" is even better. last time i tried Doxygen, its just nice for code documentation. so i dont have anything suitable as in my mind. so far the best method for me is still manual method, ie reading and understanding each code file, make a function list in the correct order "manually" and try to remember where each function resides in a file and make a logical mental map of the relation of all of them, in short word is... you got to read and understand all the source files anyway

[tggzzz]

Besides, there are customers that are not prone to let you develop your own tool during the working time.

well. since the boss cant afford to pay proper tools, you should can do it at work. when boss ask, you tell him you make a tool to do your job better, otherwise you cant do your job or just tapping feet all day long and bite pencil while reading the code bit by bit slowly. so its relevant to your "work" at work, efficiency wise at least in the long run. further why i suggested building our own tool, so we can do to our heart content, the way we want it to be. i tried the imagix 4d and sci-tool mentioned above they seem to not do what i want. imagix 4d just create some nice graph, when clicking on it it just direct me to the function location in a file, its just a nice shortcut, nothing more. the files are just "tabbed" like the other normal (free) IDE, we have to click the tab to view the next file, but the earlier file view will be gone! at most we can split into 2 view, crap. i cant test on real project since its trial version, but even with their sample project i can say it sucks. sci-tool is worst, opening a *.c file did nothing, just a normal text editor, not even map/collect the dependency files list, my "Library Manager" is even better. last time i tried Doxygen, its just nice for code documentation. so i dont have anything suitable as in my mind. so far the best method for me is still manual method, ie reading and understanding each code file, make a function list in the correct order "manually" and try to remember where each function resides in a file and make a logical mental map of the relation of all of them, in short word is... you got to read and understand all the source files anyway

The first thing you do with an impenetrable codebase is apply the prettyprinter of your choice. Without that it is impossible to read.

[AlexXL]

I would assume that topic-started have to deal with many different IDEs and compiler versions.
Why not to setup Eclipse IDE and get some sort of code-coverage + Doxygen, then you are good to go.

basically, as already stated - you would need to spend decent amount of time to get into the thing. Otherwise u can't do it.

[Mechatrommer]

not sure what is prettyprinter, maybe its a code beautifier? https://www.techopedia.com/definition/28713/pretty-printer-sap but its remind me that i can understand the codebase by printing them all on paper, with the right font and size to fit as much code in a page, when all printed (or the most important files only to save papers), each file's pages stapled nicely, i just start for each file, preferably the most relevant one ie the main() and start highlighting important area with nice yellow highlighter, such as function names, class and struct names etc. draw separating lines between each adjacent functions. this way i can make logical mental map much faster, but not 1 hour faster, it will still need several days faster.