How do you structure a large C project for embedded applications.

[diyaudio]

Hi

I've been playing around with some project concepts around a PIC24 using MPLABX, I was wondering as I come from a C# background how does one structure a large C embedded project (from a variable convention, file name layout ect.., I'm really new to C, I read the hardware datasheets, read C books and prototype simultaneously.. its working out well its just very time consuming as I don't work with embedded programming for a living, however I don't see much literature around embedded C conventions, coding style..

Can anyone comment I know this is an opinionated area, so I would just like to hear what others are doing ? 

[diyaudio]

This is a huge topic on which entire books have been written. Some key things I would suggest doing:

1. Split code into separate files. Don't be afraid to have files with one function or a few #defines in them. The more modular the better, and it makes it easier to find things in a large project. It also makes compilation faster as unchanged files don't need to be re-compiled.

2. Use a naming scheme that makes it obvious where stuff is and what scope it exists in. I use a two or three character prefix for each file, and uppercase for externally available stuff (public) and lowercase for internal stuff. For example, if I have a file called "adc.c" that has all my ADC driving code in it the function names will all start with "ADC" or "adc".

3. Come up with a consistent naming scheme. In my code I use "wake" and "sleep" in drivers. Using the ADC example I have "ADC_wake()" and "ADC_sleep()" to power the ADC on and off. Similarly I have DAC_wake() and DAC_sleep(), RF_wake() and RF_sleep() etc. in other driver files.

4. Keep scope as limited as possible. Try not to make variables and functions externally accessible if possible.

5. Watch your interrupts. Any static variable that is used by an interrupt should be named with a prefix, like "_AT" (for atomic) to remind you that it needs special handling in the main code. Providing getter/setter functions isn't a bad idea.

@mojo-chan

Thanks for the tips !

#1 Good point never thought of that. 
#2 and #3 Great, seems I have been following these conventions.
#4 Would like to hear more what others are doing or recommend for data sharing, tricks ect.. between modules. at the moment I write my global variables like this example:

Code: [Select]

extern int g_current_adc = 0;
#5 Do you mean static variable initialization in the main function?
 

[diyaudio]

learn history... of why the c, the c++ and the c#... it is the ugly truth of the very question you asked.

I'm interested in embedded C stylistics.. NOT the history.

[dannyf]

so I would just like to hear what others are doing ?

Most firms have their own coding styles and they are generally good. Here is what I do:

1) be good in C: coding a mcu isn't that different from coding a PC - just that writing to certain memory locations have particular, device-specific meanings (to control the peripherals). So you want to be proficient in C, particularly when it comes to modular programming;

2) limit your hardware access: because of 1), you want your code to be as little "embedded" as possible. That means that you want to shield your application away from the actual hardware. Any access to hardware should be done consistently and as much "single-point-of-entry' as possible. You should use the same routines to read a spi module, or send something over uart, or set a pin to output mode, for example. You shouldn't see hardware specific notations in your user code.

3) minimize writing new code: whenever you write a piece of code, think about ways of writing that code so that it can be used later by other programs, on the same mcu or different mcus. This will allow you to build up your own library and greatly speeds up code development.

Take the frequency meter code I wrote recently on a PIC. It contains a few routines:

1) configuration fuse settings: that's a file over 120KB now that sets the fuse settings for the various PICs I use. By including that in my project, I know that I will have a reasonably good starting point to work with and should I decide to change the fuse settings, the file has its own documentation and I can do so easily. Each time I work on a new PIC, I add the default fuse setting to that file and I don't have to look at the datasheet next time;
2) a gpio routine: that manages all my pin-oriented operations - setting a pin, clearing a pin, setting a pin to input or output, etc.
3) a software delay routine: standardized, frequency-conscious delays that will get me rough timing;
4) tmr0/1/2 routines: that initialize tmr0, sets up its isr, and allows a user to install his/her own handle.
5) a led display routine: that at each invocation, updates the led display.

In my application, I simply use the gpio routines to reset the pins, use the tmr0 routines to set tmr0 to trigger periodically and in the tmr0 isr, it runs the led display routine; I used th tmr1 routines to count external pulses; and tmr2 routines to gate tmr1.

The whole main() was done in 15 minutes and the whole thing was up and running in no time. and you could not see a reference to the actual hardware. So if I were to decide to port the code to AVR, or STM32F3, I can simply hook up the same routines there and hit the compile button.

Code portability is really about writing to a logic machine, improving code reliability, and speeding up development time.

Over the years, I have accumulated over 1000 such files for various functions on many mcus. That collection of routines and constant accumulation of those routines allow me to put something out much faster, and with higher reliability than I would otherwise.

[Mechatrommer]

1) be good in C: coding a mcu isn't that different from coding a PC...particularly when it comes to modular programming

didnt he said he dont like history? isnt that history?

1), you want your code to be as little "embedded" as possible. That means that you want to shield your application away from the actual hardware. Any access to hardware should be done consistently and as much "single-point-of-entry' as possible.

thats called "encapsulation" in c++ terminology. in c# "encapsulation" is taken for granted. as far as i can tell, you are teaching him history.

[diyaudio]

so I would just like to hear what others are doing ?

Most firms have their own coding styles and they are generally good. Here is what I do:

1) be good in C: coding a mcu isn't that different from coding a PC - just that writing to certain memory locations have particular, device-specific meanings (to control the peripherals). So you want to be proficient in C, particularly when it comes to modular programming;

2) limit your hardware access: because of 1), you want your code to be as little "embedded" as possible. That means that you want to shield your application away from the actual hardware. Any access to hardware should be done consistently and as much "single-point-of-entry' as possible. You should use the same routines to read a spi module, or send something over uart, or set a pin to output mode, for example. You shouldn't see hardware specific notations in your user code.

3) minimize writing new code: whenever you write a piece of code, think about ways of writing that code so that it can be used later by other programs, on the same mcu or different mcus. This will allow you to build up your own library and greatly speeds up code development.

Take the frequency meter code I wrote recently on a PIC. It contains a few routines:

1) configuration fuse settings: that's a file over 120KB now that sets the fuse settings for the various PICs I use. By including that in my project, I know that I will have a reasonably good starting point to work with and should I decide to change the fuse settings, the file has its own documentation and I can do so easily. Each time I work on a new PIC, I add the default fuse setting to that file and I don't have to look at the datasheet next time;
2) a gpio routine: that manages all my pin-oriented operations - setting a pin, clearing a pin, setting a pin to input or output, etc.
3) a software delay routine: standardized, frequency-conscious delays that will get me rough timing;
4) tmr0/1/2 routines: that initialize tmr0, sets up its isr, and allows a user to install his/her own handle.
5) a led display routine: that at each invocation, updates the led display.

In my application, I simply use the gpio routines to reset the pins, use the tmr0 routines to set tmr0 to trigger periodically and in the tmr0 isr, it runs the led display routine; I used th tmr1 routines to count external pulses; and tmr2 routines to gate tmr1.

The whole main() was done in 15 minutes and the whole thing was up and running in no time. and you could not see a reference to the actual hardware. So if I were to decide to port the code to AVR, or STM32F3, I can simply hook up the same routines there and hit the compile button.

Code portability is really about writing to a logic machine, improving code reliability, and speeding up development time.

Over the years, I have accumulated over 1000 such files for various functions on many mcus. That collection of routines and constant accumulation of those routines allow me to put something out much faster, and with higher reliability than I would otherwise.

Great advice here, it also explains one of the reason why you don't use the plib...

At the moment my project started out as a simple hello PIC24 peripheral, testing all the PIC24 peripherals now that phase is over, I'm breaking up the code into various modules, this is the motivation behind the original question. I even out of curiosity looked into the conventions used in Quake and Doom game engine, these were written in C and well documented so there is a few tips and stuff I picked up there.

[diyaudio]

1) be good in C: coding a mcu isn't that different from coding a PC...particularly when it comes to modular programming

didnt he said he dont like history? isnt that history?

1), you want your code to be as little "embedded" as possible. That means that you want to shield your application away from the actual hardware. Any access to hardware should be done consistently and as much "single-point-of-entry' as possible.

thats called "encapsulation" in c++ terminology. in c# "encapsulation" is taken for granted. as far as i can tell, you are teaching him history.

You seem to be in a very augmentative mood today.






 

[dannyf]

why you don't use the plib...

I actuall yuse a lot of libraries: I love CoIDE's libraries (and the RTE in MDK5, which is a copy of the CoIDE concept). I love PE, and I think I am one of the earlier adopter of Stellaris libraries. Libraries, particularly when the source code is available, allows you to quickly get up to speed on a complex peripheral / device.

I rarely use libraries directly in my applications: I usually put a layer on top of them.

My lack of interest in plib is more historical than intentional - though I am not a big fan of Microchip's software.

[madshaman]

Coming from C# might be a bit difficult for you as you're descending to a lower level, but good software design is language independent to a large degree.

My advice is to do it.  You can learn a lot from books or others but absolutely nothing will beat experience.  Start with an approach you feel is best and you'll make mistakes and have problems, you'll realise why you have problems and adjust your approach.

I think it's much harder for a coder to do embedded work these days because most coders are used to automatic memory management, haven't coded against bare metal before, are used to having things abstracted to the extreme and usual have RIDICULOUS amounts of system resources available both in term of processing power and memory.

People of my generation usually started programming machines with maybe 1.5K bytes of available RAM and CPU clock speed of under 2Mhz (and that CPU takes multiple cycles for even simple instructions).  Good architecture and programming practices were still required but we were forced to apply them under very tight constraints, this leads one to develop a large toolbox of tricks and techniques that quite frankly would be a waste of time to use if one were writing a Windows desktop App.

You need to start building your toolbox so my advice is to dive in and start making mistakes.

[edit: having -> haven't]

[mikeselectricstuff]

This is a huge topic on which entire books have been written. Some key things I would suggest doing:

1. Split code into separate files. Don't be afraid to have files with one function or a few #defines in them. The more modular the better, and it makes it easier to find things in a large project. It also makes compilation faster as unchanged files don't need to be re-compiled.

Splitting code into a gazillion files often makes it harder, not easier to find what you want - depends a lot on how it's split, and whether naming is logical.
Compile speed is a pretty negligible issue these days.
<flameproofpants>
My normal approach is to have all hardware-related stuff in one file, interrupts in another (ISRs and setup code), main() in another, usually quite small file, with most of the functionality in a very few other files -e.g. all filesystem stuff in 1 file, all graphics rendering in another, all UI in another etc.

And a very few global header files, included in all source files. this will contain stuff like IO port allocations, compile-time variants etc. as well as all function prototypes for all source files

But them my projects don't tend to be very big, and this approach will probably get messy at some point. 

nothing annoys me more than trivially simple example code that's been split into a dozen files each less than a page long - it makes it really hard to understand how everything relates to each other- give me a single, huge well structured file over a ton of tiny files any day. 
</flameproofpants>

As with anything there is no right answer  - the optimum way to structure a huge project to be maintained over years by many people will be very different to an 8-pin PIC job that's finished in a day.

[Q-Kernel]

I come from a C# background how does one structure a large C embedded project

Actually I think that a C# background can help. Make a diagram how you would organize your code in an object oriented language including inheritance. From there create a source file for every "object". I know that C is not OO but you can emulate the most parts of OO very simple.
Every object gets its own structure and method and properties are written like ObjRead(OBJ *p). Every "property" or "method" just includes a pointer to the structure as its first parameter.
If you need to emulate inheritance you just create the new object with the same structure as the object you want to inherit and add more data at the end.
The only disadvantage is that you have to do a lot of casting but for the rest this works great. It really helps with organizing your work, limit dependencies and makes code reuse a reality.
If you want to see how this works in practice download our open source RTIOS Q-Kernel. It is exactly build like this and it shows that this way of working does not slows you down because it is one of the best performing RTOSes.

[diyaudio]

I come from a C# background how does one structure a large C embedded project

Actually I think that a C# background can help. Make a diagram how you would organize your code in an object oriented language including inheritance. From there create a source file for every "object". I know that C is not OO but you can emulate the most parts of OO very simple.
Every object gets its own structure and method and properties are written like ObjRead(OBJ *p). Every "property" or "method" just includes a pointer to the structure as its first parameter.
If you need to emulate inheritance you just create the new object with the same structure as the object you want to inherit and add more data at the end.
The only disadvantage is that you have to do a lot of casting but for the rest this works great. It really helps with organizing your work, limit dependencies and makes code reuse a reality.
If you want to see how this works in practice download our open source RTIOS Q-Kernel. It is exactly build like this and it shows that this way of working does not slows you down because it is one of the best performing RTOSes.

Hi Q-Kernel and others, thanks for the valuable input thus far.

Nowadays in the C# world and other dynamic languages , developers are trying more and more to write behaviour against an interface, for testing and other reasonable reasons and sometimes with unreasonable justification (with thick layered abstractions resulting unreadable code ) , for a moment I thought a C# background was useless, the tips you presented make sense I will try and look into the RTIOS Q-Kernel and play around with the structure, hey... this reminds me of a convention in .NET when a function has many parameters one should use a encapsulated composite object, it offers field expansion , good abstraction and readability. 


   

 

[miceuz]

You *can* do object oriented programming in C. C itself does not support it syntactically, but nevetheless, the same principles can be applied.

For one thing, you can have typedef struct{} mytype to represent an object state and pass it to methods which are named consistently with prefixes denoting a "class" they belong to as other posters suggested.

Another thing - encapsulation - leak to the external code as little as possible, separate your code to layers. Say, I'm reading a thermocouple, I have a thermocouple library which does some math, I have a library which converts ADC values to microvolts and I have ADC library which does ADC hardware access. If I want to use the same code on a different MCU, I just rewrite an ADC level, I have the interfaces fixed in my .h file.

Interrupts are a bit tricky part as those are fairly hardware specific and introduce "nonlinearities" to the code in general. There is a school which says to keep all your interrupts in the main.c, there is a school which says you should hide them in libraries. If the project is small, I try to contain it in a single file. Next thing, I try to contain interrupt routines in main.c and refactor as much of specific code to libraries, calling them from interrupts (if it's not time-critical).

Since embedded is a resource-limited environment it's worth considering macros and inline functions where possible for smaller code size and faster runtimes.

I'm a (fairly good) programmer by trade and I was wondering about the same recently - I know really well how to build big maintainable systems, but maybe I'm doing something wrong when programming microcontrollers, maybe I don't use the right patterns? Then I've checked the book by Elecia White who was on the Amp Hour, the book is called “Making Embedded Systems”, pdf can be easily hunted down online - it's a pretty shallow book with a couple of good embedded - oriented bits.

Basically it boils down to simply doing the programming the right way. The "rightness" of the way depends on your taste/experience/religion, I for one, think that Object-oriented programming paradigm is the best (but far from perfect, not to start religious war here!) thing we have.

BUT! mcus have limited resources and you can't go nuts with all of the OO paradigm, you always have to keep in mind "how resource intense is this?"

[Jeroen3]

Program in layers with modules.

Your chip manufacturer will have peripheral drivers. These are called spi.c and spi.h which make a module.
Now, instead of having a 1000 lines long main.c, you put your application code depending on the SPI module aside in a seperate module on top of the SPI driver.
For example an mcp3008.c and .h file with all information required to use that SPI-ADC chip with the SPI module for your mcu.
Functions that leave the mcp3008 module are "init, read, deinit".
This way SPI and MCP3008 control code are separated and abstracted. Ideally you can swap SPI.c for a different architecture... But at least you have the software model supporting that action without full rewrite.

Now comes the tricky part, some people want interrupts in separate files... Some people don't. The chip doesn't care, the compiler doesn't care, you(r team) care(s). You choose.
It might have benefits keeping all "unpredictable" parts of software together. Then claling the "module_isr_handler()" in your module.

There is also the overkill version... Creating a gazillions files as mike said and using defines to rename and link functions. I hate that, your code becomes unreadable because the flow jumps from here to there without easy way to follow. Unless your IDE is that good.
Example of good (and bad) can be found online, like mbed or ChibiOS. Or the STM32F example codes, having all peripheral drivers separate with an application driver on top called by Main.
Chibios is bad and good. It attempts to have module HAL and Kernel. The kernel is fine in my opinion, but the HAL is one big mess with defined linked functions.

[David_AVD]

Splitting code into a gazillion files often makes it harder, not easier to find what you want - depends a lot on how it's split, and whether naming is logical.

+1 to that.  I've seen projects (not just in C) where there seemed to be more files than functions, or layer upon layer of abstraction that made it impossible to see what was going on.   

[dannyf]

Unless your IDE is that good.

Most modern IDEs allow "go to definition". SI is particularly good in that.

[dannyf]

A couple controversial topics:

bottom-up vs. top down coding.
I generally prefer top down coding: it provides you with an opportunity to see the big picture and how all the pieces will work together. Plus, it works likely with the modular programming style that I use.

IDE:
I do the bulk of my coding on three IDEs: Keil uvision3/C51, Turbo C, and CodeBlocks. uVision3 for its small size, and utilitarian UI. Turbo C because I have memorized the keystrokes - they are 2nd nature to me. CodeBlocks for its utilitarian UI and gcc.

Once the code is mostly done, I move to the target IDE and hook in the peripheral modules / templates and hit compile. Usually with minimum work, the code compiles on the target and I am ready for debugging. I found that to be quite efficient for me.

[miguelvp]

Just treat code as you build systems using abstractions. The more you can abstract components the easier it's to get your head around them.
If you break the abstraction and create inter-dependencies, then you end up with hard to maintain systems.

So make your discreet components modular and as independent as you can.

[amyk]

Splitting code into a gazillion files often makes it harder, not easier to find what you want - depends a lot on how it's split, and whether naming is logical.

+1 to that.  I've seen projects (not just in C) where there seemed to be more files than functions, or layer upon layer of abstraction that made it impossible to see what was going on.   

Seconding this. Keep your code simple as a whole. That doesn't mean breaking a complex design into pieces so small that each piece is trivial, because then all you've done is spread the complexity out over a larger area, not reduced it, and it makes understanding what the whole thing does much harder as you now have to chase the flow of control as it jumps around everywhere. It really means to simplify your algorithms and structure so that the whole application becomes easier to understand. You can/should break things into pieces when they get large enough, but don't overdo it.

Someone I know went into embedded development on really small systems just to get away from the "enterprise" world of grossly overabstracted software (mostly written in Java.) "Premature abstraction is the root of all inefficiency."

[hans]

I've doubts about splitting into multiple files.

I've experimented lately with writing a MRF49XA device driver into multiple files. Ultimately the structure became:
- mrf49xa.h contained prototypes, variables plus device config, frequency band, TX power, max packet size etc., plus hardware configuration (SPI bus, chip select)
- mrf49xa_defs.h contained definitions for the chip

I splitted the code up into 4 files:
- mrf49xa.c contains init, reset, modeTx/Rx
- mrf49xa_cmd.c contains functions for sending commands over SPI, reading TX/RX data, reading status
- mrf49xa_data.c contains functions for handling data a byte level, as this is how the chip works. To the application it provides a tick & ISR handler.
- mrf49xa_packet.c contains functions for handling data at packet level. At this point I've got simple functions like TxPacket(), RxPacket, RxAvailable, TxAc() etc

The total code was about 14.5kB. Could easily fit into 1 file. mrf49xa_cmd.c is only 50 lines long. I still liked how it was less scrolling around, instead of cramming everything into 1 file where in 1 part you're dealing with low level SPI transfers, and at another part you're pulling apart packet structs of the application to copy into buffers..
I guess it depends per person...


What I do like to do , is organize variables of certain "modules" using structs. This chip driver had only 2 exported variables, for HW status en SW status. The HW status is a bitmapping of the HW register, the SW status is really simple:

Code: [Select]

typedef struct rfTrcvStatus_s
{
    rfTrcvState_t state;

    rfTrcvPacket_t txPacket;
    rfTrcvPacket_t rxPacket[2];

    rfTrcvPacket_t* hwRx;
    uint8_t hwByte;

    uint8_t src; // my node ID

} rfTrcvStatus_t;
extern rfTrcvStatus_t rfTrcvStatus;
This way in the large scheme ("statics" or "variables") window I don't have to scroll endlessly through long lists of variables (that are often also slow to load via debugger hardware), but expand on the things I want to see.
I could also *in theory* hook up 2 chips to 1 MCU, by making it an struct orientated driver (it's now statically allocated). But how often do you need that?

[dannyf]

- mrf49xa.h contained prototypes, variables ...[/quote]

It probably isn't terribly wise to put variables in a .h file. typedef, yes. actual variables? no.

I splitted the code up into 4 files:

I would slice it different:

1) I would use a set of com routines (usart and/or spi) or even pin outs as the basic building block to communicate with the chiph;
2) I would put a set of routines to read from or write data to the chip, using routines in 1)
3) If you have a protocol on top of that, I would write a set of routines that communicate with the chip using routines in 2).
4) For isrs, I would write a set of routines that allow the isr to be triggered and once triggered, a user installed isr is called.

1) and 4) are non-device specific; and 2/3 are device specific. Depending on the way you communicate with the module, maybe the user sees just 3) or 2).

This approach allows the code to be ported to a different device - you simply sway out routines used in 1) for those on your target device and you are done.

[zapta]

diyaudio, regardless of general programming practices that you probably know from C# (modularity, minimizing scopes, naming conventions, etc), you will need to parallelism in some way. In C# you probably use permeative multi threading, in the embedded environment, unless you are using a multi threading package, your choices are different.  One level of parallelism is between the ISRs and the main thread. In the main thread itself you can do parallelism by parallel state machines, polling, task queues and so on. You want to avoid a long delay() that will block you only main thread while its need to do other stuff. Now that you design your parallelism, pay attention to conflict and race conditions. For example, writing a 16 bit int on a 8 bit CPU may be non atomic, or even a simple read of a 16 bit i/o register can be corrupted by a ISR routine (e.g. the AVR and it's temp register).  This kind of problems can be difficult to debug so prefer simple stuff that you understand and use as straight forward as possible code (avoid the sin of premature optimizations).

As somebody said above, it requires some experience so it will get easier over time.

[MacAttak]

Split things up in a way that groups them by feature area for the sake of having a clean revision history when someone looks at source control a year from now in order to research a bug's lineage. This also helps with avoiding devs trampling over each other's work. This does not mean throwing all code into a single file, nor does it mean going extreme in the other direction and having one file per function.

I'm presuming that you have source control in place and multiple dev team members - reasonable presumptions I think for anything described as "large".

[miceuz]

Another aspect of modularization worth mentioning is Unit testing. When modularized code can be easily tested by external test code in isolated environment. In my opinion this is one of the biggest benefits of modularization.

Just like you test how hardware responds to external signals - you set up a test bench for your code, apply external "disturbancies" to it and wach it's behaviour. Nothing beats the warm feeling that your particular code is proven to work reliably.

As a young programmer I was always thinking it's boring and stops me from going there and achieving the results, but now, when I grew older and especially after I got some experience in electronics, I just love doing that, both in production code and in small hobby projects.

[miguelvp]

+1 on unit testing at least.