Compiler: Atmel Studio v. Arduino

[jgalak]

The Arduino IDE appears to use the avr-gcc compiler.  Does Atmel Studio use something else?  Is it better?

I'm asking because I tried including a file into an Arduino project that contained a large number of variables (not declared volatile) which were initialized and assigned values, but never used.  I'd have assumed that a good optimizing compiler would simply ignore them and not actually include them in the compiled program.

But Arduino ide  gave me an error that the compiled program would exceed system ram.  This makes sense if it actually created all the variables, but that seems inefficient.

[ataradov]

Both use AVR-GCC.

And GCC is very good at optimizing stuff. So either Arduino did not pass the optimization flag to the compiler, or you did something wrong and  compiler could not tell if variables are actually used.

For example, if they are global (non-static) variables, then compiler can't optimize them, since they may be used in other files. And the only way to tell is at the time of linking.

GCC has Link Time Optimization (LTO), which would eliminate such variables, but LTO is disabled by default in most cases (I assume it is disabled for Arduino as well), since it can lead to some interesting effects, if programmer is not careful. But on the flip side, LTO saves up to 25% of flash size on large projects.

[ajb]

Have you tried the same file in an Atmel Studio project and gotten different results?

The whole basis of Arduino is a whole bunch of abstraction to hide the intricacies of the hardware, and this necessarily consumes some amount of flash and RAM.  You can examine the map file to see where all of your memory has gone.

[jgalak]

Have you tried the same file in an Atmel Studio project and gotten different results?

I've not tried Atmel Studio yet.

[ataradov]

You won't see a difference. Declare your variables static, and they will be optimized out.

[westfw]

I tried including a file into an Arduino project that contained a large number of variables (not declared volatile) which were initialized and assigned values, but never used.

What exactly do you mean by "assigned values"?

The Arduino compile (recent version) does include "link time optimization", and even before that it used "-fdata-sections" and typical optimization would remove unused data.

However, "unused data" seems to be a bit ... vague.  Modifying the Blink example:

Code: [Select]

static int ledtable[256] = {1,2,3};
void setup() {
  // initialize digital pin LED_BUILTIN as an output.
  pinMode(LED_BUILTIN, OUTPUT);
}

void loop() {
  ledtable[20] = 123;
  digitalWrite(LED_BUILTIN, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(1000);                       // wait for a second
  digitalWrite(LED_BUILTIN, LOW);    // turn the LED off by making the voltage LOW
  delay(1000);                       // wait for a second
}
Will indeed include the storage for ledtable[], even though it isn't used.  (It won't if you leave out the assignment in loop.)
I guess that's because C can't be sure that the memory referenced via ledtable isn't also referenced by other things in the called functions, even though the symbol isn't used there...

[jgalak]

That's pretty much what I meant.  The file defines a huge array that contains configuration info for a device that the Arduino will control over I2C.  The array defines all the registers, etc.  Just including the header file caused the error, even though the array is never referenced.

Not a huge deal, I can manually set the few registers I actually need, just didn't realize it wouldn't be optimized out.

[ataradov]

Can you show the file? What you are describing is not normal. If you just define a variable and never ever use it in a code, then it should be eliminated.

[jgalak]

It's a pretty large file

I'm working with the Si5351B clock chip, and the file is a register map header file generated by their clock builder program.  The bulk of it is an "array of structs" which maps all the registers and their settings.  The rest is a bunch of #DEFINEs that act as indexes into the array.  The array is declared as "const".  Not sure if that matters

I'm guessing since the array is actually populated, it forces the compiler to not ignore it.

[Bruce Abbott]

The array is declared as "const".  Not sure if that matters

'const' means the array cannot be written to, but it will still be created in RAM. AVRs don't have much RAM so it is easy to run out.

However the data to populate the array must be stored in ROM, then is copied to RAM at startup. If only you could convince the compiler to keep the array in ROM you could avoid using up precious RAM with a duplicate of the constant data. Turns out you can!

PROGMEM

Store data in flash (program) memory instead of SRAM... The PROGMEM keyword is a variable modifier, it should be used only with the datatypes defined in pgmspace.h. It tells the compiler "put this information into flash memory", instead of into SRAM, where it would normally go.

PROGMEM is part of the pgmspace.h library. It is included automatically in modern versions of the IDE, however if you are using an IDE version below 1.0 (2011), you’ll first need to include the library at the top your sketch, like this:

#include <avr/pgmspace.h>

Syntax

const dataType variableName[] PROGMEM = {data0, data1, data3…?};

The rest is a bunch of #DEFINEs that act as indexes into the array.

Exactly how do you access the array? (show us your code). This may be important when using PROGMEM, because pointers normally access RAM not ROM. You may need to use pgm_read_word() to read the array data.

   

[snarkysparky]

""For example, if they are global (non-static) variables, then compiler can't optimize them, since they may be used in other files. And the only way to tell is at the time of linking.""

Just curious, how could the compiler optimize them better if they were local.  Assuming a single threaded application.

[ataradov]

Just curious, how could the compiler optimize them better if they were local.  Assuming a single threaded application.

If they are local and not used, they will be thrown away at the time of compilation. There is no way to optimize partially used array.

On the other hand, I really don't understand the issue. Typical device header files do not need arrays of stuff. So it is still would be nice to actually see the file, even if it is big.

[westfw]

Or, if this is one of the published Arduino libraries, a link to the code would be sufficient...

[JPortici]

On the other hand, I really don't understand the issue. Typical device header files do not need arrays of stuff. So it is still would be nice to actually see the file, even if it is big.

have you ever used silicon labs stuff ?
all their RF transceivers i've used work like this (e.g. Si4455)
at POR you first have to send a long binary blob which apparently contains updates or fixes (who knows!), then you can send another binary blob which contains the configuration.

probably they do the same for other chips too..

there are two things that pissed me off about this initialization procedure
first, in their code this is hidden in a layer of two or three macros which create a 2d array based on the plain text that you have to rewrite in some other way (and they use SDCC for the examples, a PITA to port to plain C)
second, the binary blobs are generated by their part configurator software (WDS in my case) but at least you can maybe kinda come up with the configuration blob by checking out the generated code, three different datasheets (including one from a sligthly different part but with much more details) and a html file which has all the configuration bits and the set up timings explained

/rant over, now i can start my day 

[jgalak]

Progmem sounds like the right solution, maybe.  Here's some more details (not including entire file, since it's three thousand lines of code):

First, here's the struct definition:

Code: [Select]

typedef struct
{
CHAR*  name; /* Setting/bitfield name                                           */
UINT8  read_only; /* 1 for read only setting/regs or 0 for read/write                */
UINT8  self_clearing; /* 1 for self-clearing setting/registers or 0 otherwise            */
UINT8  nvm_type; /* See above                                                       */
UINT8  bit_length; /* Number of bits in setting                                       */
UINT8  start_bit; /* Least significant bit of the setting                            */
UINT8  reg_length; /* Number of registers that the setting is stored in               */
UINT16 addr[SI5351B_REVB_MAX_NUM_REGS]; /* Addresses the setting is contained in    */
UINT8  mask[SI5351B_REVB_MAX_NUM_REGS]; /* Bitmask for each register containing the setting    */
} si5351b_revb_regmap_t;
Then, here's a piece of the array:

Code: [Select]

si5351b_revb_regmap_t const si5351b_revb_settings[SI5351B_REVB_NUM_SETTINGS] =
{

/* REVID */
{
"REVID",
1, /* 1 = IS Read Only */
0, /* 0 = NOT Self Clearing */
SLAB_NVMT_NO, /* Not stored in NVM */
2, /* 2 bits in this setting */
0, /* setting starts at b0 in first register */
1, /* contained in 1 registers(s) */
{
0x0000, /* Register address 0 b7:0 */

},
{
0x03, /* Register mask 0 */

}
},
.
.
.
/* CLK7_DIS_STATE */
{
"CLK7_DIS_STATE",
0, /* 0 = NOT Read Only */
0, /* 0 = NOT Self Clearing */
SLAB_NVMT_NO, /* Not stored in NVM */
2, /* 2 bits in this setting */
6, /* setting starts at b6 in first register */
1, /* contained in 1 registers(s) */
{
0x0019, /* Register address 0 b7:0 */

},
{
0xC0, /* Register mask 0 */

}
},
.
.
.
}

Finally, here are few of the #DEFINEs:

Code: [Select]

#define             SI5351B_REVB_REVID 0
.
.
.
#define    SI5351B_REVB_CLK7_DIS_STATE 91

So if I wanted, for example, to get the address of the register which disables CLK7, I would do something like:

Code: [Select]

temp = si5351b_revb_settings[SI5351B_REVB_CLK7_DIS_STATE].addr;

I don't know if progmem can handle this, but it sounds like a nice fix.

Again, I don't _need_ this to work - of the 161 register elements defined in this file, I need, like, 3.  I can program those manually.  I was just surprised that while I included this header file in my project, but never actually referenced the array in any way, the compiler didn't just ignore it.

Edit to add:  Can I tell you how annoying it is that they stuck "revb" everywhere?  Probably to make it explictly not backward compatible, but still really annoying to have in every freaking variable name.

[ataradov]

Oh, wow. I can imagine 1000 better ways of doing this. And very few ways to make it worse.

Yes, it looks like this is another case where you need to pass on a vendor supplied stuff if you want to keep your sanity.

[jgalak]

Oh, wow. I can imagine 1000 better ways of doing this. And very few ways to make it worse.

Yes, it looks like this is another case where you need to pass on a vendor supplied stuff if you want to keep your sanity.

Indeed.

To be fair, this is totally an optional bit of code.  The more useful one is an initialization file that their software generates for you, to preload all of the registers to what you want it to be.  That one is far more useful, and compiles just fine (haven't tested in hardware yet).

Their clockbuilder software lets you define a bunch of initialization stuff and generates code for that.  It also generates the register map, in several formats - html, csv, and this C header file.  I figured, hey, might as well include it, might be handy.  Wrong.  I'm too used to programming PCs rather than uCs, where hardware constraints are rather different....