Array initialization fails

[ricko_uk]

UPDATE: I made this post with list of solutions that solved this and other issues. I hope it helps others too.
https://www.eevblog.com/forum/projects/solved-mplab-1)-slowno-variables-updates-2)-icd4-not-connecting-3)-others/

Hi,
the code below does not initialise the array (all the values are just initialised to 0x00) in the globally defined structure. Any idea why?

Code: [Select]

    typedef struct {
    uint8_t arrImage_data[2][_IMAGE_ARRAY_SIZE__IN_BYTES];
    bool bIndxImageBeingPrinted;
    uint32_t ui32IndxOfFirstByteOfCurrentlyPrintingLine;
} _image_data_struct_t; //image_data_struct;   

 _image_data_struct_t imageTest =
                            {
                                .arrImage_data =
                                    {{
                                    #include"TestImage_1.txt"
                                        },{
                                    #include"TestImage_1.txt"
                                   }},
                                .bIndxImageBeingPrinted = 0,
                                .ui32IndxOfFirstByteOfCurrentlyPrintingLine = 0
                            };

and the file TestImage_1.txt in the #include"TestImage_1.txt" has just a long list of comma-separated byte values exactly like this and nothing more:

0x11, 0x22, 0x33, 0x00, 0x00, 0x00,
0xAA, 0x55, 0x13, 0xFF, 0xFF, 0xFF,
0x11, 0x22, 0x33, 0x00, 0x00, 0x00

Thank you

[ataradov]

Why create multiple topics for essentially the same question?

What compiler? What platform? How it was tested?

Also are any warning messages when compiling? Enable the highest warning level possible.

[ricko_uk]

Thank you Alex

I use XC32 version 2.5 (I have not updated to the very recently released V3 because in the release notes it says it has some backwards compatibility issues and when I tried it the code does not compile).
The environment is MPLABX IDE (latest version).
I used the built-in simulator.
There are no compile errors.
I tried changing the warning level and even googled it but cannot find anywhere the option. I do get full error for things as simple as variables not used (instead of just a warning) so it might be already at the highest warning level, however it is set. (Just a speculation!)

Ref posting a new topic, I agree. I created a different topic only because in the past, for a similar situation, I was asked to make a new post because "although closely related it was essential a different issue/question" and the other issue I posted previously was more about not compiling. So I thought it might have been better to post a new one also because the other was solved.

Thank you

[ataradov]

I have no idea about XC32 and simulator, and have no interest in even trying to figure this out. May be someone else will.

[retiredfeline]

Just a couple of things that come to mind:

See if you can turn on the assembly language generation option to see if the initialisations made it there.

In many Harvard arch MCUs, initialised data is put into the RO data segment or the code segment, so you may have to prefix the initialised variable declaration with __code__ or something similar. I don't know about XC32.

Sorry, that's all I can suggest. Good luck.

[DavidAlfa]

#include inside a struct declaration? What the hell?
I'm not a master in C programming, but to me it looks wrong?
Why don't you do it in the correct way, declaring the arrays properly?

At least use a define:

Code: [Select]

#define IMAGE1_DATA \
0x35, 9x44, 0x11, 0x42, 0x76, \
111, 104, 84, 10, 84, 92, \
'l', 'i', 'n', 'e', '3', \
You need add "\" to make it multiline \
This is the last line (No "\")
With that, you could simply type "IMAGE1_DATA" in the initialization array.
But it's not a tidy or proper way to do that...

[ataradov]

It is fine to include the data like this. Some thing else is wrong. I suspect just misinterpretation of the results or some simulator/XC32 issue.

[DavidAlfa]

I thought it was only a C++ thing:
https://stackoverflow.com/questions/410980/include-a-text-file-in-a-c-program-as-a-char

[Nominal Animal]

the code below does not initialise the array (all the values are just initialised to 0x00) in the globally defined structure. Any idea why?

Since the code most certainly does initialize the array in a hosted environment (I checked), the reason is most likely due to the environment you are using.

It may be, that XC32 2.5 does not initialize data structures defined in RAM properly.  (In e.g. Arduino, these are initialized by storing a copy of the initialized object in Flash, and copying initialized objects to RAM during startup.)  Lessee, okay, an XC32 manual I found on the net has a section 12.3.4 about this.  It says the compiler should do this properly; it actually uses initialization records, for this.  So the underlying problem is unclear.

However, if the image data is not modified during run time, you really should put that part of the data in Flash/ROM.  According to that random XC32 manual, const suffices for this.

So, what I would recommend for you, is to use something like the following instead:

Code: [Select]

typedef  struct {
    const uint8_t *data[2];
    uint32_t ui32IndxOfFirstByteOfCurrentlyPrintingLine;
    bool bIndxImageBeingPrinted;
} image_spec;

const uint8_t imageTest_data[] = {
#include"TestImage_1.txt"
};

image_spec  imageTest = {
    .data = { imageTest_data, imageTest_data },
    .ui32IndxOfFirstByteOfCurrentlyPrintingLine = 0
    .bIndxImageBeingPrinted = 0,
};
This way, you only use RAM for the two pointers to the data, and the two other members, and the actual image data is in Flash.  (The XC32 manual said you have an unified address space, where both RAM and Flash are accessible using the same instructions, just at different addresses, so this should work without issues.)

In general, when writing code for resource-constrained embedded targets, you do need to consider what data you can keep in Flash/ROM, and what needs to be in RAM because you wish to be able to manipulate it.  Above, instead of incorporating the data into the image definition, we use a pointer to the data, so that the data can be anywhere else (RAM, Flash, wherever, since we have an unified address space).  This also means the data appears only once in Flash, so actually saves you resources there too.

[ataradov]

I thought it was only a C++ thing:

This is not what this code does, it simply includes another file. It is no different than including a header file.

[DavidAlfa]

I'm not telling you anything new, for sure you know far more more than me about C.
But this code hurts my eyes!

It's not about the fact he's including the txt file, it's about WHAT he's trying to achieve with it.
The txt file has just comma separated values, so how on earth would this work? (ignoring the fact it's being declared twice)

Code: [Select]

.arrImage_data =
                                   {{
                                   #include"TestImage_1.txt"
                                   },{
                                   #include"TestImage_1.txt"
                                   }},

#include simply tells the compiler to search for resources in that file following the line where it was declared.
But that initialization actually does this, #include does absolutely nothing inside it:

Code: [Select]

.arrImage_data =
                                    {{
                                    },{
                                    }},

It's like inserting any random preprocessor stuff:

Code: [Select]

.arrImage_data =
                                   {{
                                   #define This_does_nothing
                                   },{
                                   #define Neither_this
                                   }},

Actually, it's strange that the compiler isn't dropping any error, because a random text not following any syntax would definitely shoudl fail.
Perhabs the compiler is ignoring it due not being a standard C file extension (.c, or .h).

[mikerj]

#include simply tells the compiler to search for resources in that file following the line where it was declared.

#include inserts the entire contents of the target file, no searching is performed at that point.  I suggest the OP enables preprocessor file output and checks that.

[Renate]

Haven't you ever forgotten a semicolon in a function declaration in a header file?

Code: [Select]

int my_fun(int x, int y)   <- missing semicolonThat sometimes shows up as an error in the .c where it's included.
That shows that #include just shovels in text from the file without paying attention to anything.
In your example you could include the next set of braces in the included file or even just the opening braces (as long as the net effect with the .c file was balanced).

[ataradov]

#include simply tells the compiler to search for resources in that file following the line where it was declared.

As been said before, #include just scans the whole file as it was copy-pasted into that place. Nothing more.

This use of include is very common and useful. I use it all the time with stupider compilers, like SDCC, which does not deal well with optimizations across multiple object files. When I still want to have some structure to my code I simply include all the C files into one main source code. From the compiler point of view it starts to look like a single file.

There are other uses. And this one is valid too.

[DrG]

Hi,
the code below does not initialise the array (all the values are just initialised to 0x00) in the globally defined structure. Any idea why?

Code: [Select]

    typedef struct {
    uint8_t arrImage_data[2][_IMAGE_ARRAY_SIZE__IN_BYTES];
    bool bIndxImageBeingPrinted;
    uint32_t ui32IndxOfFirstByteOfCurrentlyPrintingLine;
} _image_data_struct_t; //image_data_struct;   

 _image_data_struct_t imageTest =
                            {
                                .arrImage_data =
                                    {{
                                    #include"TestImage_1.txt"
                                        },{
                                    #include"TestImage_1.txt"
                                   }},
                                .bIndxImageBeingPrinted = 0,
                                .ui32IndxOfFirstByteOfCurrentlyPrintingLine = 0
                            };

and the file TestImage_1.txt in the #include"TestImage_1.txt" has just a long list of comma-separated byte values exactly like this and nothing more:

0x11, 0x22, 0x33, 0x00, 0x00, 0x00,
0xAA, 0x55, 0x13, 0xFF, 0xFF, 0xFF,
0x11, 0x22, 0x33, 0x00, 0x00, 0x00

Thank you

I think you will get much better help if you will post a minimal example that demonstrates the problem that you *think* is happening and show all of the code for that minimal example.

So, in this case, the problem as you see it is that "the code below does not initialise the array (all the values are just initialised to 0x00) in the globally defined structure."

But, to my eyes, there are a whole lot of things that can be going on that are not at all revealed in that snippet. Not trying to give you a hard time, just giving my opinion. I have a little bit of experience with XC32 and I read through the thread and decided that there are just too many issues that *could* be going on and maybe I would only know about a few....but if you had posted a minimal example that demonstrates the problem, I would have looked further.

[Siwastaja]

David Alfa,

#include, by very definition, includes another file. Fancy copy-paste, ranging from very crude automated copypaste on old compilers, to a bit fancier which can track file names and line numbers to show errors/warnings properly; but still just a copy-paste).

The most typical use a beginner sees, it's used to include the headers of libraries (the actual binaries to be linked later). For this use, it's quite a crappy tool but neither C nor C++ has any better, so #include it has to be. Basically, the compiler copy pastes the declarations (but not definitions or contents, because they are not in the .h files) of all library functions every time to your code (compilation unit). If anything, this standard use would hurt the eye of someone who has seen something better in other languages.

Example shown here is different: this is the most proper use case for a file "copy paste" type of include. You have some externally autogenerated data. You don't want to manually copy-paste every time that data changes, which could be even for every compilation. You don't want to have the ugly thousand-line hex blob visible in the code because it tells you nothing. So you #include it; make the compiler copy-paste it automagically for you. This would be the most appropriate use case for #include, it should not hurt your eyes.

Preprocessor commands (starting with #) are all very simple, like automated text processing features. It's helpful to understand this.

[Nominal Animal]

@ricko_uk:

Please check if a simpler/test version of this works in your environment.  I recommend you do this as a separate project.  For example,

Code: [Select]

#include <stdint.h>

typedef struct {
    uint32_t  a;
    int8_t    b;
    uint8_t   c[2][64];
} data_struct;

data_struct  test_data_struct = {
    .a = 0xdeadbeef,
    .b = -42,
    .c = { {
         1, 2, 3, 4, 5, 6, 7, 8,
        -1,-2,-3,-4,-5,-6,-7,-8,
        32,33,34,35,36,37,38,39,
        47,46,45,44,43,42,41,40,
        16,17,18,19,20,21,22,23,
        24,25,26,27,28,29,30,31,
        55,54,53,52,51,50,49,48,
        56,57,58,59,60,61,62,63,
         }, {
         1,63,62,61,60,59,58,57,
        56, 1,54,53,52,51,50,49,
        48,47, 1,45,44,43,42,41,
        40,39,38, 1,36,35,34,33,
        32,31,30,29, 1,27,26,25,
        24,23,22,21,20, 1,18,17,
        16,15,14,13,12,11, 1, 9,
         8, 7, 6, 5, 4, 3, 2, 1,
         } },
};
The sum of test_data_struct.c[0] is 3944 (104, if you sum as uint8_t), and the sum of test_data_struct.c[1] is 1828 (36, if you use uint8_t), if you want to use a test loop to verify all the data is still there.

If you see at runtime zeroes here, then you have a minimal, complete verifiable example or MCVE, as needed for proper bug reports.

If this works correctly, then I'd expand the size, duplicating the shown data or using random data, up to around the same size as in your proper project.  If it suddenly stops working, it means that your environment has a limit of how much it can initialize data.  Perhaps it is running out of Flash/ROM, or perhaps it is some configurable limit.

It this works correctly as a separate example, but fails in your own project, then your project has a serious bug where something clears the memory after it has been initialized; most likely, your own code does an equivalent of memset(pointer, 0, size) where pointer, size, or both are bogus.  A particular case I've seen is when supplied size is negative: since the parameter is of type size_t, the size is cast to an unsigned type, leading to a very large size.

[DavidAlfa]

Damn, learned something in the wrong way, after opening my mouth and putting myself in evidence.

[ricko_uk]

Thank you all!

I now have few things to try. Also reinstalling MPLAB to see if that changes anything.

[SiliconWizard]

Just a point, XC32 is just GCC for MIPS. It shouldn't in itself do anything weird or special.

Now of course, initializers for global data are put in the .data section, and the copy in the actual objects in RAM is done in the startup code. The startup code for your particular project could be worth taking a look at - although, if it didn't copy initializers properly, absolutely no initializer in your code would work. Thus, something as simple as a global like: "int n = 5;" would not hold the value '5' either... which can be checked very easily.

Note that - even if you still should find out what is wrong here - you can try declaring your variable with a 'const' qualifier, if it's not going to be modified. Then it should be read directly from program memory and not copied in RAM. If that works, then you have an additional clue as to what could be wrong. (And BTW, if said variable is not going to be modified in your code, you definitely *should* declare it 'const' anyway.)

[ricko_uk]

I forgot to mention that (for the moment) I am not putting the large array as const at the moment only because being very large, for some reason the ICD4 takes up to 3.5 minutes to read any other variable. Even if I don't include that struct/array in the watch or variable windows (to see their value) it still takes those 3 to 3.5 minutes.

As soon as I declare it as a non-const variable or remove it completely then the ICD4 is the usual fast speed (seconds to list all the variables I want to monitor).

It will go as const in the final code.

It would be interesting to know if anybody else experience similar issues. I also ordered the JLink debugger which hopefully will not have that issue (assuming it is ICD4 related)...?

Thank you

[AlfBaz]

Apologies for this off topic question
I see the code using the dot operator without the base object. How long has this been around?
Is this something new in C or something old in C++ or ??
Is there some link I can read about the rules surrounding this built in "namespace" thing?

Sorry I don't get out much

[ataradov]

There are no dot operators here. There is only structure member initialization by name. This was in the standard since C99 or so.

[mikerj]

Apologies for this off topic question
I see the code using the dot operator without the base object. How long has this been around?

This is called designated initializers, and like ataradov says was introduced in C99.

[AlfBaz]

Thank you both

[rstofer]

I would replace the #include lines with the EXACT contents of the include file and see what happens.

[AlfBaz]

If it's based on GCC add -E to the command line and look at the files the preprocessor is spits out. This will show you what gets fed into the compiler propper

[ricko_uk]

UPDATE: I made this post with list of solutions that solved this and other issues. I hope it helps others too.
https://www.eevblog.com/forum/projects/solved-mplab-1)-slowno-variables-updates-2)-icd4-not-connecting-3)-others/

[PlainName]

UPDATE: I made this post with list of solutions that solved this and other issues. I hope it helps others too.
https://www.eevblog.com/forum/projects/solved-mplab-1)-slowno-variables-updates-2)-icd4-not-connecting-3)-others/msg3794567/#msg3794567

I'm afraid you did not. Unless there is some other EEVBlog, perhaps in a parallel universe. Perhaps you could check your links are provide working ones?

[ricko_uk]

Link updated in reply above and here (the first closing parenthesis were corrupting the URL every time I pasted the link):
https://www.eevblog.com/forum/projects/solved-mplab-1)-slowno-variables-updates-2)-icd4-not-connecting-3)-others/

[PlainName]

Thanks 

[SiliconWizard]

Link updated in reply above and here (the first closing parenthesis were corrupting the URL every time I pasted the link):
https://www.eevblog.com/forum/projects/solved-mplab-1)-slowno-variables-updates-2)-icd4-not-connecting-3)-others/

Sorry - TLDR;
So in the end, was your problem of initialization here just a problem with using Dropbox?

[ricko_uk]

It was:
- Dropbox
- ToolPack update (it was showing it was in sync in the project properties window but the ICD4 firmware was not updating) and the ICD4 was not updating
- Issue in MPLB showing all array content as 0x00 (for at least a multi-dimensional array) when mouse hovering over the array and also in the Variables Watch window. That is during both debug mode and in simulator mode. Microchip has now added that as a bug. BUT in the Watch and Variables windows if you expand the array as far as you can then it shows the data. But you have to expand it all the way to single elements.

[SiliconWizard]

OK... glad I don't use Microchip MCUs anymore.

[ricko_uk]

What do you use for Microchip micros (if you use them at all)?

I am thinking of moving to Segger IDE to replace MPLAB completely and I just made another post just about that asking for feedback.
https://www.eevblog.com/forum/programming/segger-embedded-studio-to-replace-mplab-for-pic32/msg3796088/#msg3796088

By any chance, do you happen to have tried Microchip micros with Segger IDE?

Thank you

[SiliconWizard]

I don't use Microchips MCUs anymore, haven't for a few years now. But I did use them a lot before that.

Most of the time, I was not using MPLAB. Just building using makefiles, and some programming editor/IDE for editing the source code. Was not using debuggers either usually, but on occasion just MPLAB with ICD4. Those were not recent versions of MPLAB X at the time, though, so I don't know about the latest quirks.

Never used Segger IDE.