Atxmega, reorganizing memory for programming a different processor version

[dcharette]

Hi all,

In light of the current scarcity of chips all across the board, I have a repair that I wanted to run by the forum and see if anyone had any experience on the topic.

I have an "intelligent" lighting system which has an ATXMega64D3 processor on board. This is a repair situation and I don't have access to the original source files. Fortunately, none of the lock bits are programmed, so I'm able to read everything, flash and EEPROM and am able to program a new chip without much trouble if I could get the replacement processor.

The problem with this system is that there is an area in flash memory that is not functional that appears to be critical to the system operation. There's like 10 bytes that will no longer accept programming. I think in the original application, they've mistakenly used that area in flash to write the system time on, in 10 minute increments. These things have at least 20000 hours on them, so that's at least 120,000 writes to flash. And I believe that repeated writing screwed up the flash just in those areas. I've verified it by trying to erase the chip and then write a blank file with zeros, and this ten byte area still shows FF in those locations. For whatever reason, it appears that these corruptions cause the system to cease operation after about 10 minutes. Isn't this what the EEPROM memory is specifically for? The problem seems to have been fixed in an updated firmware, which I would program to a new replacement chip, if I could get it.

So, this family of processors that the ATxMega64 belongs in, is also available as 128, 192, 256, 384. Each one with a little more flash and SRAM area. It seems I'm unable to find an AtxMega64 until mid 2023. But, I can get a hold of a AtxMega384 now.

My question is would it be possible to read the flash on the original AtxMega64 and then program a chip with more memory and still retain the system operation? Has anyone here ever manually edited a hex file to correct the memory offsets for the boot, application table areas, for a different processor flash size? i.e. cut and pasting sections with the correct offsets for the corresponding flash size being used?
It seems within Atmel studio in device programming, one can offset the memory locations independently for the flash and EEPROM. But in my case, I need to manually correct the original flash hex file to make it correctly fit in the larger memory at the right locations for boot and application table differences between the 64 and 384.

Anyone have any ideas on this topic? It's a bizarre time right now that so many chips are just unavailable until next year. And keeping things running without access to the original parts, is leading to some alternative ideas to solve issues.

Thanks for the help. 

Dan

[Miyuki]

Beware of possible different peripherals (or their addresses) if changing line
But on the same line, when not using the bootloader, the program shall run, as it boots from 0x0000
Technically only difference when migrating to bigger memory is bootloader location, everything other remains the same

//edit:
For editing, it might be better to disassemble the downloaded hex and make a new one for the new target processor

[wraper]

My question is would it be possible to read the flash on the original AtxMega64 and then program a chip with more memory and still retain the system operation? Has anyone here ever manually edited a hex file to correct the memory offsets for the boot, application table areas, for a different processor flash size? i.e. cut and pasting sections with the correct offsets for the corresponding flash size being used?
It seems within Atmel studio in device programming, one can offset the memory locations independently for the flash and EEPROM. But in my case, I need to manually correct the original flash hex file to make it correctly fit in the larger memory at the right locations for boot and application table differences between the 64 and 384.

Depends on MCU. Often different types of memory have the same address offsets within the family. In other words MCUs with lower amount of memory have more of unused address space in between, but memories start at the same address regardless of their size. In that case, you can program as is given that otherwise MCUs are compatible. Not the case with your MCUs though. EEPROM and SRAM addresses are thankfully the same for all MCUs. So how such replacement will work depends on if/how application table and boot addresses are used by firmware. I would try programming as is and see if it works.

[T3sl4co1l]

Offhand, they should be pretty similar, and I don't think the locations of peripherals move around between models, there's just more of them as applicable.  Likewise, Flash above 128k is accessible using the extension (RAMP etc.) registers, so since it's not being used in the first place, it's invisible to the program.  And everything defaults to sensible states (disabled) on boot, no init changes required beyond those needed to support the peripherals.

But as hand-waving goes, I'd feel much more confident going over the codebase itself and checking against the datasheet, and compiling with the correct headers and libraries.  A shame that's unavailable here...

For a binary image, I'd want to disassemble it and check for obvious references to peripherals, Flash, etc.  Could also be an opportunity to patch the write itself, either see why they didn't use EEPROM -- or if there's some to spare, change to it entirely.  (EEPROM is quite easy to use on these, it might not even need a jump to unused memory, just mod in situ?)

Is this a kind of thing you can test adequately, by running it through its paces, checking behavior and confirming functional code paths?  Probably, it won't crash outright, they're binary compatible; most likely outcome would be differences in peripherals, breaking some functionality -- but figuring out what, might not be so obvious.  I'd say there's a small but perceptible chance that it works as a direct drop-in replacement.

Oh uh, I forget where SRAM is mapped on those, and with respect to other peripherals/spaces, like the EEPROM address window you can enable.  I think that's its own thing, not overlapping SRAM or anything, but yeah just something to check if things move around.  Note that, although stack will be initialized to SRAM_END or something like that, if there's more SRAM on the chip, it just doesn't use it, and again, who cares, transparent to the program.

Maybe the most likely must-fix conflict is if there's a bootloader, in which case some logic will have to change around its allocation/positioning.  This might not be much: Flash can be allocated (as boot) by block I think, so you'd have to check the FUSES or NVMCTRL stuff.  Best case, actually the same location can be used, and the program again doesn't know anything is different.  Worst case, you have to stuff it in the +8kB or whatever at FLASH_END, and patch the (likely few) pointers into or out of that section.  (The above figure shows "application table section", which can I think also be allocated as boot (r/x only), effectively growing the boot section to an arbitrary number of blocks.)

Tim

[dcharette]

Hello all,

thanks for the replies and tips, they're extremely helpful even if this is not the optimum set of normal microcontroller development circumstances.

I'll add some further details.  I don't believe this system uses any unusually different peripherals, just the basics, which is why I was looking at this kind of solution. The only thing that is being used is a couple of the serial ports and Uarts, and one of the general ports for an LCD display and some pushbutton inputs.

I believe the original problem of repeatedly writing to the flash was solved with a firmware update, which I have also read and saved as hex files. It's just fixing some units that didn't have the firmware upgrade, and dealing with unavailability of the original sized processors that's my main problem to solve at the moment. So, I don't need to patch for the original mistake. 

It's just the boot location that I'm concerned with. Once I get the replacement processor, I will try to see if I can just program it directly without any hacks, and see what happens, worse case scenario it doesn't work.

But as for disassembly, and creating a new one for the target, that would be the ideal scenario. I've been using Atmel Studio 7 and an ICE for these reading and programming tests so far. I'm not in my lab now to test, so I'll ask quickly here, is it possible to simply "attach to target" of the working system, go into debug mode and get an asm listing in the main window that I could then build for my different sized target processor or are there some other better tools for such a task?

Thanks again for all the replies! 

[wraper]

But did you try flashing the new firmware into faulty MCU with worn flash? It's completely possible that faulty flash addresses are no longer used in the new firmware.

[dcharette]

Hi wraper,

it would make sense from Atmel's development and manufacturing standpoint of a family such as this to just make one design, with all the peripherals and maximum memory. And then, for the different variants, to disable them or at least access to them, as you said larger amounts of unused space on the smaller processors. I'm not an expert by any means in this field, but could that be the case here? I guess I will find out when programming the bigger chip, and whether it works straight away or not. 

[dcharette]

Hi wraper,

yes, I did try flashing the good f/w to the corrupted units, as part of going down this rabbit hole. It was a strange problem to try and figure out in the first place. I had three of these with the same f/w version crashing exactly after 9min and 50secs of power on time. And I had one good one with a higher f/w version that worked. I read the working one, and tried copying to the other ones, and still the same problem. So, I suspect the corrupt area is important some way to the operation of the unit with the newer corrected firmware. I'm certain the corrupted ones definitely have those 10 bytes of dead memory that just stay erased and won't accept any other new values.

[T3sl4co1l]

Hi wraper,

it would make sense from Atmel's development and manufacturing standpoint of a family such as this to just make one design, with all the peripherals and maximum memory. And then, for the different variants, to disable them or at least access to them, as you said larger amounts of unused space on the smaller processors. I'm not an expert by any means in this field, but could that be the case here? I guess I will find out when programming the bigger chip, and whether it works straight away or not. 

I think he meant with respect to the firmware and updates, not between MCUs...

Though that is another possibility.  FWIW, my experience with this family has been, if a bit/device/region is labeled "reserved", it's physically inaccessible, from the CPU bus at least.  Only a few bits I've seen "reserved" are even present (the registers are there), but nonfunctional (or, seemingly so).  So, unless there's undocumented chip-configuration procedures still available (say, things disabled by MFG fuses), they're pretty close to the docs.

Always fun to see when devices are more... unexpectedly functional than their datasheets, though.

Tim

[T3sl4co1l]

As for disassembly, not sure what can be done within Studio directly, but if nothing else, find where GCC is installed (somewhere within Atmel Studio?) and use:
[GCC PATH]\bin\avr-objdump.exe -b binary -m avr -D <file.bin>

But you'll have to look through it to decide what's code and data.  If from GCC, it will start with interrupt vectors (RJMP something or other?), then const (PROGMEM) data, then functions, ISRs, and main probably a big blob at the bottom, then library imports, and finally .data init at the very bottom.  Unused memory should be 0xff's and then maybe a bootloader at whatever offset(s).  Or if they're using other space, there could be other data, spare copies of firmware, bootloader(s), etc. higher up in Flash.

I forget if Ghidra, Ida Pro, etc. do this (code/data analysis, cross reference, find pointers, etc.) automatically, with AVR support I mean; offhand looks like there may be direct support, or plugins available to do it?

Tim

[dcharette]

Hi Tim,

thanks for all the info. Your suggestions and ideas are extremely helpful for someone like me that's not got the road time invested in some of these sometimes rather ornate development environment setups. It's amazing at times to witness some of the methods that different engineers accomplish identical jobs, just using different tools and routes to get to the same point. I've never heard of Ghidra or Ida Pro and went to look them up after you suggested.
I will post back, once I've tried just flashing the new bigger memory chip and seeing what happens. I think somewhere in one of the Atmel datasheets for this family, I remember reading that the code was compatible across all devices, 100%. But, I know that means when you have the source code and can compile for the specific device. But what the heck, the worse that can happen is it doesn't work quick and simple.  

[trevwhite]

The D series are usually peripheral limited variants. So I would think other than bootloader memory address, larger memory parts and C and A variants should all work.

[T3sl4co1l]

I think somewhere in one of the Atmel datasheets for this family, I remember reading that the code was compatible across all devices, 100%. But, I know that means when you have the source code and can compile for the specific device. But what the heck, the worse that can happen is it doesn't work quick and simple.  

It's a bit more complicated than that; there are subtle differences in a number of instructions, between families.  Mainly adding or shaving off a cycle here or there.  XMEGA adds XCH, LAS, LAC, LAT instructions; the biggest chips (A series?) have DES (encryption hardware support), and tinyAVR remove a number of instructions (MUL, LPM/SPM) and even registers (r0-r15 gone).  Also a few CPU-adjacent differences, like CPU flags / SFR, interrupts, etc.  Enough that, there's device-specific files for each of them, which are almost all about setting defines and memory spaces, but also include things like this, and call out the appropriate C libraries and etc.  (Even a function as simple as _delay_us(), needs to know CPU series, due to the different branch timings.)  But given that -- using the instructions common to all, not minding strict execution time, and assuming registers are in the same places, and reserved bits are respected -- it's fairly universal.

And this is all pretty normal I think; like, ARM is an absolutely massive ecosystem, but they're all binary compatible (for the given version of core, and long vs. thumb mode, etc.), and device differences are largely about where and how to access peripherals, which are almost all handled by device-specific libraries and headers.  (A given binary is almost certainly not going to run, let alone function the same, between say an LPC or STM32 core, say; but one could likely be written to do so, given a platform-detect function at the start, and enough code space (and perhaps spare execution time) to do it.)  So the same e.g. arm-none-eabi-gcc works on all them, just add the right linker scripts and so on.

Which might not be saying much after all, but yeh, depends on what level you're looking at.  At the whole-device level, less and less likely, as you move from variants within a family, to between families and product lines, to between manufacturers; but at the CPU level say, it might still run just happily -- assuming the peripherals let it run wild -- it might just not do anything useful in the process.

Tim

[dcharette]

Hello All,

So, I've got a little farther down this rabbit hole.

As I had mentioned, this post was with regards to trying to repair some lighting systems which originally contained an ATXMega64D3. The code isn't locked, so I was able to read the flash, EEPROM, and fuses using an Atmel ICE and Studio 7. I have three of these systems that are dead.
As everyone is well aware, there is a severe lack of new parts available to purchase, nearly everywhere for the next year.

I did manage to find a single ATXMega64C3. And, using the code I read from a working system, I was able to swap out the faulty chip, program it, and test it. It runs successfully and I can't seem to find anything that is compromised. The problem was in fact 10 bytes of completely unresponsive memory. I thought it was flash originally, but those 10bytes of dead memory were in the EEPROM, in the same place on all three systems. The original software logs hours, and temperature. And I think the corruptions were involving both of these items. The number of times that the system writes these values during operation, after a few years, probably goes over the spec of the number of flash/EEPROM writes.

Now, as for the other 2 dead units. The only ATXMega chips, with a 64pin TQFP package I could find in the moment that would potentially fix these, were on Atmels Xplained dev boards. These chips are ATXMega256A3BU. I realize there are more peripherals available to use, along with the increased flash and boot size/location. That aside, I was hoping that these processors would be able to run the code as well, with little to no modding on the firmware.

I first did a quick programming, only checking that the fuses were the same settings, as the 256A3 has a few more available to program. But, it doesn't work. And, I suspect it's got to do with code that's in the the boot area.

As I started having a look at the flash hex file, it seems that in the original ATXMega64D3 file, at the beginning of the boot flash location, there is a jump instruction and address.

10000:   0C 94 6D 82 FF FF FF FF...
...
...
...
10040:   0C 94 80 80 FF FF FF FF...

Since these are 16 bit addresses, how would these look if this was correctly compiled for the ATXMega256A3BU, a 256kB flash and 8kB boot?
I have already attempted to manually move the boot section as shown above from 10000, to 40000, to reflect the position as listed in the ATXMega256 datasheet. But, I realize there must be more to this since 16bits doesn't seem enough to address beyond 64kB. Is there a different jump instruction/s that is specifically for the other 3 pages of 64kB flash on the bigger chip?

While I would rather stick to the 64kB variants of this processor to get these things fixed, I'm a little stuck due to the unavailability to them, so if anyone has any ideas on what I could potentially try as experiments with the flash hex file, to make it work for the 256kB variant, please let me know, as crazy as the idea would be, I'm willing to try.

Thanks for all the help thusfar!       

[wraper]

I would buy ATXMEGA64D3 on aliexpress with "AliExpress Standard Shipping" which in average should take about 10 days to deliver, and do not waste any time going down the rabbit hole.
https://www.aliexpress.com/item/4000997106012.html
https://www.aliexpress.com/item/1005001824550920.html
They might be an old stock with poor solderability or pulled from e-waste, but for repair it's OK IMHO.

[dcharette]

Hi Wraper,

Thanks for the links and advice. I'm hesitate with aliexpress these days. At least from my experience the last 4 orders I've had ended up getting hung up one way or another either by shipping or customs, and I had been into 3-4 months of wait time, in fact now still waiting on a few unobtanium ICs, going on 4 months. I was just trying to find a dependable solution, locally and within a reasonable time frame. But, at this point, I don't have much to really lose. These lights are dead as it is, and wait time for the normal supply is next year. I'll give it a shot and post back once I have some results.

thanks again for the help! 

[wraper]

If you select "aliexpress shipping" there should be no problems.

[T3sl4co1l]

Yeah, 0C 94 xx xx is JMP longptr, with the upper bits zero, so 0x826d and 0x8080 are plausible program addresses.  You say this is in a range?  So it's an interrupt vector table?  The FF's seem suggestive of just unused memory though; maybe that would be okay for a boot vector, but it's usually jmp __bad_interrupt filling space.  Is the same or similar present at 0...0x200 as well?  Or anywhere else, uh, I forget where all the IVT can be -- but it's set by fuse I think, so it shouldn't matter very much how/where it's been duplicated?

There's also (usually?) a __trampolines_start, which, looking at some of my projects, is used for faking far calls in jump tables (without having to store 24+ bit pointers)?  It might not be generated if such code isn't used (i.e. switch statements that don't reduce to if-else, callback functions, etc.).  GCC looks to put this right after __vectors so it just looks like extra vectors, but it's not actually ever accessed by hardware.

Tim

[cv007]

And, I suspect it's got to do with code that's in the the boot area.

Skip the bootloader, leave the BOOTRST fuse unprogrammed so the app runs at boot. You then do not have any potential address issues that deal with the upper parts flash. At least it would be an easy thing to try.

[dcharette]

Hi cv007,

I had tried the bootrst fuse, in both states. It doesn't work.

What's odd though, is that on a working unit, with the 64kB flash, the bootrst fuse can be in either state and the system will boot fine and work. However, with the 256kB flash processor, it just doesn't run at all.

I've already made an order for the processors via aliexpress. While I would love to work through this problem and get it to run with the larger flash processors, it's probably best to solve it this way at least for now, until I'm really without any other option. If I had access to the source code, it would be another story and it would make sense then to create some production files, for programming different flavors of this ATXMega family, just to keep these running when this problem will probably occur again.

I've never seen a system fail like this, when only just a small bit of memory is corrupt. Usually, when memory goes, it will kill the system in whatever procedure it was at. This system however, works for 9min and 50secs exactly, and then only part of the system dies, and the UI with the attached LCD display works fine, you can look at parameters, use the pushbuttons, etc. But in order to get it to run again for another 9min and 50sec, you have to power down and start back up fresh.

Anyway, thanks for the tips and trying to help. I will post back once I have some success. Maybe revisit this topic sometime to attempt to get this running on the bigger flash, just for fun. 

[wraper]

I've never seen a system fail like this, when only just a small bit of memory is corrupt. Usually, when memory goes, it will kill the system in whatever procedure it was at. This system however, works for 9min and 50secs exactly, and then only part of the system dies, and the UI with the attached LCD display works fine, you can look at parameters, use the pushbuttons, etc. But in order to get it to run again for another 9min and 50sec, you have to power down and start back up fresh.

Likely it's how long it takes before it tries to log something for a first time after power-on. You said it logs system time with 10 minute increments.

[dcharette]

Hi wraper,

yes, the system time is logged and I know it's in 10 minute increments. But another thing it writes to memory is temperature, and I'm not sure how often it does that, it could be the same increment, or even more. I haven't been able to watch it and correlate the temp changes enough to figure out the increment. But, it's probably even more writes to memory than the system time. Even still, why the heck would the system crash if the corrupt locations were only the data values of time and temperature? But, it's a reasonable lesson in software construction, to make the code reliable enough that if this condition ever occurs, to have some error check that will at least alert the user and not break the system. I sure wouldn't want to send something like a satellite or deep sea beacon built like this that would have holes in its flash after a couple of years of operation and then have to replace the flash memory because of something as stupid as this.
 

[wraper]

Even still, why the heck would the system crash if the corrupt locations were only the data values of time and temperature?

Likely it reads the FLASH before writing or verifies the data written.

[dcharette]

Hi wraper,

you know, I've been thinking about this. And it's very possible that this "problem" is not really a problem so to speak, but possibly a planned failure. The system I've been speaking of is an "intelligent" moving LED light that you would see at a theater or music concert. And these particular ones are being using in a demonstration hall, where they are power on 24/7. The lights aren't illuminated, but the systems are powered on until which time they are needed for the light purposes, and they are sent commands to turn on the LEDs. Now, these lights all have between 15-20000 hrs of time registered in their memories. And that's pretty accurate based on what I know about the installation. But, what I'm starting to suspect is that this particular "failure" would be a pretty good way to make the light "require" service at some pre-determined time period. We all know that LEDs these days can last quite some run time. And it's very possible that the makers of these things realized that there could be an additional revenue stream by creating a legitimate failure. As you said, the time value could be first read then written to flash. If it was read and presented some erroneous value or some value out of "range" the result in code could be that it disables the function of the light that's important. At least, that's almost what it appears is going on. It wouldn't surprise me if this was a deliberate "crippling" code.