Arduino Pro Mini - two copies have different sleep currents

[Peabody]

I have two 3.3V 8Mhz Pro Minis, one a year old, and one new from a different source.  I have modified both the same way - remove the resistor to the power-on LED, and remove the voltage regulator.

I have a test sketch that turns everything off and goes into PWR_DOWN sleep, at which point I measure the current.  Nothing is connected to the Mini during the test except power and ground.  The old Mini draws about 0.4µA.  That's what it's supposed to draw.

The new Mini draws 150µA.  It works fine while it's awake and running other sketches, but I bought this specifically because it can draw very low current in deep sleep, so this isn't going to work for me unless I can fix it.

Assuming the processor isn't a fake, the only explanation I can come up with is that there must be something different in the bootloader and/or fuse settings between the two Minis, and that the fix would be to burn the stock bootloader and fuses to the new Mini.  But I've never done that before, so before risking that, I wanted to just make sure there isn't something else that I haven't thought of that could account for this problem.  Does anything come to mind?

And if anyone wants to try my test sketch on their Pro Mini, here it is:

Code: [Select]

#include <avr/sleep.h>
#include <avr/wdt.h>
int i;

void setup() {

  for (i = 2; i < 20; i++) {          // all pins to one rail or the other - power saving
    pinMode(i,OUTPUT);
    digitalWrite(i,LOW);
  }
  ADCSRA = 0;                         // disable ADC for power saving
  wdt_disable();                      // disable WDT for power saving
  set_sleep_mode (SLEEP_MODE_PWR_DOWN); // Deep sleep
  sleep_enable();
  sleep_bod_disable();                // disable brownout detector during sleep
  sleep_cpu();                        // now go to sleep
}

void loop(){
}


[Ian.M]

Another possibility is ESD damage, insufficient to stop any pins working, but enough to increase one or more pins' I/O driver leakage currents.   Try setting all pins high in your sketch and see if the sleep current changes significantly.

[DrG]

/--/
Assuming the processor isn't a fake, the only explanation I can come up with is that there must be something different in the bootloader and/or fuse settings between the two Minis, and that the fix would be to burn the stock bootloader and fuses to the new Mini.  But I've never done that before, so before risking that, I wanted to just make sure there isn't something else that I haven't thought of that could account for this problem.  Does anything come to mind?

I don't know, but I like thinking about things and looking around and thinking about things some more. So, for what it is worth....would it be willing useful to try one of the low-power libraries and see if that yields anything different?

I was looking at this one https://github.com/rocketscream/Low-Power/blob/master/LowPower.cpp and noticing how much was there. Again, FWIW.

[Syntax Error]

Assuming that both pro minis are identical, even to the clocks and fusing, I would ask, what else is eating current on those boards? What are the 3V3 regulators drawing, for example?

[DrG]

Assuming that both pro minis are identical, even to the clocks and fusing, I would ask, what else is eating current on those boards? What are the 3V3 regulators drawing, for example?

Since they are removed, I would guess very little 

[Syntax Error]

No idea then... a deep delve into the Atmel spec.

[oPossum]

I have seen leaky caps cause high sleep current. Have yet to see a defective chip cause high sleep current.

[Peabody]

I was able to learn how to use Avrdude to read out the fuse settings and the bootloader code from the old and new Minis.  They are identical, and the bootloader is the stock bootloader.  So at least that settles that issue.

Changing the IO pins to all high didn't make any difference.

So, it's a physical problem of some kind - possibly something I did when removing the LED resistor and the regulator, or maybe it arrived that way.

I'm not sure it would be time well spent, but at some point I may try to do some comparative resistance readings across caps and other things, looking for differences between the old Mini and the new one.  But whatever produces the 150µA might not even be noticeable, or might be inside the processor.  So I'll probably just save this one for some project where sleep current isn't important.

Anyway, thanks very much for the comments and suggestions.  I'll post a followup if I find the cause.

[Peabody]

Ok, I'm about to go crazy.  I had forgotten that I had ordered two of the new Minis.  So, very carefully, I unpackaged the other new one, removed the LED resistor and the regulator, soldered on the six-pin right-angle header at the end, and flashed my sleep current test sketch.

And it behaves exactly like the first one - it sleeps at 150µA.

So it's not ESD, or a leaky capacitor.  And I already confirmed that the bootloader and fuses are identical to the old Mini that sleeps at under 1µA.  So I'm at a loss to understand WTF is going on.

As far as I can tell, the PC boards are identical.  Even the silkscreen.  The markings on the new Minis are:

Atmel
Mega328P
U-TW
35473D
1947X5N

and
1936S2P

The old one is:

Atmel
MEGA328P
U-KR
35473D
1829R5G

Does anyone have an explanation for this?  My sense is that since the current is the same for both new Minis, it's something inherent in the design or perhaps the processor isn't genuine.  In theory it could be a bad batch of leaky caps, but it's improbable the leakage would be identical.   

Where would be a good place to buy Mini clones for delivery to the US?

[Peabody]

Just wanted to confirm that I checked for leaky capacitors and other stuff, and found nothing that would account for the excess sleep current.  That leaves the processor as the most likely culprit.  I hope this doesn't mean the 328P has been revised, and now uses more current.

[DrG]

Have you ever seen any changes in the datasheet to reflect that?

There is a lot about making them low power and there are certainly different pro mini clones - both 3.3 and 5. On this site https://www.iot-experiments.com/arduino-pro-mini-power-consumption/  he looked at a couple of varieties and his number correspond to you good numbers.

What does your clone look like?

In so far as a good vendor - I don't know, but I have maybe 8 of these in total and at least three 3.3v 8MHz (using resonators not crystals as I recall) that look like the one that the site shows as "The Simple". I have not done any low power mods though.

[Ian.M]

It *could* be due to surface contamination, from poor storage or production line.  Try thoroughly cleaning the board that's drawing 150uA: If you don't have a swept frequency ultrasonic cleaner, soak in IPA or solvent flux remover, scrub thoroughly with a brush that wont cause ESD, then scrub under hot water with a trace of detergent (mostly just as a wetting agent), rinse, rinse again with distilled water, shake off as much as possible then rinse with IPA, shake and let dry.  Once visibly dry, put it in a clean very warm (but not over 95 deg C) place for 24H to let any remaining moisture and/or solvents evaporate off.  If you do have a suitable ultrasonic cleaner, use it for the hot water wash stage.

The next thing to try would be, taking full ESD precautions, to desolder the good and not-so-good chips (don't get them mixed up) and fit brand new ones, purchased from MicrochipDirect or an official distributor, then thoroughly clean as above.

If the problem stays with the bad board you can investigate other components on it. If it goes away, your new supplier has bad chips, and if both are now bad, and you put back the known good chip and its still good, its looking increasingly like a specification change or batch problem so contact Microchip technical support, (but have all your ducks in a row as front-line support is usually swamped by idiots)!

[sleemanj]

Perhaps the new ones are not ATMega328P but actually ATMega328PB in disguise.  The PB has a number of new features which maybe need to be disabled, although I would have thought that in power down mode that shouldn't matter.

Other thought is maybe the brown out detection isn't getting disabled for some reason.    But even then, 150uA seems pretty high even with BOD enabled (unless it was actually in a brown out).

If it was me, I'd reset the fuses to turn off brown out detection completely, and upload using an ISP to nuke the bootloader out of the equation just to be sure. 

There's not much on a Pro Mini that can really explain it once you've lost the regulator and LEDs.

I would pull off the ARef cap, and check the pullup on reset is sensible.  Finally I would probably pull off a good chip and a bad chip swap them over good chip on bad board, bad chip on good board, to determine if it's the chip or something else on the board.

[Peabody]

Have you ever seen any changes in the datasheet to reflect that?

There is a lot about making them low power and there are certainly different pro mini clones - both 3.3 and 5. On this site https://www.iot-experiments.com/arduino-pro-mini-power-consumption/  he looked at a couple of varieties and his number correspond to you good numbers.

What does your clone look like?

In so far as a good vendor - I don't know, but I have maybe 8 of these in total and at least three 3.3v 8MHz (using resonators not crystals as I recall) that look like the one that the site shows as "The Simple". I have not done any low power mods though.

I've looked pretty hard at the datasheet and errata, and see nothing that would explain this.

The new ones I received are "The Simple".  And the old good one I have is the same.

My old Mini actually gets lower numbers on Power-Down than the guy on that site gets.  The datasheet is pretty clear - if you take all the right steps, and with no power LED or regulator, the typical Power-Down current is 0.1uA, with a maximum of 4uA.  Not 150uA.

[Peabody]

It *could* be due to surface contamination, from poor storage or production line.

-snip-

Please note that I have two of the new Mini's, and they both sleep at 150uA.  I think it's exceedingly unlikely that they are so equally dirty.

The next thing to try would be, taking full ESD precautions, to desolder the good and not-so-good chips (don't get them mixed up) and fit brand new ones, purchased from MicrochipDirect or an official distributor, then thoroughly clean as above.

I don't have any rework gear, just a soldering iron.  Replacing the processor is not something I would attempt.

If the problem stays with the bad board you can investigate other components on it. If it goes away, your new supplier has bad chips, and if both are now bad, and you put back the known good chip and its still good, its looking increasingly like a specification change or batch problem so contact Microchip technical support, (but have all your ducks in a row as front-line support is usually swamped by idiots)!

There's not much left on the board as it is.  But later today I will do some destructive testing by removing everything that's left, one item at a time, and testing again, until I get down to the processor and resonator.  Then if I can figure out how, I think I could change the fuse to make it use the internal 8MHz oscillator, then remove the resonator and try again.  My prediction is, after doing all of that, the sleep current will still be 150uA.  Film at 11.

I would have no idea of how to contact anyone at Microchip who would even talk to me.

[Peabody]

Perhaps the new ones are not ATMega328P but actually ATMega328PB in disguise.  The PB has a number of new features which maybe need to be disabled, although I would have thought that in power down mode that shouldn't matter.

Other thought is maybe the brown out detection isn't getting disabled for some reason.    But even then, 150uA seems pretty high even with BOD enabled (unless it was actually in a brown out).

If it was me, I'd reset the fuses to turn off brown out detection completely, and upload using an ISP to nuke the bootloader out of the equation just to be sure. 

There's not much on a Pro Mini that can really explain it once you've lost the regulator and LEDs.

I would pull off the ARef cap, and check the pullup on reset is sensible.  Finally I would probably pull off a good chip and a bad chip swap them over good chip on bad board, bad chip on good board, to determine if it's the chip or something else on the board.

I though maybe it was a 328 instead of a 328P.  That would account for the increased current.  But I tested the BOD disabling in software, and found that in both the old and new Minis that's a difference of 20uA.  In the 328 there would be no difference since you can't turn off BOD.  I don't know about  the PB version, or how I could tell if it was that.  Of course the markings are pretty clear that it's a 328P, assuming it's genuine in the first place.

I'll do further testing later today if I get time.  But meanwhile I've invested another $10 ordering another batch from a different Ebay source, with different markings.  Picture of those attached.  I'm a little concerned about the marking line that says 354A3P.  All the others have said 35473D.

[Syntax Error]

Although you have checked for leaky capacitors, do you know what the rated DC leakage is of the decoupling capacitor is meant to be? If your two boards show the same problem, is it another (cheap) part other than the 328 causing trouble?

f.y.i. https://passive-components.eu/understanding-capacitor-leakage-to-make-smart-things-run-longer/

[Peabody]

I finished the destructive testing of one of the new Minis.  I burned a new bootloader and fuses so it runs off the internal 8MHz oscillator, and reinstalled my sleep current test sketch.

Then I began removing everything on the board, testing the current after each removal.  In the end, I had removed everything from the board except the processor, and it still sleeps at 150uA.

So the problem is definitely the processor.  The markings look completely normal, so I don't know what the problem actually is.  I guess it could just be a bad batch.

Anyway, I should have the new batch of Minis here by the weekend, and will see what happens with them.

[DrG]

Is it irresponsible to suspect that it is not a legitimate chip?

I looked at a few that I had hanging around and they are all -AU chips. I can see the AU in the data sheet (page 635 https://ww1.microchip.com/downloads/en/DeviceDoc/ATmega48A-PA-88A-PA-168A-PA-328-P-DS-DS40002061A.pdf) I see no -KR am I missing that or is it new or ?

It is confusing because I guess Atmel changed some labeling and Microchip has their labeling.

excuse the phone pic

[Ian.M]

You don't *KNOW* its the chip till you've removed it and seen the leakage drop to zero.  It could even be a leaky board, and not necessarily surface contamination.  e.g. if the assembler's PCB house got a batch of PCB on the cheap made with reject resin or incorrect mix ratio.   The leakages for the sample of two boards you have could be pretty similar because, for the same batch of resin the leakage would be mostly determined by the track length and spacing in the layout.

A decade or so back, Microchip Technology used to do special far east market versions of some of their PICs that didn't meet the US datasheet specs, for really cost-sensitive applications.   They were supposedly marked with distinct part numbers, but we all know how that one goes when there are no westerners 'in the loop' . . .  Maybe they've started doing similar with AVRs?

[Syntax Error]

Have you removed absolutely everything else from the board including the ceramic resonator, and then fused the 328 for internal oscillator mode?

[David Hess]

That would be an awfully lot of leakage for a board.

Could there be an open CMOS input pin?  That will cause excessive power draw.

[Syntax Error]

That would be an awfully lot of leakage for a board.

Could there be an open CMOS input pin?  That will cause excessive power draw.

OP writes in Arduino code...

Code: [Select]

for (i = 2; i < 20; i++) {          // all pins to one rail or the other - power saving
    pinMode(i,OUTPUT);
    digitalWrite(i,LOW);
  }which sets all pins... so what state is pin 1 left in?

[hans]

So the problem is definitely the processor.  The markings look completely normal, so I don't know what the problem actually is.  I guess it could just be a bad batch.

I doubt it's a bad batch, otherwise that's really poor QA on Atmel's part (if it is indeed, an authentic Atmel part). Usually a part/board power consumption is a good indicator of obvious defects or faults, and having a factor 375x over in 1 particular operating mode is such one.

Is it irresponsible to suspect that it is not a legitimate chip?

I looked at a few that I had hanging around and they are all -AU chips. I can see the AU in the data sheet (page 635 https://ww1.microchip.com/downloads/en/DeviceDoc/ATmega48A-PA-88A-PA-168A-PA-328-P-DS-DS40002061A.pdf) I see no -KR am I missing that or is it new or ?

It is confusing because I guess Atmel changed some labeling and Microchip has their labeling.

excuse the phone pic

Not sure how credible  LCSC is sourcing Atmel/Microchip parts, but theirs looks identical to Peabody's: https://lcsc.com/product-detail/ATMEL-AVR_ATMEL_ATMEGA328P-AU_ATMEGA328P-AU_C14877.html
I also looked at the newest product I could find on Sparkfun with an ATMEGA328(P) their labeling looks similar as well: https://www.sparkfun.com/products/16282 (looks like end of june 2020 timeframe)

Assuming Sparkfun is using reputable parts, it's plausible this is still a genuine part.

I would also be interested in what "pin 1" does. In fact, the ATMEGA328P has 23 digital I/O + 2 dedicated ADC inputs, so it's perhaps advisable to make sure all of them are well put.
Alternatively, a dirty way of testing for I/O leakage is by touching all pins with your fingers. If any pin is floating, your body "antenna" will either force the I/O pin high/low, which may initiate a change in leakage current of an input pin.
 
However, in my experience I/O leakage is fairly in-deterministic (e.g. laying the board on a different surface can change the leakage), and seeing that each measurement seems to have little to zero effect, even on 2 boards, it may well be something completely different ...

[Peabody]

Is it irresponsible to suspect that it is not a legitimate chip?

I looked at a few that I had hanging around and they are all -AU chips. I can see the AU in the data sheet (page 635 https://ww1.microchip.com/downloads/en/DeviceDoc/ATmega48A-PA-88A-PA-168A-PA-328-P-DS-DS40002061A.pdf) I see no -KR am I missing that or is it new or ?

It is confusing because I guess Atmel changed some labeling and Microchip has their labeling.

excuse the phone pic

I don't have any Arduinos with the AU marking.  All my stuff, including Nanos, have the second line as U-{something}  Either U-TH, U-TW, or U-KR.  I'm guessing TH is Thailand, TW is Taiwan, and KR is Korea.  But I agree there's nothing in the datasheet that looks like that, so maybe all mine are counterfeits.  I just don't know.

[Peabody]

Have you removed absolutely everything else from the board including the ceramic resonator, and then fused the 328 for internal oscillator mode?

Yes I have.  All of that.  It still sleeps at 150uA.

But I have not tried to remove the processor itself from the board.  I don't have a heat gun, so I would have to use a skillet.  I'll have to think about that.

[Peabody]

That would be an awfully lot of leakage for a board.

Could there be an open CMOS input pin?  That will cause excessive power draw.

OP writes in Arduino code...

Code: [Select]

for (i = 2; i < 20; i++) {          // all pins to one rail or the other - power saving
    pinMode(i,OUTPUT);
    digitalWrite(i,LOW);
  }which sets all pins... so what state is pin 1 left in?

Oh, sorry about that.  I left D0 and D1 out originally so as not to interfere with Rx and Tx, but changed that a week ago to include them.  So it's been i=0;i<20 for a while now.  But that didn't change the sleep current.

Edit: To clarify, my memory was that the bootloader left TX at OUTPUT, HIGH, and Rx at INPUT_PULLUP.  So since both were already at a rail, I didn't need to mess with them.  But just to be sure I changed it so all the I/O pins are OUTPUT, LOW.  That works because Rx and Tx aren't connected to anything during the sleep current test.

[DrG]

Is it irresponsible to suspect that it is not a legitimate chip?

I looked at a few that I had hanging around and they are all -AU chips. I can see the AU in the data sheet (page 635 https://ww1.microchip.com/downloads/en/DeviceDoc/ATmega48A-PA-88A-PA-168A-PA-328-P-DS-DS40002061A.pdf) I see no -KR am I missing that or is it new or ?

It is confusing because I guess Atmel changed some labeling and Microchip has their labeling.

excuse the phone pic

I don't have any Arduinos with the AU marking.  All my stuff, including Nanos, have the second line as U-{something}  Either U-TH, U-TW, or U-KR.  I'm guessing TH is Thailand, TW is Taiwan, and KR is Korea.  But I agree there's nothing in the datasheet that looks like that, so maybe all mine are counterfeits.  I just don't know.

*blech* I don't know either - I was looking here https://forum.mysensors.org/topic/9388/atmega328p-au-counterfeit/16
and the AU chip (which looks like mine) turned out to be, according to the poster, a mess. He then ordered from Digikey and got the U-TH version.

AVR freaks has a thread or two showing labeling lines but I think they were changed or are inconsistent or ? e.g., https://www.avrfreaks.net/forum/fake-atmega328p

You have been using your LP code successfully but I wonder if it would not be worthwhile again to just test out that library that I linked to earlier.

It is a puzzle.

edited to add a link

[DrG]

Not sure how credible  LCSC is sourcing Atmel/Microchip parts, but theirs looks identical to Peabody's: https://lcsc.com/product-detail/ATMEL-AVR_ATMEL_ATMEGA328P-AU_ATMEGA328P-AU_C14877.html
I also looked at the newest product I could find on Sparkfun with an ATMEGA328(P) their labeling looks similar as well: https://www.sparkfun.com/products/16282 (looks like end of june 2020 timeframe)

Assuming Sparkfun is using reputable parts, it's plausible this is still a genuine part.

Yeah copy that (and see my last post) - it would be helpful to have a CURRENT version of Atmel's labeling guide.

[Ian.M]

Have you removed absolutely everything else from the board including the ceramic resonator, and then fused the 328 for internal oscillator mode?

Yes I have.  All of that.  It still sleeps at 150uA.

But I have not tried to remove the processor itself from the board.  I don't have a heat gun, so I would have to use a skillet.  I'll have to think about that.

Your photos show the ATmega32P '32A' TQFP-32 package.  To totally isolate power to the MCU to confirm its the cause of the 150uA leakage, you'd only have to lift four pins, two Vcc (4,6), one AVcc (18), and /Reset (29) (as it has an external 10K pullup to Vcc you may not have removed), which any competent tech should be able to do non-destructively, given *any* soldering iron  (that doesn't have to be heated on your kitchen gas hob ) with an appropriately sized well tinned bit, some flux and desolder braid, and free choice of an xacto knife or a sharp fine needle probe to actually lift the pin.

If you want to rework or transplant SMD chips without proper tools/equipment for SMD reflow, consider using ChipQuikand an ordinary iron.  Its not cheap, but a little goes a *very* long way.  Using ChipQuik would eliminate nearly all risk of cooking the chip, or changing board leakage (or causing board damage) by excessive heating of the laminate.


Discussion thread: https://www.eevblog.com/forum/blog/eevblog-437-removing-smd-parts-with-chipquick/

Actual desoldering starts at 9:05 and takes under 3 minutes. 
Another example, TQFP-32, similar to your MCU package starts at 18:00, removal takes 50 seconds!  N.B. keep the blobs of used ChipQuik with the kit - it will do several more chips before its melting point becomes high enough to matter.

[hans]

If you really need to get a chip off in a destructive manner (for the chip), you could also use a side cutter or a sharp stanley knife, and then cut all the pins of the package. Then clean up the pads with the remaining legs. That way you don't need to use a hot air gun, special solder wire or insane amounts of heat to get a QFP chip of a board.

In my experience, the QFP32 packages have a large pin pitch which allows you to get in there with pretty slim pair of cutters (perhaps depends on the tip design), and cut all pins.

[Ian.M]

I've never liked that approach to destructive SMD removal.  Maybe its just my bad luck but I always seem to get the boards with the weak pads and the shock from cutting them can rip pads off too easily for my liking.  OTOH its *THE* goto method for weak padded, tight plated-through-hole boards if the chip can be destroyed.

I've had better luck wicking off as much solder as possible then threading a fine wire under the row of pins, firmly anchoring one end, and pulling the other flat outwards so it wedges between the pin and the inner end of the pad while reheating the pin the wire is pulling on, moving along as the wire lifts it, to lift them all 'zipper style'.  If you see any pad start to move, stop and get some ChipQuik!

For the wire, any heat resistant fine wire that's strong enough in tension to lift a pin will do, preferably of a material or with a coating solder doesn't stick to.  I prefer fine magnet wire with tough non-solder-through enamel, as I can strip the end on an aspirin and tack-solder it where I need to fix it.

Leave opposite corner pins till last, and to avoid ripping off the last pad as the chip skews as you lift the second to last,  secure the chip in place e.g. with non-stretchy tape or bluetack  and lift the last two pins manually with a probe. 

With care, the chip can usually be salvaged by fluxing and reheating the pins with a clean bit to remove excess solder, then holding it down on the bench and realigning any laterally bent pins with the tip of an xacto knife or a very fine flat-blade jewller's screwdriver, then re-setting the pins to the bench surface plane with a shim cut from business card under the chip body.  They'll spring back up slightly, but if you didn't lift them by more than the wire diameter, the spring-back will leave them in close enough to the same plane, flat enough, and slightly below the chip body, so you wont have any problems hand soldering it

[sleemanj]

Or, just but a hot air station.  Most useful tool in the lab

  *  Desolders things
  *  Solders things
  *  Shrinks heatshrink
  *  Removes stringing from 3d prints
  *  Heats plastic for reforming
  *  Toasts ciabatta rolls
  *  Warms hands on bitterly cold mornings
  ...

[Peabody]

I'm very sorry to report that I received my new batch of Pro Minis today, and they too sleep at 150uA rather than the 1uA shown in the datasheet.  These use the same "the Simple" board as the others, but the processor markings are different.  Since I established before that no other part on the board was causing this problem, I have to conclude that the processor is the problem in all these cases.

So I have the situation where my older Mini, which uses the same board, and which has a date code marking of 2018, produces the correct 1uA sleep current, but all of the newer Minis, with date codes of 2019,  sleep at 150uA.  The test sketch, bootloader and  fuses are the same in all.

I have to conclude that either all of these newer processors are counterfeits, or there has been a change in the 328P that Microchip hasn't told us about.  In any case, unless somebody has a recent source of Minis that produce the 1uA sleep current with the power LED and regulator removed when running my test sketch, I'm going to have to give up on this and eliminate the Mini as a low-current option for battery power.  I don't know what to replace it with, except maybe the breadboard PU chip.

I'll reproduce my test code below in case anybody wants to try the same test on their Mini or their PU.  If anyone has another version of a test sketch that they think is more likely to produce the correct sleep current, please post it.  I'll be happy to try anything.

Also, if anyone knows how I might officially raise this issue with Microchip, please let me know.  I'm just a hobbyist, so there's no reason why they should talk to me.

I think this really sucks.

Code: [Select]

#include <avr/sleep.h>
#include <avr/wdt.h>
int i;

void setup(){

  for (i = 0; i < 20; i++) {          // all pins to one rail or the other - power saving
    pinMode(i,OUTPUT);
    digitalWrite(i,LOW);
  }
  ADCSRA = 0;                         // disable ADC for power saving
  wdt_disable();                      // disable WDT for power saving
  set_sleep_mode (SLEEP_MODE_PWR_DOWN); // Deep sleep
  sleep_enable();
  sleep_bod_disable();                // disable brownout detector during sleep
  sleep_cpu();                        // now go to sleep
}

void loop(){
}


[brucehoult]

That really sucks.

[Ian.M]

Unless you are willing to spend money, time and effort on swapping one of the AVRs with a genuine one from MicrochipDirect, or an official distributor that offers traceability back to Microchip, to see whether or not the genuine chip's sleep current is out of spec, you are pretty much at a dead end.   However if you do consistently find a traceably genuine chip is out of spec, Microchip support *WILL* (eventually) take an interest, even though you are a hobby user.  Register with Microchip's website, log in, then raise a support ticket at http://www.microchip.com/support/hottopics.aspx

If you conclude its just counterfeit chips they are less likely to be interested unless it helps them trace the counterfeits back to their source.   I would suggest posting good closeups of all affected chips you have to their forum, with details of your test methodology, then raising a support ticket and giving them the topic URL.  Its more difficult for them to ignore it on their own forum . . . .

OTOH if you don't want the hassle,  I think we'd all understand if you simply ordered a tube of PDIP ATmega328P chips from MicrochipDirect or a major official distributor and programmed them with your preferred bootloader configured for the 8MHz internal oscillator, and redesign your application for a PDIP AVR in place of a stripped down Arduino Pro Mini clone.

[DrG]

Did you see the last post on your AVRfreaks thread about explicitly turning off the comparator?

[sleemanj]

I too would suggest buying an "officially sourced" ATMega32P

I wonder if it's a 328 and not a 328P?  Did you test to see if brown-out-detection disabling actually works, that is, roll the voltage down and see if it detects brown out even when detection is supposedly disabled (by software, non-P can not have BOD disabled in software, so disable it in software and then test, if it still does a brown-out-reset you have a non-P 328)?  I vaguely recall that once upon a time I had some china source ATMega that were remarked as P when they were not P.

I actually just ordered some Pro Minis from Taobao, when they get here I'll check one out.

[Peabody]

Unless you are willing to spend money, time and effort on swapping one of the AVRs with a genuine one from MicrochipDirect, or an official distributor that offers traceability back to Microchip, to see whether or not the genuine chip's sleep current is out of spec, you are pretty much at a dead end.   However if you do consistently find a traceably genuine chip is out of spec, Microchip support *WILL* (eventually) take an interest, even though you are a hobby user.  Register with Microchip's website, log in, then raise a support ticket at http://www.microchip.com/support/hottopics.aspx

If you conclude its just counterfeit chips they are less likely to be interested unless it helps them trace the counterfeits back to their source.   I would suggest posting good closeups of all affected chips you have to their forum, with details of your test methodology, then raising a support ticket and giving them the topic URL.  Its more difficult for them to ignore it on their own forum . . . .

OTOH if you don't want the hassle,  I think we'd all understand if you simply ordered a tube of PDIP ATmega328P chips from MicrochipDirect or a major official distributor and programmed them with your preferred bootloader configured for the 8MHz internal oscillator, and redesign your application for a PDIP AVR in place of a stripped down Arduino Pro Mini clone.

If I could figure out which one to get, I guess I could order a few from Digikey.  But then I'm back to the problem of removing the existing chip from the board and soldering the new one.  Even better would be if I could find a 32FQPT breakout adapter.  Not likely. 

I've already started a thread on AVR Freaks, which I understand is the official forum for these parts.  I posted the markings, but not pictures.  But I get the feeling that there are no company participants there.

Yes, I could use the PDIP chip instead.  But of course that assumes I wouldn't have the same problem with them.  The Mini is just more convenient.

What concerns me is not the counterfeit issue, because that can be solved by getting genuine parts.  But if there's been a silicon revision that's causing this that's not reflected in the datasheet, then there's really no solution for that, and it would affect everything - Unos, Nanos , etc.

[Peabody]

Did you see the last post on your AVRfreaks thread about explicitly turning off the comparator?

Yes I did, and it had no effect.  The datasheet even says it has no effect in Power Down sleep.

[Peabody]

I too would suggest buying an "officially sourced" ATMega32P

I wonder if it's a 328 and not a 328P?  Did you test to see if brown-out-detection disabling actually works, that is, roll the voltage down and see if it detects brown out even when detection is supposedly disabled (by software, non-P can not have BOD disabled in software, so disable it in software and then test, if it still does a brown-out-reset you have a non-P 328)?  I vaguely recall that once upon a time I had some china source ATMega that were remarked as P when they were not P.

I actually just ordered some Pro Minis from Taobao, when they get here I'll check one out.

What I really need is an officially sourced Pro Mini.  I guess that would be Sparkfun.

It's not a 328.  I tested both the old  and new Minis with and without the BOD disable, and  both show a 20uA difference.  If I understand correctly, a 328 would have shown no difference because you can't disable BOD on a 328.

I don't know about Taobao, but it would be great if you could do that test.  You'd have to remove the power LED (actually removing the resistor is easier) and the regulator, then flash my test sketch.  Whatever the results, I'd like to see the markings on yours.

[DrG]

I too would suggest buying an "officially sourced" ATMega32P

I wonder if it's a 328 and not a 328P?  Did you test to see if brown-out-detection disabling actually works, that is, roll the voltage down and see if it detects brown out even when detection is supposedly disabled (by software, non-P can not have BOD disabled in software, so disable it in software and then test, if it still does a brown-out-reset you have a non-P 328)?  I vaguely recall that once upon a time I had some china source ATMega that were remarked as P when they were not P.

I actually just ordered some Pro Minis from Taobao, when they get here I'll check one out.

What I really need is an officially sourced Pro Mini.  I guess that would be Sparkfun.

It's not a 328.  I tested both the old  and new Minis with and without the BOD disable, and  both show a 20uA difference.  If I understand correctly, a 328 would have shown no difference because you can't disable BOD on a 328.

I don't know about Taobao, but it would be great if you could do that test.  You'd have to remove the power LED (actually removing the resistor is easier) and the regulator, then flash my test sketch.  Whatever the results, I'd like to see the markings on yours.

I re-read the entire thread this morning. No, I don't have *the* solution.

It seems to be that you have consistently demonstrated two low power setups, one bad at (again consistently) 150uA.

While not impossible, I think it is unlikely that there is an unknown board revision by Atmel that carries that kind of difference. I think it is unlikely because errata and revisions do get documented. At this point, I don't even think it is a counterfeit or bad batch of chips. Earlier markings had the board revisions marked and I can't figure out what is what largely because I lack patience - https://www.microchip.com/mymicrochip/Data/GBNG-15KQFZ896/pcn_gbng-15kqfz896_part%20marking%20guideline_part2.pdf but I think you have seen Revision D chips that were good and bad.

It smells like some peripheral is getting left on that was not left on earlier (likely correctable in software). If I am able to be be an objective onlooker it really does look that way because, as I said, you have demonstrated that they fall into two bins and each bin has a consistent low power consumption value.

Earlier you wrote "I was able to learn how to use Avrdude to read out the fuse settings and the bootloader code from the old and new Minis.  They are identical, and the bootloader is the stock bootloader.  So at least that settles that issue."

Revisiting the bootloader issue, did you actually determine that the bootloaders were identical or did you assume so because they were "stock"?

I don't know if the stock bootloaders - those that are included in the Arduino IDE have been revised. We know that there is an optiboot (with source code) that I think is separate from those stock bootloaders.

I would clone a good bootloader onto a bad board and test it that way - maybe that is just a PITA suggestion, but I think it would be a definitive test of that issue.

Buying a new Sparkfun board is fine - they revised the board and only offer the newer version. At least you have a solder blob instead of having to remove the regulator. Buying the chips from Microchip is also a good idea. In both cases, however, you are dealing with a bootloader - that *maybe* is relevant. There is so little on those board, especially after your removals, that there just has to be something that we are overlooking - or something that we are erroneously assuming.

[Peabody]

I re-read the entire thread this morning. No, I don't have *the* solution.

It seems to be that you have consistently demonstrated two low power setups, one bad at (again consistently) 150uA.

Well, I have one good old Mini at 1uA, and two new batches of bad Minis at 150uA.  The two new batches have different parts on the boards, and while the boards are the same, the silkscreens are different.  So it appears they were from two different production runs with parts sourced from different places.
 

While not impossible, I think it is unlikely that there is an unknown board revision by Atmel that carries that kind of difference. I think it is unlikely because errata and revisions do get documented. At this point, I don't even think it is a counterfeit or bad batch of chips. Earlier markings had the board revisions marked and I can't figure out what is what largely because I lack patience - https://www.microchip.com/mymicrochip/Data/GBNG-15KQFZ896/pcn_gbng-15kqfz896_part%20marking%20guideline_part2.pdf but I think you have seen Revision D chips that were good and bad.

I finally gave up on those markings guidelines too.  I just couldn't relate them to my chips. But yes, if the last letter of the third line is the revision, then both the old good chip and one of the new bad chips are both D. 

It smells like some peripheral is getting left on that was not left on earlier (likely correctable in software). If I am able to be be an objective onlooker it really does look that way because, as I said, you have demonstrated that they fall into two bins and each bin has a consistent low power consumption value.

I certainly hope it's something like that, because it would be fixable.  But of course that alone would prove there's been some kind of change.

Earlier you wrote "I was able to learn how to use Avrdude to read out the fuse settings and the bootloader code from the old and new Minis.  They are identical, and the bootloader is the stock bootloader.  So at least that settles that issue."

Revisiting the bootloader issue, did you actually determine that the bootloaders were identical or did you assume so because they were "stock"?

I read them out to hex files with Avrdude, then compared the files with my hex editor.  Also compared them to the stock bootloader for the Mini in the IDE.  All the files were identical.

I don't know if the stock bootloaders - those that are included in the Arduino IDE have been revised. We know that there is an optiboot (with source code) that I think is separate from those stock bootloaders.

I would clone a good bootloader onto a bad board and test it that way - maybe that is just a PITA suggestion, but I think it would be a definitive test of that issue.

I have done exactly that.  I burned the stock bootloader and fuses to one of the bad parts. Still 150uA.  I'll say again - the fuses, the bootloader and the sketch are exactly bit-for-bit the absolute same identical in the good and bad chips, and the fuses and bootloaders are the same as stock.  There is no difference.  The old chip works, but the new ones don't.  Unless all the new ones are counterfeits, there has been a revision.

Buying a new Sparkfun board is fine - they revised the board and only offer the newer version. At least you have a solder blob instead of having to remove the regulator. Buying the chips from Microchip is also a good idea. In both cases, however, you are dealing with a bootloader - that *maybe* is relevant. There is so little on those board, especially after your removals, that there just has to be something that we are overlooking - or something that we are erroneously assuming.

I don't want to get in the habit of paying four times as much for Sparkfun Minis.  And of course if I ordered one, and it worked, but had a 2018 date marker, that tells me nothing.  I have sent email to Digikey asking if they would be able to send me 328P-AU parts with recent date codes.  If they can, and if I can get somebody to switch out the chips on the board for me, that would settle the matter I think.  If that works, then these others probably are counterfeits, but at least I would have a source for good chips.  If it does the 150uA thing, then I think I'm screwed unless there's a software fix.

[DrG]

I re-read the entire thread this morning. No, I don't have *the* solution.

It seems to be that you have consistently demonstrated two low power setups, one bad at (again consistently) 150uA.

Well, I have one good old Mini at 1uA, and two new batches of bad Minis at 150uA.  The two new batches have different parts on the boards, and while the boards are the same, the silkscreens are different.  So it appears they were from two different production runs with parts sourced from different places.
 

While not impossible, I think it is unlikely that there is an unknown board revision by Atmel that carries that kind of difference. I think it is unlikely because errata and revisions do get documented. At this point, I don't even think it is a counterfeit or bad batch of chips. Earlier markings had the board revisions marked and I can't figure out what is what largely because I lack patience - https://www.microchip.com/mymicrochip/Data/GBNG-15KQFZ896/pcn_gbng-15kqfz896_part%20marking%20guideline_part2.pdf but I think you have seen Revision D chips that were good and bad.

I finally gave up on those markings guidelines too.  I just couldn't relate them to my chips. But yes, if the last letter of the third line is the revision, then both the old good chip and one of the new bad chips are both D. 

It smells like some peripheral is getting left on that was not left on earlier (likely correctable in software). If I am able to be be an objective onlooker it really does look that way because, as I said, you have demonstrated that they fall into two bins and each bin has a consistent low power consumption value.

I certainly hope it's something like that, because it would be fixable.  But of course that alone would prove there's been some kind of change.

Earlier you wrote "I was able to learn how to use Avrdude to read out the fuse settings and the bootloader code from the old and new Minis.  They are identical, and the bootloader is the stock bootloader.  So at least that settles that issue."

Revisiting the bootloader issue, did you actually determine that the bootloaders were identical or did you assume so because they were "stock"?

I read them out to hex files with Avrdude, then compared the files with my hex editor.  Also compared them to the stock bootloader for the Mini in the IDE.  All the files were identical.

I don't know if the stock bootloaders - those that are included in the Arduino IDE have been revised. We know that there is an optiboot (with source code) that I think is separate from those stock bootloaders.

I would clone a good bootloader onto a bad board and test it that way - maybe that is just a PITA suggestion, but I think it would be a definitive test of that issue.

I have done exactly that.  I burned the stock bootloader and fuses to one of the bad parts. Still 150uA.  I'll say again - the fuses, the bootloader and the sketch are exactly bit-for-bit the absolute same identical in the good and bad chips, and the fuses and bootloaders are the same as stock.  There is no difference.  The old chip works, but the new ones don't.  Unless all the new ones are counterfeits, there has been a revision.

Buying a new Sparkfun board is fine - they revised the board and only offer the newer version. At least you have a solder blob instead of having to remove the regulator. Buying the chips from Microchip is also a good idea. In both cases, however, you are dealing with a bootloader - that *maybe* is relevant. There is so little on those board, especially after your removals, that there just has to be something that we are overlooking - or something that we are erroneously assuming.

I don't want to get in the habit of paying four times as much for Sparkfun Minis.  And of course if I ordered one, and it worked, but had a 2018 date marker, that tells me nothing.  I have sent email to Digikey asking if they would be able to send me 328P-AU parts with recent date codes.  If they can, and if I can get somebody to switch out the chips on the board for me, that would settle the matter I think.  If that works, then these others probably are counterfeits, but at least I would have a source for good chips.  If it does the 150uA thing, then I think I'm screwed unless there's a software fix.

*drat*  I am completely and totally out of ideas. You have checked everything that I can think of and checked things that others thought of that I didn't think of and all that is left is....the correct answer. *drat*

[sleemanj]

Out of curiosity, more than any particular idea.

Would you like to try my SimpleSleep Arduino library

  https://github.com/sleemanj/SimpleSleep

Code: [Select]

#include <SimpleSleep.h>

SimpleSleep Sleep;

void setup()
{

}

void loop()
{

  for (i = 0; i < 20; i++) {          // all pins to one rail or the other - power saving
    pinMode(i,OUTPUT);
    digitalWrite(i,LOW);
  }

  Sleep.forever();
}

It should behave the same as your test code I think and produce the same results, but worth a shot.

[SilverSolder]

[...]

I don't want to get in the habit of paying four times as much for Sparkfun Minis.  And of course if I ordered one, and it worked, but had a 2018 date marker, that tells me nothing.  I have sent email to Digikey asking if they would be able to send me 328P-AU parts with recent date codes.  If they can, and if I can get somebody to switch out the chips on the board for me, that would settle the matter I think.  If that works, then these others probably are counterfeits, but at least I would have a source for good chips.  If it does the 150uA thing, then I think I'm screwed unless there's a software fix.

The picture of the "Good" processor is incorrect in that post (forum has a picture attachment bug that you may have fallen foul of)

[Peabody]

The picture of the "Good" processor is incorrect in that post (forum has a picture attachment bug that you may have fallen foul of)

I'd like to know more about the attachment bug, but the pictures look right when I click on them.  The good chip is the one with the 18...  date code at the bottom, and that's what I see when I click on it.

[Peabody]

Out of curiosity, more than any particular idea.

Would you like to try my SimpleSleep Arduino library

  https://github.com/sleemanj/SimpleSleep

Code: [Select]

#include <SimpleSleep.h>

SimpleSleep Sleep;

void setup()
{

}

void loop()
{

  for (i = 0; i < 20; i++) {          // all pins to one rail or the other - power saving
    pinMode(i,OUTPUT);
    digitalWrite(i,LOW);
  }

  Sleep.forever();
}

It should behave the same as your test code I think and produce the same results, but worth a shot.

Sure.  I'll give it a shot tomorrow.

[ADI8421]

Hi, I've skipped a few posts here so may have missed a solution, but here's my ideas:

I noticed your pro mini arduino board has a 4 ohm series resistor in the RAW power supply line. My identical boards have a 5 ohm.
There is also a diode in the same line for reverse supply protection. I trust you have bypassed these 2 components ?
What about the supplies for both circuits ? Do they use the same DC supply, including the same On/Off switch. Switch contact resistances can vary a lot. I would also remove all electro caps too.

[Peabody]

The picture of the "Good" processor is incorrect in that post (forum has a picture attachment bug that you may have fallen foul of)

I went back again and checked it this morning, and it had replaced the correct image that had been there before with a schematic that I had posted in another thread.  So I've deleted the schematic, then attached the correct picture again.  It all looks ok to me now, but that's not to say it won't get messed up again.  Is there any information on what leads to this bug or what can be done to avoid it?

[Peabody]

Out of curiosity, more than any particular idea.

Would you like to try my SimpleSleep Arduino library

  https://github.com/sleemanj/SimpleSleep

Code: [Select]

#include <SimpleSleep.h>

SimpleSleep Sleep;

void setup()
{

}

void loop()
{

  for (i = 0; i < 20; i++) {          // all pins to one rail or the other - power saving
    pinMode(i,OUTPUT);
    digitalWrite(i,LOW);
  }

  Sleep.forever();
}

It should behave the same as your test code I think and produce the same results, but worth a shot.

I tried your library and sketch, and the Mini still sleeps at 150uA.

[drvtech]

I know it's not much help but I just put a couple of chips onto bare boards (with a single decoupling cap) and they sleep at around 0.2uA according to my trusty EEVBlog 121GW meter.
The exact chip marking details are:

Mega328P U-TW
354730
1825017
Sleeps at around 0.15uA (Yes, 150nA)

Mega328PB-U
1824D TW
1824TWY
Sleeps at 0.2uA

I had to recheck the readings as they seemed unrealistically low but there is definitely a mu symbol and not an m on the display.
I added a few lines of code to send a squarewave out to one pin for 10 seconds before it goes to sleep just so I could check that I was really reading the current and hadn't forgotten to connect something! They draw about 7 mA before they go to sleep.

The boards don't have a crystal so the chips were running from the internal 8MHz oscillator
These chips were bought from RS components so they will be genuine.

As I said, it's not much help but it might serve as another data point. I guess that the 1824 might indicate a 2018 manufacturing date.

[SilverSolder]

What happens to processors that are rejected at the Atmel factory for drawing too much current during sleep....   ?   

[Peabody]

Thanks very much for that report.  Yes, 1824 is the 24th week of 2018,  And all chips of that age appear to work just fine.  My "good" chip is 1829, and Kevin Darrah has confirmed that he has chips from Digikey with the 1829 date code, and they also sleep at under 1uA.

All the chips I got in my two recent Pro Mini clone orders are dated between 1936 and 1942, and they all sleep at 150uA.  So the question is - are the new ones counterfeits, or are the new ones genuine Microchip parts, and by implication, has Microchip screwed up the 328P?  Kevin is going to try to get *current* product direct from Microchip, just to eliminate any possibility of a counterfeit, but if he isn't successful, Digikey has confirmed to me that they have stock dated March, 2020, so I can get one there if needed.

I think there's a good bit at stake here.  If Microchip has been pumping out bad parts since mid-2019, they've gone into  every Pro Mini, Uno, Nano, etc, etc, that's been made since then.  The sleep current issue may not be important in most usage cases, but any design that's battery powered, and uses sleep mode to minimize power consumption, would be affected.  And of course we don't know what else in the chip may not function correctly.  So I'm kinda hoping my chips are counterfeit.

Anybody on here have more recent 328P-AU parts on hand that they know are genuine?

[DrG]

Thanks very much for that report.  Yes, 1824 is the 24th week of 2018,  And all chips of that age appear to work just fine.  My "good" chip is 1829, and Kevin Darrah has confirmed that he has chips from Digikey with the 1829 date code, and they also sleep at under 1uA.

All the chips I got in my two recent Pro Mini clone orders are dated between 1936 and 1942, and they all sleep at 150uA.  So the question is - are the new ones counterfeits, or are the new ones genuine Microchip parts, and by implication, has Microchip screwed up the 328P?  Kevin is going to try to get *current* product direct from Microchip, just to eliminate any possibility of a counterfeit, but if he isn't successful, Digikey has confirmed to me that they have stock dated March, 2020, so I can get one there if needed.

I think there's a good bit at stake here.  If Microchip has been pumping out bad parts since mid-2019, they've gone into  every Pro Mini, Uno, Nano, etc, etc, that's been made since then.  The sleep current issue may not be important in most usage cases, but any design that's battery powered, and uses sleep mode to minimize power consumption, would be affected.  And of course we don't know what else in the chip may not function correctly.  So I'm kinda hoping my chips are counterfeit.

Anybody on here have more recent 328P-AU parts on hand that they know are genuine?

I have been following this thread and the one on avrfreaks. I was really happy to hear that you made contact with Kevin Darrah  https://www.kevindarrah.com/wiki/index.php?title=Projects: He seems very well suited to investigate the issue(s) and I am looking forward to hearing what he can uncover.

At this point,I am just going to wait before speculating too much on counterfeits or, for lack of a better term, manufacturing flaws. If there is a systematic problem due to either, it will be uncovered.

[David Hess]

It would not be the first time that parts with a high out of specification quiescent current were accidentally sold.  In most applications it would not be noticed.

[Peabody]

I have been following this thread and the one on avrfreaks. I was really happy to hear that you made contact with Kevin Darrah  https://www.kevindarrah.com/wiki/index.php?title=Projects: He seems very well suited to investigate the issue(s) and I am looking forward to hearing what he can uncover.

At this point,I am just going to wait before speculating too much on counterfeits or, for lack of a better term, manufacturing flaws. If there is a systematic problem due to either, it will be uncovered.

I contacted Kevin because he is known for his ultra-low-power stuff.  And I agree that he is better suited than I to take this to a conclusion one way or another.  In the end, I just want to find a source of Arduino Pro Mini clones that sleep properly.  I hope that's still possible.

[DrG]

I have been following this thread and the one on avrfreaks. I was really happy to hear that you made contact with Kevin Darrah  https://www.kevindarrah.com/wiki/index.php?title=Projects: He seems very well suited to investigate the issue(s) and I am looking forward to hearing what he can uncover.

At this point,I am just going to wait before speculating too much on counterfeits or, for lack of a better term, manufacturing flaws. If there is a systematic problem due to either, it will be uncovered.

I contacted Kevin because he is known for his ultra-low-power stuff.  And I agree that he is better suited than I to take this to a conclusion one way or another.  In the end, I just want to find a source of Arduino Pro Mini clones that sleep properly.  I hope that's still possible.

Yeah, I hear ya. At this point, I just don't think it is something silly and that is what gets me so interested. We shall see - probably

[Peabody]

Another interesting tidbit.  Here's a recent Ralph Bacon video about using a DS3231 to wake up a 328P-PU breadboard Arduino.



At 6:10 into the video he puts the processor to sleep, then disconnects power to the RTC, and measures the resulting sleep current.  He gets 150uA.  And of course that's exactly what I'm getting on the new AU parts.

I've written to him asking about the date code on the PU chip, and asking if he can repeat the test with an older PU.

I would just say it's beginning to look like all 328P parts may now have this flaw.

Kevin Darrah has ordered new stock from Digikey, and will test it on arrival.  So we shall see pretty soon now.  But I thought the Ralph Bacon thing was interesting.

[SilverSolder]

Have you tested ATmega328PB as an alternative?

[Peabody]

Have you tested ATmega328PB as an alternative?

No, I have no experience with the PB.

[westfw]

I noticed your pro mini arduino board has a 4 ohm series resistor in the RAW power supply line. My identical boards have a 5 ohm.

I'm pretty sure that that's a polyfuse, not a resistor...

[brucehoult]

Annoys me he keeps referring to a bare ATMega328 as an "Arduino". Too picky? Two thirds of my boards that use the Arduino API have cpus other than AVR.

[sleemanj]

As another data point, just tested in my test socket PCB I made a few years ago and forgot that I had....

 1933 date code pulled from a Chinese Nano sourced in the last 6-8 months sometime power down mode circa 150uA, idle current ( delay(...) ) 3.3mA

 1648 date code from some Chinese supplier a few years ago, power down mode less than 1uA, idle current 3.6mA

This is with the exact hardware, exact firmware, literally pulled one chip out of the socket put the next one in.  Verified with two meters, unfortunately I don't have a meter than can give me the nano amps for the 1648 coded one.

[Peabody]

Thanks very much.  So far it seems that 2019 date codes sleep at 150uA, but older ones sleep at less than 1uA.  But we still don't know if known genuine Microchip parts also follow that pattern.  Should know more about that next week.  The latest date code I have from a Chinese source that sleeps at 1uA is 1829.

[SilverSolder]

The picture of the "Good" processor is incorrect in that post (forum has a picture attachment bug that you may have fallen foul of)

I went back again and checked it this morning, and it had replaced the correct image that had been there before with a schematic that I had posted in another thread.  So I've deleted the schematic, then attached the correct picture again.  It all looks ok to me now, but that's not to say it won't get messed up again.  Is there any information on what leads to this bug or what can be done to avoid it?

There is a workaround listed here:  https://www.eevblog.com/forum/chat/website-error-reports/msg3151250/#msg3151250

The surrounding thread has been talking about the bug for a while.

[David Hess]

How could such a complex integrated circuit be cloned?  Could they have gained access to the masks or would this be another case of rejected chips being remarked and sold?

[hans]

An ATMEGA328P is not a complex chip for todays standards. Digital descriptions are easily transferred by a common format such as VHDL/Verilog, which can be synthesized for FPGAs (as soft-core, which is often done by FPGA community) or silicon. There are lots of community clones of AVR cores for FPGA's (take a look on github or opencores.org, which can be repurposed for own silicon. Cloners probably only need to add analog peripherals and some missing digital stuff, and then package it.

Analog performance of clones is often not as good, in particular ADC/voltage reference performance (e.g. less accurate, more noise) and power consumption (often higher). I suspect that cloners are not concerned that "sleep" functions only copy behaviour, but not the specifications..

[robca]

How could such a complex integrated circuit be cloned?  Could they have gained access to the masks or would this be another case of rejected chips being remarked and sold?

At the moment there are 6 Chinese clones of the STM32F103, each subtly different but broadly compatible. It is causing countless problems with people buying Blue Pills. Some chips (like the GD32) can actually be considered even better than the original, as they can be clocked higher and have a faster onboard flash

Granted, that chip is based on a M3 ARM core, so at least some part might be easier to clone, but it's a significantly more complex chip than an Atmega32p

The real question to me is "why clone a chip that is getting old and costs so little". But the Atmega 328p is still pretty ubiquitous and saving cents probably matters to the OEMs selling cheap gizmos on eBay/Alibaba

[brucehoult]

At the moment there are 6 Chinese clones of the STM32F103, each subtly different but broadly compatible. It is causing countless problems with people buying Blue Pills. Some chips (like the GD32) can actually be considered even better than the original, as they can be clocked higher and have a faster onboard flash

That appears to be not quite right :-)

It seem certain from die shots that the GD32 and GD32V (RISC-V) chips have VERY slow and cheap flash on a separate die in the package and copy it to onboard SRAM on startup.

That explains:

- why they can be clocked so much higher

- why the GD32 & GD32V take ~130 ms to come out of reset (and 120 ms to wake from the 7 uA standby mode vs 6 us from the 400 uA "deep sleep"), while STM32F103 takes 50 us to come out of the 2 uA standby mode and 5 us from the 13 uA stop mode. The GD32 is copying your code from flash to SRAM after reset or standby.

[robca]

At the moment there are 6 Chinese clones of the STM32F103, each subtly different but broadly compatible. It is causing countless problems with people buying Blue Pills. Some chips (like the GD32) can actually be considered even better than the original, as they can be clocked higher and have a faster onboard flash

That appears to be not quite right :-)

It seem certain from die shots that the GD32 and GD32V (RISC-V) chips have VERY slow and cheap flash on a separate die in the package and copy it to onboard SRAM on startup.

That explains:

- why they can be clocked so much higher

- why the GD32 & GD32V take ~130 ms to come out of reset (and 120 ms to wake from the 7 uA standby mode vs 6 us from the 400 uA "deep sleep"), while STM32F103 takes 50 us to come out of the 2 uA standby mode and 5 us from the 13 uA stop mode. The GD32 is copying your code from flash to SRAM after reset or standby.

Yes, I did oversimplify to avoid hijacking the thread (your explanation is the correct one, though). By using SRAM, de facto they have what the user perceives as "faster flash". Even without overclocking, a GD32 can fetch instructions from "flash" faster than an STM32 can (because it's not flash ). The tradeoff is the slow boot and wake up time. But there are valid scenarios where their design decision makes a lot of sense. My main point was that the GD32 is one of the better clones (including better docs than most other clones), but still not an STM32F103

[phil from seattle]

How could such a complex integrated circuit be cloned?  Could they have gained access to the masks or would this be another case of rejected chips being remarked and sold?

At the moment there are 6 Chinese clones of the STM32F103, each subtly different but broadly compatible. It is causing countless problems with people buying Blue Pills. Some chips (like the GD32) can actually be considered even better than the original, as they can be clocked higher and have a faster onboard flash

Granted, that chip is based on a M3 ARM core, so at least some part might be easier to clone, but it's a significantly more complex chip than an Atmega32p

The real question to me is "why clone a chip that is getting old and costs so little". But the Atmega 328p is still pretty ubiquitous and saving cents probably matters to the OEMs selling cheap gizmos on eBay/Alibaba

This is why I have decided not to do a product based on the Blue (or Black) Pill.  Too much variability to be able to say "buy X and install it".  Even a given merchant selling them over time never quite knows what they actually have at any point. I'll stick with the teensys - at least you get a guaranteed product every time.

And, yes, the GD F103 does appear to be quite a bit faster than the original.  At least they don't claim to have a real ST32F103...

[gamblingfetus]

• 354A3P - 5V 16 MHz version
• 35473D - 3.3V 8 MHz version

I bought both versions intentionally here.

[Peabody]

• 354A3P - 5V 16 MHz version
• 35473D - 3.3V 8 MHz version

I bought both versions intentionally here.

I have one exactly like your first one - same date code and everything.  I also have one like your second one except the date code is earlier - 1936.  Both of mine sleep at 150uA at 3.3V.  By the way, I don't think the processor itself is necessarily different for 3.3V and 5V.  They would be flashed with different bootloaders, and possibly have different fuse settings, but I think all the 328Ps will work fine at either voltage.

Well, the only way I know of to test the sleep current is to remove the voltage regulator and power-on LED, which kinda ruins the Mini if that's not the way you intend to use them.  So I'm not going to suggest you do that.  Besides, at this point I think the test that counts is the one Kevin will do on a known-genuine 328P that he should receive this week from Digikey.

[sleemanj]

My Pro-Minis from Taobao finally arrived (shipping PITA currently), date code 2002.  I won't pull a chip off yet to test but will hold onto a couple in case we still need more data.

[Peabody]

My Pro-Minis from Taobao finally arrived (shipping PITA currently), date code 2002.  I won't pull a chip off yet to test but will hold onto a couple in case we still need more data.

I'm not good at SMD rework stuff.  So what I usually do is remove the regulator and the power LED's resistor, leaving only the processor to draw current.  But that's also how I use Minis, so it all works out.  Anyway, Kevin sent me a picture of the 328P he received from Digikey, also with a date code of 2002.  Look anything like yours?

[sleemanj]

I'd have to check the  fab code in the morning but think it's the same from memory.  Did he test it?

[Peabody]

He's working on it today, but I don't know when to expect any news.  I think he plans to make a video.

[sleemanj]

Markings on mine are the same except 2002CJE instead of his 2002G1U

[Peabody]

Ok, I just got email from Kevin saying the new chip he got from Digikey sleeps properly - the same as his old 2016 stock.  It would sure be interesting to know how your new chips sleep.  If they sleep properly, then I guess all of mine were counterfeits.  Either that or Microchip produced a string of bad chips that they've since corrected.  I don't really know what to think.  But at least it's possible to replace bad chips with good ones if sleep current matters.

[SilverSolder]

Ok, I just got email from Kevin saying the new chip he got from Digikey sleeps properly - the same as his old 2016 stock.  It would sure be interesting to know how your new chips sleep.  If they sleep properly, then I guess all of mine were counterfeits.  Either that or Microchip produced a string of bad chips that they've since corrected.  I don't really know what to think.  But at least it's possible to replace bad chips with good ones if sleep current matters.

It is also possible that Microchip baked a batch of chips that work fine, but fail the sleep current test, which somehow found their way out into the wild?   In a weird sort of way, these "bad" chips are actually acceptable for use on an Arduino Mini because other devices on that board use more current anyway, so unless you modify the board nobody would ever know...   Using them this way keeps the prices down and keeps usable chips out of the landfill.   Conspiracy theory mode ON! 

[Peabody]

sleemanj, could you tell me what the location code is on your date code 1933 chip that sleeps at 150uA?  It will be KR, TW, TH or PH.  I"m testing the theory that all the bad ones are either KR or TW, but all THs are good.

[DrG]

Ok, I just got email from Kevin saying the new chip he got from Digikey sleeps properly - the same as his old 2016 stock.  It would sure be interesting to know how your new chips sleep.  If they sleep properly, then I guess all of mine were counterfeits.  Either that or Microchip produced a string of bad chips that they've since corrected.  I don't really know what to think.  But at least it's possible to replace bad chips with good ones if sleep current matters.

It is also possible that Microchip baked a batch of chips that work fine, but fail the sleep current test, which somehow found their way out into the wild?   In a weird sort of way, these "bad" chips are actually acceptable for use on an Arduino Mini because other devices on that board use more current anyway, so unless you modify the board nobody would ever know...   Using them this way keeps the prices down and keeps usable chips out of the landfill.   Conspiracy theory mode ON! 

First off, this is all completely speculative at this point. We all seem to realize that but it merits additional mentioning. That being said, I'm sorry, but  what you are saying just does not pass the smell test.

Microchip (or Atmel) puts out specifications. If, for example, the chip does not follow the specification for typical characteristics e.g., 29.1.2   Power-down Supply Current http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-7810-Automotive-Microcontrollers-ATmega328P_Datasheet.pdf BY A LARGE MARGIN, like the differences reported in the thread, it is a bad specification, and would indicate, it seems to me, poor quality control, period.

To think that Microchip (Atmel) would know that some batches of chips are OK except for failing these specifications (whatever they might be), put their marking on it, and sell them, but only to folks who promise to only use them in Pro Minis clones (or some specific clone product only), strikes me as being bizarre. Even more so since a counterfeit chip would explain the situation in a far simpler and more reasonable manner.

Reminds me of years ago, Radio Shack selling quantities of 3055 transistors that were not marked (or had uninterruptible uninterpretable markings) - everyone knew they were "seconds", but they sure did not carry legitimate manufacturer marks as I remember

But, I don't know what the story will turn out to be.

BTW: I still have those 3055s 

[SilverSolder]

Ok, I just got email from Kevin saying the new chip he got from Digikey sleeps properly - the same as his old 2016 stock.  It would sure be interesting to know how your new chips sleep.  If they sleep properly, then I guess all of mine were counterfeits.  Either that or Microchip produced a string of bad chips that they've since corrected.  I don't really know what to think.  But at least it's possible to replace bad chips with good ones if sleep current matters.

It is also possible that Microchip baked a batch of chips that work fine, but fail the sleep current test, which somehow found their way out into the wild?   In a weird sort of way, these "bad" chips are actually acceptable for use on an Arduino Mini because other devices on that board use more current anyway, so unless you modify the board nobody would ever know...   Using them this way keeps the prices down and keeps usable chips out of the landfill.   Conspiracy theory mode ON! 

First off, this is all completely speculative at this point. We all seem to realize that but it merits additional mentioning. That being said, I'm sorry, but  what you are saying just does not pass the smell test.

Microchip (or Atmel) puts out specifications. If, for example, the chip does not follow the specification for typical characteristics e.g., 29.1.2   Power-down Supply Current http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-7810-Automotive-Microcontrollers-ATmega328P_Datasheet.pdf BY A LARGE MARGIN, like the differences reported in the thread, it is a bad specification, and would indicate, it seems to me, poor quality control, period.

To think that Microchip (Atmel) would know that some batches of chips are OK except for failing these specifications (whatever they might be), put their marking on it, and sell them, but only to folks who promise to only use them in Pro Minis clones (or some specific clone product only), strikes me as being bizarre. Even more so since a counterfeit chip would explain the situation in a far simpler and more reasonable manner.

Reminds me of years ago, Radio Shack selling quantities of 3055 transistors that were not marked (or had uninterruptible markings) - everyone knew they were "seconds", but they sure did not carry legitimate manufacturer marks as I remember

But, I don't know what the story will turn out to be.

I agree with you, I don't think Atmel would have sold them as anything other than scrap...   unmarked scrap, even.  -  But someone might pay them reasonably good money for the scrap...   like Radio Shack buying the 2N3055 seconds that couldn't be sold as the "real thing".   Except in this case, they couldn't avoid marking the components so people would know what it is...

The only thing I have against this kind of e-cycling is that they should be up front about it.  I've always wondered how they can sell an Arduino Mini for less than I can buy a processor for...   this kind of thing would be one explanation.

[All of this is purely speculative idle chatter, of course!]

[sleemanj]

sleemanj, could you tell me what the location code is on your date code 1933 chip that sleeps at 150uA?

The 1933 is TW

I pulled the 2002 from one of my Pro Minis and it sleeps correctly under 1uA.

So I think one of

  1. some fabs have an issue which had not been picked up and they were sold as production normal - seems unlikely
  2. some fabs had an issue which was picked up and instead of the batch going into the trash somebody walked it out the back door and it wound up in the chinese market - this seems most feasible to me
  3. counterfeit - I don't think this is likely given I've just bought pro-minis (and nanos too) with seemingly legit chips

[Peabody]

sleemanj, could you tell me what the location code is on your date code 1933 chip that sleeps at 150uA?

The 1933 is TW

I pulled the 2002 from one of my Pro Minis and it sleeps correctly under 1uA.

So I think one of

  1. some fabs have an issue which had not been picked up and they were sold as production normal - seems unlikely
  2. some fabs had an issue which was picked up and instead of the batch going into the trash somebody walked it out the back door and it wound up in the chinese market - this seems most feasible to me
  3. counterfeit - I don't think this is likely given I've just bought pro-minis (and nanos too) with seemingly legit chips

That's great news about the 2002 Pro Mini.  And that's a TH fab.  So at least it's still possible to get Pro Minis that work right so long as you buy from the right place.  Speaking of which, could you post a link to where you bought yours?

I've always thought the counterfeit explanation was unlikely.  The 328P has been around a long time, and I've been unable to find a single confirmed case of a fake.  Of course it could be that the pirates finally got around to it, but I'm still skeptical.

Kevin has posted his video:

https://youtu.be/PlGycKwnsSw

[DrG]

Just got done watching the video. He did a good job. I think he showed, unequivocally, that your "bad" pro mini was indeed "bad". He moved it off of the pro mini board and onto his boards (with his boot loader), after demonstrating the low current sleep, and he can't get your chip to show that low current sleep. In fact, he found pretty much what you found. That's all good and convincing.

He does not, however, show any evidence that the bad chip is either genuine or counterfeit. Regardless of what he (or any of us) suspects, there is no real evidence [I am discounting the labeling difference at this point] of it being genuine or not.

[sleemanj]

Speaking of which, could you post a link to where you bought yours?

This is the supplier I got them from

  https://item.taobao.com/item.htm?id=571250487735

if you're not experienced with buying from Taobao, well, this isn't really the thread to explain it but probably spreenow is the "easiest" agent to buy through.

[sleemanj]

He does not, however, show any evidence that the bad chip is either genuine or counterfeit.

Well, there's really no way to know, short of decapping it (and even then not necessarily conclusive unless you are Microchip).

That said, if somebody here wants to decap and has an appropriate metallurgical microscope to inspect and photograph for the forum I'm more than happy to post a good and bad one to them (of course currently post from NZ to anywhere else in the world might take a while!).

[Psi]

They're probably more likely to be factory rejects than counterfeits.

However it would be cool to remove the case and compare the die to a legit chip

[westfw]

The 328P has been around a long time, and I've been unable to find a single confirmed case of a fake.

There's the LGT8f328.  It's not an exact clone of an AVR, but it has most of the "hard" parts, and I think it would be pretty easy to spin a version that was more compatible (essentially replacing the SWD-like programming interface with the Atmel SPI-based programming.  Especially at the rate that Asia seems to counterfeit themselves.

It might be interesting to poke at one of the misbehaving chips with a debugger, and see if it seems to support the full debugwire protocols (which are more secretive than the rest of the AVR architecture.  Though it has been reverse-engineered.)

[Psi]

I have seen a few AVRs permanently behave strange after having lots of programming issues.   
Like running at odd clock speeds that shouldn't be possible.
I figured it was secret undocumented fuse bits getting corrupt.


But i still think it's more likely there is a difference on the PCB to account for the additional current. Something non-obvious.

Have you tried putting the chip into sleep mode and then measuring the voltage on each IC pin with a scope. Then compare to a chip without the issue. 

[sleemanj]

But i still think it's more likely there is a difference on the PCB to account for the additional current..

That is not the case.  See my tests which confirmed the behaviour, I drop each IC into my socketed test board, no other change (also, nothing connected to any pins except for power, ground).

[Ian.M]

Ok, I just got email from Kevin saying the new chip he got from Digikey sleeps properly - the same as his old 2016 stock.  It would sure be interesting to know how your new chips sleep.  If they sleep properly, then I guess all of mine were counterfeits.  Either that or Microchip produced a string of bad chips that they've since corrected.  I don't really know what to think.  But at least it's possible to replace bad chips with good ones if sleep current matters.

It is also possible that Microchip baked a batch of chips that work fine, but fail the sleep current test, which somehow found their way out into the wild?   In a weird sort of way, these "bad" chips are actually acceptable for use on an Arduino Mini because other devices on that board use more current anyway, so unless you modify the board nobody would ever know...   Using them this way keeps the prices down and keeps usable chips out of the landfill.   Conspiracy theory mode ON! 

First off, this is all completely speculative at this point. We all seem to realize that but it merits additional mentioning. That being said, I'm sorry, but  what you are saying just does not pass the smell test.

Microchip (or Atmel) puts out specifications. If, for example, the chip does not follow the specification for typical characteristics e.g., 29.1.2   Power-down Supply Current http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-7810-Automotive-Microcontrollers-ATmega328P_Datasheet.pdf BY A LARGE MARGIN, like the differences reported in the thread, it is a bad specification, and would indicate, it seems to me, poor quality control, period.

To think that Microchip (Atmel) would know that some batches of chips are OK except for failing these specifications (whatever they might be), put their marking on it, and sell them, but only to folks who promise to only use them in Pro Minis clones (or some specific clone product only), strikes me as being bizarre. Even more so since a counterfeit chip would explain the situation in a far simpler and more reasonable manner.

Reminds me of years ago, Radio Shack selling quantities of 3055 transistors that were not marked (or had uninterruptible markings) - everyone knew they were "seconds", but they sure did not carry legitimate manufacturer marks as I remember

But, I don't know what the story will turn out to be.

I agree with you, I don't think Atmel would have sold them as anything other than scrap...   unmarked scrap, even.  -  But someone might pay them reasonably good money for the scrap...   like Radio Shack buying the 2N3055 seconds that couldn't be sold as the "real thing".   Except in this case, they couldn't avoid marking the components so people would know what it is...

The only thing I have against this kind of e-cycling is that they should be up front about it.  I've always wondered how they can sell an Arduino Mini for less than I can buy a processor for...   this kind of thing would be one explanation.

[All of this is purely speculative idle chatter, of course!]

To add to the speculation, I noted way back at reply#20

A decade or so back, Microchip Technology used to do special far east market versions of some of their PICs that didn't meet the US datasheet specs, for really cost-sensitive applications.   They were supposedly marked with distinct part numbers, but we all know how that one goes when there are no westerners 'in the loop' . . .  Maybe they've started doing similar with AVRs?

Short of decapping and side by side die inspection of a high sleep current chip and a low sleep current genuine chip straight from MicrochipDirect, it looks like we aren't going to get any answers.  If it is a batch problem and Microchip ever do respond with an errata update, odds are it will be months or even years away.

[Psi]

But i still think it's more likely there is a difference on the PCB to account for the additional current..

That is not the case.  See my tests which confirmed the behaviour, I drop each IC into my socketed test board, no other change (also, nothing connected to any pins except for power, ground).

hm..  i'm having some design issues with a product im making that uses a ATMEGA328P, not related to sleep current or the chip, but thread this is prompting me to change the chip to something less popular to avoid potentially getting fakes in the future.

[DrG]

But i still think it's more likely there is a difference on the PCB to account for the additional current..

That is not the case.  See my tests which confirmed the behaviour, I drop each IC into my socketed test board, no other change (also, nothing connected to any pins except for power, ground).

Yes, the tests run in the video as well as your tests have convinced me that the phenomenon is a real one. Those efforts, as well as those of the OP to bring the issue to light in the first place are definitely a contribution and it is an interesting.

Comments after the video also suggest decapping as, potentially, a definitive test.

The other issue that has been mentioned here and in the comments of the video, suggest this idea of factory rejects finding there way onto the boards. I just don't have the experience to know if this really does happen to any extent and while it seems at least plausible, I have not seen any documentation that something like that happened, just anecdotal evidence.

I would hope (with more of a sense that it would be nice if things worked that way, than any evidence) that some other characteristics of the "bad" chips would be found to be different - something like NOT turning off a peripheral, as being responsible.

On the other hand, it seems like there is a specific time niche that these "bad" chips were made, based on some markings - no?

Why would it be happening with only that package of the 328P (at least so far)? Why not the DIP package, for example? Is there a reasonable explanation for that?

[Ian.M]

hm..  i'm having some design issues with a product im making that uses a ATMEGA328P, not related to sleep current or the chip, but thread this is prompting me to change the chip to something less popular to avoid potentially getting fakes in the future.

Order preprogrammed ATMEGA328P chips from MicrochipDirect, with all security lock bits programmed, drop-shipped direct to your assembler.  Include a factory test mode in the preprogrammed firmware and reject any boards that don't pass it.  Include another concealed test mode you don't disclose till you have to in case they are too clever for their own good!

That of course doesn't guarantee your assembler wont substitute grey-market or fake parts, but it raises the bar considerably and prevents simply 'forgetting' to change a reel loaded with questionable chips before runing your boards, because to substitute the chip, they'd have to spend a four figure sum to recover a copy of your test firmware, and eat the overhead of programming it into every chip.

[Peabody]

Yes, the tests run in the video as well as your tests have convinced me that the phenomenon is a real one. Those efforts, as well as those of the OP to bring the issue to light in the first place are definitely a contribution and it is an interesting.

Comments after the video also suggest decapping as, potentially, a definitive test.

I don't want anyone, especially me, to spend any money decapping chips.  But I think it's worth trying to get a response from Microchip.  I'm hoping Kevin Darrah can do that.  He mentioned in email trying to contact an FAE he's talked to before.  Field Applications Engineer?  That's probably not the right guy to answer the question, but maybe he could refer Kevin to someone.  In the end, whether they are counterfeits or a bad run or whatever, I think we're all going to have to be careful about any clone we buy that uses a 328P which (so far) is dated in the last half of 2019 and which has a fab code of TW or KR.  So far nobody has mentioned having a chip with those markings that actually works properly.  And of course we don't know if sleep current is the only issue with those chips.

My original problem was that after two tries with two different sellers, none of the Pro Mini clones I received slept soundly.  But at least we know now that Microchip's current production is good, so in the worst case I could buy genuine chips from Digikey and try to find someone locally to switch them out for me.

I would hope (with more of a sense that it would be nice if things worked that way, than any evidence) that some other characteristics of the "bad" chips would be found to be different - something like NOT turning off a peripheral, as being responsible.

Yes, that's what I've been hoping for.  If there's a software fix - some bit in some register - then no harm, no foul.  I just don't know what peripheral consumes 150uA at 3.3V.  I know it's not ADC or BOD because I can measure those as using the same current on both good and bad chips.  But I don't know for sure about WDT.  In my test sketch I turn it off, but Kevin doesn't bother, I think because the Arduino bootloaders all turn it off by default.  But turning it off in the sketch, or not, has no effect on current, so I've assumed it was already off, as does Kevin.  But thinking about it, it might be worth changing the fuse setting so that WDT is always on.  If that raises the sleep current on a good chip to, say, 150uA, but has no effect on a bad chip, that would be very interesting.  But otherwise, I just don't know what other peripheral could have been left on.

On the other hand, it seems like there is a specific time niche that these "bad" chips were made, based on some markings - no?

Why would it be happening with only that package of the 328P (at least so far)? Why not the DIP package, for example? Is there a reasonable explanation for that?

Well, we may have an example of a bad DIP package.  See Ralph Bacon's video on using the DS3231 to wake up an Arduino.  He puts the 328P-PU to sleep, then manually disconnects power from the DS3231, and ends up with a sleep current of 150uA.  He blames that on the I2C lines, but he has disconnected the power to the module, so I don't see how that could be the explanation.  I've written to him asking for the markings, and asking him to try again with an older chip to see if the current changes, but have not heard back from him, and it's been a while.

[Peabody]

Speaking of which, could you post a link to where you bought yours?

This is the supplier I got them from

  https://item.taobao.com/item.htm?id=571250487735

if you're not experienced with buying from Taobao, well, this isn't really the thread to explain it but probably spreenow is the "easiest" agent to buy through.

I have no experience with Taobao, and suspect it wouldn't be a good source for most hobbyists in the US.  Maybe that seller has an AliExpress or eBay affiliate with the same current stock.

[Peabody]

I think we now have what appears to be a definitive answer.  Here's what I posted this morning on AVRfreaks:

"Kevin Darrah sent the suspect chip to a company called Sensible Micro, which has lots of experience testing for bogus chips.  They decapped both the bad chip and a known good chip and compared them.  Their report is included in Kevin's followup video:



"Their conclusion is that the part is "suspect counterfeit".  It contains the logo "Aries" instead of "Atmel", and there are other differences.  So unless Microchip engaged someone else to make chips for them, it looks like this is indeed a counterfeit.

"The bad news is that this means the entire "clone" market may now be infected with bogus processors, with no practical way to tell the difference.  I don't know whether Kevin plans to refer this to Microchip, but if anyone here has a good contact there, you might refer them to Kevin's videos.  One thing that would be nice to have is a test sketch of some kind that works one way on a good chip, but another way on a bad one.  Microchip might be able to provide something like that.  Removing regulators and LEDs to test for sleep current isn't practical for most hobbyists."


I wish I had better news.  I don't begin to understand how it could be profitable to duplicate all of the engineering for something as complicated, and cheap, as the 328P.  But it seems that has happened, and it could affect any clone Uno, Nano, Mini, or anything else that uses the 328P.

[DrG]

This is a fascinating story and my thanks to you and Kevin Durrah.  It is not an Atmel chip as I would define an Atmel chip, and as far as we know, all the Atmel chips have the 'appropriate' low power capabilities.

Looking through the thread again after seeing the results of the decapping raises some questions that may never get answered. The author of the video refuses to call it a counterfeit but, instead, calls it a suspect counterfeit, ostensibly because of the possibility of "mergers / acquisitions"  that could have resulted in a packaging of the Aries die - a die which does not meet the published Atmel specs. Furthermore, since they (the Video producers) are not the manufacturers, they lack the IP to call it a counterfeit. I get that and think that it is a responsible and objective position which I agree with.

Still, I feel like I have some common street-level understanding about the concept of liability. To me, a simple-minded consumer, if this is not a counterfeit in every sense of the word, except maybe a legal/technical sense [pending Atmel/Microchip declaring it a counterfeit], I don't know how we can determine any chip is not genuine.

This situation is far more insidious than a relabeled part see this story for example, https://www.sparkfun.com/news/364 and https://www.sparkfun.com/news/395

With all that PR $$ it would be nice if Atmel/Microchip responded.

[SilverSolder]

Wow, that is pretty amazing. 

Just over the Aries logo, I make out "Xiao Bao"  ...    Dim sum? 

[SilverSolder]

[...]  The author of the video refuses to call it a counterfeit but, instead, calls it a suspect counterfeit, [...]

...because he is not the owner of the Atmel IP, he cannot call it a counterfeit - this is a legal technicality, only Atmel is able to officially call it a counterfeit.   

He is, however, calling it a counterfeit as strongly as he can, within the legal maze that is IP!

Would be interesting to find out more about this Aries company...

[DrG]

[...]  The author of the video refuses to call it a counterfeit but, instead, calls it a suspect counterfeit, [...]

...because he is not the owner of the Atmel IP, he cannot call it a counterfeit - this is a legal technicality, only Atmel is able to officially call it a counterfeit.   

He is, however, calling it a counterfeit as strongly as he can, within the legal maze that is IP!

Would be interesting to find out more about this Aries company...

Thanks for agreeing with me as your explanation is included explicitly in my complete quote on that issue....

The author of the video refuses to call it a counterfeit but, instead, calls it a suspect counterfeit, ostensibly because of the possibility of "mergers / acquisitions"  that could have resulted in a packaging of the Aries die - a die which does not meet the published Atmel specs. Furthermore, since they (the Video producers) are not the manufacturers, they lack the IP to call it a counterfeit. I get that and think that it is a responsible and objective position which I agree with.

[SilverSolder]

[...]  The author of the video refuses to call it a counterfeit but, instead, calls it a suspect counterfeit, [...]

...because he is not the owner of the Atmel IP, he cannot call it a counterfeit - this is a legal technicality, only Atmel is able to officially call it a counterfeit.   

He is, however, calling it a counterfeit as strongly as he can, within the legal maze that is IP!

Would be interesting to find out more about this Aries company...

Thanks for agreeing with me as your explanation is included explicitly in my complete quote on that issue....

The author of the video refuses to call it a counterfeit but, instead, calls it a suspect counterfeit, ostensibly because of the possibility of "mergers / acquisitions"  that could have resulted in a packaging of the Aries die - a die which does not meet the published Atmel specs. Furthermore, since they (the Video producers) are not the manufacturers, they lack the IP to call it a counterfeit. I get that and think that it is a responsible and objective position which I agree with.

There is no merger/acquisition angle to this, since that would have had to be disclosed to investors (i.e. the public) by a listed corporation as a material fact affecting their prospects. 

In the same vein, it is not possible that Atmel could have sub-contracted or licensed its core IP in this way without disclosing that material and significant information to investors as well.

[DrG]

[...]  The author of the video refuses to call it a counterfeit but, instead, calls it a suspect counterfeit, [...]

...because he is not the owner of the Atmel IP, he cannot call it a counterfeit - this is a legal technicality, only Atmel is able to officially call it a counterfeit.   

He is, however, calling it a counterfeit as strongly as he can, within the legal maze that is IP!

Would be interesting to find out more about this Aries company...

Thanks for agreeing with me as your explanation is included explicitly in my complete quote on that issue....

The author of the video refuses to call it a counterfeit but, instead, calls it a suspect counterfeit, ostensibly because of the possibility of "mergers / acquisitions"  that could have resulted in a packaging of the Aries die - a die which does not meet the published Atmel specs. Furthermore, since they (the Video producers) are not the manufacturers, they lack the IP to call it a counterfeit. I get that and think that it is a responsible and objective position which I agree with.

There is no merger/acquisition angle to this, since that would have had to be disclosed to investors (i.e. the public) by a listed corporation as a material fact affecting their prospects. 

In the same vein, it is not possible that Atmel could have sub-contracted or licensed its core IP in this way without disclosing that material and significant information to investors as well.

I note, first of all, your refusal to validate my response which was that your "explanation" based on the only sentence of mine that you decided to quote, was, in fact, included in subsequent sentences. That speaks volumes to me. A simple little "yeah, you did say that" was a bridge too far for you.

Second, the "mergers / acquisitions" issue comes directly from the video [~8',29"]. That is why it is quotes. The author is stating that he is unaware of any such merger or acquisition that could have allowed one or the other to produce chips with that die. That you don't believe that there is no "angle" like that is something that you should take up with the author of the video and not me.

Thirdly, I am distinctly underwhelmed by your proposed expertise or credibility on all information released to investors of any of the companies involved....

and with that, I am done discussing the matter with you.

[Peabody]

I've made the point to Kevin in email that the elephant in the room here is Microchip.  So far as I know, nobody has contacted them about this.  But it seems to me that Kevin, or particularly Chris at Sensible Micro, needs to talk to the Atmel product manager, link those video, and try to get an official ruling on the bad chips.  If I were the product manager for the 328P, I think I'd like to be told about this.

[DrG]

A few hours ago (after viewing the video but before the noisy exchanges), I decided to write Microchip and simply ask them about the issue. Large semiconductor companies, like Microchip, employ marketing and PR folks that could (I would argue, should) be able to respond to the general issue and specifically, "how can a "regular" consumer draw a conclusion?".

But, who do I email? Well, I recalled that 4-5 months ago, I had a short but pleasant conversation with a "Marketing Specialist" at Microchip. Since I still had the email thread, I wrote her. Hey, I figure, why not ask?

I will wait for a good two weeks (as hard as it is to believe, people may have other things to do than answer me . I will post here if I get a response that contributes to the discussion.

[Peabody]

I'm glad you did that.  The approach I would default to, if nothing else works, is to write a snailmail letter to the president of Microchip.  I believe they are in Chandler, AZ.  This method is not always productive, but sometimes it works, and usually always gets some kind of reply.

[SilverSolder]

[...]  The author of the video refuses to call it a counterfeit but, instead, calls it a suspect counterfeit, [...]

...because he is not the owner of the Atmel IP, he cannot call it a counterfeit - this is a legal technicality, only Atmel is able to officially call it a counterfeit.   

He is, however, calling it a counterfeit as strongly as he can, within the legal maze that is IP!

Would be interesting to find out more about this Aries company...

Thanks for agreeing with me as your explanation is included explicitly in my complete quote on that issue....

The author of the video refuses to call it a counterfeit but, instead, calls it a suspect counterfeit, ostensibly because of the possibility of "mergers / acquisitions"  that could have resulted in a packaging of the Aries die - a die which does not meet the published Atmel specs. Furthermore, since they (the Video producers) are not the manufacturers, they lack the IP to call it a counterfeit. I get that and think that it is a responsible and objective position which I agree with.

There is no merger/acquisition angle to this, since that would have had to be disclosed to investors (i.e. the public) by a listed corporation as a material fact affecting their prospects. 

In the same vein, it is not possible that Atmel could have sub-contracted or licensed its core IP in this way without disclosing that material and significant information to investors as well.

I note, first of all, your refusal to validate my response which was that your "explanation" based on the only sentence of mine that you decided to quote, was, in fact, included in subsequent sentences. That speaks volumes to me. A simple little "yeah, you did say that" was a bridge too far for you.

Second, the "mergers / acquisitions" issue comes directly from the video [~8',29"]. That is why it is quotes. The author is stating that he is unaware of any such merger or acquisition that could have allowed one or the other to produce chips with that die. That you don't believe that there is no "angle" like that is something that you should take up with the author of the video and not me.

Thirdly, I am distinctly underwhelmed by your proposed expertise or credibility on all information released to investors of any of the companies involved....

and with that, I am done discussing the matter with you.

LOL ok.  I've made a note, Dr. G is always right.  It would have been simpler if you had just said that in the first place! 

[Peabody]

Kevin provides what looks to be a good test sketch to differentiate between good and bad chips without having to desolder stuff.  Bad chips are mostly FFs past the first line.


https://gist.github.com/krdarrah/6e6cb94c1df015e8e9f910ae5cf85299


Real Digikey 2020

1E BA 95 FF F C6 0 26
FF 9 FF 17 FF FF 57 39
39 31 36 31 FF 1 29 1D
17 5 12 9 13 9 FF FF


Fake Ebay  2019

1E B8 95 FF F FF FF 26
FF FF FF FF FF FF 5D FF
FF FF FF 0 FF FF FF FF
FF FF FF FF FF FF FF FF


Older Pro Mini with real looking chip 2015

1E 96 95 FF F 9A FF 26
FF A FF 17 FF FF 58 35
32 31 38 36 FF F A 13
17 1 12 6 13 6 FF FF


Real Digikey 2016

1E 87 95 FF F D0 FF 26
FF A FF 17 FF FF 58 36
30 30 33 34 FF B 21 10
17 1 12 6 13 6 FF FF


Another Fake looking ProMini 2019

1E CF 95 FF F FF FF 26
FF FF FF FF FF FF 58 FF
FF FF FF 0 FF FF FF FF
FF FF FF FF FF FF FF FF


Mine look the same:


last bad Pro Mini clone - 2019 date code
1E    BE    95    FF    F    FF    FF    26   
FF    FF    FF    FF    FF    FF    58    FF   
FF    FF    FF    FF    FF    FF    FF    FF   
FF    FF    FF    FF    FF    FF    FF    FF   


good Pro Mini clone - 2015 date code
1E    A2    95    FF    F    B8    FF    26   
FF    9    FF    17    FF    FF    57    35   
37    34    39    33    FF    7    18    2   
17    1    12    6    13    6    FF    FF   


good Nano clone - 2018 date code
1E    A8    95    FF    F    C6    0    26   
FF    8    FF    17    FF    FF    57    38   
38    35    38    39    FF    9    19    B   
17    3    12    9    13    9    FF    FF   

[Ian.M]

More on the signature bytes from 'the horse's mouth':
https://microchipsupport.force.com/s/article/Serial-number-in-AVR---Mega-Tiny-devices

I think that now you and Kevin have popped the lid on this can of worms, there are going to be an awful lot of fakes found to be out there.

Genuine Arduino Uno with a socketed DIP ATmega328P-PU, date code 1602:

Code: [Select]

boot sig dump
1E 8F 95 FF F B8 0 26
FF A FF 17 FF FF 58 35
32 34 30 33 FF 10 12 2A
17 2 12 6 13 6 FF FF
I seem to have plenty of fakes, some in PDIP.

[DrG]

I'm not sure, but if I am reading this https://microchipsupport.force.com/s/article/8-bit-AVR-MCU-with-unique-serial-number correctly, the ATmega328/P does NOT have a unique serial number...

"Most classic AVR mega and Tiny devices do not have unique serial number except ATmega328PB/48PB/88PB/168PB. And all Xmega, megaAVR0 and ATtiny0/1 families have unique serial number which can be used to identify a specific device in the field."

The ATmega 328PB is the one with the QTouch® Peripheral https://www.pololu.com/file/0J1464/Atmel-42559-Differences-between-ATmega328P-and-ATmega328PB_ApplicationNote_AT15007.pdf But, that is not the one you would find in the Pro Mini - that's the 328/P right?

In any event, if you can distinguish genuine from other, I am in favor of that. Ran the program(s) on a few Pro Minis that I have with these results"

Code: [Select]

Sparkfun (i would guess, a genuine part).
boot sig dump
1E AA 95 FF F F6 FF 26
FF D FF 17 FF FF 55 35
30 31 35 38 FF E 16 1E
17 1 12 6 13 6 FF FF

Signature :  0x1E 0x95 0xF
Serial Number :  0x55 0x35 0x30 0x31 0x35 0x38 0xFF 0xE 0x16 0x1E


DIY Mall clone
boot sig dump
1E 9E 95 FF F 9C FF 26
FF B FF 17 FF FF 58 35
32 31 38 36 FF 2 17 1D
17 1 12 6 13 6 FF FF


Signature :  0x1E 0x95 0xF
Serial Number :  0x58 0x35 0x32 0x31 0x38 0x36 0xFF 0x2 0x17 0x1D


The Simple clone
boot sig dump
1E A7 95 FF F C8 FF 26
FF A FF 17 FF FF 55 37
32 30 38 37 FF 4 17 14
17 1 12 8 13 8 FF FF

Signature :  0x1E 0x95 0xF
Serial Number :  0x55 0x37 0x32 0x30 0x38 0x37 0xFF 0x4 0x17 0x14



[westfw]

it is not possible that Atmel could have sub-contracted or licensed its core IP in this way without disclosing that material and significant information to investors as well.

I'll note that Microchip (and Atmel before them) seems to be very careful about announcing even minor changes like

"Product Change Notification - LIAL-30BVKD482 - 17 Sep 2018 - CCB 3321 and 3321.001 Final Notice: Qualification of MTAI as an additional assembly site for selected Atmel products of the 59.91k wafer technology available in 20L VQFN package."

https://www.microchip.com/product-change-notifications/#/12112/LIAL-30BVKD482

[SilverSolder]

it is not possible that Atmel could have sub-contracted or licensed its core IP in this way without disclosing that material and significant information to investors as well.

I'll note that Microchip (and Atmel before them) seems to be very careful about announcing even minor changes like

"Product Change Notification - LIAL-30BVKD482 - 17 Sep 2018 - CCB 3321 and 3321.001 Final Notice: Qualification of MTAI as an additional assembly site for selected Atmel products of the 59.91k wafer technology available in 20L VQFN package."

https://www.microchip.com/product-change-notifications/#/12112/LIAL-30BVKD482

When pension fund managers (and other investors, even private individuals) make decisions about whether (or how much) to invest in the securities of a company, the concept of risk is paramount in their minds.   They rely on accurate information from company management to make their assessments.  If the company withholds or "embellishes" material information that affects these assessments, they are in breach of securities legislation and open to class action lawsuits to be reimbursed for any losses incurred due to falling share prices that the hidden or incorrect information prevented them from predicting.  This kind of thing can get extremely expensive and career altering for the people involved.

[Peabody]

Genuine Arduino Uno with a socketed DIP ATmega328P-PU, date code 1602:

Code: [Select]

boot sig dump
1E 8F 95 FF F B8 0 26
FF A FF 17 FF FF 58 35
32 34 30 33 FF 10 12 2A
17 2 12 6 13 6 FF FF
I seem to have plenty of fakes, some in PDIP.

Well that one looks ok.  But I'd like to know more about your PU fakes.  What does their boot sig dump look like?  What markings do they have, particularly the date code?

[Peabody]

DrG, I don't think Kevin is saying the 328P has unique serial numbers.  It 's just the difference in the overall pattern after the first row of data.  Genuine chips seems to have a variety of values, whereas the fakes have almost all FFs.  Anyway, the three you posted all look good.  I'd be interested in the date code markings on those chips.  So far, all 2018 and earlier, and 2020, seem genuine, but all 2019 are counterfeit.

[Ian.M]

I believe the lot codes *SHOULD* be ASCII.  They are six characters long in byte reversed pairs starting at location 0x0E.
e.g.

Real Digikey 2020

Code: [Select]

1E BA 95 FF F C6 0 26
FF 9 FF 17 FF FF 57 39
39 31 36 31 FF 1 29 1D
17 5 12 9 13 9 FF FF


has a lot code of '9W1916'.  The wafer no. and die X,Y position (locations 0x15 - 0x17) are also informative.  Scaling from the X-ray in the video, the genuine die is about 3.35mm square. To get 255 dies across a 300mm wafer, the dies would need to be under 1.17 mm square, and 'FF FF' would be out in a corner and thus off the circular wafer, so 'FF FF FF' there is patently ridiculous, as that would also imply a batch size in excess of 16 million dies, and 36% wasted wafer area!.

Unfortunately, I suspect the Ares 'mega328 clone is likely to have flash memory in the signature byte area, so almost as soon as this 'fingerprinting' becomes widely known, we are likely to see clones with less suspicious fingerprints.   If they have the capability to individually serialise the chips, they could soon become impossible to distinguish by signature bytes fingerprint alone, unless Microchip provides a signature byte verification service, which is fraught with its own problems.

@Peabody,
I'll set up again to dump signature bytes as soon as I get around to it, but I'd rather not wear out my Uno's socket so need to build a 'breadboard Arduino' + have some 'fun' with Zadig and my PICkit 2 so AVRdude can see it as an AVR programmer.

[westfw]

I'd rather not wear out my Uno's socket so need to build a 'breadboard Arduino'

https://oshpark.com/shared_projects/57tdc7y6 ?It was mostly for fiddling with bootloaders, but it's essentially an Arduino clone with ZIF socket and jumpers to mess with things like power and auto-reset...

[SilverSolder]

Disregarding all non technical issues...  Are there any circumstances where using the Aries chip is actually acceptable?   - e.g. if they have to start selling them as "Atmel compatible" chips in the future...

[brucehoult]

Disregarding all non technical issues...  Are there any circumstances where using the Aries chip is actually acceptable?   - e.g. if they have to start selling them as "Atmel compatible" chips in the future...

They appear to be fine if you don't need really deep sleep super low power operation.

Most amateurs never use that.

It's only necessary if you want to run for a long long time off a small battery.

[Monkeh]

Disregarding all non technical issues...  Are there any circumstances where using the Aries chip is actually acceptable?   - e.g. if they have to start selling them as "Atmel compatible" chips in the future...

They appear to be fine if you don't need really deep sleep super low power operation.

Most amateurs never use that.

It's only necessary if you want to run for a long long time off a small battery.

ADC performance is likely to be suspect. Not that these chips have a good ADC to begin with, but still.

[DrG]

DrG, I don't think Kevin is saying the 328P has unique serial numbers.  It 's just the difference in the overall pattern after the first row of data.  Genuine chips seems to have a variety of values, whereas the fakes have almost all FFs.  Anyway, the three you posted all look good.  I'd be interested in the date code markings on those chips.  So far, all 2018 and earlier, and 2020, seem genuine, but all 2019 are counterfeit.

Yeah, I wasn't being very clear and was sort of thinking out loud. I was trying to understand exactly what the area that the two programs are looking at represents.

The two programs being Kevin's and the original that he got it from (according to his comment):
https://gist.github.com/speters/f889faec42b510052a6ab4be437d38ca
https://gist.github.com/krdarrah/6e6cb94c1df015e8e9f910ae5cf85299

These appear to treat some of the bytes as a UID. Looking around, there is also a library https://github.com/ricaun/ArduinoUniqueID that looks at those bytes as a UID but acknowledge that they are not "official". There is no such designation of any UID in the 328/p data sheet as noted earlier, and it does not look like the area on a genuine 328p conforms to the 328pb data sheet (right?).

Yet, there does seem to be a UID in the 328p chips, albeit unofficial / undocumented (and therefore unreliable?)- that is what I was getting at.

Your identified 'bad' chip certainly has something different in that area, namely FFs. They are not doing what the other chips are doing and I think that is what you are saying and, yes it looks different.

For example:
Signature :  0x1E 0x95 0xF
Serial Number :  0x55 0x35 0x30 0x31 0x35 0x38 0xFF 0xE 0x16 0x1E

The signature bytes are the same on all the 328p chips in the thread. The UIDs are different but "look" like they could be, in fact, UIDs...EXCEPT your Aries chips where no attempt is made to have a UID (i.e., FFs) - is that where we are at?

[Ian.M]

Something put those bytes in there (as they are not 0xFF) and it almost certainly has to be the factory wafer ATE.

I think they are *NOT* a UID because Atmel, now Microchip don't *GUARANTEE* that they will be unique.  Obviously X,Y coordinates within a wafer will repeat, and over multiple batches the wafer number will repeat, so if the lot no. isn't guaranteed unique (i.e. due to limited code space of 6 alphanumerics, which probably also encodes the fab and line) they could reuse them, returning to the start when they run out of valid lot numbers for a particular fab/line.  Also, its a management issue - if they have to restart the wafer ATE partway through a run, do they *ALWAYS* guarantee to preset the last wafer no. used correctly?  It obviously benefits them for traceability to have a near-unique serial no. but I suspect the extra costs to publicly guarantee its a UID and make it so aren't insignificant.

If my suppositions are correct, the odds of finding a serial no. collision between current production genuine chips are vanishingly small, and although such collisions between current chips and old genuine chips are possible, your chances of finding one 'in the wild' are infinitesimal, mostly due to the small number of chips that will have survived in active use that are *NOT* in a dedicated application where the firmware will never be changed again.

[Doctorandus_P]

Have you excluded simple user error?

I did not read it all, but it seems you have one AVR wich was programmed a long time ago which draws 1uA, and newer AVRs which draw 150uA. Maybe you changed something in the source code in between and the new AVR's do not enter sleep mode properly.

[Peabody]

So far, the only difference in performance that's been documented is the power-down sleep current.  And I don't usually mind using a cloned chip - I suspect I've never actually used a genuine FTDI chip.  But the MCU is a bit different.  If there's something that just doesn't work right, but it seems it ought to, could it be the fake 328P that's causing the problem?  So it becomes another thing you have to rule out that you shouldn't have to worry about.

But I would guess the counterfeits work ok for most things or issues would have been raised before.  When I tested the sleep current, it wasn't really the processor I wanted to test.  It was the rest of the Pro Mini board, and to make sure I hadn't messed up something when removing the regulator and LED.  And it was only when I received the second batch of Pro Minis from a different seller, which had been made with different parts in a different production run, but which also wouldn't sleep properly, that I began to suspect the processor.  So I suspect there are many people out there using these chips with no noticeable problem.

I wonder what's being shipped now from AliExpress, Bangood, and the large Ebay sellers.  My sellers were domestic US, and certainly not large, but could deliver in a week. SleemanJ reported receiving genuine parts on a recent order from TaoBao.  So I guess there's a possibility that this was a single production run, and new stuff will be ok.  Well, ok, just a possibility.

I still think we need Microchip to help find a reliable way for us to distinguish genuine from counterfeit.

[DrG]

Something put those bytes in there (as they are not 0xFF) and it almost certainly has to be the factory wafer ATE.

I think they are *NOT* a UID because Atmel, now Microchip don't *GUARANTEE* that they will be unique.  Obviously X,Y coordinates within a wafer will repeat, and over multiple batches the wafer number will repeat, so if the lot no. isn't guaranteed unique (i.e. due to limited code space of 6 alphanumerics, which probably also encodes the fab and line) they could reuse them, returning to the start when they run out of valid lot numbers for a particular fab/line.  Also, its a management issue - if they have to restart the wafer ATE partway through a run, do they *ALWAYS* guarantee to preset the last wafer no. used correctly?  It obviously benefits them for traceability to have a near-unique serial no. but I suspect the extra costs to publicly guarantee its a UID and make it so aren't insignificant.

If my suppositions are correct, the odds of finding a serial no. collision between current production genuine chips are vanishingly small, and although such collisions between current chips and old genuine chips are possible, your chances of finding one 'in the wild' are infinitesimal, mostly due to the small number of chips that will have survived in active use that are *NOT* in a dedicated application where the firmware will never be changed again.

I don't disagree with you although I am (literally) not sure about some of what you are saying based on my own limitations and lack of motivation to do many more hours of reading.

The problem, which we seem to be in agreement about, is that, beyond the signature bytes, the relevance of the values are undocumented. That does not mean that they can not be useful...subject to further examination. I think that is exactly what we are talking about with regard to identifying those chips that do not meet the low-power specs. If I had some Pro Minis that did not have something that looked like a UID (e.g. only FFs), I would avoid that chip or not choose to use it if I had a choice.

[HwAoRrDk]

I dumped the signature bytes from all the Arduino boards I own (or, at least, all the ones I could find ). One is a genuine Arduino Uno board, the others anonymous Uno clones. I've not had any problems with these clone boards - they've performed to expectations - but I've never done anything exotic with them that would expose them as potentially having non-genuine chips.

None of them are very recent, but I thought the data might be useful to see what typical clone board chips from prior years are like.

Genuine Arduino Uno R3, DIP-28
Markings:
1431
ATMEGA328P-PU
Boot signature dump:
      0001 0203 0405 0607 0809 0A0B 0C0D 0E0F - 0123456789ABCDEF
----------------------------------------------------------------
0x00: 1E42 95FF 0FD2 FF26 FF09 FF17 FFFF 5734 - .B•ÿ.Òÿ&ÿ.ÿ.ÿÿW4
0x10: 3431 3334 FF13 1C01 1702 1206 1306 FFFF - 4134ÿ.........ÿÿ

Clone Arduino Uno, LQFP-32
Markings:
MEGA328P
AU 1651
Boot signature dump:
      0001 0203 0405 0607 0809 0A0B 0C0D 0E0F - 0123456789ABCDEF
----------------------------------------------------------------
0x00: 1E9D 95FF 0FCE 0026 FF09 FF17 FFFF 5836 - ..•ÿ.Î.&ÿ.ÿ.ÿÿX6
0x10: 3134 3831 FF05 1B0D 1702 1208 1308 FFFF - 1481ÿ.........ÿÿ

Clone Arduino Uno, LQFP-32
Markings:
MEGA328P
U-PH
35473D
1752YSP
Boot signature dump:
      0001 0203 0405 0607 0809 0A0B 0C0D 0E0F - 0123456789ABCDEF
----------------------------------------------------------------
0x00: 1EAE 95FF 0FCE FF26 FF08 FF17 FFFF 5837 - .®•ÿ.Îÿ&ÿ.ÿ.ÿÿX7
0x10: 3734 3832 FF19 121D 1701 1209 1309 FFFF - 7482ÿ.........ÿÿ

[SilverSolder]

So far, the only difference in performance that's been documented is the power-down sleep current.  And I don't usually mind using a cloned chip - I suspect I've never actually used a genuine FTDI chip.  But the MCU is a bit different.  If there's something that just doesn't work right, but it seems it ought to, could it be the fake 328P that's causing the problem?  So it becomes another thing you have to rule out that you shouldn't have to worry about.

But I would guess the counterfeits work ok for most things or issues would have been raised before.  When I tested the sleep current, it wasn't really the processor I wanted to test.  It was the rest of the Pro Mini board, and to make sure I hadn't messed up something when removing the regulator and LED.  And it was only when I received the second batch of Pro Minis from a different seller, which had been made with different parts in a different production run, but which also wouldn't sleep properly, that I began to suspect the processor.  So I suspect there are many people out there using these chips with no noticeable problem.

I wonder what's being shipped now from AliExpress, Bangood, and the large Ebay sellers.  My sellers were domestic US, and certainly not large, but could deliver in a week. SleemanJ reported receiving genuine parts on a recent order from TaoBao.  So I guess there's a possibility that this was a single production run, and new stuff will be ok.  Well, ok, just a possibility.

I still think we need Microchip to help find a reliable way for us to distinguish genuine from counterfeit.

I wonder what these fake chips actually cost - maybe they are in range of the PADAUK processors, costing only pennies each? 

There is a place for compatible chips in the ecosystem, like Intel vs. AMD in the PC world...   but the chips need to be marked with what they actually are, rather than flying a false flag...

In a manufacturing situation you would probably want the "real thing" except for the lowest end consumer goods, unless the Aries chip somehow gets validated/tested and supplied with its own specs.


Another interesting thought is, there might be something the Aries chip actually does better than the Atmel one?  Maybe it can source or sink more current, or do well with some other parameter...  there's bound to be other differences between them in addition to the sleep current performance!

[HwAoRrDk]

I wonder what these fake chips actually cost - maybe they are in range of the PADAUK processors, costing only pennies each?

I would suspect somewhere in the region of $0.30. Clone Arduino Pro Mini boards sell for less than $2 on Alibaba, often nearer $1.50 in quantity. If you take into account that sellers are probably making their own small profit, say 10%, that means they're probably buying from the manufacturer for roughly $1.30 each. PCBs probably cost a few cents each, let's say 5c. All the supporting components are probably around 1 cent each - let's say $0.20 worth. Assembly costs are probably another few cents - call it $0.20. Add on another few cents for overheads, and we might have a price of $0.70 each. If the manufacturer is taking a 50% profit, that leaves less than $0.35 for the microcontroller.

[Doctorandus_P]

With another uC the STM32F103C8T6 which used to be popular in the "Blue Pill" boards is also plagued by dubious practices. There are currently at least 8 different clones of this chip, and if you buy the "blue Pill"boards from aliexpress / Ebay / China Direct they all have the string "STM32F103C8T6" on it, though often the ST logo is missing.
STM is the prefix for:  https://www.st.com/ But the chinese don't care about such details.

Each of those 8 manufacturers has their own prefix, and LCSC.com sells them all with the prefix of each manufacturer. Somehow these disapear on Ali / Ebay / China. There are known differences and incompatibilities between these look a likes, you never know which of the clones you get. More and more people are moving away from these boards for this reason.

The difference happened one or 2 years ago. Back then the price of these boars was around EUR 3.5, and now it dropped to below 2, but it's a price I am not prepared to pay.

There are several long threads with documentation about the STM32F103C8T6, including links to an external site with die pictures of several (maybe all?) decapped variants.

[cdev]

I have two that have the following code

MEGA328P
AU 1710

(no additional text on the chip)

Have no idea what the idle current is.

[NiHaoMike]

Another interesting thought is, there might be something the Aries chip actually does better than the Atmel one?  Maybe it can source or sink more current, or do well with some other parameter...  there's bound to be other differences between them in addition to the sleep current performance!

A good test to try would be to repeatedly write to the EEPROM until it starts to fail. I wouldn't be surprised if the clone fails way earlier than what the datasheet figures would predict.
https://hackaday.com/2011/05/16/destroying-an-arduinos-eeprom/

[Doctorandus_P]

It seems this post has reached Hackaday.
With links to youtube video's and die pictures.
The die has a "Aries" logo on it instead of "Atmel" (@08:20 on the 2nd video).

https://hackaday.com/2020/09/15/deep-sleep-problems-lead-to-forensic-investigation-of-troublesome-chip/

[DrG]

I wonder what these fake chips actually cost - maybe they are in range of the PADAUK processors, costing only pennies each?

I would suspect somewhere in the region of $0.30. Clone Arduino Pro Mini boards sell for less than $2 on Alibaba, often nearer $1.50 in quantity. If you take into account that sellers are probably making their own small profit, say 10%, that means they're probably buying from the manufacturer for roughly $1.30 each. PCBs probably cost a few cents each, let's say 5c. All the supporting components are probably around 1 cent each - let's say $0.20 worth. Assembly costs are probably another few cents - call it $0.20. Add on another few cents for overheads, and we might have a price of $0.70 each. If the manufacturer is taking a 50% profit, that leaves less than $0.35 for the microcontroller.

Your estimate is generally consistent (maybe a bit high) with this advertisement https://szaries.en.alibaba.com/product/60782607389-807067064/_Electronic_Components_atmega_328p.html?spm=a2700.icbuShop.41413.15.40fa769dCfBf3H

[TomS_]

Edit: Nevermind, already covered in Doctorandus_P's post

Haven't read the whole thread, but I just came across this video on YouTube. Very likely a high probability of a fake chip?

https://youtu.be/o0rEzcKYzGw

[Mecanix]

All parts used by the mass (popular ones) are cloned, not only by China but pretty much anyone/anywhere. Some even reported seeing those populars sold by authorized/licensed resellers for the price of a genuine. Trick to avoid being victim of this global incentives-hungry-phenomena is to avoid all parts that are "hyped" on the internet. Use/engineer with the equivalent from another manufacturer instead. I'd certainly avoid those 328 and other atmel duino thingy at all cost... regardless of where they come from

It is what it is...

[elecdonia]

I'm bumping this topic because I have several projects which use AVR devices: ATmega328P, ATmega324, ATmega644, ATmega1281, ATmega2560.

Some are used as "bare chips" in my own projects.

Others are "factory-installed" on a variety of Arduino-style boards including genuine original Arduino UNO, Chinese clone UNO-style boards, sparkfun RedBoards (UNO form factor), and adafruit Metro (UNO form factor). I also have a large quantity of Chinese clone Pro-mini and Nano style boards of various ages from recent back to 5 years old. Some of my Pro-mini boards cost <$4 USD each, leading me to suspect they do not contain genuine Atmel AVR chips.

My hoard of AVR devices provides a large variety of devices to explore. Several are guaranteed to be "genuine Atmel" because I purchased them as bare chips from Digikey 8-10 years ago, well before the Chinese began supplying huge quantities of (cheap) clone Arduino boards.

I plan to measure sleep currents and will obtain the entire 32 bytes of signature data from each AVR device.

Another item  I will measure is the frequency and stability of the 8MHz internal clock. I'm doing this because I frequently use the 8MHz internal RC oscillator in my projects. I wonder how much the accuracy/stability of the internal oscillator varies when comparing "genuine Atmel" to clone AVR chips?

Here are My first results from reading the entire 32 byte signature area from my AVR chips:

     Note: I edited “Lot#” and “Wafer info” after reading the Atmel app note

              This is from a genuine Atmel ATmega1284 I purchased from Digikey. 40-pin DIP package. Date code: 1940

Code: [Select]

AVR boot signature dump for ATmega1284
 Signature: 1E 97 06     OscCal: 90
Hex data:
  1E    90    97    FF    06    FF    FF    FF   
  FF    FF    FF    F7    FF    FF    57    39   
  36    31    32    36    FF    15    13    0F   
  17    01    12    FF    13    FF    FF    FF   

 Lot# 9W1662    Wafer info: 15  13  0F
           Chinese clone UNO board

Code: [Select]

AVR boot signature dump for ATmega328P
 Signature: 1E 95 0F    OscCal: AE
Hex data:
  1E    AE    95    FF    0F    E0    FF    26   
  FF    0A    FF    17    FF    FF    57    38   
  31    32    34    30    FF    0D    14    01   
  17    01    12    09    13    09    FF    FF 

 Lot: 8W2104     Wafer info: 0D 14 01
                 Chinese clone MEGA 2560 PRO board purchased in 2023

Code: [Select]

AVR boot signature dump for ATmega2560
 Signature: 1E 98 01    OscCal: 94
Hex data:
  1E    94    98    FF    01    FF    FF    FF   
  ED    FF    FF    F7    FF    FF    57    36   
  30    31    36    31    FF    16    15    0D   
  17    08    12    07    13    07    FF    FF 

 Lot# 6W1016    Wafer info: 16 15 0D
               Chinese clone NANO board      "DIY   more"      About 4 years old

Code: [Select]

AVR boot signature dump for ATmega328P
 Signature: 1E 95 0F    OscCal: AB
Hex data:
  1E    AB    95    FF    0F    DD    FF    26   
  FF    0C    FF    17    FF    FF    58    35   
  37    33    31    36    FF    12    0B    1D   
  17    01    12    06    13    06    FF    FF

 Lot# 5X3761     Wafer info: 12 0B 1D
                 Chinese clone Pro-mini board    "DIY more"   purchased about 4 years ago

Code: [Select]

AVR boot signature dump for ATmega328P
 Signature: 1E 95 0F    OscCal: A4
Hex data:
  1E    A4    95    FF    0F    C6    FF    26   
  FF    0A    FF    17    FF    FF    55    38   
  38    30    32    32    FF    10    06    06   
  17    01    12    09    13    09    FF    FF   

 Lot# 8U0822     Wafer info: 10 06 06
         Another Chinese clone Pro-mini board    "DIY more"   purchased about 4 years ago

Code: [Select]

AVR boot signature dump for ATmega328P
 Signature: 1E 95 0F    OscCal: 8A
Hex data:
  1E    8A    95    FF    0F    E3    FF    26   
  FF    08    FF    17    FF    FF    57    38   
  32    33    36    39    FF    0B    1B    14   
  17    01    12    09    13    09    FF    FF   

 Lot# 8W3296     Wafer info: 0B 1B  14
    So far, this next board has the only signature I’ve discovered yet where most of the bytes are 0xFF

            Another Chinese clone Pro-mini board      "The Simple"      purchased in 2023

Code: [Select]

AVR boot signature dump for ATmega328P
 Signature: 1E 95 0F    OscCal: C8
Hex data:
  1E    C8    95    FF    0F    FF    FF    26   
  FF    FF    FF    FF    FF    FF    58    FF   
  DF    FF    DF    FF    FF    FF    FF    FF   
  FF    FF    FF    FF    FF    FF    FF    FF

Lot#  X          Wafer info: FF FF FFThe only ASCII character in the 6 byte “Lot#” field is the letter “X” 
(The rest is  of this field is either 0xFF or 0xDF - not ASCII characters)
“Wafer info” is all 0xFF

[elecdonia]

I believe the lot codes *SHOULD* be ASCII.  They are six characters long in byte reversed pairs starting at location 0x0E.    Real Digikey 2020:

Code: [Select]

1E BA 95 FF 0F C6 00 26
FF 09 FF 17 FF FF 57 39
39 31 36 31 FF 01 29 1D
17 05 12 09 13 09 FF FF
has a lot code of '9W1916'. 
The wafer no. and die X,Y position (locations 0x15 - 0x17) are also informative

Did Atmel ever release any documentation about these reserved locations in the signature area?

    EDIT: Yes they did. See my newer post for details:
              https://www.eevblog.com/forum/microcontrollers/arduino-pro-mini-two-copies-have-different-sleep-currents/msg4799735/#msg4799735

Interesting: The “lot code” ASCII string is stored in “little endian” fashion (byte order reversed). This doesn’t surprise me because AVR machine code is also stored in flash as “little endian.”

I’m in the process of gathering signature dumps (all 32 bytes) from all of my AVR devices and Arduino boards.
So far I’ve identified only one board where most of the data is a string of 0xFF. It is a recently manufacturerd Chinese clone Pro-mini board labeled “The Simple.”

[elecdonia]

Did Atmel ever release any documentation about these reserved locations in the signature area?

Answering my own question: Microchip/Atmel technical article (summarized here):

Code: [Select]

​​​​​For Mega and Tiny T devices with “Serial number,”
  they have similar signature addresses for
  Lot Number, Wafer Number and X, Y coordinates.
 Byte Address --------- Description
    0x0E --------- Lot Number 2nd Char
    0x0F --------- Lot Number 1st Char
    0x10 --------- Lot Number 4th Char
    0x11 --------- Lot Number 3rd Char
    0x12 --------- Lot Number 6th Char
    0x13 --------- Lot Number 5th Char
    0x14 --------- Reserved
    0x15 --------- Wafer Number
    0x16 --------- Y-coordinate
    0x17 --------- X-coordinateThis section of signature data is 10d bytes in length.

 First 6 bytes:    ASCII text stored in “little endian” (bytes reversed)
                             ( addr 0x14 “reserved” )
 Last 3 bytes:    Wafer data (wafer #, X & Y coordinates)

Link to Microchip/Atmel technical article:
    https://microchip.my.site.com/s/article/Serial-number-in-AVR---Mega-Tiny-devices

[LC333]

    So far, this next board has the only signature I’ve discovered yet where most of the bytes are 0xFF

            Another Chinese clone Pro-mini board      "The Simple"      purchased in 2023

I've just ordered 10 of these marked "The Simple".  I had clean forgot about the prevalence of copied 328P chips as it's been a while since I've ordered complete PCBs like this, will check them when I receive them.

[Pacjunk]

I know this is an old thread, but I thought I would share my experience...

I have built several battery powered items over the years that use deep sleep, so I know my code is good. This particular system used one of the "The simple" type boards, which seemed to do things as expected. After only getting 3 weeks battery life for something that I expected to run for a year+, I pulled the thing apart looking for the issue. Once I had it down to the bare pro mini board, it was still taking 2.5mA in sleep. I know it was in sleep mode as it was doing the 8 second cycle as expected.

I tested the board with the suggested sketch and it showed lots of FF's. I managed to find a non-simple board in an old project and wired it up.
Same code is now 1.5uA in sleep!

So this post is really just to say, don't necessarily expect the 100-150uA sleep for these counterfeit chips. It can be much higher than that!

[SilverSolder]

Did Atmel ever release any documentation about these reserved locations in the signature area?

Answering my own question: Microchip/Atmel technical article (summarized here):

Code: [Select]

​​​​​For Mega and Tiny T devices with “Serial number,”
  they have similar signature addresses for
  Lot Number, Wafer Number and X, Y coordinates.
 Byte Address --------- Description
    0x0E --------- Lot Number 2nd Char
    0x0F --------- Lot Number 1st Char
    0x10 --------- Lot Number 4th Char
    0x11 --------- Lot Number 3rd Char
    0x12 --------- Lot Number 6th Char
    0x13 --------- Lot Number 5th Char
    0x14 --------- Reserved
    0x15 --------- Wafer Number
    0x16 --------- Y-coordinate
    0x17 --------- X-coordinateThis section of signature data is 10d bytes in length.

 First 6 bytes:    ASCII text stored in “little endian” (bytes reversed)
                             ( addr 0x14 “reserved” )
 Last 3 bytes:    Wafer data (wafer #, X & Y coordinates)

Link to Microchip/Atmel technical article:
    https://microchip.my.site.com/s/article/Serial-number-in-AVR---Mega-Tiny-devices

[FX: begging bowl in hand]  Would you be able to share a sketch that dumps the information to serial, or something like that?

[Peabody]

The sketch I use was provided by Kevin Darrah.  It works on the 328P.  I don't know about the other Atmels.

Code: [Select]

#include <avr/boot.h>
#define SIGRD 5
void setup() {
  Serial.begin(9600);
  Serial.println("");
  Serial.println("boot sig dump");
  int newLineIndex = 0;
  for (uint8_t i = 0; i <= 0x1F; i += 1) {
    Serial.print(boot_signature_byte_get(i), HEX);
    Serial.print("\t");
    newLineIndex++;
    if (newLineIndex == 8) {
      Serial.println("");
      newLineIndex = 0;
    }
  }
  Serial.println();
}

void loop() {
}