Electronics > Beginners
Pulling my hair out. Circuit boards stop working once shipped to client and more
(1/21) > >>
Jackster:
So I designed a small product a few years ago and it has been selling well.
But around 5-10% that get shipped end up not working once received by the end user.

The product is pretty simple and only requires power to work.
But a small % just don't work once they are in the hands on my clients.

All they have to do is plug in power, which is either via a battery (a standard used in the industry) or via USB.

I did the whole PCB design, case CAD and offloaded the software to a freelancer.
In house (literally my house) this all works fine and passes all our QA testing.



Another problem I have just had is that I needed more PCBs the other month.

I wanted to improve 1 thing which was when installing the PCB this corner was getting in the way a tad.
So I chopped off this corner in PCB design software and re-poured the ground plane.

And once the PCBs arrived, none of them would allow me to burn the boot loader onto the ATMEGA via the programming header.
Had to pull all the ATMEGAs off and flash them in a socket.
Put them back on and 90% of the boards I made just don't work as expected.
This is after P&P and hand soldering over £300 worth of components >.<


I just don't know what to do any more.

Would love to just hand off the PCB design to someone else but I don't have enough money to fund that right now.
I don't understand why one minute the thing works but then dies as soon as it arrives at the end user.

Any advise on what I should do?



[edit]
For clarification as I know I have not added a lot of details here.


Any boards that have failed have been replaced with working boards. Full warranty was provided.

The two issues above are not the same issue/related.
 I have done 3 batches of boards.
 Batch 1 out of a total of 50 boards, 2-3 failed and were replaced.
 Batch 2 we have had 3-4 fail and were replaced. One has failed again, we are investigating.
 Batch 3 with a slight PCB change, all but a handful have failed to work as expected. None have shipped.

Boards that arrived back here were examined but I don't have the tools or knowledge to go deep into scoping pins or anything like that.

The flashing of the boot loader is only done once, which is why we can flash them on a socket and then P&P.
The USB interface is used to flash firmware and updates.
I am aware that there might be underline issues with batch 3, which is why we are not shipping any of these boards.

My clients are aware that this is a project being done out of my garage and that I am not a "pro" at this.



Psi:
Note: I am adding to this list as i think of things. So you may want to re-read it in case i have added more since you read it last.

Questions
- Have you got any units back that a client says don't work? If so do they work for you or are they dead.
- Is your QC test automated? If so are you sure it is not faulty and letting dead units out the door? Maybe a manual test is needed so you know for sure all units leaving you are 100% working.
- Is there anything unusual about your location or places you ship to.  Some places irradiated all their mail with high energy x-rays and this can destroy electronics.
- Does the MCU/software system interface/talk-to something that is different in different parts of the world.  Here's examples of what i mean. Maybe bluetooth to a phone  or maybe comms to a desktop PC app. Maybe some people have their phone/desktop set to a different timezone/country/language and may your app is incompatible with this.
- Do you have the sourcecode to your ATMega or does the freelancer hold that?

Few possibilities i can think of.
- Are users connecting the battery around the wrong way and damaging the device. (9V around wrong way for a split sec, that sort of thing)
- Do you have any floating inputs pins on the MCU? Maybe the code only works if an input is read as either low or high but it keeps changing with ambient noise. When built it might stay in one state but in noise environments maybe it floats to high and stops the code running. (Floating inputs should have MCU pullups enabled in software but maybe they are not set in your code?)
- Have you tried powering the device from 4.5V and with lets say 50mA current limit.  Not all USB ports are created equal. Maybe your product is quite critical on power and not all USB ports can power it.
- Where are you getting your parts from, maybe you are getting lots of fake ICs
- Could be a PCB track routing issue where tracks run too close to a hole or board edge and sometimes get cut by the drill/router.  etc Some PCBs work, some don't, some intermittent.
- Are you sure you have the ATmega Fuse Bits set correctly, maybe the startup delay, brownout detector or crystal settings are wrong and this is making it run intermittently.
- Does the product have protection from ESD or PSU spikes, like a TVS?  Does the product get used in a location where it might need this.  etc  automotive/industrial
- How are you programming the MCUs?  I one had a crappy USBASP programmer that would brick 2 our of 5 AVRs it flashed. Not sure why, maybe clock was out of spec and kept erasing fuse bits.
- Does your MCU programming system include a verify check?
- There is one AVR MCU, cant remember which, that comes with fuse bit set to put it into a compatibility mode where it pretends to be a different AVR chip. Some of the IO/peripherals don't work until you get it out of that mode.   :palm:  (my guess is they have a supply agreement to sell a compatibly chip for 25 years for MIL/MED/AERO) EDIT: All ATMega128 pretend to be a ATMega103 until you change the M103C fuse bit


--- Quote from: Jackster on June 01, 2019, 10:56:11 am ---And once the PCBs arrived, none of them would allow me to burn the boot loader onto the ATMEGA via the programming header.
Had to pull all the ATMEGAs off and flash them in a socket.
Put them back on and 90% of the boards I made just don't work as expected.

--- End quote ---

This makes me lean towards a PCB/SCH issue.
Are you using MISO MOSI SCK pins for anything else other than programming?
You can use them for other things too, but you need to make sure you don't load the lines so much that programming is effected.
It can become intermittent if you load them or have caps on the line to gnd.

Also, grab one of those boards that doesn't program and use DMM to check the tracks between the programming header pins GND VCC MOSI MISO SCK RESET and the ATmega pads for those pins. Also check none are shorted together.



Help
- Are your PCB files in Altium? if so i'm happy to take a look at your SCH/PCB/CODE and see if i can spot any potential problems.


vk6zgo:
I remember many years ago, a separate State branch of the organisation I worked for were tasked with making boards that would automatically ring various phone numbers  when required.

We duly received our portion of those devices, but unfortunately they didn't work.
When we complained to the other State, they protested:
 "But we tested them & they all rang up who they were supposed to!"

Yup! They dutifully programmed the whole number needed to call those sites from their State into the PROMs.
Those additional numbers, of course, weren't needed in the State they were intended for, & would "freak the exchange out".
iMo:
Hard to help without details. Anyhow, the fact the 5-10% that get shipped end up not working is an indication there is something wrong with the product or processes around and the manufacturer should have stopped shipping such a product.
dmills:
What does your power input stage look like? Ceramic cap and LDO by any chance?

If so, try plugging in the power with the supply already switched on, and with reasonably long power leads, you might be blowing the regulator due to ringing in the LC circuit formed by the power cable and the input cap (Cure is a jellybean electrolytic about 10 times the value of the input ceramic in parallel with it, the ESR damps the ringing).

Are ALL of your external IO lines fitted with some form of ESD protection?

No floating inputs on the micro?
Is everything run well within datasheet ratings?

I had an issue with a production board once where we suddenly started getting a very high failure rate, turned out the spi bus was being driven on the wrong clock edge and the first batch of the peripheral chips just happened to work with zero hold time, the next ones not so much.

You need to get a few duds back and investigate.

Regards, Dan.
Navigation
Message Index
Next page
There was an error while thanking
Thanking...

Go to full version
Powered by SMFPacks Advanced Attachments Uploader Mod