Nasty bug found to be from the PCB fab

[jolshefsky]

I have been chasing a bug for two days now and finally found it: on my test rig.

I was trying to diagnose a problem identified at another site. They sent me the device to check and I couldn't reproduce their error, but was noticing a different one. It seemed the whole board was bad (the assessment I was given was that it was an intermittent problem that indicated a cold solder joint or a cracked trace, so I was already planning to replace the whole board.) So I replaced the board ... and encountered exactly the same problem again. This was part of a circuit path that I had never previously tested, so I assumed it was a problem in the board design or some such thing. After poring over schematics and doing continuity tests, I eventually determined why I was getting the problem: I had a short where there should be no short. It didn't take long before I had the test jig detached and was debugging it. I had an inexplicable short which I thought might be due to some bodges, but upon tracing it, I found the problem: See "Before.jpg".

The copper pour had a manufacturing defect which attached it to a via. I don't mean to bash OSHPark because their service's primary function is cheap prototypes, so they omit testing. On our main boards, our manufacturer does full electronic testing which, I guess, would have handily caught this error and I never would have seen the board.

Scratching away with an X-acto, I cut the offending trace and fixed the problem: See "After.jpg".

I guess this is just a cautionary tale, but I'm not sure what to take away from it. The test rig I was using I had no problem with for 2 years, but since I was in a new circuit path, I encountered a problem I didn't expect.

I think it would behoove me to keep a notebook. I should write down what the expected result is, and what the measured result, marking all issues where the results were far astray, even if (especially if) it was only a momentary problem. While this would not have solved my problem, a more methodical approach may have led me to an answer quicker and with less frustration.

[rx8pilot]

Have you closely inspected your gerbers? Sometimes the gerber creation can end up with some extra geometry.

[DerekG]

Yep, check your Gerbers very closely.

If your Gerbers are good (open them using two different viewers to be sure), then the problem either occurred at the fab shop due to the use of incompatible apertures or the photo plots/film were damaged before the boards were made.

[Monkeh]

Those vias barely worked, why are your annular rings so small?

[krivx]

Those vias barely worked, why are your annular rings so small?

Yeah, with that much clearance around the track and via the ring could stand to be much larger...

[JuKu]

(open them using two different viewers to be sure)

+1. I've seen a bug that was the same in generation and view.

[wraper]

Those vias barely worked, why are your annular rings so small?

+1, Such gerbers should be rejected by factory if checked as they are against design rules of any factory I'm aware of.

[Psi]

Damn, there really is no angular ring there at all.

[jolshefsky]

Re: Gerbers

OSHPark sent 3 boards and the other one I have here does not have the problem (it looks normal with normal clearance). As a spot-check, gerbview shows no error. And the OSHPark render does not show the error.

Re: Annular rings

Good idea. I'll update the design rules. Although it really doesn't need to be much at all (save for higher-currents) since the connecting trace provides the copper to tie it together. Nonetheless, an annular ring will prevent rare but un-findable bugs from appearing.

[Monkeh]

Re: Gerbers

OSHPark sent 3 boards and the other one I have here does not have the problem (it looks normal with normal clearance). As a spot-check, gerbview shows no error. And the OSHPark render does not show the error.

Production error, it does happen.

Re: Annular rings

Good idea. I'll update the design rules. Although it really doesn't need to be much at all (save for higher-currents) since the connecting trace provides the copper to tie it together. Nonetheless, an annular ring will prevent rare but un-findable bugs from appearing.

OSHPark specify a 7mil annular ring for 2-layer boards, and that's with good reason. The drill size varies and the accuracy is not perfect.

[rx8pilot]

Re: Annular rings

Good idea. I'll update the design rules. Although it really doesn't need to be much at all (save for higher-currents) since the connecting trace provides the copper to tie it together. Nonetheless, an annular ring will prevent rare but un-findable bugs from appearing.

The reason for oversized annular rings is not [just] for current, it is to make room for manufacturing errors. The holes for a via are drilled in a setup that is entirely different from the process that makes the copper traces. The drill machine must align itself based on panel fiducials but that is not always perfect. The drill itself can and will wander a little as it makes contact with the PCB. The spindle runout, geometry of the tool, the age of the tool, the RPM, and the feed rate will all contribute to the variations from absolute position of the hole. If the annular ring is properly sized, it will be slightly bigger than the sum of all the position tolerances. The cheap PCB's do not get much of a gerber inspection, and unlikely to get any other inspections which is why they are cheap. The vast majority of the time, it's fine. I have learned that I get a much better pass rate if I design the board so it is well inside the manufacturing limitations.

[steve_w]

Hey jolshefsky,

Any chance you can put your gerbers up?

would be interesting to see the difference.


regards

SW

[IconicPCB]

This looks like a lift off problem.

Lift off refers to a perfectly imaged panel which then goes into further processing ( developing, protecive zinc plating ,pattern plating etc...)

During the early process zinc is plated protect copper.

Sometimes photo resists lifts off inadvertently allowing unwanted copper to be zinc plated. What You see there looks just about right to be called liftoff problem.
QA/QC problem in manufacturing. Should have been picked up by electrical testing of the bare board.

[T3sl4co1l]

If you're ordering from bottom-dollar sources like Seeed, check the boards yourself.  They sure as hell aren't making enough money to check them for you!

Also apply generous design rules.  I typically go with 10/10 width/space rules and 10 or 15 mil annular rings, for anything that doesn't need fine pitch or compact size, regardless of who's manufacturing it.

Tim

[IconicPCB]

Tessla Coil.... Spot on

I see too many designs where people use components or shop capabilities to the limit just because they are there.

The designer ought to consider the cost of that extra couple of thousands of an inch.

I have seen designs where quite senior designers mix 0201 components with power ferrite chokes simply because that's what's to hand despite having large areas of the PCB unused.
 Similarly 0.3 mm via holes despite the fact that the PCB density does not warrant such small vias.

Common sense ought to dictate that a challenge is not always needed to bring out optimal results.

[wraper]

This looks like a lift off problem.

Lift off refers to a perfectly imaged panel which then goes into further processing ( developing, protecive zinc plating ,pattern plating etc...)

During the early process zinc is plated protect copper.

Sometimes photo resists lifts off inadvertently allowing unwanted copper to be zinc plated. What You see there looks just about right to be called liftoff problem.
QA/QC problem in manufacturing. Should have been picked up by electrical testing of the bare board.

At the same place on entire batch of the boards?

[IconicPCB]

Possibly. we have no indication what the real artwork looks like and therefore no idea whether it may have been rubbish on the photo tool.