Intermittent FPGA board problem/repair suggestions needed

[twitzelbos]

Hello all,

I have been looking at a problem with a FPGA board for a while. The board is over 15 years old and now has the problem that it doesn't power on correctly one in three times. The other two times it works perfectly fine. It is powered of 5V but has a local 3.3V supply consisting of a LT1086-3.3 regulator. Initially I have noticed that the voltage is kind of low (3.2V) when the board doesn't work. Now I have lifted the output pin of the LT1086 off the board and spliced in a DMM. I can confirm that the LT1086 outputs 3.3V dead on when not loaded. The board draws 100.5 mA from the regulator in every instance that it works. In cases when it doesn't work it draws about 520 mA. To make it work its enough to turn off the 5V supply to the board and turn on again or keep trying till it works. There is no mechanical interaction with the board at all. This problem started a couple of months ago after over 15 years of reliable operation. Besides the Altera FPGA there is a few logic IC, an Analog Devices DSP and a MAX811T on the board, the rest is all bypass caps and a few resistors.
I have no idea what component would show this behaviour as failure mode, so I'm a bit at a loss and would appreciate any ideas on where to look. I have always thought when something is broken it should be permanently broken, not work perfectly fine most of the time. It should be noted, when it powers on correctly it can stay on working correctly for days on, the failure occurs exclusively at the time of power on.

Thank you for your input.
Thomas

[Le_Bassiste]

admittedly not being an expert in fpga affairs, this sounds to me like hefty overclocking. maybe a clock source's gone wild, starting at an overtone every now and then? 
 

[Kilrah]

When the board draws 5x more current, you should be able to feel which component is getting warmer than usual. Would be a good starting point.

A possibility could be faulty caps that cause improper voltage ramp-up sequences on the different rails.

[twitzelbos]

Thanks,

I did already on a hunch replace the wet electrolytic capacitors with no effect. Next would be the tantalums. Sadly this is a VME board with multiple power pins, so its hard to reach in action for feeling heat. I found a cheap riser card on amazon and when that gets here maybe its time for some FLIR action. There is about 25 or so tantalums on this board, so I don't want to replace them all...

Thomas

[singapol]

Thanks,

I did already on a hunch replace the wet electrolytic capacitors with no effect. Next would be the tantalums. Sadly this is a VME board with multiple power pins, so its hard to reach in action for feeling heat. I found a cheap riser card on amazon and when that gets here maybe its time for some FLIR action. There is about 25 or so tantalums on this board, so I don't want to replace them all...

Thomas

Get a cheap MESR100 V2 esr meter or the combo cap tester. I'll  bet there are a few tantalums with higher esr. If you value this FPGA board then change all the tantalums which uses the same batch of tantalums which sooner or later may develop high esr. Yes I know it's going to cost money...your choice or take a gamble. PS check the solder joints for the obvious cold solder or cracked joints.

[twitzelbos]

The plot thickens....

I have used an ESR meter and found all tantalums and electrolytic capacitors to be in a good shape. A new bit of information arose from using the VME riser card. While the board was on the riser, sticking out of the chassis, I could NEVER get it to fail, after many many power cycles. So I decided it might be temperature related, and I put an antistatic bag over the board for 10 minutes to let it warm up a bit. I noticed that the supply current dropped from 100 mA to 99 mA very steadily while the bag was on, and as soon I removed it, I started rising again. Long story short, the board temperature increased to 92 Fahrenheit in those 10 minutes (87F before), and a voila on the next power cycle it failed. Interestingly, the only component that changed temperature in failed state besides the 3.3V regulator, was the ADSP-21065L DSP (normally 90 F, in failed state 109 F). I power cycled one more time, the board booted and the ADSP went back to 91F temperature. Thats where I'm at now.....

T

[helius]

Intermittent high-current draw from power-on is probably latchup. This is a difficult thing to fix without schematics.
One thing to consider is that the board should have a power supervisor reset circuit. The way these commonly work is that the board is held in reset until a constant current source can charge a capacitor to a value that trips a comparator. So if this timing capacitor is no longer the correct value, or if the comparator's reference (a zener), or the comparator itself is bad, then the supervisor may allow reset to go high too soon (making some other chip latch up). You should be able to follow this with a scope.

[Jwalling]

Intermittent high-current draw from power-on is probably latchup.

Yup, that's what I think as well. I'll bet there's one chip that gets very warm when the failure occurs.

Jay

[helius]

Yup, that's what I think as well. I'll bet there's one chip that gets very warm when the failure occurs.

Another approach is to wire a tact switch onto the reset line. As long as you hold it in, it pulls reset low (through a 100 ohm or something like that). Before powering the board up, hold that switch down, then release it a few seconds later. If doing this completely fixes the latchup, then you know the problem is in the reset circuit.

[twitzelbos]

I believe the board has a MAX811T for exactly that purpose. Its not getting warm. I'll scope it. The only part getting super hot when there is the suspected "latchup" is the ADSP-21065L DSP itself, and the 3.3V regulator (LT1086CS-3.3), everything else stays the same, as confirmed by FLIR.

-T

[Jwalling]

I believe the board has a MAX811T for exactly that purpose. Its not getting warm. I'll scope it. The only part getting super hot when there is the suspected "latchup" is the ADSP-21065L DSP itself, and the 3.3V regulator (LT1086CS-3.3), everything else stays the same, as confirmed by FLIR.

-T

If it does turn out to be the DSP, I hope it's not the BGA package. 

Jay

[twitzelbos]

Its the 208 pin MQFP, still near impossible to repair for me as the board has a lot of components that cannot go into reflow (and green wiring)...

-T

[Jwalling]

Its the 208 pin MQFP, still near impossible to repair for me as the board has a lot of components that cannot go into reflow (and green wiring)...

-T

Still, it's possible to do. I've done solder work (fractured joints) on the ACQ memory controllers that are in TDS700X scopes. They are 308 pin quad flat-paks which I think have a .5mm pitch. Magnification for the eyes and a few beers to steady the hands are key for me. I would recommend a hot air rework station to remove the old part, though...

Jay