Shorted industrial controller

[MultiMike]

I'm a millwright and an electrician (millwright first).
I do a lot of repairs, and I do major electrical installations.
I'm getting into electronics as a hobby.
I'm trying to fix an industrial controller stopped working.
The controller has one major board which takes various AC voltages from a small multi-tap transformer.  These are fed through a bank of diodes acting as half-wave rectifiers.  There is a 7805 and a 7815 on a heat-sink.   My problem is with the 5V bus (of course).   
The output from the 7805 is only about 2V.  So I immediately replaced the 7805, which was not the right call (as most of you probably know).  The new VR does the same as the old, because there is a short somewhere on the 5V rail.   Now I need to find it.   I'm trying to decide how to do that.

I've been beating my head against it at home, because my electronic stuff is there.   Now I think I've hit a wall and I need advice on how to find this short.   I'm going to check my meters at work to see if I have one that will do at least 0.01 ohm resolution.   Might have one.   I'd love to build a Shorty but there is a long lead time on the chips from the supplier I checked.   I have a power supply at home, and a thermal camera at work.

Any thoughts?

Mike

[coromonadalix]

you have to find the "short"  by isolating sections ....  or if you have access to a thermal camera it could help ?

[kripton2035]

my shorty "clone" uses standard IC that are available AFAIK
http://kripton2035.free.fr/Projects/shorty-display.html

[TheMG]

Since it's not a dead short and you're still getting 2V out of the 7805, something has got to be getting warm (other than the 7805 itself which is of course going to be quite hot).

As long as there's no high voltages lurking about on the circuit board, you can always use you fingers to feel the temperature of various components, ICs especially but don't overlook any MLCC bypass caps I've seen those go bad and load down a 5V supply.

Another trick is to dab some isopropyl alcohol on the components, it will evaporate far more quickly from any hot areas.

[PaulAm]

If you have access to a milliohm meter, you can put one lead on ground and follow the +5 around until you find a  minima, which will be at the problematic part.  Dead shorts are a little easier, but it should still work.

If you don't have one of those, it's probably worth building the shorty mentioned by kripton2035.  It's a pretty simple project and will make finding the problem a whole lot easier.  If you don't want to mess with a microcontroller, there are discrete milliohm meter projects available online.

Otherwise, it's divide and conquer.  Isolate the problem to a board by disconnecting things or using thermal cues and persevere.  The right tools can make finding the problem an easy 15 minute task.  Without them, it just feels like this: 

[WattsThat]

The 7805 will go into shutdown and limit its output power, making it difficult to impossible to find the offending component thermally.

One way to find a shorted component would be to remove the 7805 and clip in a circuit limited bench supply capable of at least 2-3 amps at 5 volts. Turn up the amperage, start at an amp so you don’t damage any traces. You’ll find the shorted component quickly with a thermal camera or your finger. Just don’t press hard, you’ll be amazed at at how hot a small blob of epoxy can get with 5-10 watts.

Admittedly, this is a production level troubleshooting technique that may not be a good idea with vintage or one of a kind expensive or otherwise valuable gear. But it sure finds a shorted op amp or similar components quickly.

[MultiMike]

Since it's not a dead short and you're still getting 2V out of the 7805, something has got to be getting warm (other than the 7805 itself which is of course going to be quite hot).

At this point the 7805 is out of the board.  I'm injecting power directly to the bus on a suspicious-looking spot that may well have been used for this purpose before (the solder mask has been scraped off).   I guess the question now is this - how much power can I pump into this thing without wrecking things?  Should I say "well, the 7805 is rated for 1.5A so 1.5A is the maximum I can put in there."  Or if that doesn't heat anything up enough can I go beyond that with the assumption that something is going to be taking the lion's share of the current and the devices that are okay are not going to be harmed.

I can take the thermal camera home and see if I can get anything to poke its head up.

As long as there's no high voltages lurking about on the circuit board, you can always use you fingers to feel the temperature of various components, ICs especially but don't overlook any MLCC bypass caps I've seen those go bad and load down a 5V supply.

Geez.... This thing has a dozen of those caps.   Ah well.

Thanks for the reply.  I  have more to read, but I'm looking forward to taking another swing at this thing this evening after work.

Mike

[MultiMike]

If you have access to a milliohm meter, you can put one lead on ground and follow the +5 around until you find a  minima, which will be at the problematic part.  Dead shorts are a little easier, but it should still work.

I checked my meters at work and none of them do better than 0.1 ohm (including the Flukes).  As I was digging through my old meters I found that I do, however, have an ancient Micronta/Radio Shack analog meter with a 1 ohm range that might be able to do it.   I remember buying that thing when I was around 13 years old.   That's a looooong time ago.

If you don't have one of those, it's probably worth building the shorty mentioned by kripton2035.  It's a pretty simple project and will make finding the problem a whole lot easier.  If you don't want to mess with a microcontroller, there are discrete milliohm meter projects available online.

I would LOVE to build a Shorty, but I looked up the two chips on Mouser and I got 23 weeks for one and 56 weeks for the other.  Not really reasonable.  I'll check again with Digikey and other suppliers this evening, but so far it looks like a non-feasible option at this point in time unless there are cross-over equivalent components that I just don't know about (very possible!) which are more common, but I wouldn't want to use a different chip and have the device end up not being nearly as accurate.

Mike

[shakalnokturn]

I have a power supply at home, and a thermal camera at work.

The easiest would be to have both together you should find the defect(s) in no time with both.

As said by TheMG with 2V (not a dead short) you should be able to touch the heat somewhere.

Otherwise rig the lab supply to the 5V line, set a safe voltage (around 4.5V). Then bring up the current to a reasonable value depending on power rail track thickness.
The problem should show up in no time with the thermal camera.
Remove or replace the cause and re-check until you can reach 5V within 7805's current range.
Keep in mind that some components may have multiple power rails and may be damaged by a missing rail if there is enough current available on an input.

[PaulAm]

Here's a milliohm adapter for your dmm that has a handful of parts:

https://www.edn.com/simple-circuit-serves-as-milliohmmeter/

Nothing special about the parts, you should be able to throw it together if you have any junkbox parts.  The text is informative.

Here's one that just uses a dmm with a clever technique:

https://www.edn.com/quickly-find-pc-board-shorts-with-low-cost-tracer-technique/

Lots of good stuff at EDN

[kripton2035]

Quote from: MultiMike on Today at 18:09:39

I would LOVE to build a Shorty, but I looked up the two chips on Mouser and I got 23 weeks for one and 56 weeks for the other.  Not really reasonable.  I'll check again with Digikey and other suppliers this evening, but so far it looks like a non-feasible option at this point in time unless there are cross-over equivalent components that I just don't know about (very possible!) which are more common, but I wouldn't want to use a different chip and have the device end up not being nearly as accurate.

with MY shorty (with display) clone
AD8628 is in stock by mouser
MCP3421 is in stock by mouser (9000+ ex...)
an arduino nano is available everywhere from ebay to aliexpress to bangood, but needs 15-30 days to get to your home
same for an i2c oled 0.96" display.
plus some resistors and capacitors.

[MultiMike]

I have a power supply at home, and a thermal camera at work.

The easiest would be to have both together you should find the defect(s) in no time with both.

So, I brought the thermal camera home and found the short in about 5 seconds.

Geez.  I feel pretty silly having spent frickin' hours pulling up one leg on a bunch of caps on the weekend.  Sigh.
Another little annoying factor is that that chip had a red sticker on it.  A few of these chips have blue stickers on them, and this one had a red sticker.  Maybe somebody in the past had already figured out this chip was bad?  Next time I'll pull out chips with red stickers first.
Okay, so here's the crappy part... the defective component is a chip, and it appears to be a programmable CMOS chip.  It's a PALCE 16V8H-25-PC/4.
I can buy the chip, but who's to say what was on it?   I need to spend some time with the data sheet and see what is used to program them, and then I can decide whether that's a dead end or not.

I'm not going to bother putting the rest of this board back together until I figure out whether this chip is replaceable or not.

Thank you all for the help!   It is greatly appreciated.

Mike

[shakalnokturn]

A programmable logic array, a part used to define custom logic functions. They're easy to program with the adequate hardware, that can be found cheaply now. The programming involves a .JED file containing a "fuse map" for the device that defines the internal interlinks between basic logic gates.

Unfortunately there are not that many options...
The easiest is having access to the original .JED file for the specific use in that circuit. (You'd be very lucky...)

Having access to an identical board with a working PALCE that the fuse map can be read from. (More often than not they are read protected though.)

After that there aren't many other options than reverse engineering by either logic probing a working board or having the schematics and exact I/O function of the board to try to deduce the logic needed in the PAL.

[perieanuo]

take a good power variable supply with 10-30 amps capability, inject voltage on that pcb starting from 0V raising slowly with current limitation let's say 1-2 amps (depending on your pcb, could be more, in laptops i used sometimes over 10-12A) and your dead rabbit will heat
or maybe the bad component heats already with your existent 5V 7805 regulator, try with back of your hand
but to repair stuff like that you should be more experienced 

[PaulAm]

It's possible that some other part than the GAL is bad and is drawing enough current to raise it's temperature.  Because it will be nearly impossible to burn a new one, I'd follow that avenue first.  Go back to tracking where the short is.  If you have a dmm that can resolve 1-10mV,  get a flashlight battery, a resistor and read the second EDN link I posted above, them spend 15 minutes tracking down where the short it.  If you get to the GAL, well then you have a problem.

The problem is not programming the GAL, programmers are easily available and GALs are cheap, the problem is getting the data file to program it with.

[MultiMike]

It's possible that some other part than the GAL is bad and is drawing enough current to raise it's temperature.  Because it will be nearly impossible to burn a new one, I'd follow that avenue first.

I pulled out that chip and the short went away, so it seems to be the culprit.  I have contacted the manufacturer and it looks like they might be able to provide a new chip.  This chip, by the way, is from 1995.  Approaching 30 years old.  Not a new one.  Newer than *me*, though.   

I'm waiting for a quote for a new chip.  If I can get one I will try to copy the programming first (a programmer is only $15 or less) which may or may not work, but if that chip died for a reason then it would be worthwhile to be able to replace it without going to the manufacturer.  Might not work, I know.

Mike

[PaulAm]

If you can get a new chip, you may be able to reverse engineer it.  There are a few tools available, here's one set from gitnub that may work.

https://github.com/psurply/ReGAL

Unfortunately, it's not like an eprom that you can just read out.  Good luck

[james_s]

Can you tell what it is used for in the circuit? Is there a chance that there are multiple chips of the same type that serve the same purpose? It's kind of a long shot but it's possible. People do also manage to reverse engineer the logic inside programmable gate arrays but it requires an understanding of how the circuit is supposed to work, followed by some educated guesses and luck.

[MultiMike]

Can you tell what it is used for in the circuit? Is there a chance that there are multiple chips of the same type that serve the same purpose? It's kind of a long shot but it's possible. People do also manage to reverse engineer the logic inside programmable gate arrays but it requires an understanding of how the circuit is supposed to work, followed by some educated guesses and luck.

Final update:

   I was just speaking to the manufacturer today, and they told me that the EEPROMs in these units from the mid 90's had some problems.   They offered to repair the unit.  For free.   This thing is almost 30 years old!  I am... gobsmacked that they offered to fix it, but I'm not holding out for pride!  They can fix it and I will HAPPILY use it.   I learned a lot in the process of trying to fix it, so it's not time wasted.  It will come back re-wired with a new (old, refurbished) motherboard and in full fighting trim.   Excellent. 

Mike

[james_s]

Well it's hard to argue with an offer like that. The manufacture should certainly be applauded, quite a few would not provide any information at all or even be willing to touch it.

[shakalnokturn]

And the manufacturer with good customer support is called?

[MultiMike]

The supplier is a specialty manufacturer that I doubt 1% of people on this board would have ever heard of.   Unless you're pretty familiar with gravimetric blending it just won't be in your world.
I'm not certain why I am feeling hesitant to name the manufacturer.  It's almost like I don't want to do anything that could possibly 'upset the apple-cart' before receiving the controller back in a repaired state.  I'm not claiming that my hesitancy is sensible... but the offer to fix it for free was so totally out of left field that I just.... I don't want to poke at it.

Mike

[shakalnokturn]

No worries... You're right there's next to no chance I've ever heard of them. Anyway nice to know some customer service still exists.

[pcwrangler]

...They can fix it and I will HAPPILY use it.   I learned a lot in the process of trying to fix it, so it's not time wasted.  It will come back re-wired with a new (old, refurbished) motherboard and in full fighting trim....

 Did they tell you how they fix the problem? Do they replace the chip or apply a whole new solution, preventing the issue in the future? If it's just a new chip you might want to dump a copy of it if it's not read-protected. Great support though! Definitely a  company to stick with.

[james_s]

It sounded like they had problems with a bad run of chips, so I suspect they're replacing it with a new, ostensibly more reliable chip.