troubleshooting sagging power rail (overloaded?)

[cnt]

I'm trying to fix the flaky GPIB board on my keithley 197. 

The 1972 board has its own supply and is isolated, so the issues are thankfully constrained to the 1972 board and not the whole meter.  I originally thought my supply issues were a bad cap so I recapped the whole thing (the main filter cap was -50%)  but that didn't fix my problem.  Upon closer investigation, I've noticed that the power supply rail randomly sags down to 4ish volts from 5.1 and the GPIB controller IC and transciver ICs get much hotter than I'd consider normal for such slow logic (maybe 40C-50C, judging by touch).  The diode bridge also gets warm which is another indication to me that too much current is being drawn.  The bus transceivers all idle at >3v which I believe is the resting state so I'm not sure why they'd be hot. That said IDK whats hot and not hot for 1980's ICs.  The 6800 micro doesn't get warm at all though. 

Does anyone have thoughts on next trouble shooting steps?  When the supply is at a stable 5v I can get the GPIB to work properly so most stuff does seem to work still even if something has a fault that's causing excess current draw.

attached is a photo of the board with the warm chips circled.  I took it while recapping, but rest assured all the caps are in there of correct size and voltage and are high quality.

EDIT here's a scope capture of teh 5v rail doing its bad thing.  It sits at 5v for a while so i setup a single shot trigger at 4.5v to catch this.

[Refrigerator]

Is that scope capture supposed to represent a regulated 5V rail? Because if so it's terribly noisy.
I've had something similar happen with Philips PM5131 function gens, where the decoupling cap on the -22V rail regulator output went bad and caused the regulator to start oscillating at 1.5MHz.

[cnt]

lol yeah.  It falls to 3v in that capture.  I took a step back and looked at the input to the 7805 regulating that 5v, which is stepped down, rectified and filtered mains AC (this board gets its own winding). 
Ignore the high frequency stuff though.  that's 100mhz probe resonance aliased in because of my probing.  I can't seem to reduce that in my environment but using one of those spring grounds eliminates it when I care to use it.

It saggs and follows the 5v rail one dropout above.  I'm starting to think its not the circuit loading though.  I don't see any 120hz ripple when the 5v rail gets taken down so low.  so its not enough current to make the 2200uf filter cap insufficient.  but I did notice I can sometimes make a mechanical correlation to the rail dropping out.  and its so slow an "analog" like I'm started to suspect it's not a short or pin latchup or something.

Now i seem to be able to get even more correlated voltage sags by poking the mains transformer.  I think maybe just that secondary winding is having issues. 

[fzabkar]

Do you have any spray freeze?

Have you measured the load resistance on the 5V rail?

[cnt]

I don't have freeze spray or air cans. 

I did pull the board off and wire it across with jumper wires (only 6 wires) to measure the RMS AC load load current ~350mA.  I then replaced it with a power resistor of the best value I had laying around (33ohms about 250mA rms).   I was able to capture this wave form with a bit of poking on the transformer and it seems to get worse if I let it heat up.

I'm no transformer-oligist but I don't think they're supposed to do that. 

Could I have intermittently shorted windings on the secondary?

[Refrigerator]

Is it a toroidal transformer or just a regular E-I type?
I've seen videos of bad toroidal transformers in test gear.
But it is quite unusual that your scope probe would cause so much noise that it makes the measurement so misleading.
Anyways, i think the power supply instability could be causing a brown out or a latch up in your board.
It might be worth trying to power the board externally to isolate the power supply issue.
Does the transformer get hot/warm after some time running?
Also could you measure the unloaded output of the transformer when you poke it?

[m k]

Is in any better upside down?

[cnt]

Is it a toroidal transformer or just a regular E-I type?
I've seen videos of bad toroidal transformers in test gear.
But it is quite unusual that your scope probe would cause so much noise that it makes the measurement so misleading.
Anyways, i think the power supply instability could be causing a brown out or a latch up in your board.
It might be worth trying to power the board externally to isolate the power supply issue.
Does the transformer get hot/warm after some time running?
Also could you measure the unloaded output of the transformer when you poke it?

It's mounted on the PCB and E-I style. 

IDK whats causing it but its pretty typical for me to get 100-200mv of noise when using the long grounding lead.  It always goes away when I use the little spring ground wire. 

I don't have a low impedance source of AC right now.  so I can't fully power it externally, since it has a -8v rail too.  but I squinted at the schematics and I think not supplying -8 is probably okay. If not okay I think only the analog output on the board would catch fire which is not important to me.  <hold while i hook it up...>  everything's okay applying 8.5v to the output pins of the bridge rectifier (and of course not attaching the transformer)  about 270mA DC, very steady.  I'll let it run for a couple hours and see whats what


Transformer gets warm.  I'd say 35C, using my fingers.

Unloaded and cold it doesn't glitch that I've seen.  I tried the resistor dummy load to see if the voltage was sagging because of transient loading and that doesn't seem to be the case, or at least not the only case two things can be broken at once or one cause fault in the other.

[Refrigerator]

Is it a toroidal transformer or just a regular E-I type?
I've seen videos of bad toroidal transformers in test gear.
But it is quite unusual that your scope probe would cause so much noise that it makes the measurement so misleading.
Anyways, i think the power supply instability could be causing a brown out or a latch up in your board.
It might be worth trying to power the board externally to isolate the power supply issue.
Does the transformer get hot/warm after some time running?
Also could you measure the unloaded output of the transformer when you poke it?

It's mounted on the PCB and E-I style. 

IDK whats causing it but its pretty typical for me to get 100-200mv of noise when using the long grounding lead.  It always goes away when I use the little spring ground wire. 

I don't have a low impedance source of AC right now.  so I can't fully power it externally, since it has a -8v rail too.  but I squinted at the schematics and I think not supplying -8 is probably okay. If not okay I think only the analog output on the board would catch fire which is not important to me.  <hold while i hook it up...>  everything's okay applying 8.5v to the output pins of the bridge rectifier (and of course not attaching the transformer)  about 270mA DC, very steady.  I'll let it run for a couple hours and see whats what


Transformer gets warm.  I'd say 35C, using my fingers.

Unloaded and cold it doesn't glitch that I've seen.  I tried the resistor dummy load to see if the voltage was sagging because of transient loading and that doesn't seem to be the case, or at least not the only case two things can be broken at once or one cause fault in the other.

If the transformer is soldered to the board through pins i'd check the pins and also the wires going to those pins from inside the transformer.
I don't like transformers that get hot but i don't know this bit of kit well enough to be certain about the temperature.
The unloaded voltage would fall if windings were shorting, but if it doesn't then perhaps it's a series resistance problem (a break in continuity).
So far this looks like a transformer problem.

[cnt]

IDK if its manufactured this way or if there was rework done on the transformer in the past but there is uncleaned flux on the transformer joints.  It's also odd that the two highest current windings (Judging from the filter cap sizing GPIB takes twice the current vs the whole DMM system) have the smallest traces. 

It looks like at least one of the pins is getting more heat than the rest judging from the browning and the state of the flux.  I'm tempted to bodge-wire in two 22awg wires in parallel with those tiny traces and see if that fixes it. 

Also seems weird to me that those secondary windings were put on the mains side of the bobbin instead of the secondary side, but again idk much about transformers so maybe that's common.

[cnt]

ok i'm pretty sure there's a crack or something in that right trace.  I clipped to the transformer pin and the board to board connector pin and the resistance was all over from 0.6ohms to 5+ ohms (leads are about 0.4 ohms shorted).

Also attached is 6 seconds of the voltage across the trace.  the big dip at T=0.8s is me poking the transformer right above the offending pin.  I think a bodge wire is in order.  I'm going to solder it up and leave it running for a day or two and see how it goes.

[Refrigerator]

Those traces look unnecessarily thin.
I'd suspect those too, they immediately caught my attention.

[m k]

My pick is also that right red, first suspicious thing being the lack of brightness.
I've used wall mount Cat5 for those wire repairs, its insulation is easy and conduit soft.

Above right red you seems to have two after soldering cutted legs.
There you can guess how things were done back in the day.
It doesn't say that the transformer is new, maybe they were just a bit lazy at that part of the production line.

Left violet seems to be cracked.
Maybe it's just a picture.

You can still clean those Transformer solderings and possibly see the crack clearly.
Make a brush, stiff roll of toilet paper or some other soft tissue and tape it around, not from bottom but close.
Now get a contact cleaner and if it's evaporating fast use more and spray into the brush also.
Then brush the flux away.
You'll get some mess of paper shreds of course but that's easily cleared and the board will be clean.

[cnt]

Its been about a day and its been running like a champ.  The top of the case gets slightly warm over the area with the 7805 and gpib chips.   With the case open they were running about 45c which seems hot to me, but I don't have any reference to know whats normal for this meter.

I'm wondering if there's damage to those chips too (eg bad bus transceiver chips loading down the gpib controller chip).  not sure how to test that without pulling the chips or cutting leads.