HP 3456A Repair (Selftest Error -3)

[Kinkless Tetrode]

I scored a broken HP 3456A for cheap and saved it from being eWaste.  It starts up and displays all dashes (------) and the selftest fails at test #3.  It's a 2201 series, in reasonably good condition, and I am working through the service manual.

The unit must have been in storage for a while because all the PCBs are peppered with a pulverized foam.  This powdery foam is sticky and does not blow off easily with compressed air.  Looks like all the boards can use an isopropyl wash.  Does foam tend to be conductive?

I took a look at the inguard power supply rails and the voltages look normal, except that the +33V to 46V rail has a 2V peak-to-peak 60 Hz ripple on it.  The ripple on the other supply rails are in the millivolts.  Looks like the PS will be getting an electrolytic cap replacement.  Could this large ripple be contributing to the test #3 failure?

Anyhow, moving through the service procedure, Service Group A, 8-A-34, for all dashes and error #3, points to an outguard isolation logic board problem.  So, this is where I am confused: paragraph 8-A-38-(b) states "Connect the vertical input of an oscilloscope to the core of transformer A3T1.  Using the scope..check for the waveform shown...(Figure 8-A-7. Transmitter Output).

1. How does one connect the scope to the core of transformer A3T1?  The transformer has no visible test points and is fully insulated and varnished.  I am confused.

2. So, instead I put the scope probe at pin 8 of U19 because this leads into the primary of A3T1. The waveform looks similar to, but not the same as, the ideal transmitter output waveform according to section 8-A-39.  The ideal waveform is 2 high, 3 low, and 4 high.  But it appears that the U19 waveform instead is 2 high, 2 low, and 4 high.  Please see photo below, with upper trace waveform.  The signal I am seeing appears to be missing a tick.  Perhaps, the counter U1 is bad?

3. Finally, I noticed that U13 has some greenish copper oxidation on the pins. Corrosion?  I see this on a few other places on the board also and it is preventing a clean contact with scope and multimeter probes.  Has anyone seen something like this before?  Would desoldering the devices and resoldering them back into place be thing to do for this?

Anybody have any ideas?

[Kleinstein]

The number of pulses transmitted and the shape can depend on the data transmitted so it may look a little different. It would be unusual for a counter to fail by sending out one pulse to little.
Checking the core should likely be looking at one winding, possibly to put an extra loop through.


The last picture looks like corrosion, e.g. about what one gets from a leaking cap of NiCd cell.

In sensitive areas it may need unsoldering parts from the effected areas.
Cleaning would be initially with water, maybe dilute vinegar (if from a leaking NiCd cell) and some mechanical help from a brush. IPA would be only a last step, more to dry things up. The corrosion products are better soluble in water.

[Kinkless Tetrode]

The last picture looks like corrosion, e.g. about what one gets from a leaking cap of NiCd cell.

In sensitive areas it may need unsoldering parts from the effected areas.
Cleaning would be initially with water, maybe dilute vinegar (if from a leaking NiCd cell) and some mechanical help from a brush. IPA would be only a last step, more to dry things up. The corrosion products are better soluble in water.

Agreed, it sure looks like leaking battery acid.  Fortunately, the 3456A does not have any battery to worry about.  So, this corrosion is a bit odd to me.  Some of it has begun to eat into a few traces surrounding some of the pullup resistors.

So I'm going to have to desolder, clean, and then resolder more than a few parts back on to the board.

I guess, while I have the chips removed anyway, I should test each of them on a breadboard before cleaning them and putting them back into the PCB.

[Kleinstein]

For simple parts (simple chips, caps, diodes and resistors) it may be easier to replace them with new ones. Especially carbon resistors often drift quite a lot over time.  At least the digital part is usually not that sensitive.

The corrosion could be due to just water or similar during storage. 

[Kinkless Tetrode]

For simple parts (simple chips, caps, diodes and resistors) it may be easier to replace them with new ones. Especially carbon resistors often drift quite a lot over time.  At least the digital part is usually not that sensitive.

The corrosion could be due to just water or similar during storage. 

Very true with resistor drift. A couple of the 1K pull-up resistors were measuring 600 ohms. Looks like I will be becoming very familiar with each of the parts in this meter.   

[Kinkless Tetrode]

I have been troubleshooting this 3456A error -3 and it certainly is a very generic error.

I have been following the service manual and it has taken me through the power supply boards, the Inguard and the Outguard logic boards.  The Inguard and Outguard interface transmitters and receivers look ok, but they are faithfully conveying a bad signal through the channel.

Also, the power supply voltage doubler caps C7 and C8 were replaced, due to a large ripple on the +45V rail.  The other rails, including the +5V digital, all look good with ripple in the microvolts.

So, tracing the source of the bad waveform has been a very educational experience. Each time it seems that I may have found the problem, such as corrosion on the leads of a chip, new power supply caps, or cold solder joints, the suspect chips so far has been proven to be innocent after all.  Tracing the signal back through boards A30, A40, A4, and  A3, the origin of the nonconforming waveform appears to be A4U15. The MC68A00P microprocessor on the Main Controller board. The inputs to the suspect microprocessor look normal and the main clock is at the right voltage and frequency, but looks very noisy.  But I am not completely sure the uP is at fault since I have not experienced a failed microprocessor before and I don't know what it should look like.  Could an noisy clock signal create such nasty uP output? 

The photos below show scope waveforms from the A4U15 outputs.  I don't have an HP Signature Analyzer, so, can anyone tell if this looks like a bad microprocessor chip?

Photo 1:
Here is the noisy A4U15 uP clock input.  Is this a good clock signal? Looks very noisy:


Photo 2:
Here is the crystal input to this clock chip.  Does not look right to me:


Photo 3:
Just to confirm, here is the crystal signal from the A30 board.  Now this looks good:


Photo 4:
Here is address line output from the A4U15 uP, address line 0.  No way this is good:


Photo 5:
And, finally, this is address line output from the A4U15 uP, address line 9.  Also, bad:


The scope waveforms of the uP output are not static but are jumpy and noisy, and I see the same similar noisy waveforms after powering the system off and back on again.

There are likely many other problems also, but it's no way that this A4U15 MC68A00P uP is functioning close to properly, right?

[jklasdf]

Don't be so quick to consider A4U15 bad. Photo 1 seems fine, depending on how you're probing. It looks like there is some ringing on the rising and falling edge caused by too long of a ground lead. Where are you probing for Photo 2. Based on the fact that the clock input to the uP A4U15 (the output from the clock generator) looks good, I'd suspect the crystal input to the clock generator is good. There is one more phase of the clock input to uP A4U15 to check.

Photos 4 and 5 look good, the address line transitions are spaced about 0.7us apart if there is a transition. Sometimes there is a transition and sometimes there isn't (not a fixed pattern), so that would explain why it's "jumpy".

In general there isn't much that go wrong with these meters. Older serial numbered multimeters had EPROMs which would bit rot, but yours is a newer serial number. The most common problem is dried out electrolytic capacitors in the power supply section. These can lead to weird errors. Make sure you measure the ripple output carefully and replace any that show ripple (may be worth it to just do a group replacement of all of them if any need replacing).

[Kleinstein]

The clock shows quite some ringing. This could also be due to a proor ground lead connection. Still it is a digital signal, so it may be acceptable.  Still I consider this the worst of the waveforms shown.  One could try some RC snubber or similar. One could also check the supply, if there is ringing on the 5 V. Sometimes replacing caps with modern low ESR ones can make things worse.

The crystal signal looks a little odd, but the crystal signal are usually rather high impedance and thus easy to upset with the probe. It's also possible to have an odd loading to the crystal so that the next mode of the crystal is close to 3 times the fundamental. From the waveform I would more suspect the crystal or the caps than the µC. However the clock output looks OK (stable clock), so I would not worry about this. One could still check the caps at the crystal for breaks or cold solder joints.

The address line signals look good and relatively clean. The signal is not simple periodic which causes the superposition in a kind of eye diagram like style - this is normal for a µC running more than short loop.

[Kinkless Tetrode]

Don't be so quick to consider A4U15 bad. Photo 1 seems fine, depending on how you're probing. It looks like there is some ringing on the rising and falling edge caused by too long of a ground lead. Where are you probing for Photo 2. Based on the fact that the clock input to the uP A4U15 (the output from the clock generator) looks good, I'd suspect the crystal input to the clock generator is good. There is one more phase of the clock input to uP A4U15 to check.

Thanks, for the verification that these waveforms are not bad.  I didn't realize that a good clock signal could look so messy.  Yes, both clocks look similar.  The photo 2 clock signal is from test point TP3 on the A30 board.

Photos 4 and 5 look good, the address line transitions are spaced about 0.7us apart if there is a transition. Sometimes there is a transition and sometimes there isn't (not a fixed pattern), so that would explain why it's "jumpy".

I see, I didn't realize that the address line transitions would have a pattern like that.

In general there isn't much that go wrong with these meters. Older serial numbered multimeters had EPROMs which would bit rot, but yours is a newer serial number. The most common problem is dried out electrolytic capacitors in the power supply section. These can lead to weird errors. Make sure you measure the ripple output carefully and replace any that show ripple (may be worth it to just do a group replacement of all of them if any need replacing).

Good idea. I shall replace the remaining old capacitors in the power supply.  Thanks for the helpful advice. 

[Kinkless Tetrode]

The clock shows quite some ringing. This could also be due to a proor ground lead connection. Still it is a digital signal, so it may be acceptable.  Still I consider this the worst of the waveforms shown.  One could try some RC snubber or similar. One could also check the supply, if there is ringing on the 5 V. Sometimes replacing caps with modern low ESR ones can make things worse.

Yes, you, and jklasdf, both are right about the poor ground lead causing ringing.  I found that the ringing is slightly reduced when I use a different and shorter set of leads to connect to the digital ground.  But there is still substantial ringing remaining.  The digital 5V rail has less than 20 mV of ripple so, though it could stand for a cap replacement on the power supply board, it doesn't seem to be the cause of such waveform distortion.

The crystal signal looks a little odd, but the crystal signal are usually rather high impedance and thus easy to upset with the probe. It's also possible to have an odd loading to the crystal so that the next mode of the crystal is close to 3 times the fundamental. From the waveform I would more suspect the crystal or the caps than the µC. However the clock output looks OK (stable clock), so I would not worry about this. One could still check the caps at the crystal for breaks or cold solder joints.

Thanks, I shall check the LCR values attached to the crystal.  But considering that the clock frequency is correct and solid, it appears that the uP clock is probably good.

The address line signals look good and relatively clean. The signal is not simple periodic which causes the superposition in a kind of eye diagram like style - this is normal for a µC running more than short loop.

Thank you, I now have a better understanding of how the uP address line signals should look.   


*********

Well, it looks like the situation has changed with the 3456A earlier today.

After re-seating the A30 board back into the chassis, I powered on the meter and the display was all blank and all the button LEDs were lit.  I powered the unit off and on again and the display showed all 1s, "-1.1.1.1.1.1".  Then powering off and on another time showed a display of all Fs, "-F.F.F.F.F.F".

It seems the 3456A has now become a slot machine showing random combinations on the display.  But now it is blank most of the time with only the power on LED on and all other LEDs off.

Furthermore, one of the relays K101, K102, or K103 on the A20 board starts buzzing on and off randomly.  I cannot tell which one it is, but it sounds like an intermittent low frequency buzzing.  But when I put the scope probe on one of the data lines of the A4U15 out guard uP or the Inguard uP the relays stop buzzing.  This is totally confounding me.  Either the bad part has changed character or some other component has now become damaged from my powering the unit off and on many times.   


Oh well, I shall restart troubleshooting from the beginning and start by replacing all the power supply capacitors first. The problems are now growing each day...   

[SilverSolder]

I have no experience with this model so just a couple of general thoughts: 

Especially with vintage tech, it is worth eliminating connectors and ground points as possible suspects - i.e. literally unscrew the PCBs from the frame, clean/deoxit under the screws, and tighten everything back down again.  Disconnect all ribbon connectors etc., deoxit, reseat.  This includes the power supply, if it is on a separate board, the front panel, everything.

Even if it doesn't fix the problem, at least it gives a "solid foundation" where you can have a higher degree of trust in the signals you are measuring.

E.g. the 3325A signal generator, good though it is, is very prone to these kinds of issues which can cause anything from a garbled display to freezing during operation. With that instrument, there is no point even turning on the scope until all of the above has been done...  and often enough it fixes the problem, so you don't need to turn the scope on at all.

I once spent way too long analyzing CPU signals only to find the problem was the GPIB address DIP switches had gone open circuit, leading the CPU to read a bad GPIB address during boot, which led to bad behavior once the unit was running!

[Kinkless Tetrode]

I have no experience with this model so just a couple of general thoughts: 

Especially with vintage tech, it is worth eliminating connectors and ground points as possible suspects - i.e. literally unscrew the PCBs from the frame, clean/deoxit under the screws, and tighten everything back down again.  Disconnect all ribbon connectors etc., deoxit, reseat.  This includes the power supply, if it is on a separate board, the front panel, everything.

Even if it doesn't fix the problem, at least it gives a "solid foundation" where you can have a higher degree of trust in the signals you are measuring.

E.g. the 3325A signal generator, good though it is, is very prone to these kinds of issues which can cause anything from a garbled display to freezing during operation. With that instrument, there is no point even turning on the scope until all of the above has been done...  and often enough it fixes the problem, so you don't need to turn the scope on at all.

I once spent way too long analyzing CPU signals only to find the problem was the GPIB address DIP switches had gone open circuit, leading the CPU to read a bad GPIB address during boot, which led to bad behavior once the unit was running!

Thank you, SilverSolder. That’s an excellent idea.  . At minimum, the procedure may help reduce a few variables and unknowns.

[Kinkless Tetrode]

Update:  I finally have this HP 3456A now working.   

1. I re-started troubleshooting from the beginning.  To start, I replaced the old Nichicon capacitors on the power supply board.  That cleaned up the rails quite a bit and reduced the ripple on the +33V rail by half.

2. I picked up an HP 5005A signature multimeter on ebay at a reasonable price, so now I can follow the service manual steps.  I couldn't figure out how to interpret the scope waveforms for the microprocessor so there was no way for me to determine if what I was seeing was good or bad.  But the signature analyzer is exactly what I needed.  The service manual describes how to connect the signature analyzer to components and test points on the board and you touch a component with the test probe to get a digital signature.  You then compare that measured signature to the signature value published in the manual; if the value matches, it's good, else it's bad.

3.  Using the signature analyzer, the CPUs tested good, address bus good, clock good, etc...  But line D2 on the data bus was reading a bad value.  Following the suggested troubleshooting flowcharts, it suggested bad ROMs on the A4 board.  However, the ROMs all showed a bad signature on pin 11 with all the other pins reading good.  Having all three ROMs become defective on the same pin?  That seems unlikely, and it just happens that pin 11 on all three ROMs correspond to line D2 on the data bus.  So, some component connected to the 8-bit data bus was bad somewhere in the system.

4. The 8-bit data bus connects from the A4 board to the A3 board, the board where I saw the bad corrosion when I first opened the lid.  So, I replaced chips U1, U6, U13, U15, and U17, since they were involved with processing the data bus and had gates used across the entire board in different places.  No change.  So, finally, I replaced chip A3U10, the SN74LS165N the 8-bit shift register, and powered it up.

Success!  The meter came alive, passed the self-test, and started showing measurements!   

The meter seems to have maintained its calibration well and it gives readings close enough to my Fluke 8505A and HP 3457A with my 5V voltage reference.

I guess I could have fixed it without the signature meter, but I would have probably replaced many more chips that were good before I found the bad one(s).  But the Signature Analyzer does give a definitive answer of whether the CPU, memory, bus lines, and timing lines are good.  It clears the innocent parts but it alone does not find the true problem(s).  But it's a great tool to help you know where to look for the problem and not waste time looking where you should not be looking.  At least, now I have learned what good waveforms look like in the microprocessor lines.

Also, assuming that A3U10 was the only problem, I could have found the problem more quickly had I strictly followed the service manual steps. Service Group A does say to replace A3U1 and U10 if the transmitter waveform pattern doesn't have the correct number and combination of highs and lows, 2 high, 3 low, 4 high.  I was seeing 2 high, 2 low, 4 high, in the defective A3 transmitter output.

Finally, in this specific case, I could have solved the problem the moment I opened the lid, had I merely replaced all the chips on board A3 that had any green corrosion on their leads.  Chips A3U10, U13, U15, and U17 all had green corrosion on their leads and traces.  In fact, removing U10 and U13 exposed some significant decay of traces underneath the chips that were not visible until I removed those chips from the board.  Please see the photos.

It seems that moisture had gotten on to the chips on the A3 board since A3 is exposed to the environment through the bottom grilled panel. Board A30 has the metal cover panel so that board was protected.  The top side panel doesn't have vent holes on the cover panel so A4 and A40 were not exposed to moisture either.  I shall be alert to attack corroded chips first in the future.

Thanks to all who provided me the helpful advice to get this boat anchor sailing the measurement seas again.   

[Vgkid]

Nice job. Now put the 3456 in 6 digit mode, that is what it deserves.

[Kinkless Tetrode]

Nice job. Now put the 3456 in 6 digit mode, that is what it deserves.

Thanks!  Yes, indeed.  The 3456 deserves the full 6 digits!   

[SilverSolder]

Nice work.  Surprising how often problems turn out to be "mechanical" (chemical?) in nature...

[Kinkless Tetrode]

Nice work.  Surprising how often problems turn out to be "mechanical" (chemical?) in nature...

Thanks!  Yes, it’s interesting that many of the simple problems are of a non-electrical nature.

I guess when we get a new piece of equipment the first thing to do, as you suggested, is to open it up and peek inside to look for anything odd and reseat all the boards and connectors.  Clean out any debris inside and perhaps put DeoxiT on all the switches before pulling the power switch.   

[jklasdf]

Good work, these meters are usually pretty reliable, but I guess not when there's corrosion involved!

The clock edges (and any digital signals with fast-transitioning edges) will look better, with less ringing, if you have a nearby ground and can use a spring ground tip thingy (e.g. something like this:

). This minimizes inductance in the ground lead, see http://cdn.teledynelecroy.com/files/appnotes/lab_wm-1026.pdf for example waveforms.

[Kinkless Tetrode]

Good work, these meters are usually pretty reliable, but I guess not when there's corrosion involved!

Thanks!  I learned a lot by fixing it.

This 3456A is a really nice piece of equipment.  It's solid as a rock, physically and electrically, and built to last.  It's perfectly repairable and maintainable; calibration is in hardware, not software, so no backup battery to rot or to replace, or risk of losing calibration data in memory; standard TTL obtainable parts, mostly; simple ROMs instead of impenetrable FPGAs or unobtainable ASICs. Best of all, it has a nice bright and very readable LED display.  I like LEDs, I cannot read LCDs as easily even if they were backlit.  Nice solid push buttons that make a nice loud snap when pressed, no need for the audio or visual haptic feedback necessary for soft buttons.  Just a very fixable meter, you gotta love it.

I have to admit that I like the HP 3456A and Fluke 8505A much better than my fancier feature packed  HP 3457A.

The clock edges (and any digital signals with fast-transitioning edges) will look better, with less ringing, if you have a nearby ground and can use a spring ground tip thingy..(...). This minimizes inductance in the ground lead, see http://cdn.teledynelecroy.com/files/appnotes/lab_wm-1026.pdf for example waveforms.

Yeah, my cheap scope probes and leads are causing quite a bit of ringing.  That induction reducing  grounding spring looks like a good idea.  Looks like we can use a simple paper clip. 

Thanks for the very interesting Teledyne-Lecroy App Note.  It's a good read. 

[cncjerry]

Hello, great job, I was a little worried when I saw the corrosion.   You need to somehow wash or clean that board or it will continue.

[Kinkless Tetrode]

Hello, great job, I was a little worried when I saw the corrosion.   You need to somehow wash or clean that board or it will continue.

Thanks, cncjerry.  It does look like nasty stuff. I hope that it doesn’t continue eating away at the board traces over time.  I think I will have to give that A3 board an isopropanol bath and get it clean as possible underneath the other chips. Maybe that will take care of any residual junk.   

[cncjerry]

I had a board on a 3455 meter that once something got under the coating it ran down the trace. Maybe if you painted nail polish on the bad area, assuming it is digital and not analog it will keep the oxygen out.  I'm sure there are better varnishes or coatings.

[Kinkless Tetrode]

I had a board on a 3455 meter that once something got under the coating it ran down the trace. Maybe if you painted nail polish on the bad area, assuming it is digital and not analog it will keep the oxygen out.  I'm sure there are better varnishes or coatings.

Great idea! Thanks. 

[Kleinstein]

Hello, great job, I was a little worried when I saw the corrosion.   You need to somehow wash or clean that board or it will continue.

Thanks, cncjerry.  It does look like nasty stuff. I hope that it doesn’t continue eating away at the board traces over time.  I think I will have to give that A3 board an isopropanol bath and get it clean as possible underneath the other chips. Maybe that will take care of any residual junk.   

For cleaning of such residue pure alcohol may not be very effective. With some parts just plain water is more effective. So the better cleaning would be water first (and plenty of it) and only alcohol later, mainly to get things dry without concentrating the rest in drops.

[Kinkless Tetrode]

For cleaning of such residue pure alcohol may not be very effective. With some parts just plain water is more effective. So the better cleaning would be water first (and plenty of it) and only alcohol later, mainly to get things dry without concentrating the rest in drops.

Thanks, I will do that.  So, the final alcohol is to remove the remaining water?