Agilent(/HP/Tektronics) 54642A Oscilloscope repair

[daveaep]

Hi!
Before I start with the questions, let me introduce myself.

I'm Dave, and engineer working for the University of Twente and I'm quite ok in repairing equipment.
Now I'm having quite some trouble with one of our old oscilloscopes.

I'll start with describing  what DOES work, and later on tell what I've done to repair it.

So when the power of the scope is turned on, the lights of the button/knobs panel blink once, and afterwards the 'More' button keeps lit.
This is all. With another scope (54622A) this happens as well, but then it starts 'booting' up.

I have taken apart the scope, and started measuring the voltages as described by the repair manual.
These voltages should be: 5.1V, -5.2V, +15.7V and +3.4V.
Now all these voltages match, except for the -5.2V. This was around -8V. According to the manual I should already replace the power supply, but as this thing is 1000 years old, getting by one might not be easy. So what I did was to replace (almost) all aluminium capacitors and try again.

I have not yet retested the voltages, as I;m not quite sure that this single voltage would mess it all up. Does someone else have the same experience with this machine? Or am I missing some important stepts? This is the first scope that I'm trying to repair, but luck has not yet been on my side.

For the people still reading, thanks for doing so, and I hop you might have some insight on how to repair this ancient device.

Dave

[SilverSolder]

Without knowing much about the 54642A, it it not unlikely that the bad voltage could be preventing the system from booting, not to mention causing all kinds of other issues.  I would definitely get the supply voltages back to spec before digging further.

Something to check:  I have a 54622D which needed a new battery for its long term memory.  Not sure if a dying/dead battery could potentially leave bad values in memory that stops the thing from booting?

[JFJ]

If you haven't done so already, then it's worth trying a 'key down power-up reset'. Hold down any softkey and switch the oscilloscope power on. Release the softkey when the display appears.

[daveaep]

Hi!
Thank you both for the replies.
Today I just thought of replacing the backup battery. I removed the old one (2.45V) and placed a removable 2030/2032 battery holder.
Would this suffice? It's easier to remove the battery this way, and I had no other option at this moment.

I will try to place everything together when I'm sure the 2032 will suffice.

Dave

[daveaep]

So I went with the 2032 3V cell and put it all back together.
When I turn the scope on, I still get the 'More' button lighting up, while nothing else happens.
I do hear the tube getting the high voltage field, but no display.

When I hold one of the soft buttons, and turn on the scope, nothing different happens. Just only that annoying 'More' button.

Voltages of measurement points are still all in spec, except for the -5.2V. This is still -8.4 after replacing all the aluminium caps.

Anyone has some ideas?

[dogbert]

I got a 54641D that it turned on one day with the same problem: it got stuck at lighting up the "More" button. Used to boot fine before. So I bet it's an aging problem.

I checked the signals around the system and noticed all the digital chips, including the processor, has no activity despite the clock is OK (e.g. the motorola processor has a 25Mhz clock after division).

What gets even more mysterious is that the unit boots properly with a failed crystal I put in, which was extracted from another unit that failed "timing chip error" self-test, obviously because the clock measures 214Mhz despite the markings says 200Mhz. Of course it doesn't really solve the problem because a 214Mhz, despite it boots the computer, screws up the timing. The unit does not boot whenever I put the proper 200Mhz chips extracted from two known good units.

I've swapped the clock divider HEL32 with a known good working unit well. It's not the culprit. The chip worked fine on the other board.

Hope it narrows down a bit for you.

[SilverSolder]

That sounds truly weird, the magic smoke is dancing for us here.

Are all the supply voltages on spec?

[dogbert]

Yup. The voltages are on fleek (I followed the service manual. I have the good -5.18V unlike the OP). It has the same behavior even when I swap the power supply with other 54640 series cases.

I even made a socket for the XO to swap test my observation. The scope boots 100% of the time with the wonky XO but always get stuck with "more" button lit up (like the OP's scope's symptom) when I put in the known good XO.

[SilverSolder]

This kind of stuff is almost enough to make one believe in ghosts, magic, etc. right?

Something has aged and gone marginal...   can we find any capacitors to blame?

What if one or more bypass capacitor(s) near one or more of the digital chips has gone marginal...   there is enough capacitance left that you can boot at 214MHz as the cap doesn't have time to discharge during some heavy I/O operations, but it does have time to get drained at 200MHz and crashes the chip (or a neighbouring chip)...   Maybe scope the rails during boot to see if there are obvious problems?

Maybe check all the capacitors on the Vcc rail for ESR...  or just change all of them on principle - any capacitor that touches the power rail, whether in the power supply or on the main board, or any daughter boards, should be checked / changed?

[dogbert]

Spooky, but not enough to make me superstitious (although I believe in Murphy's law. lol). I've fixed a TDS754A that gives noise spikes at random times that seemed to correlate with heat, cold, or spanking the edge of the unit ... turns out it's a flaky PCB that skipping one screw fixes it.

I had the same hunch as you did before I posted on the forum and swapped the tantalums around the rails and around the XO with a working unit. Nothing changed.

I scoped the rail voltages of MC68EC020 and the clock. Clean and nice just like a good 54642A that boots. That's actually the first thing I did. I ended up catching that wonky 214Mhz XO that causes another unit to fail timing in self-tests because I compared the clocks between good and bad units.

There are no daughterboards for 54640 series. The scope has only 3 parts (keyboard aside): power supply, CRT driver and the main board. I swapped the main board with a working unit, and the problem follows the main board. The power supply + CRT driver is good for either cases.

The XO runs on ECL signals (it's a NEL HS819), so I paid close attention to -5.2V rails. Found nothing unusual.

Another fun fact: the CRT driver's horizontal hold is linked to the XO, as well as the 25Mhz that goes to the MC68EC020. I'll need to adjust H.Hold when I put in the wonky 214Mhz crystal.

[SilverSolder]

Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. 
-- Sherlock Holmes  (Arthur Conan Doyle)

Wow, Murphy is really working overtime on this one...

So what else can go wrong with a perfectly volted digital circuit...   

We know that it actually starts, since it gets to the point of lighting up the "More" button...  a lot has to have happened for it to get to get that far...   and then BANG, it must run into an invalid bus condition - what else can cause processing to stop altogether?  I don't know much about the Motorola processor, does it use a multiplexed bus or anything like that?

The problem might follow one address line.  The CPU could start at an address that doesn't use the bad address line, only for everything to go bad when some subsystem or other requires that line in order to be addressed.   For example, we could have two chips responding to the same address commanded by the CPU if one of them is misinterpreting the address, causing a collision...

It might be possible to work out exactly what subsystem the CPU was trying to initialize when it croaked, which might help identify the faulty chip.

Maybe it is possible to read the state of all the address lines after the unit gets stuck with a lit "More" button - and directly see the address that it was trying to access?


Some random thoughts:

Address line on ROM or NVRAM going marginal, so the processor eats an instruction that it chokes on?  That could also be timing sensitive.   RAM chip gone marginal?   Any supporting logic chips / glue logic anywhere to be found? Are some of the address lines being used for chip select purposes etc. via supporting chippery?  Are there any active termination arrays on any of the buses that could be playing with the signals?

Freeze spray / heat application on various chips while unit is booting on the fast oscillator, to see if can be provoked into faulting on the fast clock?

[dogbert]

Murphy works post-mortem.

According to the datasheet (https://www.nxp.com/files-static/32bit/doc/ref_manual/MC68020UM.pdf):
"The asynchronous bus structure of the MC68EC020 uses a nonmultiplexed bus with 32 bits of address and 32 bits of data"

I desoldered PC97338, the peripheral I/O chip, to see if the system boots without it (even with the wonky 214Mhz XO). And indeed the 214Mhz XO gets stuck at the "more" button as well (same goes for the proper 200Mhz XO). Then I swapped the IC with another good unit, and confirmed that this IC is not faulty.

So it might be something that causes the boot process to get stuck, which I have no idea how to inspect it. When I powered the unit, the address line and data lines of the MC68EC020 started with regular pulses (startup?) for <1s and quickly turn completely quiet when I have the 200Mhz XO. The address/data lines immediately moves on into data pattern when I have the 214Mhz crystal.

There are tons of 7400 series glue logic everywhere, and also a Xlinx FPGA. I suspected the CMOS SRAM (k6t4008c1b) before I have the wonky 214Mhz XO and have that swapped with a good unit too, so it was also ruled out.

[SilverSolder]

Something is definitely stopping the boot process.

And it happens very early in the boot process. 

Maybe the processor documentation explains what order things happen in - and whether there is an external power on delay or something like that, which may be playing up?

Short of a logic analyzer or a fast MSO to record what is happening on the data bus, perhaps liberally spraying glue logic chips with freeze spray and heating them in an attempt to provoke misbehavior while the scope is running could reveal something?

[dogbert]

I have a MSO, Agilent 6054A, but have little experience troubleshooting microprocessors outside embedded software level. I have relatively little experience in digital electronics compared to analog. I know enough to hook up a circuit to program it, but not troubleshooting timing at binary level. It might be a good excuse for me to learn this.

Can you suggest any materials that I should learn first before diving into the datasheet of M68020, so I know what to look for in the datasheet and the MSO?

EDIT: I just tried poking around with hot air heating each chip (excluding the ones that clearly has to do with the analog/data acquisition/trigger/logic analyzer that exhibits symmetry). Nothing stops the wonky chips from booting and nothing makes the proper chip boot, as I expected.

[SilverSolder]

That sounds like a nice MSO, wish I had one!  :-)   Might be the only tool you need.

I have no experience at all with this processor.  But armed with a nice MSO,  and given that the broken scope stops after only 1 second, the MSO may be enough to troubleshoot this.  Presumably it could capture pretty much the whole process using its digital capture ability?

I just downloaded this document on the M68020:

https://www.nxp.com/files/32bit/doc/ref_manual/MC68020UM.pdf

Section 5 describes how the bus transfers data.   As it says in section 5,  all the peripherals must follow the handshake protocol given in that section for things to work.

The most likely bet is that the processor is getting stuck because some flakyness somewhere, is not permitting the required protocol to happen as the boot code is trying to read or write something during the boot process.

Understanding how that protocol works is what I would do next, as it will probably lead to some ideas for things to look for with the MSO.