Attempting to repair a Tektronix 2430

[siggi]

Hi y'all,

I recently adopted a very sick 2430. I intentionally bought a sick scope for the experience of debugging and hopefully fixing the problem. I've previously done small repairs on a 2465 and slightly more invasive fixes on a 2467, both of which are very repairable, with beautiful service manuals - so getting another scope from a similar era seemed like a good idea.

Now this 2430, when I powered her on, she'd do different things. I'd get anything from nothing, to beeping, to blinking error codes on the front panel LEDs. Sometimes I'd get a dot on the CRT, sometimes I'd get wiggly scribbles, so I'm hopeful the display machinery is still good.
When I opened her up, the most apparent problem was that the lithium battery had leaked, to the point where the nearest component leads had corroded away.
I cleaned all that mess up, but that didn't get me any joy on powerup - same behavior, pretty much, so I'm starting to think that's not the primary failure in this poor old thing.

Now, I've gone through the "dead start" procedue in the service manual. In particular the "kernel" tests, both for the main CPU and for the waveform CPU, and they both manage to run through the address space(s), no problem. The address decoding machinery all works, and the data busses look clean.
Sometimes when I power up, it seems the main CPU goes as far as enabling interrupts. I observe this by looking at the delivery and the CPU's ack of the 500Hz. The ack is done by addressing a register, which means the CPU's reading the interrupt vector and getting to sane instructions on that schedule...
Now, in my process of trying to elminate things, I've found that pulling all the bank-switched ROMs will allow deterministic execution to a stable front-panel LED display. This "makes sense" as a documented part of the power-on self checks is to checksum the ROMs.

What doesn't make sense is that when I re-populate the ROM sockets, populating one particular socket (the U690 socket), with any of the ROMs leads to random execution again. I replaced the address decoding logic that generates the ROM CS signals, on the theory that this is the only thing unique to the individual ROM slots, but no change.
Why this particular slot should cause this I don't understand, it's physically in the middle of the pack, and all the address & data lines are a dead short to their counterparts on the other slots...

So, I'm starting to wonder if this could be as simple as borked decoupling. On the 5VD rail I see periodic fuzz to the tune of 100-150mV, which seem to coincide with the timer interrupt clock - 500Hz.
The decoupling on that board is a 47uF/25V electrolytic, plus a bunch of those little weird diode-looking capacitors scattered all over. I figure it's easy enough to change the electrolytic, but the little'uns are going to be a PITA if I have to change them all.

Has anyone here debugged something like this? Any hints, tips or tricks?

Siggi

[David Hess]

Heya Siggi, not enough help over at TekScopes@yahoogroups.com? 

Did you confirm that it was the battery that leaked and not just electrolytic water damage?

My advice is still to start by working outwards from the most damage areas.  I would replace at least the SRAMs (which I believe you have done) and the address and data bus transceivers which connect to them.

The 2430 series is probably the worst candidate having full service documentation that I can think of for a random repair job.

[siggi]

Hey David,

fancy "meeting" you here, somehow I feel like I owe you a drink .
I haven't replaced the RAM yet, as I'm still waiting for the parts. I made the mistake of getting USPS "Express" delivery to the frozen tundra of Canada-land - they might show up tomorrow, or never.
I have however sucked the old chips out, cleaned up and installed sockets, so I can swap them around and run with/without. If the RAM chips themselves are bad, this might clear up once I get new ones - though I'm still hard put to explain why it'd get wobbly when I populate one particular ROM socket.

I figure at this stage I'm debugging borked TTL-level logic, more so than a Tectronix scope, per-se, so might get useful hints from here...

I think the address buffers are fine, looking at the signals they seem quite clean and robust, and all the dowstream address decoding seems to work just fine.
The data buffers are reasonable suspects, as are perhaps the outputs from all the memory devices. I've looked across the data buffers by differentially probing their ins/outs and they look just fine. I can only really look at the read direction with the kernel tests and a regular scope. Now that you mention it, one mode of fallout due to the corrosion could have been RAM and data buffers fighting on write cycles.
Perhaps it's worth putting the logic probe on them, see whether the reverse direction is borked - or I might just switch them out. I guess a soldering iron is sometimes quicker than a logic probe .

Assuming that e.g. a CS line has shorted to another signal with the corrosion gunk on there, then I would have had two or more memory devices, and/or RAM and data buffers waging war on the data bus.
Would the device outputs usually survive this, or would I likely be looking at burned-up outputs?
Any practical difference between the ROMS/RAMs? The NVRAM is a CMOS device, the other one is TTL, I believe - any practical difference there?
I figure if the ROMs are baked then this'll get a lot harder to debug and fix, though I might rig up something to read them and see whether they checksum OK or something. From looking at the data lines under kernel test, they do look OK, e.g. I don't see any lines wedged high/low or undriven.
It's kind of cool being able to look at the address space in the time domain - there are neat troubleshooting features in those scopes.

I have to admit that the 2430 isn't as viscerally pleasing as the 2465/2467s.
When I bought it, I was under the impression that it was dual path, e.g. a 150MHz analog path as well as the digital path. Turns out it's all digital display, although the details are still pretty fascinating to me. Using those CCDs for capture and storage, then digitizing at a much lower rate seems like a pretty scrappy thing to do.
I guess the killer feature of this scope at the time must have been pre-trigger signal display, cause there's not much else to it?

Siggi

[PaulAm]

I have a 2432a in the queue that randomly locks up and hangs while doing a selfcal.  It's about number 3 in the queue right now.  I'll be very interested in seeing what you track down.

[siggi]

Heya Siggi, not enough help over at TekScopes@yahoogroups.com? 

Did you confirm that it was the battery that leaked and not just electrolytic water damage?

To answer your question, I'm pretty sure the battery leaked. The gunk seems to emanate from the battery, and there was a mess of corrosion and gunk on the PSU's aluminum cover plate. This is straight below the battery. One of the battery's leads was corroded away, as were a couple of component leads right next to it.

However, there was also corrosion around and under the NVRAM chip. The manuals that came with the scope were all musty and mouldy and dank.
So, it seems as plausible as anything - to me - that this all happened while the scope was in storage, and that the primary failure was something else altogether. The eBay picture of it had the OSD listing a litany of POST failures, starting with system RAM...

[David Hess]

fancy "meeting" you here, somehow I feel like I owe you a drink .

I am just hanging out here to evaluate the possibility of a dedicated Tektronix or test and measurement forum after the Yahoo fiasco.  That is my story and I am sticking to it.

To answer your question, I'm pretty sure the battery leaked. The gunk seems to emanate from the battery, and there was a mess of corrosion and gunk on the PSU's aluminum cover plate. This is straight below the battery. One of the battery's leads was corroded away, as were a couple of component leads right next to it.

However, there was also corrosion around and under the NVRAM chip. The manuals that came with the scope were all musty and mouldy and dank.
So, it seems as plausible as anything - to me - that this all happened while the scope was in storage, and that the primary failure was something else altogether. The eBay picture of it had the OSD listing a litany of POST failures, starting with system RAM...

The reason I think it was water damage and not electrolyte leakage is that it follows the areas where the standby voltage was present and is too wide spread.  Water by itself can explain the corrosion damage to the ground and power battery terminals.

I think the address buffers are fine, looking at the signals they seem quite clean and robust, and all the dowstream address decoding seems to work just fine.

The data buffers are reasonable suspects, as are perhaps the outputs from all the memory devices. I've looked across the data buffers by differentially probing their ins/outs and they look just fine. I can only really look at the read direction with the kernel tests and a regular scope. Now that you mention it, one mode of fallout due to the corrosion could have been RAM and data buffers fighting on write cycles.
Perhaps it's worth putting the logic probe on them, see whether the reverse direction is borked - or I might just switch them out. I guess a soldering iron is sometimes quicker than a logic probe .

Signal integrity is one of the easier things to check.  I hope that the problem is in a SRAM or bus driver and not any ROM or EPROM which would be more difficult to replace.

Assuming that e.g. a CS line has shorted to another signal with the corrosion gunk on there, then I would have had two or more memory devices, and/or RAM and data buffers waging war on the data bus.
Would the device outputs usually survive this, or would I likely be looking at burned-up outputs?

TTL outputs are *not* overload tolerant.  They can definitely be damaged by over current conditions.

Any practical difference between the ROMS/RAMs? The NVRAM is a CMOS device, the other one is TTL, I believe - any practical difference there?

There is not any difference from the perspective of the interface.  In the quest for backwards compatibility with TTL levels, the these CMOS devices have adjusted threshold voltages like the HCT logic family.

I figure if the ROMs are baked then this'll get a lot harder to debug and fix, though I might rig up something to read them and see whether they checksum OK or something. From looking at the data lines under kernel test, they do look OK, e.g. I don't see any lines wedged high/low or undriven.
It's kind of cool being able to look at the address space in the time domain - there are neat troubleshooting features in those scopes.

Doesn't one of the self test routines check the ROM checksum or CRCs?  This assumes that execution gets that far.

I have to admit that the 2430 isn't as viscerally pleasing as the 2465/2467s.
When I bought it, I was under the impression that it was dual path, e.g. a 150MHz analog path as well as the digital path. Turns out it's all digital display, although the details are still pretty fascinating to me. Using those CCDs for capture and storage, then digitizing at a much lower rate seems like a pretty scrappy thing to do.

I guess the killer feature of this scope at the time must have been pre-trigger signal display, cause there's not much else to it?

The 2232 was the high end of combination analog and DSO from Tektronix.  The non-real time 2252 does not really count and I suspect that is as good as they could have done in the 2465 form factor.

Any DSO can support a full pre-trigger display.  That was not new.  The killer feature was the real time sample rate which supports the full oscilloscope bandwidth.

At the time CCD digitizing systems were an order of magnitude faster than using a fast converter and memory at a reasonable price.  The limitation was actually the memory bandwidth which is why the 2232 is only 50 MS/s in dual channel mode even though it uses a 100 MS/s converter for each channel.  Tektronix continued development of this design for years with the TDS600 series which were much faster, 2 GS/s versus 250 MS/s to 1 GS/s) than their non-CCD contemporaries.

[lowimpedance]

 From my 2430 perspective, when repairing the one I found in the "bin" the main supply had a range of bad electrolytics to completely shot.
Basically would not even power up at the start. After the supply refurb and a very good clean to remove the considerable dirt build up it seemed to work and passed all start up tests. But one time I turned it on all I got was squiggly lines on the screen. Poking around the Rom area seemed to make a difference so I removed the IC's and reseated in the sockets and all is good again. So be wary of dodgy sockets and corrosion.

[siggi]

Poking around the Rom area seemed to make a difference so I removed the IC's and reseated in the sockets and all is good again. So be wary of dodgy sockets and corrosion.

Ah interesting. I've cleaned and re-seated all the CPU board ROMs on this one, as well as the hybrids and socketed ICs on the main board. There are, however, also a couple of socketed chips on the readout board that I haven't touched. It's an easy thing, and worth trying - thanks.

[siggi]

Doesn't one of the self test routines check the ROM checksum or CRCs?  This assumes that execution gets that far.

Yes, I believe the ROM CRC checks are the first self-test. The test numbering is ROMs: 1XXX, registers: 2XXX, RAM: 3XXX and so on, and this appears to match what I see on the logic probe.
By removing the ROMs I'm able to provoke what looks like an early POST failure reliably, which is why I'm baffled at the non-determinism associated with populating one of the sockets...

[siggi]

So my RAM finally arrived, and when I plugged it in I get OSD - sweet! I guess the random execution with that ROM socket populated was some sort of incidental effect on borked RAM chips.
For science, I tried powering on with only the system RAM socket populated to see how POST behaves with borked NVRAM. This makes the POST violently unhappy, all I get is single, very bright horizontal line on the CRT. I'm a little surprised at this, seems that the "NVRAM" is at least in part used as working memory during POST.

The reason I think it was water damage and not electrolyte leakage is that it follows the areas where the standby voltage was present and is too wide spread.  Water by itself can explain the corrosion damage to the ground and power battery terminals.

I think you may be right about this being water damage, which presumably occurred in storage after the scope was retired. However, I think the battery also leaked, e.g. I believe the corrosion eventually penetrated the battery and caused it to spill it's guts.

Now all the POSTs are failing except for ROM and WP. At least I'm to square zero, as the state of the scope now matches the picture on the eBay listing, so I can start working on the primary - or initial - failure.

I'll have to take another close look at the schematics and read the error code flashed at startup to see where to begin, but it should be a little easier from here on .

[David Hess]

Now all the POSTs are failing except for ROM and WP. At least I'm to square zero, as the state of the scope now matches the picture on the eBay listing, so I can start working on the primary - or initial - failure.

A whole bunch of the POST items will fail when the calibration data is missing.  If that is the only problem, then it should operate.

There is a load factory calibration or defaults command somewhere in the calibration menu that should be executed.

[siggi]

A whole bunch of the POST items will fail when the calibration data is missing.  If that is the only problem, then it should operate.

There is a load factory calibration or defaults command somewhere in the calibration menu that should be executed.

If only it were so. Three of the failing POSTs are the front panel processor - so the FP is dead - the system registers, and the system RAM. I figure if I can get the FP going, I can individually do the other tests. Presumably something resulting in failures across registers, system RAM and the FP is pretty central, so ought to be easy (famous last words) to find.

Alternatively if the GPIB is functional, apparently I could intitiate individual tests from there. I'd need to get a USB-GPIB controller, though, and those look relatively pricey compared to the money I have in this scope .

[siggi]

Now all the POSTs are failing except for ROM and WP. At least I'm to square zero, as the state of the scope now matches the picture on the eBay listing, so I can start working on the primary - or initial - failure.

I took another look at the original eBay listing, and turns out I'm at square -1, as the register POST wasn't failing when the photo was taken. Maybe this is more fallout to the water/corrosion damage. Best case this'll clear up when I fix the battery circuitry...

[David Hess]

Now all the POSTs are failing except for ROM and WP. At least I'm to square zero, as the state of the scope now matches the picture on the eBay listing, so I can start working on the primary - or initial - failure.

I took another look at the original eBay listing, and turns out I'm at square -1, as the register POST wasn't failing when the photo was taken. Maybe this is more fallout to the water/corrosion damage. Best case this'll clear up when I fix the battery circuitry...

My suspicion is that you will find significant electrolytic corrosion or the products of electrolytic corrosion under other ICs in that area causing problems.  That is why I said I would work outward from the center of the damaged area adding sockets and replacing ICs.

[siggi]

As much as I hate to say it, it appears you're right. This evening I briefly got a PASS on registers and FPP, which then reverted for no apparent reason. Guess I'll be pulling the registers and replacing them to see what's up...

Good news is that the FPP is OK, and the damage is hopefully constrained to the CPU board.

[siggi]

Progress! Also some embarrassment...

After debugging the FPP ad nauseum, I figured it was right as rain. Both the self tests worked like charm, but I couldn't figure out why it was bailing on the 4400 "self test byte" test. I put a logic probe on it, and the FPP was returning a high on one of the lower four data lines, which are required to be all low.
Reading through the SM again, it hinted that if all voltages are good, the FPP must be bad.
So, turns out the FPP reads the battery voltage, and when the 10M resistor from the battery to the TL74s input has rotted away, the TL will float. Mine chose to go negative, which makes the FPP self check unhappy, as it were...
Rigging a positive voltage to the TLs input fixed it, and I have passing FPP, REG etc.


The CRT intensity is pretty wobbly, so perhaps reason to look at the supplies again. Still, menus and stuff!

After multiple cold boot and self cal cycles, I'm past the NVRAM CRC failures, but I'm still failing all 7/8/9000 tests except CCD efficiency. Also acquisition is producing ... fuzz ... though not altogether unlikely fuzz. The vert/horiz knobs are glitchy as hell, more corrosion for sure. Guess I should pull and scrub the remaining hybrids.

The most recent symptoms lead me to I suspect the DAC works may be gunked, but at least I'm done with the CPU board for now .

[David Hess]

If it has that kind of wide spread damage then this is the time where I would reconsider whether it is repairable.  On the other hand, it is not like you have anything to lose.

[siggi]

If it has that kind of wide spread damage then this is the time where I would reconsider whether it is repairable.  On the other hand, it is not like you have anything to lose.

Yeah, I don't have high confidence that I'll make her whole.
As a software guy, this makes a fun hobby, though, as it gives me a reason to wield an oscilloscope and a soldering iron in anger. Also, these 'scopes seem pretty well designed, and pouring over the schematics may lead to some learnin'. The troubleshooting is fun, though mostly after the fact, and perhaps not so much during.

That being said, I'm still hoping I'll find another central fault in there somewhere, as it'd be nice to have a DSO sometimes. It should be easier from here as I can now exercise and troubleshoot the different subsystems in isolation through the self-tests.

[David Hess]

Given the apparent cause of the damage, operating the oscilloscope in a wet environment, I suspect most of the treatment is going to come down to disassembly and cleaning.

[siggi]

More progress. Turns out that if you flip the 4-pin connector carrying the CCD signal out to the DAC, you severely compromize the 'scope's performance . This is what I did during initial cleaning - the coax had wicked some of the corrosion gunk out to the main board, so I pulled it and cleaned it. When I plugged the connector back in, evidently I wasn't paying attention to what I was doing.
Somehow a trace of the signal was able to make it through from the main board to the DAC, even though I'd grounded the output drivers. Presumably the drivers are low enough impedance to lift the ground on the DAC, or some such. Also, many thanks to the Tektronix engineer who made those drivers short-proof.
After correcting this small problem, and after cleaning and re-seating the trigger hybrid, I have a working scope that passes all self tests - sweet!

The attenuators are in a bad way though. They both click with likely sounds at likely times, but channel 1 seems to be frozen in one configuration. The nominally 400mV P2P calbrator signal is reading 1.4V P2P on the 500mV/div vertical setting - I can't figure out what configuration of the relays would cause that?

Channel 2 seems better, although the 50Ohm switch is not making contact.

As a corrolary to this, the scope fails a random subset of the self-tests if I have a signal connected to it during testing, which had me confused for a while.

I've seen postings where people claim to have had success with opening and cleaning the attenuators, but no pictures of what's in there. I think the 2430 has pretty much the same attenuators as the 2465-series.
Anyone, anyone, Bueller?
Not that it matters, as I've clearly nothing to lose to attempt it .

Now that it seems she's viable, I figure I'll take the trouble to replace the battery and fix the circuit. The main board also needs cleaning, so I guess I'll pull that and give it a scrub.

[David Hess]

The attenuators are in a bad way though. They both click with likely sounds at likely times, but channel 1 seems to be frozen in one configuration. The nominally 400mV P2P calibrator signal is reading 1.4V P2P on the 500mV/div vertical setting - I can't figure out what configuration of the relays would cause that?

Channel 2 seems better, although the 50Ohm switch is not making contact.

The relays are magnetic latching types.  The only failures I know of are sticking and driver failures.

As a corollary to this, the scope fails a random subset of the self-tests if I have a signal connected to it during testing, which had me confused for a while.

This should not happen but if one of the magnetic latching relays is not operating correctly and leaving the signal source connected, this is what I would expect to happen.

I've seen postings where people claim to have had success with opening and cleaning the attenuators, but no pictures of what's in there. I think the 2430 has pretty much the same attenuators as the 2465-series.

The attenuator assemblies are practically identical between the two series of oscilloscopes.  I am sure you could substitute one for the other.

I suspect the preamplifiers are the same as well but there are part number errors in the 2465B documentation so I am not positive about that.

[siggi]

I pulled the main board out last night to find an ugly mess on the solder side. Also pulled the front panel switch board, which is all crusted with gunk. You seem to be absolutely right that this scope was operated and/or stored in a wet environment, maybe even on a ship...

I'm not sure how I'll get at the switch internals to clean them, guess I'll just have to dunk the whole thing in IPA? The channel 2 vertical control switch is so glitchy it's unusable, and the scope just jumps around randomly when I wiggle it.

The relays are magnetic latching types.  The only failures I know of are sticking and driver failures.

I pulled the channel one attenuator and tried to exercise it's relays, but most were stuck.
Pulled the thing apart, the internal construction of those isn't too scary.
I cleaned the surface of the circuit, the backside of the rocker arms and the inside of the relays with IPA. I fixed the sticking for sure, but I haven't measured it yet to see whether it works right now.
Attaching some pictures of the attenuator guts pre-cleaning.

[siggi]

IPA scrubbing fixed the sticking, but some of the contact pads were apparently so far gone with oxidation that they just wouldn't make contact. For some reason this was true for both 50Ohm contact pads, but only for one 10X attenuator. I managed to find some Deoxit out here in the frozen tundra, and this cleaned it up, it seems. I had to really scrub, and apply the Deoxit a couple of times to break through, though.

I think when you set the scope to GND, what it really does is AC coupling, 50Ohm termination and perhaps 10X^2 attenuation for good measure. Losing the 50Ohm termination definitely seems to compromize the GND input setting...

Also, for anyone looking at my pictures from above, mind that when you remove the attenuators that you desolder the components from them, and don't leave them attached as I did. The pins out of the attenuators (and the preamps) are really, really fragile and brittle, and you really don't want to jiggle them anymore than you must. I messed up both attenuator and preamp on channel one to learn this.

[David Hess]

The same switch which connects the 50 ohm termination also disconnects the AC coupled path so only the relatively high value precharge resistor connects the vertical input to the first attenuator stage.  There is no 50 ohm AC coupled mode as such.  The only four configurations are 1M AC, 1M DC, 50 ohm DC, and GND which disconnects the input but leaves the 50 ohm termination connected internally and also acts as the precharge mode.

[siggi]

The same switch which connects the 50 ohm termination also disconnects the AC coupled path so only the relatively high value precharge resistor connects the vertical input to the first attenuator stage.

Ah, that's what that resistor on the schematic is for. Thanks, somehow I couldn't put that together myself .

So as I read it, the upshot is that without the 50Ohm  resistor, the GND coupling puts the AC decouping cap and the precharge resistor in-line with the 10X attenuators.
A fair amount of signal will evidently bleed through. This messes up my self-cal randomly if I leave a signal on the inputs during calibration.

With the 50Ohm resistor in the picture, it makes the bottom half of a voltage divider against the decoupling cap and the precharge resistor (aka effective short, I assume?). I get no visible calibrator signal bleeding through, and my self-cal succeeds even with a signal on the input.