Keithley 2000 repair - ADC behavior is weird. Looking for ideas or suggestions.

[petemate]

Hi guys, sorry in advance if this post becomes a bit too implicit and detailed. I am addressing people who knows about these meters in detail and the questions will be quite specific. I will be making references to specific ICs as seen in the following schematic: https://xdevs.com/doc/Keithley/2000/K2000.pdf


As some of you know, I am trying to repair a Keithley 2000. Currently, I have the following problem: The meter never initializes. There is never any measurement carried out. The meter just shows "---.---- mVDC" and freezes if I try to change range. Relays are not set to the correct setting and so on. There is something that isn't initialized. I am however able to run the self-test, which fails at 100.1. Luckily I have borrowed a working unit, which I am able to use as a sort of cheat sheet to see what should be going on.

Probing around inside reveals that the ADC circuit doesn't function. In a working unit, the ADC output(pin 6 on U137, page 7 in the schematic) is a triangular wave, which is then zero-cross detected. The zero-cross detection pulses are then fed to the digital circuit and the density(i guess) of these pulses are what produces the actual digital value. At the same time the data bus should constantly be sending out different commands to switch various voltages to the ADC circuit, along with configuration of relays and dividers when changing range and/or measurement. This doesn't happen in my unit, but happens in the working unit. In the broken unit, the data bus basically sends out the same command all the time, at regular intervals, as if it is asking for something but never getting a reply. The initial commands are identical on both units, but the working unit "moves on" and starts setting up the relays and so on. I suspect it is somehow waiting for an ADC result before being able to proceed with actual setup, but I can't verify this. I do however think it sounds plausible given the "repeating the same command" scenario.

From the schematic(page 7) it can be seen that the triangle is generated with two currents, being switch in and out to the ADC. Those voltages are ICB+ and ICB-. My theory is that they flow into(and out of, depending which is engaged) the ADC, causing charge to added to or removed from C171. After all, thats the usual method by which an opamp integrator works.

Looking at the very first, initial data pulses, I get the following result on the working unit:

Ch2 is the data bus, CH3 is pin 1 on U148-A, which is responsible for switching in ICB+ to the ADC, and Ch4 is pin 6 of U137, which is the ADC output to be zero-cross detected. It is clearly seen that the ADC triangular waveform follows pin 1 and produces the correct pattern.

However, on my non-working unit the result is a bit different:


Channels are as above. While the ADC output(Ch4) on the working unit starts dropping at a fixed rate, it can be seen that the faulty ADC output immediately drops to its negative supply rail, but I am unable to pinpoint the cause of this. The data bus(Ch2) is identical in both cases and so are the initial patterns on pin 1(Ch3). However, the duty cycle ends up being different. I am sort of thinking that the control system is limiting the duty cycle in order to try and get the ADC level back up around zero, but I don't know that for sure. One thing to notice, though: Whenever Ch3 is not high, the opamp does start to ramp up. This can be seen by the two small sawtooth-like waveforms right after Ch4 goes to -15V. It is hard to see from the picture due to the limited resolution, but I have verified that this is the case. It also fits with the idea from the working unit, that whenever Ch3 is low, Ch4 raises(this corresponds to ICB- being coupled to the ADC).

This behavior is very weird to me. I was thinking that the cause could be the magnitude of the currents flowing into the ADC(ICB+ and ICB-), so I tested the custom resistor network R271, which is generating the currents(its a voltage applied to a resistor). The custom resistor network works well and nothing is shorted or out of value. Due to the negative feedback at pin 2 of U138 I can not verify ICB+ and ICB- directly, but I do believe that the switching action performed by the XOR gates(U148-A and -B and the U149-A, -B, -C and -D) is correct.

I am puzzled by this and I have no idea where to go next.. I could start looking at the opamps of the ADC, but I don't think that they are to blame. Negative feedback keeps pin 2 of U138 at zero, which sort-of indicates working performance. The two small sawtooth pulses mentioned previously supports this. I have ordered replacements of PA177GS, but it will take some time before they arrive. And, like I said, I don't think that either U138 or U137 are to blame. I have tested other circuits(Q132 and Q133), and I guess they operate as they should, in terms of when they are switched on and off. They work much less frequent than the ICB+ and ICB- switching, so they should have no effect on the generation of the triangular waveform on the adc output.

I have checked all the digital logic from the controller(U165) through to the switches(U149), and the signals are consistent all the way. The thin duty cycle clearly emanates from the controller itself. It is not due to a defect gate or anything. The four switches work inversely of each other, like one would expect, so that isn't the issue.

I welcome all answers and ideas and will happily try whatever I can to try to fix this issue. I can't borrow components from the working unit, though.

Thank you for the read!

Edit: for some reason I wrote Keysight instead of Keithley in the subject. Sorry about that.

[TiN]

Stupid question, is the reference voltage okay (6.9ish V) ? Also check all power rails if they are okay.

[petemate]

Stupid question, is the reference voltage okay (6.9ish V) ? Also check all power rails if they are okay.

Hi Tin, thanks for replying. Yes, all rails are ok. I have had some issues with the reference voltage not being correct (the +/-14v regulated incorrectly), but this is not the case during the measurements of the ADC output. But i will re-check this, just to make sure.

[Kleinstein]

It looks like the ADC is going into saturation (e.g. to positive an input signal).  The slope down to the negative side is really fast, so this can not be the normal input current, even if the input it all the way to the positive rail. It also already goes down rather early, likely before the actual measurement, more like in the zero phase.  This makes me suspect that Q132 or something around that path (e.g. CR110) could have failed.  Q132 is responsible for a reset of the integrator before the actual measurement. An interesting test-point might be the drains of Q132/Q133.
Another point to check would be the input voltage to the ADC.

The odd thing is that the integrator is no totally stuck to the negative side - so it is not just Q132 fused.

Another possible cause of error could be a broken U149 (e.g. gate to source link at U149B).

The integrator itself looks like it is working. So U138 is likely not broken.

[petemate]

Hi Kleinstein, thanks for your suggestion.

First of all, as per TiNs suggestion, I verified that the reference voltage is indeed a stable 7V while all of this is going on.

I checked that there were no problems with CR110. It appears to be fine, with 0.3V across it when tested with a multimeter, which matches the info in this datasheet: http://www.redrok.com/Diode_Schottky_HSMS-282x_1pF_100pS_12O_0.34V_-15V_AVAGO.pdf.

I then had a look at pin 3 of Q132/133, like you suggested:



Please note that I switched from looking at that first, initial data pulse to look at the repeating data pulse, since I believe the behavior to be the same in both cases(=the rapid drop of the ADC slope).

Ch1 is gate of Q132, Ch2 the data bus, CH3 is the drains of Q132/Q133 and Ch4 is U137-6. The gate voltage initiates the reset like you said, which makes sense, dragging the voltage of U137-6 to zero. The voltage at the drain(Ch3) is zero due to Q133 being on(its inverse of Q132). Then, when Q132 turns on, the voltage goes negative because now R234 and R225 divides down the -15V at U142-6. That gives about -1.2V, but thats not the level indicated, but at least its negative. But when Q132 then turns off again, the drain level rises to ~100mV, as seen in the above picture. Now that is weird, since Q133 is operated inverted from Q132. Could this maybe be the error?

Anyway, zooming out reveals something interesting: There is a triangular pulse sometime later.



Zooming in on this pulse reveals that it has "nice" slopes:

(channels same as before)

I then try to figure out what causes the drop in voltage on Ch3, which is odd since the gate of Q132 is low.



Channels are similar as before, except for Ch1, which is now gate of Q131.

This is very confusing to me.. Any ideas at what to look at? Im not sure what to make of that triangular pulse afterwards, but right now the interesting most thing in my opinion is Q133.. I'll compare with the working K2000 that I have and see if I can make out something. Please let me know if you have any ideas for what to investigate! Thank you!

[petemate]

I just tried replacing U133(I have a K2700 that i can borrow parts from) and got the following results:




As usual, Ch2 is data, Ch3 is drain of Q132/Q133 and Ch4 is U137-6. Ch1 is gate of Q132. Unlike before, the drain-voltage(Ch3) now rises much higer. However, why is the drain voltage still high(even higher, now), when Q133 is pulling it to ground?



Zoomout, with same channels as before. Notice that the "base level" of Ch4 is now around -5V, not -15V. Ch1 goes high and Ch3(drain of Q132/133) and Ch4(U137-6) goes to zero. Then Ch1 goes low again. which means that Q133 gate goes high and thus Ch3 should drop to zero. However, that doesn't happen.



Zoom of that triangular spike. Its shape is now completely different than before and it returns to about -5V instead of -15V like before.

Finally, my measurements on the working K2000 do of course confirm that whenever Q133 is on, then the drain voltages should be low. I can't understand why that doesn't happen..

Bonus question: VN0605T is quite hard to get a hold on. Would you guys agree that I can use 2N7002 FETs as an alternative? Thanks!

[Kleinstein]

The behavior around Q133 looks really odd. Looks a little like a broken Q133 with gate to drain leakage / short. 
Still this alone would not explain why the integrator goes negative so fast - maybe the very short time before Q132 is turned of is enough for this.

The 2N7002 should be OK as a replacement, at least for Q133 or a first test. Probably it would be OK at other positions too. The specs on both the 2N7002 and VN0605 are not that tight and not that specific to the actual use. Thus parts selected / checked for low leakage might be a good idea, though likely most samples will work OK.

I don't understand the different behavior with the different U133 logic chip. Maybe slightly different timing, due to different firmware versions ?

The triangular shaped part could be something like the rundown phase.

[petemate]

Thanks for your reply, Kleinstein. Im afraid that I won't have time to do more bughunting in the coming weeks(will be traveling), but I will definitely keep you updated.

[petemate]

An update has been a looong time coming.. Im sorry about the loong wait, but I have been really busy with a new job and other stuff. I hope that some of you Keithley experts will be able to provide me some pointers and hints.

Anyway, I replaced the four FETs Q127,Q131,Q132,Q133 (with 2N7002's) and I now have a start-up sequence that looks like the first image i posted, taken from a working unit.

Image from working unit:


Image from my unit:

Again, Ch2 is the data bus, CH3 is pin 1 on U148-A, which is responsible for switching in ICB+ to the ADC, and Ch4 is pin 6 of U137, which is the ADC output to be zero-cross detected. I tried to match it to the original, working picture, for reference.

It shows a nice ADC output which does get zero-cross detected and sent to the ASIC. But I still don't get any updates to the display. Its "frozen" with only ---.--- mVDC written. There are no updates.

Running a 100.1 test fails, even though I assume I get correct waveform. Pressing shift after the test reveals the number of counts, which is "-1358692021". Here is a screenshot of a 100.1 test:


And a close-up showing the output of the ADC:



Now, here's the really weird thing: After a break, I turned on the device again. It "worked". It was updating the display and counting(as it does when the inputs are unconnected). However, it wasn't reacting to shorting the input. It just kept counting.. After some time, it then went back into the "frozen" mode.

Now the big question is: What do I do now? I am sort-of out of ideas and I am afraid that the ASIC is fried, since the ADC appears to be working correctly. However, the fact that it "sometimes work" makes me think that it isn't the case. But what now?

[Kleinstein]

The some times blocked state is a difficult to find fault, like with most intermittent faults. It could be something like a cold solder joint, but also a boarder-line signal level at some point.  The frozen state makes me suspect the communication between the ADC and ground based part - so this could be the ASIC, but also the opto-couplers. Still reacting to keys suggest the ground connected part is still working.

Continuously counting up, but not reacting to a short would indicate something like an input problem (e.g relay), JFET control.

[petemate]

Thanks for your input! I'll have a look at it tonight. Do you know if anyone has done some work on how the ASIC communicates with the MCU? Any specific protocol, etc?

I thought I had eliminated problems in the front-end and narrowed it down to the ADC, but I will have a look at it again. I had some trouble around U115 and the FETs that it controls, but I'll have a look if I can get it to go into this "runing but not counting" mode. It however only happens after being powered off for some time(I think, haven't investigated this much), so it might be something that gets charged up somewhere, but will run when its initially turned on.

What are your thoughts on the slope screenshots ? Are they looking correct?

Again, thanks for your input. I will keep this topic updated. Hopefully it won't be as infrequent as previously

[Kleinstein]

The Integrator slopes look fine,  very similar to the working unit. It still looks a little odd in the time before what looks like the runup-phase with the regular up/down triangle pattern.  However this is the same in the other unit. So it could be some kind of slope check done before every conversion.

The waveform for channel 1 looks. I would guess thus is again U148 pin1 and thus a supposedly digital signal. So have a 3rd stable level is odd and could indicate a problem with the supply or possible some loading of the output.

[petemate]

Hi, thanks for your feedback. If I recall correctly, Ch1 is the TP105, which is the input to the ADC, so it makes sense that its shifting around.

Anyway, I found something that may be (partly?) causing the problem: FSRES is behaving very weird. According to my calculations, it should be a steady voltage of about -4.7V. It comes from the TF245 resistor package and is basically a voltage division divider the +14V and -14V. However, when I measure the voltage level with the oscilloscope, its close to zero. Both +14V and -14V are as they should be. If I use a multimeter, I am able to trigger all sorts of weird behavior on the ADC slope. Now, the FSRES is just a voltage divider between an 15k and the 30.1768k resistors inside the TF-245. Then there is a 1.28M resistor to the input(virtual ground point) of the ADC. I don't see why measuring that node would cause so much trouble ? I know its part of the feedback loop, but Its not that high impedance and it isn't high compared to a probe. What is the purpose of the FSRES signal? Its some sort of constant slope added to the ADC, but I don't get why. And it seems weird that its off and can be manipulated so easily..

[petemate]

Jesus! I made it work! The problem was indeed the TF-245 FSRES signal!

Its really weird, because the problem wasn't the resistors themselves. I could measure from pin 3 to 4(14k + 30.1768k) just fine, But I got OL or several megaohms from pin 3 to 14 and pin 4 to 14. So it must be the pin to IC PAD that was the problem. It was sort of confirmed because I could press *really* hard and get some sensible, but off readings from pin 14.

Anyway, I took the TF245 from the Keithley 2700 I have and put it in the 2000. It works just fine now! Crazy!

Now I just need to get hold of another one for the 2700(Or rather, a new one for the 2000 as I would avoid some calibration on the 2700 by putting it back). Thats probably gonna be a problem, though :/

Edit: in total the solution required replacement of Q127,Q131,Q132,Q133, some JFETs(Q104,Q105, etc - forgot which ones) and an LM399(which drives the JFETs). And then the TF247.

[Kleinstein]

I don't know for sure, but the 1.28 M resistor from the FSRES signal is likely there to produce a fine slope for the ADC during run-down. The exact actual value should not be that critical as very likely there is an extra test measurement to measure the actual value of the divider +1.2 M resistor. However I don't know how close the value really has to be. Chances are there is some 2-10% of tolerance.

So there is a chance that one could replace the 2 resistors inside the TF245 array with 2 external one to create the FSRES level. I don't think theses divider is that critical, so that 2 normal thin film (maybe even thick film for a 1 st test) resistors should be good enough.
Like changing the complete resistor array, it might take some kind internal calibration call, maybe self-test.

[petemate]

One of the resistors has an exact value of 30176.8k, so that has to be important. Anyway, the problem is that the resistors are fine, its just the pin out of the IC thats faulty. So I would be throwing away to perfectly good resistors(and one calibration) if I had to replace them. All i need is the internal connection to the outside. Thats gonna be a hard one to solve, though. I'd rather see if I could find a replacement part, but I doubt it.

Anyway, thank you for all your help. I do really appreciate having someone to help me out and provide feedback.

[Kleinstein]

One of the resistors has an odd value, but there relevant number is the current provided through the 1.28 M resistor, which is specified with 1% tolerance. I don't think the 1% is accurate enough for the fine slope (though it might be just good enough). It is kind of standard to use the ADC to do a internal calibration of the exact value. However I don't know when this is done: It could be enough to be done at power one (though the ADC is not yet warmed up - so not a good time for an accurate adjustment). It could be during calibration runs, self tests, or maybe even before every conversion or every change in range.  So chances are very good that the resistor ratio is not that critical - the odd number cold be just a nominal value.

Normally the fine slope part should be responsible for something like 0.01% (maybe less) for a 1 PLC conversion and 10 times less at 10 PLC. So the DMM would like to know the slow slope to something like 0.1%-1% or better.  So there is a slight chance the 1% tolerance 1.28 M resistor could be sufficient, but it is also easy to measure for the ADC, even in cold ge the state. So a quick measurement during power up would make sense. Another related (maybe even slightly more important) parameter would be the +14 V to -14 V reference ratio - so this would likely be measured together.

A slightly wrong value and no internal measurement to use the actual value would add some medium range DNL/INL, that could be tested with a reasonable simple setup  (measure a slow slope at a small fraction of the range).

So unless there is a good source for the TF245 resistor array, I would consider using 2 external resistors. Even 1% types could be good enough - no real need to much better than the 1.2 M.

This seems to be not the first time that the TF245 resistor array fails - so maybe someone else already knows how to get a replacement.

[petemate]

I have written Tektronix to ask, but they won't answer until next week day.

Anyway, since the internal resistors work, but the tap-out for FSRES doesn't, I will just add two resistors in parallel to the already-functioning 15k and 30.1768k resistors. This will keep the internal resistors in the feedback path, and the parallel resistors can provide the tap out for FSRES. The low-impedance +14 and -14 around the resistors shouldn't have any problem driving those.

I'll do a quick match of a before-and-after measurement of a resistor, just to compare how much the calibration actually changes.

The 1.28M is 0.1% by the way, but I don't think theres much of a problem if I hand-select the resistors. And since I have a very nice working DMM, I will be able to select them precisely.

Now I just gotta remember where my bag of 30.1768k resistors is

[Kleinstein]

The resistor ratios are such that the fine slope would be at around 1/128 of the normal slopes. So there is a slight chance that they did not measure the fine slope, but assume the nominal value.

Having a wrong resistor ratio would not effect the overall calibration, but would add nonlinearity errors, repeating on a fine scale, like 1/10000 of the full scale. So the corresponding test would be testing a range of maybe 0 - 1 mV in fine steps, e.g. with a good quality divider and measuring before and after the divider.

[petemate]

This is the hack I ended up making - I managed to match the 30.1768k resistor with 4 SMD resistors. I actually matched it to around 30.176k. So 0.8R off. But that number is obviously only as accurate as the use of the new TF-245 in the old unit. I matched the 15k resistor to about 15.004k.



Here are the results of going back to the old TF-245 with the added external resistors.

With the TF-245 from the 2700, I measured 989.882R on a random 1k resistor.
With the old TF-245(the one originally in the instrument), I measured 990.081R.

Thats a difference of 14mR. Its a 4-wire measurement. Thats a difference of 0.02%. Taking into consideration that this unit probably hasn't been calibrated since it was made(IC date code of the TF-245 is 9741, meaning that the unit is over 20 years old), im pretty satisfied. Last cal date was 17th december 1997.

I will need to get some calibrated resistors to test it out. I did a quick test on the 1k resistor of the new TF-245. It shows 999.514R. Thats outside the 110ppm guaranteed after one year, but again, this unit is over 20 years old. And to be fair, I don't know the original value of the resistor. However, the fact that I am about 0.5R below the other results on the xDevs web site is a cause for concern, though. On average I am 0.05% off those values. Thats pretty nice for a unit that hasn't been calibrated in 21 years

[Kleinstein]

Unsoldering and soldering the resistor array is quite some stress to the part. So one can expect a measurable change in resistance from the procedure. So this alone would invalidate a calibration - maybe comparable to the 20 years of time passed since last cal.  It is more like the change in the other resistors that would effect absolute value.
Comparing with a different resistor array the results could be off way more - I would consider the small change coincidence and would not even worry about a 1 % difference. The absolute tolerance of the resistor array can be quite high - well more than 0.1% - these arrays are chosen for there stable and sometimes (if adjusted) accurate ratio.

The hacked in new divider would mainly cause linearity effects: A possible test would be measuring a slow ramp like 0 - 5 mV, measured in the 10 V range. The expected type of INL error would be a sawtooth like feature with maybe a 10-500 µV period and amplitude (ideally < 1 µV, maybe a few 10 µV if bad) depending on the accuracy of the resistors. To really see it one might have to start with an intentionally bad state (e.g. 5% off) to see the pattern to later look for.  Despite of higher noise the effect should be more visible in 1 PLC mode (maybe with averaging / filtering).  A quick crude test would be watching a large capacitor to discharge from a few 10 mV and look for significant jumps that might repeat.

[petemate]

Thanks for your input and your guidance. I managed to fix the 2700 as well, so now I am up and running with a great lab setup. I just need to get verified calibration somehow. If I am lucky I can borrow a meter at work and calibrate according to its readings.