DMM Noise comparison testing project

[martinr33]

I put the files on the Xdevs FTP site under K2010martin

Here are the 10V results:
10V 10NPLC   STDEV 500nV average -600nV
10V  1NPLC        STDEV 830nV average -680nV

As with the other K2010 here, and another unit I have, the SD is quite low at 500nV.

I included the zero voltage because I find it surprisingly low for a unit this old. I don't see any evidence of calibration.
S/N 0743311
Rev: A09 A02


60Hz 110VAC ambient 70 degrees F
Keithley low thermal short applied
Been warming up for weeks
Absolute readings - not relative (unit has good zero if it is allowed to sit)
Unit came from Japan. Display is tired.
No work done on meter yet. May need caps, fan and so on.


SCRIPTING NOTES - the script is also in the xdevs directory.
K2010TEST is the script. Works OK, except some readings are too slow. Needs timeout increased to 90 seconds.
See notes at beginning of program.

Most important thing is removing the EGR COMP config, as the 2010 does not support it.

[martinr33]

Here's the distribution of readings from my 10V NPLC 1 session on the Keithley 2010.

The readings are quantized in steps of 560nV.

[maxwell3e10]

Here is the Allan variance of Martin's Keithley 2010 for 10V range 1 plc data - it has exactly the same bump as  DMM7510! This is the clearest evidence yet that the Autozero problem is deep inside their algorithm and applies to all meters, not just DMM7510. No wonder they have trouble fixing the problem.

[saturnin]

I think mentioned autozero effects can be clearly seen during warmup phase of Keithley's multimeters. They all exhibit the same saw-like warmup characteristic as autozero mechanism corrects large thermal drifts which are present just after startup.

See attached plots of warmup phase of my K2010 and K2001 multimeters (no filters used, sample rate 0.5 Hz) + data from HighVoltage's measurement from the previous page (first two plots): https://www.eevblog.com/forum/metrology/dmm-adc-noise-comparison-testing-project/msg1317367/#msg1317367

Btw. K2010 stabilizes fairly quickly compared to K2001. Even then K2001 exhibits the worst noise performance from the 2000 series - it is basically useless as 7-1/2 multimeter without heavy filtering.

[dr.diesel]

Here is the Allan variance of Martin's Keithley 2010 for 10V range 1 plc data - it has exactly the same bump as  DMM7510! This is the clearest evidence yet that the Autozero problem is deep inside their algorithm and applies to all meters, not just DMM7510. No wonder they have trouble fixing the problem.

The k2002 is probably in the same boat.  Lots of data sets in over at TiN's site, but probably best to let him choose the right one to include in the comparison.

[TiN]

Let me know what conditions are needed, I can capture fresh log, as I have both meters currently idle.

[maxwell3e10]

The saw-tooth behavior during warm-up was also observed in DMM7510, https://www.eevblog.com/forum/metrology/dmm-adc-noise-comparison-testing-project/msg1081865/#msg1081865

Its a good question if this periodic relatively slow temperature compensation is the source of the AZ problem. I believe the evidence points against it. In particular, there are no spikes in the Fourier transform of the signal, as one would expect for a periodic correction. The most likely explanation for AZ bump is that the zero value for autozero correction is calculated as a running average over a long time. Forcing the triggering of the meter can eliminate the bump, probably because it resets the running zero average.

The data for K2002 meter are analyzed in https://www.eevblog.com/forum/testgear/keithley-dmm7510-smu-2450-2460-problems/msg1083291/#msg1083291
It shows similar behavior, but K2002 short-term noise is higher, so the bump is not as pronounced.
 

[Kleinstein]

The saw tooth like curve on warmup is likely due to internal corrections done with something like temperature measurements every 70 seconds or the like. At least it looks like those corrections are done quite good. It might be worth to compare the warm up with a 10 V signal and with a short, as there are both effects on the scale factor / reference and offset drift. I would expect main an effect on the scale factor - at least with AZ mode active.

These corrections (especially those effecting the offset, and if not implemented well) could also lead to some extra noise in the same frequency range as the extra noise peak. So there is a chance that a not so good implementation if those temperature corrections is causing the problem. This might be hard to fix and might meed HW modifications, e.g. to get better temperature readings - though with a reasonable stable ADC/amplifier the demands on the internal temperature reading should not be that high. Also triggering can change things more points to a problem due to the AZ procedure and not temperature corrections.

The 2001 meter is different from most other meters, as it uses a separate short time reference (zener-diode on the ADC board) for the ADC and long time reference (LM399). So there somehow needs to be a regular extra measurement of the long time reference (LM399) - this could be as frequent as after every reading. There are other options, like switching between the AZ and 7 V reading - thus requiring some averaging for the AZ readings.

This procedure can lead to rather good stability (similar effect as a frequent acal 72 on the 3458) but also more lost time and thus higher noise. Also the data rate might be lower than expected from the PLC setting. AFAIK the 2001 is rather high noise, but good stability / linearity.
Chances are also that the noise contribution from the reference has a different spectrum than just a LM399 - if well done it could be better, as the extra short time reference could be used for a digital filtering of the LM399 reference to reduce the noise to the level of the possibly lower noise short time reference.

[Andreas]

Hello,

same behaviour here with a K2000 on a 7V reference.
Saw tooth up to 10uVpp (except when warmup drift is too large).

Warmup shown for the first 3 minutes with 10 NPLC (~0.6 seconds / measurement).
X-Axis: number of measurement.

with best regards

Andreas

[saturnin]

Some time ago, I debugged the ADC in K2001 (at first I thought it was faulty - too noisy). When I connected test leads to a node on the ADC board, there was often an immediate shift in readings - a few tens of ppm. Of course, this is not so surprising since conditions in the ADC circuitry were changed by attached test leads. What surprised me was K2001 reaction - with test leads still connected it returned to the initial reading very slowly and reluctantly. IIRC, the return to the initial value consisted of several discrete steps (it was not definitely smooth).

I suspect AZ calibrates zero/gain of the ADC and averages these calibration values over a given time period. Therefore several measurement cycles are needed to adapt to a new state of the ADC circuitry. Using averaged zero/gain calibration values rather than actual ones might lead to excess of noise in K2001 ADC. (AZ might not be able to seamlessly compensate the drift since it looks too much to the past due to the averaged values.)

The 2001 meter is different from most other meters, as it uses a short time time reference (zener-diode on the ADC board) for the ADC and long time reference (LM399). So there somehow needs to be a regular extra measurement of the long time reference (LM399) - this could be as frequent as after every reading. There are other options, like switching between the AZ and 7 V reading - thus requiring some averaging for the AZ readings as I observed.

Kleinstein, I would correct you at this point. K2010 model uses an auxiliary reference too (its nominal value is 6.4V, see attached photo). This reference is used for definition of CB currents and the constant current source depends on it too. The master reference (LM399) is used to calibrate the whole multimeter.

[Kleinstein]

Some time ago, I debugged the ADC in K2001 (at first I thought it was faulty - too noisy). When I connected test leads to a node on the ADC board, there was often an immediate shift in readings - a few tens of ppm. Of course, this is not so surprising since conditions in the ADC circuitry were changed by attached test leads. What surprised me was K2001 reaction - with test leads still connected it returned to the initial reading very slowly and reluctantly. IIRC, the return to the initial value consisted of several discrete steps (it was not definitely smooth).

I suspect AZ calibrates zero/gain of the ADC and averages these calibration values over a given time period. Therefore several measurement cycles are needed to adapt to a new state of the ADC circuitry. Using averaged zero/gain calibration values rather than actual ones might lead to excess of noise in K2001 ADC. (AZ might not be able to seamlessly compensate the drift since it looks too much to the past due to the averaged values.)
...

Using some averaging for the gain calibration is OK, as this can reduce the noise, especially ADC noise and higher frequency noise of the long term reference. Averaging determines the cross over from the long term reference to the short therm reference at the ADC.  However averaging for the zero reading used for the offset compensation (AZ) is not such a good idea: it reduces the higher frequency noise, but on the back side it adds low frequency noise, that is higher due to 1/f noise. Ideally the AZ readings can suppress much of the low frequency noise just like a chopper amplifier. However this does not work well anymore when the zero readings are averaged. This seems to be a problem with quite a few of the Keithley meters.

[Mickle T.]

The master reference (LM399) is used to calibrate the whole multimeter.

No. The master reference is used to calibrate the ADC FS on a periodic basis. K2100 does the same thing via the same DG408 MUX.

[saturnin]

Kleinstein,

thank you for explanation. That's probably right - AZ gain calibration is not likely culprit in this case: there is significant noise even with 0V input signal in K2001.

The master reference (LM399) is used to calibrate the whole multimeter.

No. The master reference is used to calibrate the ADC FS on a periodic basis. K2100 does the same thing via the same DG408 MUX.

Well, I maybe used a little bit vague formulation, but my point was to mention there is the auxiliary reference (6.4V) in K2010 too. On the other hand, there is very stable master reference (LM399) that serves as a reference point for the whole multimetr (i.e. all functions depend on it - ok, except frequency). That's obvious the ADC is used to compare reference value with measured ones... 

[maxwell3e10]

Given the fact that K2010 and DMM7510 have very similar noise, I wonder if the analog part is almost the same, only a faster processor for the DMM7510. Does anyone have both meters to open?

[martinr33]

There's a good 7510 teardown video on

The DAC seems to be living inside an Actel ProASIC3. The version here has 250,000 gates - enough for an ARM core. The previous DAC was eventually placed in an ASIC, but the new one looks like it adds capability. On top of that - two PowerPC processors. Quite a lot of processing.

I don't think that the K2K units had a processor in the gate array. Too early for that - best capability in that device would be a simple state machine. Not enough to do any averaging.

[Kleinstein]

The K2010 and DMM7510 have similar performance. So they might share some of the electronics. However the 7510 seems to use the better reference (LTFLU) compared to LM399. On first sight it is mainly the graphics display with the DMM7510. Not shure for what reason they need that much processing power, maybe the OS for the graphical system.

From the teardown pictures so far, it looks like the new ADC uses a fast ADC chip for the ADC internal feedback. The newer HP meters seem to use a very similar method, at least they use similar parts. Likely it is much like an improved 34401, with an ADC to get the residual charge on the fly. This would need quite some processing for the ADC already and thus the more powerful FPGA. Something like the AZ mode or adjustment measurements would be already low data rate and could thus very well be in the ground referenced part.

p.s.:
I looked at the picture from 2010 and the DMM7510: there are very different designs. The 2010 is more like an improved 2001 - the 7510 looks much more modern. So not much in common at all.

[maxwell3e10]

On the 1V and 100 mV scale the noise for DMM7510 is about a factor of 2 lower than for K2010 (and also better than for HP3458A). So the amplifier in DMM7510 must be better. But the 10 V range has exactly the same noise as K2010, so I guess ADC performance hasn't changed even if the processor is faster.

[Kleinstein]

The ADC in the DMM7510 is not just using a larger / faster FPGA. From what it looks like, the 7510, like the Keysight3446x/70 is using extra ADCs and likely a more continuous mode like in the old 34401. So integrate with feedback (using fast ADC Chip in the new meters) and measure the residual charge with an ADC. The ADC in the 2010 very much looks like the typical older Keithley ADCs (e.g. 2000,2001). So the more classical way with integrate and separate rundown phase. In contrast to the high noise 2001 they seem to have found a good way to use the secondary reference to get low noise even with just a LM399 as the main reference.

So the 2010 and 7510 do not have very much in common and the very similar noise level is more like coincidence, maybe due to the same target noise during design. I would consider the 7510 a modern replacement for the 2010. The 2010 looks rather similar to the 2182 nV meter for the ADC (and reference) part. The rather similar noise peak suggests that they use the same AZ algorithm.

[maxwell3e10]

It's interesting that the intrinsic noise of the ADC (on 10V scale at 1 to 10 PLC speed) is virtually the same on all meters (HP3458A, 34465/34470, DMM7510, K2010, K2002), within roughly 50%. I don't think its just a design target. Certainly if someone had come up with a way to significantly beat 3458A noise, they would have run with it. There must be a pretty hard hardware limit.

Its not obviously a limit coming from analog input. A 10 nV/Hz amplifier would give at 10 PLC a noise of 2.5ppb, more than a factor of 10 less than typical meter noise. Its not obviously the voltage reference noise, if the signal is near zero volt. Its not obviously timing jitter in an integrating ADC, 100 psec jitter would give 0.6ppb noise. What are other possible fundamental sources of noise?

[Kleinstein]

There are quite a few contributions to the noise of those integrating ADCs. As typical with an optimized design it is not one big noise source, but several of similar size. A few obvious ones are:
 
1) The resistors at the input of the integrator. For the 3458 this are 40 K for the input and about the same for the reference currents. So effectively 80 K and thus about 36 nV/Sqrt(Hz). Other DMMs often use larger resistors.
2) noise voltage of the integrator OPs relative to the input signal.  Due to the low frequency (e.g. 25 Hz range or lower) there can be quite some 1/f noise from JFET OPs. A BJT based OP like the OPA177 will add some current noise too.
3) Higher frequency noise of the integrator and following comparator (if used). The comparator might need to be rather fast and thus the effective bandwidth can be rather high and thus quite a noise contribution. Some modern ADCs use and extra ADC chip instead. This might be an advantage due to an effective lower BW. Not sure on how much of the given opportunities they actually use. This would be especially true if an ADC is used in combination with a separate rundown phase ( I don't know such a design).
4) The charge zero phase at the start of conversion might add some noise for the starting point too - this part might be overlooked in some designs. Again continuous versions with ADC might not use a zero Phase.
5) Charge injection from the switches for the reference might also add some noise, especially with fast switching.
6) Clock or control circuit jitter. For a low noise it needs signal jitter well below 1 ns. It is not so much the full time, but with a rather fast reference switching frequency, a single conversion can have several 10000 switching events. So clock jitter adds up. Frequent switching is attractive to keep other noise (and INL) contributions small.
7) The balance of the positive and negative references adds some noise (e.g. the OP and the resistors used for this)
Power supply noise and similar signals coupled in might be a factor too
9) for the critical low frequency part (e.g. 1-100 mHz range) thermal stability can be a factor too - this often is what makes the Alan variance curve go up after some point.
10) Higher frequency noise from the reference (e.g. modulation frequency band) can enter.  Though filtering is easy I have not seen it with most DMMs. 
11) A non perfect sync to the line frequency can contribute

There are other contributions too, some can be hard to find and specific to some ADCs.
For the AZ mode the second measurement also adds to the noise - it is hard to avoid for the longer time scales.

The typical noise level of a LTZ1000 reference also sets a noise level, that might lead to a common design target. There is not that much advantage of having an ADC that is much lower noise than the reference. Similar there are not many signal sources (e.g. calibration sources) that have a much lower voltage noise.

[maxwell3e10]

I just got a used Agilent 34420 nanovoltmeter, seems to work fine. On 1mV and 10mV ranges the input noise is 5nV/sqrt(Hz), so about a factor of 2 better than chopper amplifiers. The Allan variance looks good too, crosses 1 nV at about 10-20 sec.
I wanted to compare the performance to the Keithley 2182A. TiN, I saw you use one recently, could you take some shorted input noise data, primarily on 10mV and 100 mV scales.

[TiN]

Ok, will do, however one I have now is plain 2182, non-A. And it's non-proofed one, might be sub-par performance, as it was after repair.
It's currently used together with other meters to log my proto FX references, but I do plan to compare it's noise performance versus K2002, K2002-1801(A10) and K182M I have.
I'll be recalibrating it as well, it's currently +3.5 ppm off on 10DCV.

[maxwell3e10]

Thanks. The specifications that they list for 2182 and 2182A are identical as far as noise is concerned.

Keithley specifies the noise performance in the most asinine way possible, though, as peak-to-peak noise for a time=10x response time, where "Response time = time required for reading to be settled within noise levels from a stepped input". Neither of these is a well-defined quantity.

There were some earlier reports here by alanambrose (#332)  of the K2182A performance that show very anomalous behavior of rms noise vs. PLC. So I am curious to look at the raw data.

[Kleinstein]

From the pictures the K2182 ADC looks very much like the K2100. So it might be interesting to not just look at RMS noise vs. PLC, but also look at the Allan variance curve. I would be surprised not to find that extra noise in the 10-50 seconds range. Those old data from Alanambrose look a little odd.

Independent from the meter used, for less than 1 PLC it might be interesting to separate true noise from 50/100 Hz coupled in. Just looking at the RMS values can be misleading if it is dominated by a 50/100 Hz contribution, that can also vary between units.

[maxwell3e10]

I made a new summary plot showing the noise for a bunch of DMMs with shorted inputs. I picked 100 PLC integration time since it gives a good estimate of the uncertainty after a few seconds of data taking. The voltage rms noise is plotted vs. voltage range. This allows one to distinguish the noise of the front end vs. the noise of the ADC.

Part of my motivation was to see where HP3457A would fall on the plot. The 10 V range in it is added using a voltage divider to 3V range https://www.eevblog.com/forum/metrology/extra-10v-dc-range-on-hp3457a/msg1521082/#msg1521082.

EDIT: Added a couple more meters and fixed some errors in the spreadsheet
Some conclusions:
Best modern low-noise DMM: 34461A
Best nanovoltmeter: 34420A
Best low-cost used low noise DMM: 3457A