DMM Noise comparison testing project

[MiDi]

...
Usually it's  completely sufficient to take 10 .. 16 samples to get a good estimate for the quantity to be measured, and also that implies that the StD is sufficiently 'stable'.

Taking more samples to 'improve' the StD is counter productive, as you will get instabilities of higher order into your measurements, e.g. mid- and long term drifts.
...
As you cite my measurements, I just want to refer to the Allan Deviation method, where you get a good picture of instabilities or noise over different timescales.
...

If the noise is just white noise, relatively short sets of readings give good estimates for the standard deviation. However if there is some extra 1/f or popcorn noise or drift or a other superimposed signal the RMS calculation can fluctuate and different lengths may show different values. In this case the Allan deviation plot may be more helpful than just the standard deviation. A single number is just not sufficient to characterize complex noise.

So I would interpret the fluctuations seen in the stD calculated over 100 samples each as an indication that there is not just white noise.

...
RMS noise and StD share basically the same formula, so under some precautions, like observation of the different noise sources, the StD gives a good estimate for the noise figures, as I have demonstrated with my diagram for apertures ranging from 1.4µs to NPLC of 1000.
I can't observe, that there is a large spread, compared to the hp specification, and also within the whole graph.
Please also take notice, that such measurements are always plotted on a logarithmic scale, so small variations do not play a role.
...

For comparison, the dataset was filled with pure white noise (16154 samples, SD 100nV, mean 1.53µV - same as original Data).
With several runs with different white noise sets I get (limited represantative):

	rolling SD(10)	rolling SD(100)	rolling SD(1000)	SD(16154)
multiple runs white noise	~+-50...100%	~+-20...30%	~+-5...10%	100nV <+-1%
original dataset	-72% +80%	-26% +26%	-7% +13%	100nV

An Example with white noise is attached.

The spreading of rolling ACRMS/SD over white noise for short sets with 10 samples are in the range of ~0...200nV.
In the context of comparison I would not call that good estimates for given ACRMS/SD of 100nV over all samples.
If this is my misconception of statistical methods for this purpose: I can suffer, so slap me hard on the head

In contradiction for large sets the ACRMS/SD may be dominated by other noise sources, as you already stated.
I am wondering if it would be appropriate to use rolling ACRMS/SD with short sets on a large dataset and take e.g. the Mean of all rolling ACRMS/SD?
Intention is to have a high-pass filter to get rid of probably dominant LF Noise Sources (Drift, TC, popcorn-noise?) whilst maintaining good approximation of the remaining mostly white noise part.
Clearly there is not one number to deal with "complex" noise, but if it is possible to separate appropriate into a couple of numbers for different noise types and other influences, this would have value.


The shorted DMM seems to have mostly white noise, but you are right, for deeper insight the Allan deviation would be helpful.

We want to compare the ACRMS/SD of shorts for different DMMs at given PLC and range.

Who is 'WE'? Plural Majestic? 

'WE' meant in the sense the participants in "DMM Noise comparison testing project"
I am pretty shure you got the point 

[MiDi]

Trying to answer the remaining question:

...
I also do not understand, which theory you mean, which gives an idea about the 'trustworthiness'  of the StD.. sounds very strange to me.
...
Reference, FLUKE: Applying Measurement Uncertainty to Digital Multimeter and Clamp Meter Calibration
http://download.flukecal.com/pub/literature/webinar-uncertainty-presentation-Dec%202011.pdf

What I meant is the transition from student t to standard distribution for higher count of given values (degrees of freedom).
In the linked Fluke document on p. 54 the adjustment factors are plotted for different degrees of freedom.
For a coverage factor of 1 to 3 (Sigma) with 100 degrees of freedom the adjustment factors are not significant (<<10% deviation).
From this I derived the misconception that 100 values would be sufficient to get ACRMS values with sufficient confidence ("trustworthy") for comparison - seems I am not the only one: this is the point of my discussion.

PS: I had that figure only in mind, but good to have the confirmation from the Fluke document p. 54.

Thank you all for the valuable suggestions & links.

[Kleinstein]

In contradiction for large sets the ACRMS/SD may be dominated by other noise sources, as you already stated.
I am wondering if it would be appropriate to use rolling ACRMS/SD with short sets on a large dataset and take e.g. the Mean of all rolling ACRMS/SD?
Intention is to have a high-pass filter to get rid of probably dominant LF Noise Sources (Drift, TC, popcorn-noise?) whilst maintaining good approximation of the remaining mostly white noise part.
Clearly there is not one number to deal with "complex" noise, but if it is possible to separate appropriate into a couple of numbers for different noise types and other influences, this would have value.

The shorted DMM seems to have mostly white noise, but you are right, for deeper insight the Allan deviation would be helpful.

Using the average over the SD calculated over shorter subsets is kind of doing some high pass filtering, taking out very slow processes like drift and some 1/f noise. So this may be appropriate. However it should not help with the general limitation of all noise estimators. It just takes quite a lot of samples to get a low uncertainty for the noise estimate. So 100 samples is well good enough to need to worry about the limited number of degrees of freedom and thus the correction factor.  However the SD calculated from 100 samples can still show quite some noise, even with the ideal white noise data.  So for a normal measurement 100 samples are OK to calculate the uncertainty - usually one does not really are if an measurement is +-1.0 % or +- 1.2 %. It is not uncommon that the uncertainty is not known very well.  However when noise is the main interest 10 or 12 may be a significant difference.

In this sense the Fluke Webinar contains a flaw: they use just 6 readings to calculate the statistical part of the error as +-25.5 mA. However these +-25.5 mA have a large uncertainty - it could as well be 15 mA or 50 mA with quite some probability. So rounding to 30 would probably be appropriate. With so much uncertainty in the error, there is little sense in calculating the exact correction factor from the limited degrees of freedom.
6 readings may be OK to get the value, but they are not at all good to get a good estimate of the statistical error.

With 100 readings resulting in +-25% (I assume this are the extremes) scattering, this may be just the lower limit to get a useful noise estimate. More would definitely be better, though it should only improve like the square root. At least for the shorter PLC settings 1000 readings are possible - with longer integration this gets slow and more sensitive to drift.

[3roomlab]

TTEST stats function in opencalc
hmmm fascinating
i tried this in 1 of my old data
TTEST(x;y;2;2) = 0.13497 (n=70, NPLC10 10v, 1.5s interval per sample)
TTEST(x;y;2;2) = 0.2917 (n=25 .... )

samples from near the top, and samples from near the middle in a span of 3600 samples.

[branadic]

AD774x series are capacitance to digital converters (CDC) based on sigma delta ADCs, thus you want to express noise in terms of the input parameter, which is capacity here.

-branadic-

[jchw4]

My 34401A. I bought it used, history is unknown.

Code: [Select]

*IDN?
HEWLETT-PACKARD,34401A,0,7-5-2
cal:count?
+42
cal:str?
"15 FEB 1996 26.2C"
(Why 26.2C? Who knows?)


Script:

Code: [Select]

*RST
*CLS
CONFIGURE:VOLTAGE:DC 10V,MIN
VOLTAGE:DC:NPLC 1
SENSE:DETECTOR:BANDWIDTH MAX
SENSe:ZERO:AUTO ON
INPUT:IMPEDANCE:AUTO ON
TRIGGER:SOURCE IMMEDIATE
SAMPLE:COUNT 50
Then

Code: [Select]

READ? in a loop.

I repeated the same script for NPLC 10 and 100.

Horizontal axis is minutes from logging start.

34401A User Manual has this table:

Code: [Select]

Integration time Resolution
1 NPLC           0.000003 x Full-Scale
10 NPLC          0.000001 x Full-Scale
100 NPLC         0.0000003 x Full-Scale

It seems that the noise is below the stated resolution, which is definitely good!
I wonder why the meter has negative offset for 1NPLC, positive offset for 10NPLC and no offset at all for 100NPLC.

[Kleinstein]

Chances are the zero adjustment was done with 100 PLC - so no offset there is no surprise.
Than having some offset with 10 PLC is a surprise indeed, as the 100 PLC more is supposed to be the average over 10 of the 10 PLC readings.
The PLC can be a little different, with more effect from amplifier settling / switching transients.
There could also be different zero constants with the faster modes not adjusted as careful / recent.
The temperature vales should be the internal temperature during the last calibration way back. The absolute value may not be very accurate.

[jchw4]

This is Mooshimeter (https://www.eevblog.com/forum/crowd-funded-projects/mooshimeter-wireless-smartphone-multimetre/, https://moosh.im/).

I bought one of the last stock from DigiKey half a year ago. Now it's all gone everywhere. Time to do some noise measurements 

I don't have proper short for it, so I used Keithley 8640 plugged in as  "AUX + VOLT" ~ Input, "VOLTS + CURRENT" ~ Sense (if it makes any sense, but you can imagine the meter oriented vertically with sockets on the bottom, and 4-wire short oriented vertically).

I also did not check the temperature, but hopefully it was well isolated wrapped in a bubble wrap and put into a drawer.

Configuration:
- "VOLTAGE DC 60V" range for main input (available ranges 60V and 600V).
- "AUXILIARY VOLTAGE DC 1.2V"  for the second input (available ranges 1.2V, 300mv, 100mv).

"RPS" =~ "Records per Second" to an SD card.

Q(0.9) = 90% Quantile of the StdDev over the graph.

Horizontal axis is minutes. Averaging is per-minute, so each graph is 30 points.

It looks it's decent 5.5 digit meter on main input (I would ignore 6.5 possible with <2 RPS) and 6.1 digit on AUX input!
(with up to 6.5 and 7.1 digit possible with 1 - 0.5 RPS on each channel).

WDYT?

[MegaVolt]

Once I saw just such a picture for the coolest 3458a multimeter and wanted to repeat it.



Here are the results:

Keithley DMM7510 noise:

Just a little more words and details here: https://www.eevblog.com/forum/testgear/all-about-keithley-dmm7510-bugs-and-features-recipes-advice-notes/msg3115140/#msg3115140

[Kleinstein]

Does a 1 PLC reading with the 6500 really take 50 ms. Nomally I would expect just a little over 40 ms, e.g. some 42 ms.
The vertical scale seems to be in volts (not ppm) - still the noise for the 100 PLC mode looks quite high, because of the keithley typical hump at some 20 seconds.  It is odd that they have still not fixed this - this seems to go back quite a bit, likely a left over from the brown 19x series.
The 1 PLC mode with averaging seems so be slightly better than 100 PLC directly, though not much.

[guenthert]

Agilent 34410A noise

10VDC, 100NPLC, Auto-Zero, Fluke 4W short.

[..]

     This is an awfully long thread and I'm sure it has been explained before, but perhaps for readers as lazy as me, it might be worth to restate (every 10 pages or so ;-} how this measurement was made.  Variations in measurement less than 0.1 ppm on a 6.5 digit meter are worth expanding on, methinks.

[Kleinstein]

Most modern higher resolution DMMs (that is newer than the early 1980s) usually have higher internal numerical resolution and use a numeric scale factor for the calibration.
So 1 LSB step of the ADC no longer corresponds to 1 display step. The PC interface often give data with the full internal resulution (e.g. more digits than actually valid from the internals).

The display resolution is choosen depending on the noise and accuracy (linearity and zero stability), but there is no simple universal rule what to choose. Especially for the longer integration time the internal resolution and noise limit can be quite a bit better than the display resolution.
The HP34410 is quite a bit lower noise than the predecessor 34401, but still sold as a 6.5 digit meter - in part because the reference and accuracy are limiting, not so much the noise. The noise also depends on the integration time. 100 PLC is usually something like the average of 10 conversins at 10 PC and thus some factor 3.2 lower noise than 10 PLC (except for many Keithly meters that get the averaging /auto zero wrong).
Modern meters often reach there nominal 6 digits with 1 PLC while some old meter needed 100 PLC and maybe even longer.

The relatively common LM399 referene is usually considered not good for more than 6.5 digits and there are thus only few meters with more digits, even though the noise can be quite a bit lower than the early 6 digit meters or comparable to 7 digit meters. The main difference between the 6 digit KS34465 and 7 digit KS34470 is the bettter reference in the 34470.
Purely from the ADC noise the 34410 is lower noise than even some old 8 digit meters (e.g. Solartron7081 and Keithly 2002 for some cases). The noise test with a short is not everything.

[bsw_m]

V7-54/2 6.5 DMM developed in MNIPI
20V range, 2.56s integration time.
Sorry, cannot calculate Allan variance for this dmm. 

[Kleinstein]

V7-54/2 6.5 DMM developed in MNIPI
20V range, 2.56s integration time.
Sorry, cannot calculate Allan variance for this dmm. 

The data somehow have insuficient resolution. There is still some variation, but just +0 and -0.  So one would need to change the data format. One may have to use shorter integration to get useful data, that are no longer limited by ouput quantization.
So far one can only conclude that the peak to peak noise seems to be less than 10 µV and thus internal RMS noise less than some 2 µV. This is relatively good for a 6.5 digit meter with a 20 V range.

[bsw_m]

V7-54/2
2 volts range, 2.56s integration time. About 34 hours log:

[Kleinstein]

V7-54/2
1 volts range, 2.56s integration time. About 34 hours log:

For me the data just show zero - so no use full information at all. There seems to be the same problem with insufficient number of digits in the data to measure the noise.

[bsw_m]

57 data lines with +0.000001 from 47982 lines.
About resolution: this is maximum resolution for ADC in this metter.

[Kleinstein]

Even with a few 1 µV reading one can not tell very much from the data. One can give an upper bound for the peak to peak noise of about 2 µV and thus some 330 nV for the RMS noise.
One might get slightly more noise and thus a better chance to see something with shorter integration.

[bsw_m]

shorter integration.

Shorter integration time for this type ADC = less resolution.

[Kleinstein]

With a limited number of digits at the output, the simple normal distribution picture for noise no longer works.  Describing the noise would need more than just the std. deviation. One could take the quatization effect seprate and try to estimate the noise in the transition before the quantization. It is a little more tricky, but still possible. It is a little more like the methods used for low resolution ADCs (e.g. 8 bits) where it is more common to have the quantization dominating the noise. The added difficulty here is that the DMM is relatively slow and drift may be an issue.

One gets essentially 2 possible readings at a given input voltage level and the result is the percentage of readings showing the higher value for something like a few 100 readings.
One can than apply a small constant voltage (one the order of a few 100 nV) and than observe how the result (percentage of higher readings) changes. This way one kind of checks the distribution function at a few different offsets. To get an idea of the steepness of the step one would ideally like some 2-3 points between 20 % and 80%.

Limiting the resolution only to those digits that are really stable adds a bit quatization noise. This is usually OK to do with the display for manual reading, but for the computer it is nice to have the raw data without the extra rounding noise. 12 digits are overdoing it a bit, but 1-2 extra digits at the computer interface are good practice. The PC can still decide to save fewer digits in case memory is low.  For a modern 6 digit DMM i don't think that the ADC's LSB steps are directly linked to the display steps (e.g. like in the old ICL7106 based DMMs). So there should be a slightly higher internal resolution to apply a numerical scale factor without extra rounding errors.

[TiN]

Little fun test with nanovoltmeters:



Settings for each instrument:

Keysight 34420A

Fixed ranges 10mV, 1mV (and 100mV,1V,10V present in RAW data but not plotted).
Meter warmed up (weeks of runtime).
Channel 1 used, NPLC 50, MAX 7.5 digit resolution
Original Keysight 34420A shorting plug used as zero source

Keithley 2182A

Fixed range 10mV.
Meter warmed up (weeks of runtime).
Channel 1 used, NPLC 5, MAX 7.5 digit resolution
Autozero and front-end autozero enabed. ACAL was not performed during whole log duration.
Second original Keysight 34420A shorting plug used as zero source.

Keithley 1801 (digitized and powered by Keithley 2002)

Fixed range 20 uV, later switched to 200uV towards the end of the plot.
Keithley 1801 amplifier was placed in thermal chamber and cycled from +18 to +30°C.
NPLC 20, 8.5 digit mode, rear terminals used. Digital filter 10.
Analog amplifier filter set to SLOW (50uF) at the beginning of the log, later switched to FAST.
Bare copper wire short mounted right at the amplifier input M3 rods.
No warmup, turned on minutes before datalog started.

RAW data in CSV-ish format (semicolon separated).

[bsw_m]

24 hour data for 0.2V range
Multimeter: V7-54/2
Setting: sample per 2.56s (6.5 digit mode)
Environment: total uncontrolled (about 7 Celsius temperature different, dirty line voltage and other antimetrology things. Now this DMM used in metal processing workshop with heavy machine tools, like a milling machine weighing 60 tons, and welding machines )
 

[Kleinstein]

The noise data for the V7-54/2 look about normal for a 6 digit meter. Not especially good, but not bad either. At least not Keithley typical extra hump in the 10-100 s range. The reading rate is rather slow, but the amplifier noise looks about normal for an input amplifier. As expected the 0.2 V range is more limied by the amplifier and not the ADC's quantization.

[bsw_m]

0.2 V range is more limied by the amplifier and not the ADC's quantization.

You right!
Some interesting fact for this input amplifier. Please note to input bias current.
Amplifier is little bit noisy, but have very low input bias current (~0.07pA).

[MegaVolt]

Amplifier is little bit noisy, but have very low input bias current (~0.07pA).

Is there a schematics or other details?  What transistor is at the input?  How are the protection circuits made?