DMM Noise comparison testing project

[maxwell3e10]

I fixed a couple of errors in the plot in the previous post, going back to the raw data when possible.

[serg-el]

Хмм.

У меня получились другие данные.

I got other data.

Edit: added data
WTF!!!

Edit2: ManateeMafia HP3458A пересчитано

[maxwell3e10]

Проверьте что вы вычисляити стандартное отклонение а не среднее, у Тина есть ошибки в таблице

[serg-el]

В каких данных ошибка?
Я пока не нашёл.
Вычислял stddev (rms).
СТАНДОТКЛОН в русской версии exel.

Edit: нашел.
Новые данные выше.

[TiN]

EEVBlog fixed unicode support for forum? Wowski

I had two versions of Excel file, with older peak-peak values and then later with RMS Stddev.
Also there was issue of double division by range on some test.

[maxwell3e10]

Old neutrons 34461A, pilelie 34420A - analyze original data, summary table shows rms, not stddev.
barnacle2k Solartron -shows integration time in seconds, not nplc

[serg-el]

https://xdevs.com/datashort/Pipelie/
Вычислял сам.

https://xdevs.com/datashort/OldNeurons/
Вычислял сам.

barnacle2k Solartron это, да. Неправильно. Но график не сильно изменится, если пересчитать 3,2 сек. в plc. Будет равно 160 PLC. Если 50 Гц.
Или 192 PLC если 60 Гц.

RMS это и есть stdev

Edit: OldNeurons_RAW_25_344641A вычислено заново.
Edit: ManateeMafia_HP3458A__ID111 вычислено заново.

[maxwell3e10]

нет, Stddev=sqrt(rms^2-ave^2)

[serg-el]

Physical scientists often use the term "root mean square" as a synonym for standard deviation when it can be assumed the input signal has zero mean, i.e., referring to the square root of the mean squared deviation of a signal from a given baseline or fit.[5][6] This is useful for electrical engineers in calculating the "AC only" RMS of a signal. Standard deviation being the root mean square of a signal's variation about the mean, rather than about 0, the DC component is removed (i.e. RMS(signal) = Stdev(signal) if the mean signal is 0).

https://en.m.wikipedia.org/wiki/Root_mean_square

[dl1640]

checked my 6.5 digit DMM today.
with copper short on hi to lo.

setting:
dcv 200mV range, AZON, 25PLC (500ms, 50Hz line), 25 is the largest settable number.
sampling interval 1100ms, single trigger on each and every 1.3s with totally 3600 data.

STDEV for the last 1800 data around 0.102uV, the display resolution is only 0.1uV, not bad.

 

[knightzdw]

What meter is used in this case if you don't mind to share?

[MiDi]

Is there ootb script/tool for 3458A and Solartron 7081 for Linux/RPI awailable?
Or at least sourcecode, so that it is easy to derive script?

[MiDi]

Is there ootb script/tool for 3458A and Solartron 7081 for Linux/RPI awailable?
Or at least sourcecode, so that it is easy to derive script?

Seems I have to start from scratch or did I miss something in this long thread or elsewhere?

If there is enough interest, I would get some motivation to make a universal python script to adapt different DMMs easy.
Target is primary linux-gpib and perhaps COM-Port.

[maxwell3e10]

This thread has been dormant for a while, but it contains a lot of useful information. In particular, it had been noted here that the shorted input noise is quite small for DMM7510 on low DC voltage ranges, much smaller than for 34461-70 Keysight meters. It turns out a lot of the noise for Keysight meters is due to air flow over the input stage.
https://www.eevblog.com/forum/metrology/turning-fan-off-to-reduce-dmm-voltage-noise/
This effect was first pointed out by OldNeurons earlier in this thread. https://www.eevblog.com/forum/metrology/dmm-adc-noise-comparison-testing-project/msg665603/#msg665603
With proper air current shield or turning the fan off the noise can be reduced quite a bit. Here is a comparison of Allan deviation for DMM7510, DMM6500 and Keysight 34461 with air cover. One can see that for short averaging times DMM7510 is still the best. But because it does not average well, at longer times 34461 can do just as well.

[iMo]

34401A shorted inputs.
Stddev 42.1nV.

[3roomlab]

this pdf cpem-2016-keysight.pdf was mentioned in another thread about noise analysis
of interest is my incomplete understanding about the use of allan variance in finding a certain range of sampling which gives the best SNR
at some point in the 3458A plot, the SNR seem to get smaller

eg : for 10v, the best sampling is 10-20 samples? eye popping 150dB SNR
(150 SNR ~ 26 bits. 26bits @ 10v = 0.15uV per step. the best set of data I have is 0.6uV/24bits for similar range but needs 100x more averaging)
on most plots here in this thread contributed by users, it does not seem like 100NPLC is more noisy on 3458A than 10 NPLC
maybe theirs (on paper) is a different way of measuring?

on K2015, my previous messy data seem to suggest the best capture block is around 8NPLC. weird? but looking into the K2015 pdf, it also seem to suggest similar. with that, back then i then started to play in blocks of 2/4/8 binary increment NPLC blocks with interesting results.

in the other thread about the maths use, I used to think about the use of kurtosis. but after seeing allanvariance, i am imagining some kind of algorithm where there is a real time allanvariance calculation which adjusts NPLC as time progresses to get the most optimum sampling over time. or maybe the user specify a SNR and the software auto determines averaging and NPLC rate for fastest capture possible.
currently all of our captures are fixed NPLC.averaging, but noise and ambient intrusions are variables.

i found 2-3 xls online which has simple allan variance calculations. i tried to do something similar for noise but i could not get the allan variance to work. im sure my math is wrong somewhere 

lets hope keysight doesnt do this first before one of us does and names it "true-nplc"

[Kleinstein]

It is quite normal that there is an optimal integration time to get the best SNR. This because there can be source of 1/f noise in the ADC. In auto zero mode, from some point on it is more effective to use shorter conversions and averaging than one (actually 2 for the AZ mode) long conversions.  The like many modern DMMs the 3458 uses this, and the normal 100 PLC setting is actually the average of 10 readings at 10 PLC. AFAIK there is a way to force it to do linger integration at a piece, but this gives more noise.
The lowest noise point may be even at shorter integrations, but INL may be better at 10 PLC than 1 PLC.
For my ADC design I get best block size at 1 PLC and the DMM7510  seems to be best more around 2 or 3 PLC. It is a balance between 1/f noise and limited resolution / error in reading the final charge.

There is no real need to do some complicates read time calculations to choose the best setting. Close to the optimum it does not make a big difference which setting is used and the best settings are fixed for a given instruments. So just using averaging from a certain point on (e.g. 10 PLC for the 3458) is a good enough solution.  Things get more interesting with those meters that include an extra analog low pass filter before the ADC (e.g. DA1281, Keithley 2182) - there using shorter conversions allows a faster settling analog filter. The filter helps reducing the noise bandwidth seen for the input signal - this does not help much with a short in the 10 V range, but it can help (get the same noise level in half the time) if the signal source itself is noisy or in the low ranges where the amplifier noise is more important than ADC noise.

[Dr. Frank]

Using APER 1 instead of NPLC 50 gives a single conversion,  what you can directly observe in OHM mode, probably valid in DCV mode also.

[MiDi]

Made some measurements with 3458A on 10V and 100PLC.
It seems - at least for this unit - that 100 or even 1000 samples are not enough for "trustworthy" results of StdDev.
For a rolling SD over 100 samples we get values from 74nV to 126nV (+-26%), for 1000 there are 93nV to 113nV (-7% +13%) with overall SD of 100nV for 12184 samples.
At least for me this is surprising, because theorie claims that 100 samples should give "trustworthy" results for SD 





For comparison with Dr. Franks 3458A (100 samples):




Edit: added Dataset

[Dr. Frank]

Obviously you have some misconception about statistical methods.

StD as such is only an estimate for the stability of the quantity to be measured, and it's therefore not useful at all to assign itself an uncertainty value.
Usually it's  completely sufficient to take 10 .. 16 samples to get a good estimate for the quantity to be measured, and also that implies that the StD is sufficiently 'stable'.

Taking more samples to 'improve' the StD is counter productive, as you will get instabilities of higher order into your measurements, e.g. mid- and long term drifts.

I refer to a FLUKE seminar about measurement uncertainty, where this is explained nicely. (will link it here, as soon as I find it again)
I also do not understand, which theory you mean, which gives an idea about the 'trustworthiness'  of the StD.. sounds very strange to me.

As you cite my measurements, I just want to refer to the Allan Deviation method, where you get a good picture of instabilities or noise over different timescales.

This method might be, what you are really looking for, and I've done that also on my 3458A.
Here's an example of the combined stabilities of an LTZ1000 and the 3458A.

The 2nd diagram is a similar method, which shows the noise / instability of the 3458A zero reading at different time scales, about 16 samples each point, which is fully sufficient to reconstruct the datasheet.

Frank

For example
Reference, FLUKE: Applying Measurement Uncertainty to Digital Multimeter and Clamp Meter Calibration
http://download.flukecal.com/pub/literature/webinar-uncertainty-presentation-Dec%202011.pdf

[Kleinstein]

If the noise is just white noise, relatively short sets of readings give good estimates for the standard deviation. However if there is some extra 1/f or popcorn noise or drift or a other superimposed signal the RMS calculation can fluctuate and different lengths may show different values. In this case the Allan deviation plot may be more helpful than just the standard deviation. A single number is just not sufficient to characterize complex noise.

So I would interpret the fluctuations seen in the stD calculated over 100 samples each as an indication that there is not just white noise.

[MiDi]

We want to compare the ACRMS/SD of shorts for different DMMs at given PLC and range.
What would be a good method to do this?
For me it seems that this method gives large spread and for that reason even comparing meter to itself seems not appropriate.

[Kleinstein]

There are a few difficulties with just using the RMS value for the short. One is that the RMS value fluctuates and may also depend on the length of the interval used. So one should use at least the same number of readings.
There is another complication with some meters: the readings done in a short sequence may not be fully independent, as something like the zero reading in AZ mode may be averaged / filtered over a longer time. The worst example is an unnoticed running average filter or settling from sigma delta converters.
With more than just simple white noise it would take more than just 1 number, more like the Allan deviation curve - though difficult to compare between more meters. Reducing it to one number the StD may still not be that bad. The other common choice to give noise is the peak to peak value, but this naturally fluctuates quite a bit and may give to much weight to a few points (especially popcorn noise events).
For the noise I see mainly 3 contributions:
1. simple white noise that is easy to to measure and describe with the StD.
2. slow variations and drift, like thermal effects that may depend on environment.
3. popcorn type noise with more of less sudden jumps that may appear rather infrequent - so a fast test naturally can not capture this correct and it would take really long time to get a reliable picture.

[maat]

At least for me this is surprising, because theorie claims that 100 samples should give "trustworthy" results for SD 

I believe you are referring to the sigma of the normal distribution, which goes with 1/sqrt(n). This is only true for truly random events, that have a white noise floor. Most quantities one measures are well above the noise floor and far away from that pesky 1/f part and therefore tend to be more Gaussian in shape, hence a bit of averaging helps. As soon as you hit that 1/f part in the noise spectrum, no averaging will help you there and the standard deviation no longer scales with  1/sqrt(n).

To sum it up, you can always use the RMS value + the bandwidth to compare results. A better picture can be seen using a noise spectral density plot or in this case the Allan Deviation as it a measure for the deviation of successive samples.

[Dr. Frank]

We want to compare the ACRMS/SD of shorts for different DMMs at given PLC and range.
What would be a good method to do this?
For me it seems that this method gives large spread and for that reason even comparing meter to itself seems not appropriate.

Who is 'WE'? Plural Majestic? 

RMS noise and StD share basically the same formula, so under some precautions, like observation of the different noise sources, the StD gives a good estimate for the noise figures, as I have demonstrated with my diagram for apertures ranging from 1.4µs to NPLC of 1000.
I can't observe, that there is a large spread, compared to the hp specification, and also within the whole graph.
Please also take notice, that such measurements are always plotted on a logarithmic scale, so small variations do not play a role.

And again, for noise measurements the Allan Deviation is the appropriate, broadly accepted, and maybe more 'exact' method of choice.
This is mostly used in time measurements, like stability of oscillators, but can as well be applied to any other stability / noise measurements.

The freely available programs PLOTTER by Ulrich Bangert, or STABLE32 both have all necessary tools on board.

http://www.ulrich-bangert.de/html/downloads.html
https://ieee-uffc.org/frequency-control/frequency-control-software/stable32/

John Miles TimeLab also contains Allan Deviation, together with acquisition software for many T.I. and frequency counters, but unfortunately not for voltage instruments. But maybe there you can find further information about stability / noise measurements.

http://www.ke5fx.com/timelab/readme.htm

Therefore you have to write your acquisition program for your 3458A on your own.
Frank