DMM Noise comparison testing project

[Kleinstein]

With a working AZ mode, the noise curve should go down even for long times even beyond the 300 seconds. There is not need to choose an integration time near the minimum of the curve - one can always do later averaging on the data. Most modern DMMs get the longer (e.g. > 100 PLC) integration times from averaging anyway.
The use of faster sampling is more question of data rate and memory / file size - not a big problem anymore. So even with longer time logging one can use quite a fast sampling and do filtering / averaging as needed later. Also there often is no choice of time scale - it is set by the experiment.

Also keep in mind that with real, non zero readings, there will be additional noise from the reference. This is not captured in the thread so far - but it can be quite important at a slower time scale. Most refs. show quite some flicker noise, so the curve including the reference will go up after some time. So for comparison one could include the noise of typical refs (e.g. LM399 and LTZ1000). Looking at the zero point noise in the 10 V range is a little odd point to test, as for a real signal near zero one would often use a smaller range.

[maxwell3e10]

Looking at the zero point noise in the 10 V range is a little odd point to test, as for a real signal near zero one would often use a smaller range.

This is generally true, but we are talking about 7.5-8.5 digit meters, where the whole point is to resolve a small fractional signal. Otherwise for small signals near zero one can always just use a pre-amplifier. For 10V range, 10^(-7.5)=0.03 ppm=300 uV, which is exactly where DMM7510 is starting to behave funny. So one can say it just barely satisfies its specs of giving a result to 7.5 digits. Actually it has rms noise specifications in the datasheet and the measurements that have been posted here are right on the edge, some are results are OK but some are slightly worse than the datasheet specs.

[VintageNut]

Looking at the zero point noise in the 10 V range is a little odd point to test, as for a real signal near zero one would often use a smaller range.

This is generally true, but we are talking about 7.5-8.5 digit meters, where the whole point is to resolve a small fractional signal. Otherwise for small signals near zero one can always just use a pre-amplifier. For 10V range, 10^(-7.5)=0.03 ppm=300 uV, which is exactly where DMM7510 is starting to behave funny. So one can say it just barely satisfies its specs of giving a result to 7.5 digits. Actually it has rms noise specifications in the datasheet and the measurements that have been posted here are right on the edge, some are results are OK but some are slightly worse than the datasheet specs.

I think you mean to say 300nV.

[Kleinstein]

Using the 10 V range near zero is a little misleading, as one has to expect a higher noise if used at something like 5 V or 10 V. This may not be so much with the DMM7510 but I would expect quite some extra noise for the 34465 or Keithley 2010 with it's LM399 reference. Measuring near zero just excludes most of the reference noise - especially the 1/f part. This is one of the 3 main noise sources - in the low frequency or longer time range very well possibly the dominating one of one is not near zero. In the Alan variance plot this would give a slope of sqrt(time) up from somewhere around 1 s or so, scaled with the faction of the full scale.

[3roomlab]

With a working AZ mode, the noise curve should go down even for long times even beyond the 300 seconds. There is not need to choose an integration time near the minimum of the curve - one can always do later averaging on the data. Most modern DMMs get the longer (e.g. > 100 PLC) integration times from averaging anyway.
The use of faster sampling is more question of data rate and memory / file size - not a big problem anymore. So even with longer time logging one can use quite a fast sampling and do filtering / averaging as needed later. Also there often is no choice of time scale - it is set by the experiment.

Also keep in mind that with real, non zero readings, there will be additional noise from the reference. This is not captured in the thread so far - but it can be quite important at a slower time scale. Most refs. show quite some flicker noise, so the curve including the reference will go up after some time. So for comparison one could include the noise of typical refs (e.g. LM399 and LTZ1000). Looking at the zero point noise in the 10 V range is a little odd point to test, as for a real signal near zero one would often use a smaller range.

based on your info, i am starting to understand more about integration time and noise. i went to find the longest log i have, and TADA ! well at least i know that is not how to use "point (c)"
but i did find an alternative curve which may fit a point (c) in an alternate way.
by changing the confidence level from 1 sigma to 3 sigma and using flicker FM boundary. the range where noise is included or precluded can be seen, and there is a low position where a "point (c)" could be found on top of the upper bound tips. by longer integration, it captures more and more LF noise which is what is going on as the boundary expands at the tail (esp for noisy DMM).

plot 2015Nov02_2357_21DD.gif is 0.1v, 1NPLC (sigma = 3). 30k samples
plot 2015Nov01.gif is 10v 10 NPLC (sigma = 3), 20k samples
plot 2015Nov01b.gif is 10v 10 NPLC (sigma = 2), 20k samples
for ease , we assume all samples to be taken at 1sample per second.

these are plots i think way before any mods, so the DMM is quite noisy. in the 3rd pic using sigma = 2, it seems by using an even longer integration beyond 1000seconds, the 100nV resolution can become "statistically" useful. if i use sigma = 3, between 1000-2000s of integration barely makes 100nV of some use (which is more than total effective NPLC of over 10,000. i guess this is why this DMM was sent to the dumpster?). in the 100mV scale, it only needs 32seconds to make 100nV useful (sigma = 3, which is effective total NPLC of only 32). so if i am using a noisy DMM like this in 100mV for long term logging, i will assume it be best to integrate to 32seconds to obtain usable (and repeatable) data down to 100nV in order to exclude the most amount of noise.

would my intention of using allan variation in this way be applicable? it makes sense i would think?
so my guess for 3458a to resolve 10nV, a suitable allan plot can be taken to find the optimal integration time too?
it will also be interesting to turn on all the noisy appliances in the house and do an allan plot vs 1 that has nothing on.

this link talks about a diff allan variation program, but it has more details about the different noise types and the integration slope leading to lowest noise
http://www.stable32.com/paper2ht.htm which gave me the idea of the lowest point thingy

**edit, but looking at Tin's 7v log "7v_3458_nplc200_tin_goodA3.csv". we see that the noise is now an increasing slope. by using allans variance, we can see that in order to resolve 1uV, we should sample below 16seconds. does this make metrological sense?
same in the case of the 10k resistor, "time_10k_dmm_3458_nplc100_tin.csv", to resolve to 0.01ohm, we need to sample within 64s, in the case of the resistor, there is a point (c) @ 2~4s.

[Andreas]

**edit, but looking at Tin's 7v log "7v_3458_nplc200_tin_goodA3.csv". we see that the noise is now an increasing slope. by using allans variance, we can see that in order to resolve 1uV, we should sample below 16seconds. does this make metrological sense?

Hello,

I guess there is something wrong with the measurement setup (or the alan deviation calculation)
I know that Frank has usually below 200nV standard deviation when measuring a LTZ1000 with his HP3458A at 100NPLC and AZERO on (>=4 seconds)

see also here: (where the plot starts at 130 nV.)
https://www.eevblog.com/forum/projects/project-kx-diy-calibrator-reference-sourcemeter/msg592170/#msg592170

And yes: it makes no sense with a 3458A to average more than around 10 measurements.
(the X-scale is in measurements so a 1 corresponds to 5 seconds with triggered measurement).

Your LTZ/3458A plot begins with 400-500nV which is far too high.

with best regards

Andreas

[3roomlab]

i dont know why its high, im too new to understand it esp measureing of VREFs. but what do you think of application of allan variation to measurement noise? this isnt something applied to volt metrology widely, only popular in time metrology. with my limited knowledge it do seem like it can be useful, a systematic way to determine limit of sampling to what resolution? since the estimated/statistical limit of where the noise stops can be presented/plotted. or in another way of looking at it is the plot of boundary of certainty vs uncertainty

[maxwell3e10]

Before calculating the Allan variance its useful to look at the time trace of the data. If the trace looks flat and just has  uniform noise, the Allan variance will look good, go down as 1/sqrt(time). If there is any drift in the trace or any spikes, level shifts, etc, the Allan variance will not look good, either flat or going up in time. One can choose to remove spikes or fit out the drift, depending on the purpose of the analysis.

Regarding testing with finite voltage, it would be good to come up with a standard setup that is not too specialized, so many people can use it. Perhaps use a big low-pass filter, say 1 MOhm and a large capacitor, to remove some short-term drift in whatever reference one has. Does anyone know if electrolytic capacitors would cause too much drift in this filter application?

[Dr. Frank]

**edit, but looking at Tin's 7v log "7v_3458_nplc200_tin_goodA3.csv". we see that the noise is now an increasing slope. by using allans variance, we can see that in order to resolve 1uV, we should sample below 16seconds. does this make metrological sense?

Hello,

I guess there is something wrong with the measurement setup (or the alan deviation calculation)
I know that Frank has usually below 200nV standard deviation when measuring a LTZ1000 with his HP3458A at 100NPLC and AZERO on (>=4 seconds)


And yes: it makes no sense with a 3458A to average more than around 10 measurements.
(the X-scale is in measurements so a 1 corresponds to 5 seconds with triggered measurement).

Your LTZ/3458A plot begins with 400-500nV which is far too high.

with best regards

Andreas

By using the LTZ design of Andreas, I  measure about 0,047ppm noise, or about 330nV, see here:
This includes medium term fluctuations (hours time scale), short term (10min) noise may be around 200nV.



A high precision VDC measurement should be made over about two minutes averaging time, that's a recommendation from NIST / PTB, I think.
Therefore , averaging over 25 samples of NPLC 100 is fully sufficient.
By using built in statistics, you will get mean and STD values; here I achieve between 100nV..500nV  noise for 7.15V or 10V, typically around 200nV.
Longer averaging may only make the STD value more stable, but that's not useful.
Also, one might already measure drift effect, instead of noise.

Instead, such long termed measurements with equidistant measurements, analysed by the Allan Distribution, reveals the different stability effects, as you can already identify in the diagram.
That is the initial drift of the 3458A for the first 2h (0.5°C change => ~0.2 ppm drift), then the short term noise of < 0.05ppm, and medium term fluctuations, over 10minutes.

Frank

[TiN]

7v_3458_nplc200_tin_goodA3.csv

That's very very old data, just to check A3 stability back there.

Suggest to use recent 7V LTZ test. Attached log over 10hours with HP,pair of 2001s, pair of 2002's. Ambient temp within 1c overall. Should have more accurate result.

[Kleinstein]

The changed shape of the variance curve when measuring an external reference come from the extra noise from both the external ref to be measured and the DMM internal ref. This extra noise is relatively high compared to the other meter internal noise (at least for the longer times).
This noise can also depend on the environmental conditions as some of this can be temperature effects. Not all of the refs of the same type are the same with respect to noise.

A relatively easy to reproduce finite ref voltage might be a pack of NiCd cells (e.g. 6 to 8 cells to get 7.4 to 9.8 V. They are supposed to be very low in noise, if at a stable temperature and not moved around. It might need a certain treatment to get a really stable state, like charge fully, discharge by 10-20% over a day or so (load with a suitable resistor) and then let them sit for a few days. The voltage will not be absolutely the same and one might have to check for drift, but chances are good to get a quite stable voltage for a few hours.

A resistor and cap for filtering is tricky. This can work for higher frequency noise (e.g. higher than 0.1 Hz) but it can not work for longer times. The electrolytic cap is also likely as sensitive (or worse) to environmental changes as a battery. So one would need similar care and still only has a limited capacity and frequency range. For the reference noise, I would be more interested in the 1-100 mHz range - just where a capacitor is not very practical any more.

The 2010 meter is a little special in the way AZ is done. The AZ mode does not include the input buffer, as the is chopper stabilized. The plateau if the curve for the 10 V range suggest there is some kind of averaging used in the AZ mode of many Keithley instruments - not just the DMM7510. For the 100 mV range it is not visible any more because the input amplifier noise dominates.

[maxwell3e10]

A relatively easy to reproduce finite ref voltage might be a pack of NiCd cells (e.g. 6 to 8 cells to get 7.4 to 9.8 V.

NiCd? Mercury cells are known to be stable, and I still have a few of them, but they are impossible to get now.

[Andreas]

Instead, such long termed measurements with equidistant measurements, analysed by the Allan Distribution, reveals the different stability effects, as you can already identify in the diagram.

Frank

Hello Frank,

Do you have the Allan diagram for this measurement, or can you provide the raw data?

With best regards

Andreas

[Andreas]

Suggest to use recent 7V LTZ test.

Hello Illya,

thats much better.
now it starts below 200nV standard deviation.

The time scale seems to be unstable.
The time stamps have 6-10 seconds difference between the steps.
I think for allan deviation you should use equal time stamps.

with best regards

Andreas

[Dr. Frank]

Instead, such long termed measurements with equidistant measurements, analysed by the Allan Distribution, reveals the different stability effects, as you can already identify in the diagram.

Frank

Hello Frank,

Do you have the Allan diagram for this measurement, or can you provide the raw data?

With best regards

Andreas

Yep. Latter one. Dammit. xlsx not allowed. xls too big. therefore csv data. Sorry.No csv format, therefore txt. Rename it, please.
Frank

[TiN]

The time scale seems to be unstable.
The time stamps have 6-10 seconds difference between the steps.
I think for allan deviation you should use equal time stamps.

First I was puzzled by this comment, as code just takes samples as fast as they arrive, talking to each meter in sequence, but then things got cleared out, as there is ACAL correction function, to trigger ACAL DCV (which takes 2 minutes, at while code waits) every 0.2c of temperature change. Sorry for that.

[3roomlab]

does timing inconsistency add to flicker noise? or its a math thing? (the comparison @ 1024 mark)
what else could be possible to add to Tin's incremental flicker noise? 
i was hoping to plot in opencalc the way andreas presented his plot, but i couldnt understand the equation well :/

[Kleinstein]

The uneven time data should not add extra noise. it is more like smearing out the time axis a little. As the curve usually is not that step (usually  sqrt(1/tau) to sqrt(tau)) this does not introduce a large error.

The ACAL run from time to time might add some extra error though, as this also causes small jumps in the curve. In a certain way this is part of the instrument noise - just not fixed, but under user control.

[maxwell3e10]

Based on this information http://www.basytec.de/Literatur/temperature/DE_2002.htm
it looks like NiCd batteries would have a sweet spot somewhere on the discharge curve. But Pb batteries should also work, they have small temperature dependence and one can get them in much higher capacity and thermal mass, which should  help.

[maxwell3e10]

Here is a comparison of the finite voltage measurements. Here Keithley 2002 perform pretty well, they start at higher noise, but then average down to maybe even slightly better performance than HP3458. I also tried to look at differences between 3 simultaneous meters in TiN data, but there is no improvement, so I think the meters have the dominant noise contribution.

[Andreas]

Yep. Latter one. Dammit. xlsx not allowed. xls too big. therefore csv data. Sorry.No csv format, therefore txt. Rename it, please.
Frank

Hello Frank,

.txt is perfect (as the plotter program of Ulrich Bangert needs .txt).

here the diagram.
There are some glitches in the .txt with 0 V and -273 deg C which I have removed.
Also the drift of the first 2 hours is removed. So the diagram is the last 7 hours.

First: diagram over time.
2nd: histogram showing a near perfect gaussian distribution
3rd: Allan deviation starting at 136 nV standard deviation for short time stability.
(near the theoretical limit of 100nV of the instrument alone).
Above around 10 averaged measurements stability is determined by drift.
X-Axis shows number of measurements (4-5 seconds each for 100 NPLC).

So for me the main usage of the Allan diagram in volt-nut mode is to determine
a useful integration time (averaged measurements) for lowest noise.

with best regards

Andreas

[3roomlab]

would my assumption be correct that this integration time is around 25s ? base on the the first slight dip at tau =5?
then what about the huge dip going on at tau =700, do you know what to make of that? this is all so new 

[TiN]

Got data for last test from fairy module before shipping to the voltnut. 8 hours with stable ambient w/o ACALs, then drop temperature/ACAL
Two meters this time only, 3458 (black line) and 2002 (purple). Keithley gets some credit, as after ~5C ambient temperature change reading does not change much. Meters are sampling different KX LTZ modules.

[Dr. Frank]

would my assumption be correct that this integration time is around 25s ? base on the the first slight dip at tau =5?
then what about the huge dip going on at tau =700, do you know what to make of that? this is all so new 

The sampling spacing is about 4sec, as NPLC 100 in our 50Hz grid equals 2sec measurement plus 2s AZ time.

Therefore, about 20s averaging gives best noise figure for the 3458A.
That characteristic can be observed in many other setups also, like NPLC 10, or so.

That artefact at tau=700 will be related to that initial drift on the first 2h, when the 3458A inside the room had at first to stabilize.

I'd expect that if you skip the first two hours, you'd see a smooth curve.

Anyhow, for these long time constants, the Allan Deviation gets more and more sensitive to disturbances and drifts. Or in other words, these high tau values are based on too few data points.
For a more stable graph, it would be necessary to measure over a much longer period of time, let's say about 10 times longer at least as the tau value to be observed.

Frank

[Andreas]

That artefact at tau=700 will be related to that initial drift on the first 2h, when the 3458A inside the room had at first to stabilize.

Hello Frank,

not in this case, as already mentioned above I have removed the first 2 hours from the diagram.
I think more the 700 tau artefact is something generated by the unusual stable environment conditions in your lab.
That is something that you cannot generally expect from a setup.
I never regard the Allan deviation above the first minimum.

So with the first 2 hours we have the following picture:
(again removed the two outliers with 0 V).

1. slighgly drift over time
2. distribution shows a inclined distribution due to the drift
3. allan deviation again starts at 0.136uV going up to higher levels due to drift.
    (the artefact at 700 tau remains).

with best regards

Andreas