Bench DMM Noise & Stability Comparisons with TestController

[KungFuJosh]

A more systematic way is to use the Allan deviation instead. Instead of just 1 PP or RMS value one gets an uncertainy for different time differences. It assumes averaging where possible. So using the filter does not change much at the Allan deviation curve (more cut away the the very fast part).
I consider it somewhat similar to looking at different frequencies with a FFT, just in the time domain with different filters.

For a good curve one tends to still need a relatively long tests. Knowing how the curves for the Keithley meters usually look, one would want the curve up to some 20 minutes and to get a reasonable stable curve at least 10 x that time for the data set. So someting like 3-4 hours for a test.

The simple time domain data make some sense if one has multiple meters at the same time to compare.

How do I setup and calculate Allan deviation? I've never tried that before.

Thanks,
Josh

[iMo]

If ADEV/MADEV has not been included into the TestController application (I do not use the TC yet) you may use TimeLab for example, it reads the voltage data off a .csv file, for example..

https://en.wikipedia.org/wiki/Allan_variance

[Salient666]

Here is my 9+ hours test with the following settings:


Keysight: NPLC100, AutoZero OFF, No filter
Keithley: NPLC10, AutoZero OFF, Averaging filter with 10 readings (to match the NPLC=100 as closesely as possible)

Meaured device: CircuitValley ADR1399 10V output with months and months of burn-in time and almost a week 'warm-up' time

My conclusions:
- The meters have inverted temperature coefficients
- The Keithley is generally less sensive to temperature changes
- The Keysight still has less overall noise

[iMo]

You cannot compare the meters that way, imho.
With shorter and longer "integration times" (the "tau" in the ADEV) the STD goes up (always an "U" shaped graph).

To compare "noise" somehow you have to calc the STD over a limited time (the tau), like those tau=30-40secs I mentioned above. When you start to use the ADEV you will see the minimum in the ADEV graph may be found around that tau for most (perhaps all) DMMs equipped with the zener references.

Looking at your STD calculated over 9hours of samples is like to look at the ADEV at the very right hand side where the uncertainty is gigantic (look at the ADEV graphs above) and any environmental factors may affect its level (temperature fluctuations for example).

Imho, your numbers (like those stds - 12uV and 25uV etc) shown in the above tables say very little about the meters (or may be even misleading), I am afraid.

So your TestController shall implement the std calculation over that "tau=XX secs" (optionally to make an average of many such STDs, or a graph with rolling STD over that tau), or/and it shall include the ADEV/MADEV graph where you may see the noise levels (stds) for various noise types for all the taus (like from the tau=1/samplingerate to the tau=total_measurement_length)..

PS: added the ADEV expl.

[Kleinstein]

The std dev makes less sense if there is noise from different frequencies and drift.
Calculating a std for a limited time window (e.g. the last 100 points) would make sense. For the final representation the Adev makes more sense, but this need quite some computation and long data sets (like 50000 points). So it is nothing to do in real time like with testcontroller. If really wanted there a Python libraries around to do the math (Allantools).

The ADEV curve can be seen as doing averaging over different times and than calculate the sdev over very short sets from that and average that STD over the set.

Another way to look at the data is a FFT. So separate the noise in frequency. The ADEV curve is similar to a smoothed FFT with the x axis changes to 1 over frequency.

[iMo]

Yep, the easiest way is to calc a "rolling/sliding window STD" over last N samples (keep a buffer with last N samples), where the N shall be set such it fits into the tau, say, 30secs. You will get the STD and you will also see how the STD evolves over time or how it depends on other stuff (my favourite tool).
ADEV - there are libs in java for that, sure..

PS: for example with my 34401A at 10PLC (0.404sec sampling interval) the N=75 for tau=30sec (75x0.404=30secs). So keep last 75 samples in the buffer and calc the STD over the last 75 samples after each new sample enters into the buffer.
With 100PLC the N=8 for 30secs (8x4.04=32)..

Thus you have to agree on the time tau (I vote for 30secs ), not the number of N, as the N varies with the sampling_interval (=1/sampling_rate). And always use the 30secs for comparison.

[HKJ]

Calculating a std for a limited time window (e.g. the last 100 points) would make sense. For the final representation the Adev makes more sense, but this need quite some computation and long data sets (like 50000 points). So it is nothing to do in real time like with testcontroller. If really wanted there a Python libraries around to do the math (Allantools).

Calculating a sliding window with std dev is a standard math function for TC and can be done on the fly.

[KungFuJosh]

You cannot compare the meters that way, imho.

Read the title of the thread: Noise & Stability. Yes, you can calculate that exactly as he did. He specified that one was more temp stable while also being the more noisy of the two.

No, it's not the only way to compare, and that means more fun.

[Salient666]

Calculating a std for a limited time window (e.g. the last 100 points) would make sense. For the final representation the Adev makes more sense, but this need quite some computation and long data sets (like 50000 points). So it is nothing to do in real time like with testcontroller. If really wanted there a Python libraries around to do the math (Allantools).

Calculating a sliding window with std dev is a standard math function for TC and can be done on the fly.

I haven't found it yet But I guess I will figure it out.

In the meantime I have a new DMM to put through its paces. I got fed up looking at my broken Siglent SDM3065X so I got myself an SDM4065A to replace that one.

[HKJ]

I haven't found it yet But I guess I will figure it out.

It is on of the functions for the Math channels: StdDevRunning

[KungFuJosh]

There are a lot of great math features built in. Go to the Math tab, click add, name it if you want, and select a formula, or add your own.

Hopefully, soon to include Allan deviation too.

[KungFuJosh]

Here's an example with the math channel added to the popups and chart.

ETA: DAQ @ 10NPC with Hybrid Filter set to 10.

[Salient666]

I am still running into the problem that whenever I start up TestController, the DMM is defaulting to D5 instead of SI7.. I am using the latest version and checked the textfile, it is set to SI7.
Is there some kind of cache in Windows that I have to clear/take care off?

[KungFuJosh]

I am still running into the problem that whenever I start up TestController, the DMM is defaulting to D5 instead of SI7.. I am using the latest version and checked the textfile, it is set to SI7.
Is there some kind of cache in Windows that I have to clear/take care off?

Make sure you don't have any old definitions. But also, make sure you're definitions are being loaded from where you think they are.

For example, you might think it's a folder on your desktop, but actually in your documents folder or C root.

ETA: Edit the TestController batch file with the location you want it to be.

Code: [Select]

start javaw  -jar TestController.jar dataDir=C:\TCData %1 %2 %3 %4 %5 %6 %7 %8 %9


[Salient666]

I am still running into the problem that whenever I start up TestController, the DMM is defaulting to D5 instead of SI7.. I am using the latest version and checked the textfile, it is set to SI7.
Is there some kind of cache in Windows that I have to clear/take care off?

Make sure you don't have any old definitions. But also, make sure you're definitions are being loaded from where you think they are.

For example, you might think it's a folder on your desktop, but actually in your documents folder or C root.

ETA: Edit the TestController batch file with the location you want it to be.

I guess there were old files in the DEVICES folder. It had 247 files while the latest testcontroller has 240. I deleted everything and copied the files from the new testcontroller realase aaaaaaand.. it works.
Now I have to figure out the rolling std deviation math function on how to implement that.

[KungFuJosh]

I guess there were old files in the DEVICES folder. It had 247 files while the latest testcontroller has 240. I deleted everything and copied the files from the new testcontroller realase aaaaaaand.. it works.
Now I have to figure out the rolling std deviation math function on how to implement that.

That's what I figured.

For the math, the two things I left out are:
1. Set the device as part of the formula (see attached)
2. Set SI for the math function on the current values tab


Thanks,
Josh

[Salient666]

I figured it out, but I added: ,alan(v). This probably does not do anything then??
These are based on 30 samples, hence the 30 in the name of the math function.

[HKJ]

Also note that this running StdDev can be added to saved files, it is not necessary to do a new test.

[iMo]

Note: the sliding STD has to be calculated upon tau=XX seconds (you have to agree on the XX, it may be between 30-60secs, ask the experts in the Metrology section on the best number). That is because the minimum of STD for most DMMs (and voltage references) is at that XX seconds. Thus good for comparison.

The number of samples in those tau=XX seconds may vary because of the actual DMM's sampling rate (like with 10PLC it may be 100 samples for 40secs, with 100PLC 10 samples for 40 seconds for a specific DMM).

The average of many sliding STDs (each made over the tau=XX secs) is similar to the STD you may see in the ADEV at tau=XX seconds.
The ADEV makes in principle the same, for all applicable taus (based on the total measurement length and sampling rate).

The ADEV averages that STD measurements for a specific tau, the longer the total measurement the smaller the errors.
In the ADEV graph above you may see the error bars (vertical bars) indicating the error spread of the calculated STD values.

The longer the tau the larger the STD's error/spread as you have averaged a smaller number of STDs with that longer taus within you total measurement duration.

I think the error goes similar to the FTT like 1/sqrt(N) where the N is the number of averaged STDs (doublecheck this plz!).

For example in order to measure the stability at tau=1hour with a 10% STD error you have to provide a 100-hour long continual measurement.. 

[TUMEMBER]

Is this what this is about?

[iMo]

Yes.
There is a lot of variants of that ADEV.. graphs. [
The ADEV with its U-shape applies for any measurement of any physical quantity.

Those ADEV.. graphs provide not only the STDs (=sigma(tau)) but also the shape of the graph is important - the shape indicates the type of the noise(s) prevailing at the specific regions of taus..
Some bulging on typical lines or unusual slopes of the lines may provide indication of some issues as well (strong popcorn, jumping/telegraphing, oscillating, EMI, EMF, etc). Rocket science otherwise..

http://www.stable32.com/Techniques.pdf

PS: MADEV (ModifiedADEV) can better distinguish between white and flicker (1/f) noise at lower taus - see below.

W - white
F - flicker (aka pink aka 1/f)
RW - random walk

[Salient666]

In the meantime I've let the test run overnight and during the day when at work, so here is a 20 hour graph of all three meters.
All are at NPLC=10 and AutoZero=ON, MovingStdDeviation=30samples (which is 6 seconds, right?)

[HKJ]

In the meantime I've let the test run overnight and during the day when at work, so here is a 20 hour graph of all three meters.
All are at NPLC=10 and AutoZero=ON, MovingStdDeviation=30samples (which is 6 seconds, right?)

You generally have to be a bit careful with the charts on GridPanel when doing long runs. They may not collect all the, but only the last x minutes of data (The documentation says how much they keep).
You hit the limit on both Min/max chart and histogram charts.
The regular chart handles long run by only storing 1 in X samples, at the start X is one, but it can increase to a arbitrary large value.

[Kleinstein]

10 PLC with AZ on, with most meters mean 2 conversion of 10 PLC and thus a little (some 1%) less than 2.5 or 3 conversions per second.
so 30 samples would be some 10 or 12 seconds depending on the mains frequency (60 / 50 Hz).

How fast the meters read at 10 PLC depends. E.g. the old Keithley19x tend to use 3 conversions in AZ / MUX mode. Also meters with an SD ADC may take longer or less, depending on how the AZ mode is implements.

[iMo]

 theYep, so you have to set the sampling_period for the calculation of the number of samples in the sliding STD (for the optimal tau=30-40secs), and/or for the ADEV calc properly.
Sampling_period of ie. 10PLC may have ie. 4 values as Kleinstein wrote above.

A note on the tau (why the tau in seconds is important when talking STD) and other ADEV stuff for younger makers here:

Example:
I have my HP3458C set properly, I have my ideal DUT say 10V reference based on an LTZ3000.
I set the 10PLC and I knew the sampling_period (ie. for 50Hz mains, AZ ON) and I set it properly for ADEV calc, I ran the measurement over night, I got many thousands of samples with nice values around those ideal 10V.

Off the TestController I got a nice ADEV graph.

Now, I want to know what the f...ing ADEV and those taus are about and how are they related to the "stability" of my meter.

Let say I want to make (anytime) a first voltage reading with my setup "now", and the second one in a) 3 seconds after, or in b) 40secs after, or in c) 1hour after the first one reading.
Thus I want to know how is the stability of my DMM between two readings which are some time apart.

So I look into the ADEV graph and I see for example:

a) the sigma(3) (std at tau=3secs) is 400nV
b) the sigma(40) (std at tau=40secs) is 150nV
c) the sigma(3600) (std at tau=3600secs) is 800nV.

How to interpret that (with a little bit simplification, ie. neglecting the ADEV error bars)?

a) it says that ".. 68.3% of any two readings made 3 secs apart will fall into an interval 10V+/-400nV.."
b) it says that ".. 68.3% of any two readings made 40 secs apart will fall into an interval 10V+/-150nV.."
c) it says that ".. 68.3% of any two readings made 3600 secs apart will fall into an interval 10V+/-800nV.."

Why 68.3% there ?? Because the +/-1 sigma covers 68.3% under normal distribution curve (aka STD in Vrms).

I want to have it in peak-peak voltage!!

Then (ie. gaussian noise):

a) it says that ".. 99.9% of any two readings made 3 secs apart will fall into an interval 10V+/- (3.29*400nV).."
b) it says that ".. 99.9% of any two readings made 40 secs apart will fall into an interval 10V+/- (3.29*150nV).."
c) it says that ".. 99.9% of any two readings made 3600 secs apart will fall into an interval 10V+/- (3.29*800nV).."

Why 3.29 there ?? Because the +/-3.29 sigma covers 99.9% under normal distribution curve.

And what about the "noise"?? Where is the lowest noise in the above ADEV result??

The lowest noise is at tau=40secs, because the 68.3% of any two readings made 40secs apart fall into an interval with the lowest spread (only +/-150nV). So the lowest noise with the STD=150nVrms is at the tau=40secs.
And I want that noise in peak-peak Volts!
150nVrms*6.6=990nVpp for tau=40secs.