Ultra Precision Reference LTZ1000

[exe]

Looks like a very good result to me too, but I'm not a voltnut.

[niner_007]

Yes I know that This is not a serious measurement, just a sanity check. Three or more multimeters (DMM7510 and 3458A) sampling the reference is what I had in mind for the serious measurement, and proper cables and shielding.

The reference used here is an old and unmodified A9 board from an 3458A. DMM7510 is what I had conveniently located near the power supply.

That's a too elaborated approach, which will create new problems.
 
Parallel measurement with several DMMs is unnecessary, complicated, and will give big interference noise from one DMM to the other.
We had this failure recently, during our MM 2020 already, so do not repeat the same mistake.

Keep everything simple, i.e. one good DMM, make a clean setup, do the basic shielding, and then you're done.

Frank
https://www.eevblog.com/forum/metrology/volt-nut-meeting-2019-in-stuttgartgermany/msg3091382/#msg3091382

Not paralleled, I was thinking of switched across 3 or more DMMs with a scanner

[MegaVolt]

For comparison: I get 850 nV RMS for 1000 samples with NPLC = 5 for direct measurement of Fluke 732a vs DMM7510.
And just 300 nV RMS noise when measuring one Fluke 732a against another a range of 0.1 in the same conditions. And this is completely not caring about shielding using the most ordinary wires from a multimeter.

From this we can conclude that the direct measurement of a good reference voltage source directly against the DMM7510 does not make any sense. Because we only measure the noise of the multimeter

[Dr. Frank]

Am I the only one that doesn't see any problem?

   niner_007  ->  ΔY = 1.73uVpp  (unkown interval)
   Dr. Frank  ->  ΔY = 1.33uVpp  (** 1 hour interval)

   (** assuming 7V absolute value, and +/- 0.95ppm Δ)

So results doesn't seem too bad considering power cables, and board and DMM placement.

Hi folks,
That discussion is quite interesting now. 

The LTZ is specified as 1.2µV_pp noise, which is mathematically equivalent to 420nV_rms.
The measurement from niner_007 shows 2µV_pp or 700nv_rms, if you look more carefully at the cursor position, which is definitely too high, and indicates additional noise.

Typical LTZs compared against a 3458A, or also against a 7510 (which has an LTFLU reference inside!?) only show 150..300nV_rms.
A 732A should measure even lower.
Please have a look into the 'DMM noise' measurement campaign, we did a few years ago. You will get a good estimate of the noise figures of the different DMMs.
Maybe the 7510 is more noisy for other reasons, (I don't like the limitation to NPLC 5), I do not remember that precisely any more:
https://www.eevblog.com/forum/metrology/dmm-adc-noise-comparison-testing-project/


These values depend on the settings, the equipment and the environment, and then also on the quality of the LTZ / DUT and the reference inside the DMM.
So to get comparable results:

1. Averaging time of the DMM: use NPLC 100, or 1.6 .. 2 sec averaging/sampling time for one data point
2. Always use statistics: Take 16 datapoints of NPLC100, note average and StD.
The StD is also equivalent to the rms noise value, on this time scale.
With more and longer averaging data, especially the StD, will be biased by timely or temperature drift effects, or from other noise sources like Popcorn noise.
3. You may distinguish the different noise or (in-) stability effects described above by using Allan Deviation statistics
4. Digitization error/noise: For sub-ppm measurements, at least 0.1ppm resolution @ 10V is required, but the 7510 seems to have 0.5µV only
5. Use a DMM with an equivalently noisy internal reference, like another LTZ1000, or an LTFLU. The real noise figure of the DUT is then about 1/SQRT(2) of the StD
6. Avoid or mitigate external disturbance, like E.M.C. and temperature change, which will both falsify / contribute to the StD value.
7. For precise determination of the absolute value of the DUT, use low e.m.f. cables, and reverse the leads at the DUT to cancel these.

On our MM2020 meeting, we discussed this data evaluation deeply.
Many measurements on the different LTZ1000 circuits (~22EA) showed 500nV_rms, or more StD. That was a clear indicator that something was wrong, and these measurements should have been discarded at once.
In the end we identified the root cause(s), so that we could reduce our disagreement of uncertainty between our reference groups from 2ppm to about 0.5ppm.

Scanning the DUT with different DMMs seems like a good idea, but the scanner itself will introduce a lot of e.m.f. errors / noise, or you invest $$$ in a nV scanner.
Notice again, that we're discussing sub-ppm absolute values also.
The other way round, i.e. scanning several LTZs with one DMM, also depends on the quality of the scanner.

Frank

[MiDi]

The LTZ is specified as 1.2µV_pp noise, which is mathematically equivalent to 420nV_rms.

Usually the rough conversion from peak to peak noise to rms is by dividing by 6.
That would give ~200nV_rms for LTZ1000.
Could you please explain the difference?

https://youtu.be/-KcODSYXiZA

[Dr. Frank]

The LTZ is specified as 1.2µV_pp noise, which is mathematically equivalent to 420nV_rms.

Usually the rough conversion from peak to peak noise to rms is by dividing by 6.
That would give ~200nV_rms for LTZ1000.
Could you please explain the difference?

https://youtu.be/-KcODSYXiZA

The conversion depends on the actual noise waveform, that's  missing in this video also.
If you have sinusoidal noise, instead of that Gaussian noise, then the rms value would be: U_pp/(2*SQRT(2)).
For simplicity, I take this as an upper estimate, if a peak-peak value is given.
Btw.: Zener noise is mostly not Gaussian, that's why you can't filter it well.

I prefer the  direct StD measurement over p-p, because that quantity is better defined, or better established in error/stability calculus. DMMs mostly calculate this value.

Also, the formula for the StD is nearly the same as the rms calculation for a random waveform, so giving an estimate for the rms noise for a short observation time.
This will only be the case, if the mean value, i.e. the DC component is zero. That's the reason, why longer observation times will give wrong StD values concerning noise, as drifts will give a non-zero average.

Coming from frequency stability calculation, the Allan Deviation calculus is the best and most accepted method to measure and separate the different noise types  / stability figures.
peak-peak values are also not used there. In the DMM noise project, we had successfully established the Allan Deviation also for voltage noise characterization.

PS: This problem with different types of noise vs. conversion factors of rms <=> pp is briefly mentioned here:
https://www.analog.com/media/en/training-seminars/tutorials/MT-048.pdf

Frank

[Kleinstein]

Measuring the RMS value usually gives less scattering values compared to the peak to peak numbers. For good peak to peak numbers it would take several test intervals (e.g. 100-1000 points each) and than take some mean.

There can still be some justification of the peak to peak value if one has a large contribution of popcorn (random telegraph) noise. In this case the simple approximate factor 6 does no longer apply.
For the actual use the worst case and thus the peak to peak value may be more relevant.
With more noise source (here at least the external reference and the meter internal reference) the noise gets more normal, so that one can better use RMS and the simple statistics.

[BU508A]

Typical LTZs compared against a 3458A, or also against a 7510 (which has an LTFLU reference inside!?) only show 150..300nV_rms.

Yes, the DMM7510 has a LTFLU inside.
https://www.eevblog.com/forum/metrology/ultra-precision-reference-ltz1000/msg2922618/#msg2922618

[niner_007]

Results seems to be about the same for me on the DMM7510, 100 sample averaging filter, and 15NPLC

[dr.diesel]

15NPLC

In regards to the lowest noise region of the 7510, from the user manual:

[niner_007]

Yeah, I mean to try that, I remember the discussion about the weird DMM7510 noise in the DMM noise thread, I have better meters, this one was the closest

[Grandchuck]

On our MM2020 meeting, we discussed this data evaluation deeply.
Many measurements on the different LTZ1000 circuits (~22EA) showed 500nV_rms, or more StD. That was a clear indicator that something was wrong, and these measurements should have been discarded at once.
In the end we identified the root cause(s), so that we could reduce our disagreement of uncertainty between our reference groups from 2ppm to about 0.5ppm.

Frank
[/quote]

What kinds of issues were identified as root causes?  I will guess that EMI is/was a factor.

[branadic]

What kinds of issues were identified as root causes?  I will guess that EMI is/was a factor.

Turned out to be a simple issue with one button in the wrong place. Guard button on K3458A was in "low" position and not in "open". Guard of the cable was connected to case of the reference and Guard jack of K3458A, but on GND on K2002. This created a Guard loop, which created the differences.
I was able to recreate this problem today. Having multiple meters in parallel can then influence the readings.

-branadic-

[SilverSolder]

A "Guard Loop" - there's a new one!

[Dr. Frank]

What kinds of issues were identified as root causes?  I will guess that EMI is/was a factor.

The 2nd definite effect were two 3458A in parallel. They were not triggered synchronously, therefore the charge injection of one would every time influence the readings of the other, and vice versa.

The StD value was extremely high, in the µV range, instead of 150..250nV, as usual.
Shows, that StD can be used as a sanity check, provided you have comparable or identical situation.
Frank

[SilverSolder]

What kinds of issues were identified as root causes?  I will guess that EMI is/was a factor.

The 2nd definite effect were two 3458A in parallel. They were not triggered synchronously, therefore the charge injection of one would every time influence the readings of the other, and vice versa.

The StD value was extremely high, in the µV range, instead of 150..250nV, as usual.
Shows, that StD can be used as a sanity check, provided you have comparable or identical situation.
Frank

Interesting.  I have tried 4DMM in parallel but they were all triggered in sync via GPIB, that might be why I didn't see any particular problems with that method.

[niner_007]

Over 1 hr of data, 2.0189uVpp, 394nV std

[e61_phil]

What kinds of issues were identified as root causes?  I will guess that EMI is/was a factor.

The 2nd definite effect were two 3458A in parallel. They were not triggered synchronously, therefore the charge injection of one would every time influence the readings of the other, and vice versa.

The StD value was extremely high, in the µV range, instead of 150..250nV, as usual.
Shows, that StD can be used as a sanity check, provided you have comparable or identical situation.
Frank

Std. Dev. of 1 µV would mean ~6µVpp. If you further assume 6nApp from the 3458A that would lead to an output impedance of 1k.

I think the GUARD loop is much more likely to explain the high std. dev.

[niner_007]

How do you synchronize multiple 3458A in parallel, that’s not possible is it? Why not scan the inputs with a scanner? What’s paralleling giving you? nV scanner are available, relays with low EMF are also available or can be constructed

[e61_phil]

How do you synchronize multiple 3458A in parallel, that’s not possible is it?

External Trigger, GPIB GET (That's what I'm using most of the time)

Why not scan the inputs with a scanner? What’s paralleling giving you? nV scanner are available, relays with low EMF are also available or can be constructed

Sometimes you want to compare if some disturbance was caused by the DMM or the source. And in these cases you want to integrate over the exact same point in time. INL comparisions are such an example.

Reasonable scanners are also not available to everyone.

[e61_phil]

The conversion depends on the actual noise waveform, that's  missing in this video also.
If you have sinusoidal noise, instead of that Gaussian noise, then the rms value would be: U_pp/(2*SQRT(2)).

Sorry, for beeing that late with my comment. But this is a fundamental topic and worth to discuss it deeper, I think.

In the video, they're talking about noise which creates a bell curve in the histogram and that is it what matters. The distribution is important not the signal over time. If you take all the values of your sinewave and mix the order of the values it will result in the same RMS value. Therefore, it doesn't make sense to talk about sinusoidal noise. It is more important how the shape of the histogram looks like.

The histogram of a sinewave might make sense as a worst case, because the highest abundance is at the maxima (makes absolute sense if have a look at the derivative), but that is a very unlikely shape for noise.


The 1/f (pink) noise means that there isn't only uncorrelated noise (white), but also some correlated noise. Which means the values you measure are dependend from the past (finite correlation length).

It is also wrong to say, that a gaussian histogram indicates white noise. The histogram is just the probaility density function and 1/f has also a bell shape.


@Frank: I'm absolutely sure that everything from that is known by you, I just want to make it more clear, that the signal over time is less important than the appearing values.


Edit: The standard formula for the standard deviation will give also the correct numbers, if you feed in a sinewave. There is no need to differentiate here.

[Kleinstein]

The 1/f (pink) noise means that there isn't only uncorrelated noise (white), but also some correlated noise. Which means the values you measure are dependend from the past (finite correlation length).

It is also wrong to say, that a gaussian histogram indicates white noise. The histogram is just the probaility density function and 1/f has also a bell shape.

With 1/f noise it depends on the details. One may have a bell shaped histogram, but the histogram can also look quite different (e.g. with dominant popcorn noise - which also gives an approximately 1/f spectrum). If the 1/f part is more like some random walk the measured histogram may also look different from the gauss bell curve. It is possible to a bell shaped histogram with 1/f noise, but this is not always the case.

The histogram and frequency spectrum are kind of complementary. So one would need to look at both - there is no way to convert one to the other.

[e61_phil]

I played with some simulated random walks. To evaluate "how wrong" 6x the standard deviation compared to the "real" peak to peak value is, I created 1000 random walks. Every walk was a million steps long. I calculated the standard deviation and the peak to peak value for every walk and compared them.

The mean ratio of 6x std. dev / pp was 1.38 (+/- 0.19). Which means you will almost ever overestimate the peak to peak value by taking 6x the standard deviation in the case of a random walk.

Btw.: The standard error of the standard deviation is S/sqrt(2(n-1)) which means you will already have 0.18 sigma (relative to a standard deviation of 1) if you only take 16 measurments with your 3458A.

[dietert1]

What happens to the stdev/pp ratio for real (shorter) measurement series?

For example in my current observation of two LTFLU 10 V references i see a stdev = 0,029 ppm of the daily averages, so times 6 gives an expected pp of 0,174 ppm. After 38 days i have an observed pp of 0,11 ppm (minimum -0,065 ppm, maximum 0,046 ppm). So that gives a ratio of about 1.6 (overestimation of pp by 6x formula). In this case the factor seems to be more like 4 instead of 6.

Maybe this case is not a random walk, but something else. Maybe air pressure changes.

Regards, Dieter

[Kleinstein]

Popcorn noise can give a rather low ratio for peak to peak values relative to RMS value.
Some periodic background (e.g. day / night temperature or pressure variations could also give a small ratio).