DMM Noise comparison testing project

[TiN]

That is a question there, how to show diffent X scales for different settings? If use same scale, graphs end up thin line for base ranges and slow NPLC and spread all over for fast NPLC and high voltage ranges.
Group by only by NPLC does not solve it either. Need a better way. Matrix of graphs with own scales on each range perhaps?

[Dr. Frank]

well I think, as this should be comparative, the x-scale should always be the same, to show the width  of the distribution.
Naturally, the NPLC determines greatly the noise figure, therefore, this parameter should be indicated always, and also in a comparative manner, i.e. same NPLC at the same graphical position.

Btw.: I've finished the 3458A measurements, no surprise, as specified, and obviously the best performer here, currently.

Frank

[ManateeMafia]

The script should issue one "READ" to initiate measurements, then go into the big loop doing "DATA?" repeatedly.

I will test this out on the 8846A tonight. I have found an issue with the 8846A script, it seems that I am calling some commands too often and it is hanging on the second 100NPLC acquisition.

Anybody who has run this script has probably seen it already. I should have tested this more thoroughly.

TiN,

Maybe all the test scripts can be listed on the first page to keep everything in one place? I can send them to you after some more testing with macboy's recommendations.

[jlmoon]

Will make every attempt to gather data from my 3458.. as of yet have not had a reason to use the NI / 488 interface in my computer.  Hope I am able to climb up the steep learning curve on that one. 

JLM

[Dr. Frank]

Hi,

I did 12 measurements, NPLC 0.1 / 1 / 10 / 100 for DC 0.1V / 1V / 10V.

The standard deviation for each measurement equals the RMS noise.

The attached diagram clearly summarizes the HP3458A noise specification and the real measurements.

(csv files uploaded)

Frank

[macboy]

Hi,

I did 12 measurements, NPLC 0.1 / 1 / 10 / 100 for DC 0.1V / 1V / 10V.

The standard deviation for each measurement equals the RMS noise.

The attached diagram clearly summarizes the HP3458A noise specification and the real measurements.

(I will upload the csv files later.)

Frank

I like this chart, it is a very informative way to present the data. It clearly shows the tradeoff between integration time (i.e. sampling speed) and noise or effective resolution. It also illustrates that the 100 mV range on this instrument is practically limited by the ten-fold increase in noise over the 1 V range, and that the 1 V range still has good noise performance. I would be curious to see at what point the noise takes off for other insturments.

I think that it would be quite easy to compare multiple instruments on a single chart too. Lines in shades of blue for one instrument, red for another, green for another, etc.

We can see here that noise is inversely proportional to the square root of the integration time (100x longer ~= 1/10 noise). We know from the histograms that the noise is essentially gaussian - it has a nice bell curve. We also know that if you have gaussian noise in your samples, you can reduce noise by averaging samples; the reduction in noise is proportional to the square root of the number of samples(*). So, in theory, we should see a similar noise level when averaging 100x 1.0 PLC samples per reading, as when taking 100 PLC readings. We should be able to confirm this by applying the math to the raw data, provided there are sufficient samples.

*refer to Silicon Labs AN118 for theory and application.

[Dr. Frank]

Hartlijke Groeten van de Koningsdag in Ouddorp!

Frank

[6thimage]

@TiN: thanks for the offer of a short board, but its difficult to find decent plugs here. I considered what to do for a short while, and then bought the fluke 884x-short.

I've run some initial data collection on my 34461A - I aimed at an hour for each of the PLC settings on 10 V, with the meter sampling as fast as it can. I tried to figure out the number of measurements that would take an hour for all of the PLCs, which seemed to work quite well with the exception of the 0.02, which ended up being around 45 minutes.

This is the python script I've been using to perform the measurements:

Code: [Select]

import time

#import usbtmc as tmc
#inst=tmc.instrument(tmc.KEYSIGHT_34461A)
import ethernet
inst=ethernet.instrument('192.168.0.19')

def error_check():
    while True:
        error=inst.ask('syst:err?')
        if error[:2]=='+0':
            return
        print error

print 'ID: {}'.format(inst.ask('*idn?'))
uptime=inst.ask('syst:upt?').split(',')
print 'Uptime: {} {:02}:{:02}:{:02}, line frequency: {} Hz, temperature: {}'.format(
        int(uptime[0]), int(uptime[1]), int(uptime[2]), int(uptime[3]),
        int(inst.ask('syst:lfr?')), float(inst.ask('syst:temp?')))
print 'Cal string: {}, cal date: {}, cal temperature: {}'.format(
        inst.ask('cal:str?'), inst.ask('cal:date?'), float(inst.ask('cal:temp?')))

inst.write('*rst')
inst.write('*cls')
inst.write('abort')

inst.write("sens:func 'volt:dc'")
inst.write('volt:null:stat off')
inst.write('volt:zero:auto on') # auto zero on
inst.write('volt:imp:auto on') # enable high impendance for 0.1, 1 and 10 vdc
inst.write('disp:stat off') # disables the screen

# triggering
inst.write('trig:sour imm')
# as fast as possible
inst.write('trig:del:auto on')
# once per second
#inst.write('trig:del 1')

inst.write('volt:range 10')

error_check()

#dcv ranges - 0.1 1 10 100 1000
#plc - 0.02 0.2 1 10 100 (rdgs/s 1000, 300, 50, 5, 0.5)
nplc=         ['0.02', '0.2',  '1',    '10',   '100']
sample_count= ['1000', '300',  '50',   '5',    '1']
trigger_count=['1800', '1800', '1800', '1800', '900']

for i in range(0, 5):
    print 'nplc: {}, samples: {}, count: {}'.format(nplc[i], sample_count[i], trigger_count[i])
    print 'temperature: {}, time: {}'.format(float(inst.ask('syst:temp?')),
            time.strftime('%d%m%y-%H%M%S'))

    # measurement set up
    inst.write('volt:nplc {}'.format(nplc[i]))
    inst.write('samp:count {}'.format(sample_count[i]))
    inst.write('trig:count {}'.format(trigger_count[i]))

    error_check()

    inst.write('init')

    num_results=int(sample_count[i])*int(trigger_count[i])

    # reading
    results=[]
    while len(results)<num_results:
        resp=inst.ask('r?')
        header_length=int(resp[1])
        data_length=resp[2:2+header_length]
        if int(data_length)==0:
            time.sleep(1)
        else:
            results+=resp[2+len(data_length):].split(',')

    print 'temperature: {}, time: {}'.format(float(inst.ask('syst:temp?')),
            time.strftime('%d%m%y-%H%M%S'))

    with open('34461a-10vdc-{}-{}.dat'.format(nplc[i], time.strftime('%d%m%y-%H%M%S')), 'w') as f:
        for result in results:
            f.write('{}\n'.format(result))

error_check()
The second temperature and time output (after the reading loop) was added after I had started the first run.

I've thrown the data files up onto dropbox https://www.dropbox.com/sh/ubsqjr8d5uiaufj/AACm2P6GARq0H5nqs6Y39ybfa?dl=0 - the meter was allowed to warm up for just under 3 and a half hours with the short in the front connectors.

I have attached two plots - one is the histograms with a fitted normal distribution, the other is the same histograms but with the density plotted on top. The 0.02 range shows some strange behaviour - with some strange oscillations. The data on dropbox, also has a second run on the 0.02 range completed after the first run through, which does exactly the same. So I'm not sure what is going on there.

[blackdog]

Hi Dr Frank,

You are looking good 
But i expected at least a 4,5 Digit DMM under your arm, scored on the flea market, not Silly Willie and his wive 

Kind regarts,
Blackdog

[OldNeurons]

The 0.02 range shows some strange behaviour - with some strange oscillations. The data on dropbox, also has a second run on the 0.02 range completed after the first run through, which does exactly the same. So I'm not sure what is going on there.

I already observed that behaviour with my 34461A / 50Hz, but this does not occur only at 0.02 PLC. If you look carrefully, you will observe the same at other PLC n°.
The "step" value at 10V, 0.02 PLC is close to 100µV, but not exactly. A bit less ... Hummm ...

I can't yet explain it. Something to do with the sampling / AD mechanism?
I'am close to 60 years old but still an "apprentit" voltnut. May be the "senior" voltnuts here have some ideas ?

Below, some datas at 10 Volts and 0.02 PLC.

[LaurentR]

The 0.02 range shows some strange behaviour - with some strange oscillations. The data on dropbox, also has a second run on the 0.02 range completed after the first run through, which does exactly the same. So I'm not sure what is going on there.

I already observed that behaviour with my 34461A / 50Hz, but this does not occur only at 0.02 PLC. If you look carrefully, you will observe the same at other PLC n°.
The "step" value at 10V, 0.02 PLC is close to 100µV, but not exactly. A bit less ... Hummm ...

I can't yet explain it. Something to do with the sampling / AD mechanism?
I'am close to 60 years old but still an "apprentit" voltnut. May be the "senior" voltnuts here have some ideas ?

I wonder how the resolution really works (not familiar with the ADC of the 34461A). The real 10V range is 11V, so we should see 1.1mV steps but you;re seeing slightly less than 100uV...
Below, some datas at 10 Volts and 0.02 PLC.

Is the issue the discrete steps? If you believe the resolution tables, e.g.:
http://rfmw.em.keysight.com/bihelpfiles/Truevolt/WebHelp/US/Content/Misc_Definitions/Range-Resolution_Relationships.htm#kanchor581
The 34461A at 0.02PLC has a resolution factor of 100ppm, which would mean 1mV on the 10V range.

So I am not surpised to see discrete steps, but I am surprised to see them at 100uV...
As far as the jitter still noticeable at 0.02PLC, it could be due to AutoZero being ON? Even at 10V there will be small offsets that will be compensated and create jitter (although the offsets should also be read out with the same 0.02PLC resolution). I wonder if you'll see pure discrete steps if you turn it off.

[OldNeurons]

The 0.02 range shows some strange behaviour - with some strange oscillations. The data on dropbox, also has a second run on the 0.02 range completed after the first run through, which does exactly the same. So I'm not sure what is going on there.

I already observed that behaviour with my 34461A / 50Hz, but this does not occur only at 0.02 PLC. If you look carrefully, you will observe the same at other PLC n°.
The "step" value at 10V, 0.02 PLC is close to 100µV, but not exactly. A bit less ... Hummm ...

I can't yet explain it. Something to do with the sampling / AD mechanism?
I'am close to 60 years old but still an "apprentit" voltnut. May be the "senior" voltnuts here have some ideas ?

Below, some datas at 10 Volts and 0.02 PLC.

I wonder how the resolution really works (not familiar with the ADC of the 34461A). The real 10V range is 11V, so we should see 1.1mV steps but you;re seeing slightly less than 100uV...
Is the issue the discrete steps? If you believe the resolution tables, e.g.:
http://rfmw.em.keysight.com/bihelpfiles/Truevolt/WebHelp/US/Content/Misc_Definitions/Range-Resolution_Relationships.htm#kanchor581
The 34461A at 0.02PLC has a resolution factor of 100ppm, which would mean 1mV on the 10V range.

So I am not surpised to see discrete steps, but I am surprised to see them at 100uV...
As far as the jitter still noticeable at 0.02PLC, it could be due to AutoZero being ON? Even at 10V there will be small offsets that will be compensated and create jitter (although the offsets should also be read out with the same 0.02PLC resolution). I wonder if you'll see pure discrete steps if you turn it off.

Thank's for your feedback.

I moved your sentence "I wonder how the resolution really works (not familiar with the ADC of the 34461A). The real 10V range is 11V, so we should see 1.1mV steps but you;re seeing slightly less than 100uV..." at the right place.

A resolution factor of 100ppm at 10V range means a resolution of 100µV. Am I correct?
I ran a short trial with AutoZero OFF, and it looks like it is the same. But to be confirmed.
You put me on a new track, and I will collect more datas to get the exact step value.

Anyway, this "issue" is may be off topic...

[Kleinstein]

This looks like limited resolution of the raw ADC data at the high speed reading. For calibration the raw data are multiplied with a floating point number and that rounded to a fixed number of decimal places. So each bin of the histogram may not cover the same number of possible values from the raw ADC reading. Some bins may correspond to just 1 ADC reading and other to 2.

Some of the curves can not be explained with this. My guess, is there is also some digital filtering involved that may some readings even less likely.

So these strange looking histograms are cases where DNL erorrs are rather large.

[Dr. Frank]

The 0.02 range shows some strange behaviour - with some strange oscillations. The data on dropbox, also has a second run on the 0.02 range completed after the first run through, which does exactly the same. So I'm not sure what is going on there.

I already observed that behaviour with my 34461A / 50Hz, but this does not occur only at 0.02 PLC. If you look carrefully, you will observe the same at other PLC n°.
The "step" value at 10V, 0.02 PLC is close to 100µV, but not exactly. A bit less ... Hummm ...

I can't yet explain it. Something to do with the sampling / AD mechanism?
I'am close to 60 years old but still an "apprentit" voltnut. May be the "senior" voltnuts here have some ideas ?

Below, some datas at 10 Volts and 0.02 PLC.

I'm a relative Junior volt-nuts  , but all HP DMM will deliver lesser resolution by shorter aperture / NPLC.

The 3458A goes down from 8 1/2 digits at NPLC10 to 4 1/2 digits at 100kHz, therefore up goes the noise, and very similar behave all other HP DMMs.

It's all in the digitizing specification, I think.

Frank

[ManateeMafia]

I uploaded the latest file for the 2001 to TiN's FTP server. I made a lot of adjustments based on macboy's recommendations. For now, it will not be posted it here. Perhaps TiN could create a central location if he wishes.

I had to do a lot of reading from different operator's manuals to understand what macboy meant. The Read? command will only read from the external buffer, where the INIT / Data? / Fetch read from the internal buffer which is much faster.

The 8846A was relatively easy to configure. The Keithley meters are so configurable, it took me a while to find enough sample code to get it functional. I have been making improvements to both scripts but the 2001 is first to be uploaded. I would like TiN, macboy, or anyone with spare time (and 2001) to test it while I finish the 8846A. I can email it if someone can't find it on the ftp server.

Any more suggestions are welcome.

[TiN]

I will try it as well as process data we gathered on friday, we have national holiday here , so will finally get some time to do something. 
Will also test it on 2002 and 2400 for you.

[barnacle2k]

I will try it as well as process data we gathered on friday, we have national holiday here , so will finally get some time to do something. 
Will also test it on 2002 and 2400 for you.

Oh dear, that means i gotta finish my data collection on the 7081 till then.
All the super long integration time stuff is done so it should be quick, i am gathering about 2.5k samples per range&integration-time.
Edit:
All done, file attached.

[OldNeurons]

Hi Volnuts gang's members !

I have collected a lot of data during the past days but I am dealing with 2 issues:
I reported the first one in this topic : https://www.eevblog.com/forum/testgear/34461a-line-frequency-setting-wrong-after-front-panel-reset/
I don't know how much of my data collection was created with the wrong LF value, and don't know how it really affects the noise measurement.
If anyone here owns a 344xxA model running on 50 Hz power line, please test it, and give me your thoughts about this bug.

My second issue is related to quick temperature drifts affecting the 10 and 100 PLC datas series (Added: for 0.1 and 1 V ranges). Again, it seems that this fan is bringing more trouble than benefit, at least in a normal room temperature condition.  (see post #61, page 4). The story is certainly different in a rack configuration in a hot environement where a forced cooling system is necessary.

I ran a test with different cooling configurations and will report later ...

Jean

[6thimage]

I don't know how much of my data collection was created with the wrong LF value, and don't know how it really affects the noise measurement.
If anyone here owns a 344xxA model running on 50 Hz power line, please test it, and give me your thoughts about this bug.

For the data collection, were you performing a front panel reset before hand? And how were you collecting the data - front panel or remote script?

My second issue is related to quick temperature drifts affecting the 10 and 100 PLC datas series. Again, it seems that this fan is bringing more trouble than benefit, at least in a normal room temperature condition.  (see post #61, page 4). The story is certainly different in a rack configuration in a hot environement where a forced cooling system is necessary.

I ran a test with different cooling configurations and will report later ...

I have ran some tests of my own (I need to upload the data), and whilst covering the sides seems to effect the offset value, it does not affect the noise - the histograms with and without the sides covered look almost identical.

How much of a temperature drift are you talking about? I've noticed that if the ambient temperature changes by 1 to 2 degrees (C) the histogram will be double peaked.

It is very easy to blame the fan, but with regards to noise and warm up time, I haven't seen any difference with the sides covered or not. I did notice similar behaviour to the graph in your post (#51), but this seems to just be a change in the offset value.

[OldNeurons]

I don't know how much of my data collection was created with the wrong LF value, and don't know how it really affects the noise measurement.
If anyone here owns a 344xxA model running on 50 Hz power line, please test it, and give me your thoughts about this bug.

For the data collection, were you performing a front panel reset before hand? And how were you collecting the data - front panel or remote script?

My second issue is related to quick temperature drifts affecting the 10 and 100 PLC datas series. Again, it seems that this fan is bringing more trouble than benefit, at least in a normal room temperature condition.  (see post #61, page 4). The story is certainly different in a rack configuration in a hot environement where a forced cooling system is necessary.

I ran a test with different cooling configurations and will report later ...

I have ran some tests of my own (I need to upload the data), and whilst covering the sides seems to effect the offset value, it does not affect the noise - the histograms with and without the sides covered look almost identical.

How much of a temperature drift are you talking about? I've noticed that if the ambient temperature changes by 1 to 2 degrees (C) the histogram will be double peaked.

It is very easy to blame the fan, but with regards to noise and warm up time, I haven't seen any difference with the sides covered or not. I did notice similar behaviour to the graph in your post (#51), but this seems to just be a change in the offset value.

Line frequency issue:
I collected the data from remote scripts, but over several days, playing with different scripts. I am now looking to all files and from the recorded time data, I should be able to extract those with the right frequency line setting.

Fan:
I suspect that the fan is creating some turbulences into the DMM resulting into non homogeneous temperature over sensitive components.
I am surprised that you don't see any difference with the sides covered or not. I ran yesterday another test from front panel, with the correct LF setting and manual recording of the DMM internal temperature. To cover the sides, I used some rectangular pieces of magnetic rubber.

Everything is summarized in the graph below.

The results are, for me, obvious, and best results are achieved with sides covered and fan off ...

Regards,
Jean

Edit: Added DMM screen capture from cooling experiments

[6thimage]

Fan:
I suspect that the fan is creating some turbulences into the DMM resulting into non homogeneous temperature over sensitive components.
I am surprised that you don't see any difference with the sides covered or not. I ran yesterday another test from front panel, with the correct LF setting and manual recording of the DMM internal temperature. To cover the sides, I used some rectangular pieces of magnetic ribbon.

Everything is summarized in the graph below.

The results are, for me, obvious, and best results are achieved with sides covered and fan off ...

I do see a difference in the offset, but not in the noise. I've attached a few graphs - the first two show the histograms with either a fitted normal distribution or the density. The last (density-plot-comp.png) is a comparison of the densities of the covered and non-covered histograms. The covered is blue, the non-covered is red. There are very little differences between the two.

Numerically comparing the two:

	non-covered	covered
min	-9.024061e-07	-6.228938e-06
max	7.918318e-07	-4.592069e-06
range	1.694238e-06	-5.314253e-06
mean	-1.44677e-08	1.636869e-06

Data is on dropbox again @ https://www.dropbox.com/sh/ubsqjr8d5uiaufj/AACm2P6GARq0H5nqs6Y39ybfa?dl=0

Now, why are we seeing different results? My guess is either the rate of change of ambient temperature or the method for blocking the sides (I used paper held in place by the straps of my accessory bag). I am going to try disconnecting the fan to see if that produces any difference.

Something that I meant to ask before - are you shorting the front or rear terminals?

Additionally, when you use the front panel to make measurements, do you start the measurements then leave it alone? I have noticed, when doing other measurements (resistance), that moving near the input jacks can affect the readings.

[6thimage]

@OldNeurons

I also just noticed that you are using the 100 mV range - have you been consistently using 100 mV for these readings?

All of the readings I have been doing are on the 10 V range (as it will have the best accuracy for measuring the noise of the ADC).

[OldNeurons]

@OldNeurons

I also just noticed that you are using the 100 mV range - have you been consistently using 100 mV for these readings?

Yes ...!

All of the readings I have been doing are on the 10 V range (as it will have the best accuracy for measuring the noise of the ADC).

I have no problem with 10 V range as I already mentionned.

[6thimage]

@OldNeurons

I also just noticed that you are using the 100 mV range - have you been consistently using 100 mV for these readings?

Yes ...!

All of the readings I have been doing are on the 10 V range (as it will have the best accuracy for measuring the noise of the ADC).

I have no problem with 10 V range as I already mentionned.

Where the hell did you mention that? It seems I've been comparing apples and oranges

Right then, I will try doing some measurements on the 100 mV, and probably 1 V, ranges.

[OldNeurons]

Something that I meant to ask before - are you shorting the front or rear terminals?

Additionally, when you use the front panel to make measurements, do you start the measurements then leave it alone? I have noticed, when doing other measurements (resistance), that moving near the input jacks can affect the readings.

Front terminals, but front or rear, why should that make a difference? It's a short.