-
Keysight X1000 oscilloscope (edit) averaging bug
Posted by
maxwell3e10
on 13 Oct, 2018 07:56
-
It didn't take me long to find some bugs in Keysight EDUX1002A scope!
1. Set the scope to Acq. mode: Averaging, # of averages: 2, turn on AC RMS-FS measure with nothing plugged in on 1V/div scale. It gives RMS values as low as 200 uV. Without the averaging mode it gives 20 mV AC RMS, which is what one would expect for 8-bit ADC noise. Hard to believe average of 2 measurements reduces the noise by 2 orders of magnitude
.
For comparison I saved the data to a reference waveform and expanded the scale of it. The RMS noise looks like 2-3 mV - still too low to be an average of 2 numbers.
2. When the scope is either in Averaging or High Resolution mode, the maximum length of saved file is 7680 samples - a weird number. When the data are saved, their sampling rate does not agree with the acquisition rate reported on status screen.
-
#1 Reply
Posted by
MrW0lf
on 13 Oct, 2018 08:09
-
Maybe it does boxcar? Does it work single shot? Also 100k/7680=13,02083... and 7680*13=99840. Prob it all somehow aligns with screen pixels.
-
-
For random noise it shouldn't matter how to do the average. But yes, even single shot it behaves weird. It seems that turning on Averaging mode also turns on high-resolution mode at the same time. But even apart from funny averaging, it doesn't seem to report properly the standard deviation of the final waveform it got.
-
-
Here is a comparison of 4 data sets taken over a couple of minutes under identical conditions, with input shorted, averaging mode with 2 averages, 1V/div vertical scale. The scatter looks dramatically different. Its probably the case of excessive averaging (much more than 2) of bit noise which itself is not entirely random. This certainly would be quite confusing if one was trying to understand why noise in some circuit is changing in time.
-
#4 Reply
Posted by
Fungus
on 13 Oct, 2018 10:39
-
It didn't take me long to find some bugs in Keysight EDUX1002A scope!
Impossible!
-
#5 Reply
Posted by
Performa01
on 13 Oct, 2018 10:58
-
I guess that in normal acquisition mode, the scope uses its full memory and noise is high. For Average mode, the scope doesn’t use the full sample buffer anymore, but just screen buffer or some tiny secondary buffer with just a couple kSa. This drastically reduces the sample rate, hence also the noise, even without averaging, which then reduces the noise even further.
As I said, just a guess – but if I’m right, then it’s not a bug, it’s a feature.
-
#6 Reply
Posted by
MrW0lf
on 13 Oct, 2018 11:24
-
-
-
This is an EDUX scope, so its memory spec is 100k points. In reality for normal acquisition the buffer size is 50k and under either Average or High-resolution mode, the buffer is 7680 points. I don't know how these numbers behave for a DSOX scope.
The the rms noise should not change with the acquisition rate unless one is doing some averaging or anti-alias filtering, which scopes don't normally do unless specifically set to. So when the average is set to 2 and the noise decreases by 2 orders of magnitude something is clearly happening under the hood. They maybe trying to be helpful, but it is more confusing. How do you explain to students "Averaging your signal 2 times reduces the noise by Sqrt(2), ahm..."
Besides that, there is a limit on how much boxcar averaging in "high-resolution" mode can improve your resolution. Maybe 8 bits to 12 bits if one averages 16 times. Here from the bit size it looks like they are averaging something like 160 points, so inevitably one gets some subtle artifacts from non-random bit noise. This results in the drastic fluctuations of the noise level.
-
#8 Reply
Posted by
Performa01
on 16 Oct, 2018 08:50
-
Yes, right, the difference is way too much to be explained by different sample rates. Even though the bandwidth is much lower with the short memory (hence low sample rate) in average mode, all aliased signal/noise content still adds to the result.
Of course I have been curious and when I tried to reproduce a similar situation on a Siglent SDS1104X-E, I only got about 120µVrms noise in normal acquisition mode (without averaging) which wouldn’t be a good candidate for demonstrating further noise reduction by averaging.
At 1V/div, one LSB of the ADC equals 40mV and the measurement hints on a rather stable LSB that only toggles once every ~300 samples on average.
If I got you right, then the Keysight EDUX1002A shows ~20mVrms which means the LSB would toggle for every other sample.
Even though this is not the primary topic here, I should explain that one reason for this huge difference might be that the 1V/div is a sweet spot for the SDS1104X-E, because at that setting, the first attenuator is in effect and the VGA operates at minimum gain. In this scenario, there is very little analog noise to trigger the (much higher) granular noise of the ADC.
With average acquisition mode it gets a bit tricky.
Using average acquisition mode on the SDS1104X-E doesn’t change much. As it seems, there is just some numeric base noise that cannot be undercut, probably due to limited resolution of the measurements and/or math. Not a problem at all, just to be kept in mind.
On the Keysight EDUX1002A, the averaging kills all the granular ADC noise and the measurement falls back to a similar numeric noise as on the Siglent – which is quite baffling, to say the least.
I’ve long thought about that in order to find a plausible explanation and the best guess I can come up with is that the Averaging mode on the Keysight EDUX1002A isn’t meant for resolution enhancement, hence the averaging result is truncated to 8 bits. Only this would result in a threshold effect as you’ve observed.
-
-
When one is looking at noise that is below 1 bit of native resolution it can be quite unstable. If the offset happens to match a bit transition edge, it could toggle every bit and give RMS noise equal to 1/2 of the LSB. For a slightly different offset it may not toggle at all and give zero RMS noise. Sometimes people on purpose add random analog Gaussian noise to dither the input to the ADC and improve the performance after averaging.
The confusing part here is that the noise on Keysight in "Averaging" mode appears much larger than the bit noise, yet drifts in unpredictable way. There maybe several layers of averaging that interact in unintended manner.
-
#10 Reply
Posted by
David Hess
on 19 Oct, 2018 13:45
-
I guess that in normal acquisition mode, the scope uses its full memory and noise is high. For Average mode, the scope doesn’t use the full sample buffer anymore, but just screen buffer or some tiny secondary buffer with just a couple kSa. This drastically reduces the sample rate, hence also the noise, even without averaging, which then reduces the noise even further.
The sample rate itself does not affect the noise bandwidth but averaging sure does. The RMS value is the same as the standard deviation which does not change with decimation. Noise above the Nyquist bandwidth folds over and is still measured which is how GHz+ wideband RMS voltmeters can get by with a sample rate of only 50kHz.
Some other things to consider:
Modern oscilloscope front ends which use integrated CMOS impedance converters can have incredibly high levels of 1/f or flicker noise so low frequency measurements are much more unstable than the wideband noise measurement would indicate.
I have been told that these Keysight DSOs and most modern low cost DSOs make measurements on the display record. The processing which produces the index graded display record corrupts RMS noise measurements.
-
#11 Reply
Posted by
Performa01
on 19 Oct, 2018 14:34
-
The sample rate itself does not affect the noise bandwidth but averaging sure does. The RMS value is the same as the standard deviation which does not change with decimation. Noise above the Nyquist bandwidth folds over and is still measured which is how GHz+ wideband RMS voltmeters can get by with a sample rate of only 50kHz.
Yes, thank you for the confirmation - I've already realised it myself, see my last reply #8, first paragraph.
Since the visual impression is driven by the peak to peak noise, I'm mostly thinking of that. But I know that's not a good parameter for noise measurements/comparisons for various reasons.
Modern oscilloscope front ends which use integrated CMOS impedance converters can have incredibly high levels of 1/f or flicker noise so low frequency measurements are much more unstable than the wideband noise measurement would indicate.
That's certainly true, but it is nowhere near as bad to explain the effects described here. The 1/f noise is nicely documented by the graphs provided by the very same author maxwell3e10 in his noise comparison thread.
I have been told that these Keysight DSOs and most modern low cost DSOs make measurements on the display record. The processing which produces the index graded display record corrupts RMS noise measurements.
Well, it depends what you define as low cost DSOs. Keysight uses some secondary buffer, probably just a few kPts, but certainly more than the screen buffer - and some others do the same. But then again, Siglent is certainly low cost when compared to Keysight, yet they had a comparably big secondary buffer of about 70kPts on their older models [SDS1000X(+), SDS2000(X)] but provide full memory measurements on the new platform scopes [SDS1000X-E and upcoming SDS5000X).
Apart from that, I very much doubt that the rms (or any other) measurement has to be seriously flawed because of the intensity grading. If properly implemented, it is just some sort of Histogram over many acquisitions in the third (intensity) dimension and I have carefully checked accuracy of all automatic measurements on various Siglent scopes (which happen to have superb intensity grading). All the amplitude measurements have always been within expectations (thus well within specifications).
A huge error like the one discussed in this thread because of intensity grading appears rather unlikely, especially not on a "real Scope"
-
-
Just to clarify, I don't think anymore there is a error in the actual calculation of RMS value for a given data set. Its just that the noise is is widely fluctuating in time. So unless one captures exactly the same shot, it looks like the plot does not agree with the calculated RMS value.
-
#13 Reply
Posted by
Performa01
on 19 Oct, 2018 18:20
-
Just to clarify, I don't think anymore there is a error in the actual calculation of RMS value for a given data set. Its just that the noise is is widely fluctuating in time. So unless one captures exactly the same shot, it looks like the plot does not agree with the calculated RMS value.
Now I think I''ve finally got it.
So you are saying that even if everything is kept the same (particularly vertical offset), the LSB of the ADC toggling might show a different pattern from one measurement to another and in some (many?) situations it does virtually not toggle at all. And you just attributed the vastly different measurements to the acquisition mode (normal/average) initially, which now has turned out has essentially nothing to do with it.
I bet the bevaior would be different if you choose one of the higher sensitivities, where the real analog noise from the frontend starts serving as a dither for the ADC. Then the textbook example for noise reduction by averaging should work reasonably well.
-
-
I apologize in advance if I missed a comment or am repeating someone here.
I'm also not sure if this is new information or not, but I believe it's been discussed publicly. If not,
.
First - yes, the InfiniiVision scopes use the on-screen data for measurements, not the full record length. We have some algorithms to make sure we're not doing anything crazy.
For measurements and some acquisition modes, we have what we call "eavesdrop memory" that's a sub-sample of the overall acquisition. That's what we use for the measurements, math, etc. We do this to keep the update rate fast and not overload any of our processing.
In averaging mode, memory acts differently. You essentially switch out of a "save all the data" mode and into a "give me an average." It does a waveform-to-waveform averaging, and I think it's based just on the plotted screen data itself. The design philosophy is essentially "since you're in averaging mode, you probably don't care a whole lot about super high resolution. So our averaging method will do it's best to get you a nice, clean average that's not a ton of memory points."
High Res mode can change behavior based on your time/div setting. It does use a boxcar averaging mode.
I did put together an article a while back on the different modes:
https://www.electronicdesign.com/test-measurement/why-you-should-care-about-oscilloscope-acquisition-modes There's also a video on it, but it was one of my first (so don't judge
):
http://bit.ly/2Oyhypg
-
#15 Reply
Posted by
David Hess
on 20 Oct, 2018 02:27
-
I have been told that these Keysight DSOs and most modern low cost DSOs make measurements on the display record. The processing which produces the index graded display record corrupts RMS noise measurements.
Well, it depends what you define as low cost DSOs. Keysight uses some secondary buffer, probably just a few kPts, but certainly more than the screen buffer - and some others do the same. But then again, Siglent is certainly low cost when compared to Keysight, yet they had a comparably big secondary buffer of about 70kPts on their older models [SDS1000X(+), SDS2000(X)] but provide full memory measurements on the new platform scopes [SDS1000X-E and upcoming SDS5000X).
Apart from that, I very much doubt that the rms (or any other) measurement has to be seriously flawed because of the intensity grading. If properly implemented, it is just some sort of Histogram over many acquisitions in the third (intensity) dimension and I have carefully checked accuracy of all automatic measurements on various Siglent scopes (which happen to have superb intensity grading). All the amplitude measurements have always been within expectations (thus well within specifications).
A huge error like the one discussed in this thread because of intensity grading appears rather unlikely, especially not on a "real Scope"
I think the Rigol DS1000Z series suffers from this problem. I am not sure about other Rigol models or other manufacturer's DSOs. It is one of the first things I would check now with a DSO.
Last year when I was making calibrated RMS noise measurements of various old oscilloscopes, I was surprise how well old analog and some old digital storage oscilloscopes did; most were better than any comparable results I could find from modern DSOs. They were more limited by maximum vertical sensitivity and trace geometry than input referred noise.
-
-
In averaging mode, memory acts differently. You essentially switch out of a "save all the data" mode and into a "give me an average." It does a waveform-to-waveform averaging, and I think it's based just on the plotted screen data itself.
Yes, this is how the average mode usually works. However, when the number of averages is set to 2, it should reduce the noise by sqrt(2), not 2 orders of magnitude. What I think is happening here is that the averaging mode also turns on the high-resolution mode. So first it averages the raw ADC data into a shorter buffer (which appears to be 7680 points) using boxcar averaging and then does the number of waveform averages specified.
-
#17 Reply
Posted by
Performa01
on 20 Oct, 2018 10:25
-
Yes, this is how the average mode usually works. However, when the number of averages is set to 2, it should reduce the noise by sqrt(2), not 2 orders of magnitude. What I think is happening here is that the averaging mode also turns on the high-resolution mode. So first it averages the raw ADC data into a shorter buffer (which appears to be 7680 points) using boxcar averaging and then does the number of waveform averages specified.
If you are right, then this completely defies the purpose of average acquisition mode, which is supposed to have
no effect on bandwidth and/or frequency response, but only decreases the max. rate of waveform change, i.e. the modulation bandwidth.
This is why we choose average mode for repetitive signals that are fairly static where we want to get rid of some unwanted modulation due to interference and/or noise.
We need to resort to HiRes (boxcar averaging) or Eres (FIR filter) for dynamic signals or single shot captures, but then we are aware that this has a low-pass filter effect on the signal, which of course also reduces noise.
-
#18 Reply
Posted by
Fungus
on 20 Oct, 2018 10:32
-
Yes, this is how the average mode usually works. However, when the number of averages is set to 2, it should reduce the noise by sqrt(2), not 2 orders of magnitude. What I think is happening here is that the averaging mode also turns on the high-resolution mode. So first it averages the raw ADC data into a shorter buffer (which appears to be 7680 points) using boxcar averaging and then does the number of waveform averages specified.
If you are right, then this completely defies the purpose of average acquisition mode, which is supposed to have no effect on bandwidth and/or frequency response, but only decreases the max. rate of waveform change, i.e. the modulation bandwidth.
I think somebody's getting the names mixed up. I often get them mixed up myself.
I think the Rigol DS1000Z series suffers from this problem.
And many others. Some compromises have to be made when you're selling 4-channel scopes for $350.
Not happy? Spend more (please!)
-
-
Sometimes spending more does not result in a better scope, that is why this forum is so useful.
Keysight entry level scopes, while at a slight premium compared with Chinese brands, supposed to have the same basic analysis functions as their higher level scopes and hopefully be free from bugs.
-
-
So, I did some digging on this. Here's what's going on:
When in Average mode, InfiniiVision scopes first turn on/do a pass in High Resolution mode and then do the Average per the average number set. This was an intentional design decision, but I'm still digging into the benefit of this approach. If I had to speculate, I'd say that when in averaging mode one generally doesn't care about little random noise spurs and cares more about general trends. Switching to high res mode smooths out the random noise and will give a cleaner average. At this point it's speculation, though.
Also, the ASCII XY max length in this mode is different across the InfiniiVision series.
1000X and 2000X: 7,680
3000A and 3000T: 16,000
So that will explain some of the different file size values.
-
-
Thank you for looking into it, I think this explains most of the behavior. Using high-resolution mode with averaging can be a good approach in many cases, but maybe not all. In the manual it just says "Use this mode for reducing noise and increasing resolution of periodic signals without bandwidth or rise time degradation." - without mentioning high-resolution boxcar averaging.
Regarding high-resolution boxcar averaging, do you have information on how many points are averaged together? It may be too many points are averaged, resulting in funny non-stationary noise.
-
-
I realize I may have been not 100% clear. The noise reduction is coming from the combo of high res + averaging. If you start in high res and switch to averaging, the noise will drop 1/2, like you'd expect.
The number of points being averaged depends on the sample rate, which is dependent on the time/div setting. Slower time/div means more averages get sampled. From the 3000T user's guide:
"
In High Resolution mode, at slower time/div settings extra samples are averaged in order to reduce random noise, produce a smoother trace on the screen, and effectively increase vertical resolution.
High Resolution mode averages sequential sample points within the same
acquisition. An extra bit of vertical resolution is produced for every factor of 2
averages. Random noise is reduced by ½ for every factor of 4 averages. The
number of extra bits of vertical resolution is dependent on the time per division
setting (sweep speed) of the oscilloscope.
The slower the time/div setting, the greater the number of samples that are averaged together for each display point.
"
If I recall correctly the ADCs are always running full speed, and the acq mode determines what happens with that data.