Oscilliscope memory, type and why so small?

[Someone]

As mentioned earlier in the thread there are scopes with deep memory and scopes with high throughput, there aren't scopes that do both simultaneously.

Right.  But why?  What is it about the basic architecture of the above that is somehow inadequate to the task?  I'm not arguing that the architecture above will work.  I'm trying to figure out why it won't, because I presume the third option I mentioned previously is the one that's in play here.

Ok, simple thought experiment. You have a 100,000,000 sample acquisition and a 1000px display, at 10GS/s the acquisition is filled in just 1/100th of a second.

How quickly can you get that 100Mpts drawn to a 1000px wide display? Pick anything you like from pure hardware to pure software or anything in-between and see how much blind time you come up with (and the resource cost required to do so). Throwing away samples is not an option, every single sample needs to be inspected or you miss glitches. Now try the same with a 1Mpts acquisition depth, how much faster/cheaper was it?

[nctnico]

@kcbrown: you and some others are massively overestimating the problems. Streaming data from an ADC into memory is peanuts. Doing this while displaying many waveforms/s is also peanuts (hint: many if not all scopes use double buffering). I create very similar data acquisition systems for one of my customers.

Hmm...double buffering takes care of a lot of problems, I have to admit.  But the double buffering has to exist at the bus level, because you're fighting bus contention.   As long as that's the case, it should be fine.  Even segmented memory capture can bounce between the two banks to avoid contention. 

And I suppose as long as your trigger conditions can be limited to a fairly short amount of memory (enough that you can buffer it separately), you won't have trouble there, either. 

If this problem is as easy to deal with as you say, then why haven't manufacturers stepped up to the plate with massive capture memories based on DDR3 architectures?   The design only has to be done once, right?

Manufacturers are already using double buffering.   So what's the holdup for them?

• There's a really good and non-obvious reason that the obvious solution is not being used.

Just look at GW Instek and Keysight for the answer. GW Instek leverages new technology with lots of processing power to come up with a scope which can deal with the large memory in a useful way. Keysight OTOH rehashes old technology which uses a slow processor so it can't do much with large amounts of data. The hold up is in the engineering costs of creating firmware for a scope. If a manufacturer is too invested into inflexible technology they are unlikely to change even if they are on a dead end road.

This reminds me of a meeting I attended many years ago at one of my former employers. We where looking to buy some DSP IP from a company. However it turned out that company wrote all their algorithms in assembler for a DSP which was declared obsolete. If we wanted to use their algorithms we would basically have to rewrite them from scratch so that deal didn't happen.

[Keysight DanielBogdanoff]

@kcbrown: you and some others are massively overestimating the problems. Streaming data from an ADC into memory is peanuts. Doing this while displaying many waveforms/s is also peanuts (hint: many if not all scopes use double buffering). I create very similar data acquisition systems for one of my customers.

Hmm...double buffering takes care of a lot of problems, I have to admit.  But the double buffering has to exist at the bus level, because you're fighting bus contention.   As long as that's the case, it should be fine.  Even segmented memory capture can bounce between the two banks to avoid contention. 

And I suppose as long as your trigger conditions can be limited to a fairly short amount of memory (enough that you can buffer it separately), you won't have trouble there, either. 

If this problem is as easy to deal with as you say, then why haven't manufacturers stepped up to the plate with massive capture memories based on DDR3 architectures?   The design only has to be done once, right?

Manufacturers are already using double buffering.   So what's the holdup for them?

• There's a really good and non-obvious reason that the obvious solution is not being used.

Just look at GW Instek and Keysight for the answer. GW Instek leverages new technology with lots of processing power to come up with a scope which can deal with the large memory in a useful way. Keysight OTOH rehashes old technology which uses a slow processor so it can't do much with large amounts of data. The hold up is in the engineering costs of creating firmware for a scope. If a manufacturer is too invested into inflexible technology they are unlikely to change even if they are on a dead end road.

Actually, the cost (and time) is in designing and creating a new ASIC, if it was firmware it definitely would have been done already. For the InfiniiVision scopes (which I assume we're talking about, because Infiniium scopes have huge memory depth), all of the acquisition memory is proprietary and built into the ASIC itself.[/list]

[kcbrown]

You do have to get all the points from the acquisition to the screen or otherwise you won't see the interesting characteristics

You mean you have to get some representation of them to the screen.  Yes, I agree, but:

• The rate at which you have to get changes written to the screen is capped by the limitations on human perception.  30 times per second is more than enough for this.
• For continuous display purposes, it is perfectly acceptable to be one buffer's worth of time behind
• More importantly, the point is that DDR3 memory bandwidth is not likely to be the limitation on this

, two ways to do it:
Double buffer the acquisition and let a slow process render the display (such as a CPU), while waiting for that you can't see any of the samples coming in.

Right.  But that just means that the display is behind the actual current samples by some amount of time.  How much time depends on a lot of factors, e.g. segment length divided by sample rate (i.e., the amount of time represented by a single segment).

Under what conditions would that not be acceptable?  Note that there is a difference between the amount of time the screen is behind the current samples being taken and the rate at which data is being refreshed on the screen.

or
Have a realtime hardware system render the view, no (or much fewer) missed samples but limited in memory size as .

Well, missed samples is an entirely different thing, unrelated to double buffering (this assumes that switching banks is sufficiently fast to not cause samples to be missed, and if you have a fast enough FIFO in the pipeline, you can eliminate that as a problem).  That has much more to do with trigger rearm time.

Now, all this presumes that the decimation/conversion process can keep up with the sampling subsystem.  It doesn't need to be looking at the current samples, but the rate has to be the same.

Even just looking at a long capture with Mpts+ you need to see the details so you know where to zoom in for further study (or have some intelligent tool search for you) so approximations such as envelopes (min/max etc) aren't always enough.

Are we talking about a capture after you've stopped the scope, or while the scope is currently capturing?  I would presume the latter, since in the former, the scope's processing subsystem no longer has to compete with the sampling subsystem for memory bandwidth.

For the latter, the system has to be able to keep up with the sampling rate.  Beyond that, it can get away with operating on half the memory (while the sampling subsystem operates on the other half), or more generally, it need only avoid attempting to read from the bank of memory that the sampling subsystem is currently writing to.
 

The 9000/90000 series have quite slow update rates and large blind time, since they just pile the acquisition up into a big buffer and then slowly read it out for processing and display, they don't have the realtime hardware but as you say a very wide memory bus to queue the acquisition in. On the issue of cost notice how they ship with all the memory installed and you unlock the deeper memory with a software license

Yes, but bus width can just as easily be used to advantage on the processing side of the equation, at least for the purposes of getting the data out of memory.   It sounds like the issue here isn't with the memory bandwidth at all, but rather with the processing speeds that would be needed in order to perform decimation at a rate that keeps up with the sampling rate.  You'll get no argument from me on that, but again, that means that the memory itself is not the limiting factor in this equation.



It should be noted that there's a difference between the amount of memory available to the system as a whole and the amount of memory used in a single capture.  Obviously, the more memory you use for a single capture (i.e., continuous data before and after the trigger point), the slower your screen update rate can be.  But that's not a limitation of the memory or of the processing (though it could be), but rather the direct result of the fact that more memory used in a capture, while the sample rate is kept constant, automatically represents more time.   Generally, in that situation, what is displayed is a small subset of what is captured, and so the processing side of the equation will generally be quite fast (it's only attempting to display a portion of the waveform).  In the case of decimation of the captured data, e.g. FFT, it's not necessarily the case that the entirety of the buffer needs to be processed for that -- a subset may well prove sufficient.  That will obviously depend on the nature of the operation.  Nothing prevents you from doing decimation on a subset of the buffer, as if the size of your buffer were smaller than it is.  But the biggest difference, and the biggest benefit of large amounts of memory, is that with the larger amount of memory, you'll have a larger period of time captured in the buffer that can then be reviewed once the scope is stopped and, perhaps more importantly, you can perform decimation processing on the entire buffer once it is stopped, thus giving you all the advantages of deep memory at that point, even if those advantages can't be realized during capture.

[kcbrown]

Just look at GW Instek and Keysight for the answer. GW Instek leverages new technology with lots of processing power to come up with a scope which can deal with the large memory in a useful way. Keysight OTOH rehashes old technology which uses a slow processor so it can't do much with large amounts of data. The hold up is in the engineering costs of creating firmware for a scope. If a manufacturer is too invested into inflexible technology they are unlikely to change even if they are on a dead end road.

Instek's memory size isn't particularly large, certainly nothing like what we're discussing here, so I don't see how this argument is terribly compelling.  They do seem to make very good use of the processing power of the system, but we're still left with the question of why the amount of memory they're using in their low-end (GDS-1000 and GDS-2000 series) is measured in megabytes instead of gigabytes.

[Someone]

You do have to get all the points from the acquisition to the screen or otherwise you won't see the interesting characteristics

You mean you have to get some representation of them to the screen.  Yes, I agree, but:

• The rate at which you have to get changes written to the screen is capped by the limitations on human perception.  30 times per second is more than enough for this.
• For continuous display purposes, it is perfectly acceptable to be one buffer's worth of time behind
• More importantly, the point is that DDR3 memory bandwidth is not likely to be the limitation on this

You can draw any number of waveforms to the screen per second, that the screen only updates 20/30/50/100 times is irrelevant. You don't just draw the last waveform but all the waveforms possible on top of each other.

, two ways to do it:
Double buffer the acquisition and let a slow process render the display (such as a CPU), while waiting for that you can't see any of the samples coming in.

Right.  But that just means that the display is behind the actual current samples by some amount of time.  How much time depends on a lot of factors, e.g. segment length divided by sample rate (i.e., the amount of time represented by a single segment).

Under what conditions would that not be acceptable?  Note that there is a difference between the amount of time the screen is behind the current samples being taken and the rate at which data is being refreshed on the screen.

You confuse latency and throughput, its ok to take time to do the processing but if that is done on a CPU or other sequential processor that is blocked during that time then you are not able to continue sampling and have dead time that the scope (and user) simply miss. To capture all the events/data you need to process the incoming stream at a high enough throughput to minimise the missing gaps in the data that get missed (or get that down to zero blind time).

Yes, but bus width can just as easily be used to advantage on the processing side of the equation, at least for the purposes of getting the data out of memory.   It sounds like the issue here isn't with the memory bandwidth at all, but rather with the processing speeds that would be needed in order to perform decimation at a rate that keeps up with the sampling rate.  You'll get no argument from me on that, but again, that means that the memory itself is not the limiting factor in this equation.

Thats the limitation for some uses of a deep memory scope, if you just a single capture and want to spend a lot of time processing or looking through it then its not a problem and you just worry about the slow user interface. Processing is cheaply available in distributed forms, its the memory bandwidths and latencies that limit that processing.

[kcbrown]

You can draw any number of waveforms to the screen per second, that the screen only updates 20/30/50/100 times is irrelevant. You don't just draw the last waveform but all the waveforms possible on top of each other.

That's part of the decimation process itself, which I've noted multiple times has to be able to keep up with the sampling rate.

You confuse latency and throughput, its ok to take time to do the processing but if that is done on a CPU or other sequential processor that is blocked during that time then you are not able to continue sampling and have dead time that the scope (and user) simply miss.

That is the entire point of double buffering: to ensure that processing isn't blocked by the sampling subsystem.

I don't believe I'm confusing latency and throughput here.  Indeed, my point is that you can sacrifice latency to get throughput without any real detrimental effect.

To capture all the events/data you need to process the incoming stream at a high enough throughput to minimise the missing gaps in the data that get missed (or get that down to zero blind time).

Yes, but the point is that you don't have to be processing the head of the stream.  It's perfectly fine to be processing something downstream, as long as you're processing samples at the same rate as the sampling subsystem is writing them.

Thats the limitation for some uses of a deep memory scope, if you just a single capture and want to spend a lot of time processing or looking through it then its not a problem and you just worry about the slow user interface. Processing is cheaply available in distributed forms, its the memory bandwidths and latencies that limit that processing.

In what way do memory bandwidths limit processing rates here?  We're talking sequential memory write rates (which are slower than read rates, from what I understand) in the gigabytes per second range per bank.  You already have to go to a multi-core CPU architecture in order to make the memory a bottleneck with a 64 bit memory bus, do you not?

[Someone]

You confuse latency and throughput, its ok to take time to do the processing but if that is done on a CPU or other sequential processor that is blocked during that time then you are not able to continue sampling and have dead time that the scope (and user) simply miss.

That is the entire point of double buffering: to ensure that processing isn't blocked by the sampling subsystem.

I don't believe I'm confusing latency and throughput here.  Indeed, my point is that you can sacrifice latency to get throughput without any real detrimental effect.

Double buffer the data, its easy enough with a dedicated memory subsystem (a big fifo). But you wont be able to do anything practical with it, the Gs/s of data just keep coming. If you have a processor taking 10 times longer to plot the data than it takes to fill the fifo then you've got a blind time of 90%.

You cant mask processing time (or blocking latency) with more memory, the processing has to keep up and requires its own memory separate to the acquisition buffer. Its this memory where you try and take the 10's of Gs/s and analyse/plot/search/etc that is expensive and doesn't scale well, try it for your self and design a system which can take in 10's of Gs/s and actually do something with it rather than just park it sequentially into memory (the easy bit).

The acquisition buffer can be built with readily available memory parts in scales that are affordable, Lecroy scopes are the obvious example of this with their HDO line or the Agilent 9000/MSOS/etc, they take long captures and then process through each capture slower than they arrive. The blind time can be substantial when at high sample rates, at lower sample rates or accepting the blind time they can do amazingly deep analysis of the waveforms.

[Lukas]

I definitely agree that the highest-end scopes that are doing 40GS/s (like the Keysight 90000 series) will almost certainly require something faster than DDR3, unless the memory is organized in parallel access banks or something equivalent.  That kind of throughput does require a super-fast special-purpose ASIC.   But we're talking about lower end scopes that do sample rates of something like 2GS/s, are we not?

Instead of guessing, just take a look at the service manuals of the 90000A/X series scopes. They clearly mention DDR2 acquisition memory.

[kcbrown]

Double buffer the data, its easy enough with a dedicated memory subsystem (a big fifo). But you wont be able to do anything practical with it, the Gs/s of data just keep coming. If you have a processor taking 10 times longer to plot the data than it takes to fill the fifo then you've got a blind time of 90%.

This misses the point of double buffering.  The point is to eliminate contention, to keep one subsystem from having to wait for another as a result of both attempting to access the same thing at the same time.

Yes, clearly the processing subsystem has to be fast enough to properly deal with the data stream, but:

• It can always sample a smaller amount of data than is actually available if it wouldn't otherwise be fast enough.  After all, the presumption is that it would be faster if the amount of memory were smaller, right?  But it cannot sample data that isn't there.  Hence, more memory is always better than less, as long as there aren't other architectural considerations that would arise.
• The memory is not the bottleneck here.  If it were, then it would be a bottleneck for the acquisition subsystem as well.  But here, we're talking about the problems of the processing subsystem keeping up with the acquisition subsystem.

You cant mask processing time (or blocking latency) with more memory, the processing has to keep up and requires its own memory separate to the acquisition buffer. Its this memory where you try and take the 10's of Gs/s and analyse/plot/search/etc that is expensive and doesn't scale well, try it for your self and design a system which can take in 10's of Gs/s and actually do something with it rather than just park it sequentially into memory (the easy bit).

Sure, but what has that to do with having a larger amount of memory available?  Just because you have more memory available to you doesn't mean you have to use all of it for everything you do.

The acquisition buffer can be built with readily available memory parts in scales that are affordable, Lecroy scopes are the obvious example of this with their HDO line or the Agilent 9000/MSOS/etc, they take long captures and then process through each capture slower than they arrive. The blind time can be substantial when at high sample rates, at lower sample rates or accepting the blind time they can do amazingly deep analysis of the waveforms.

Right.  But the memory subsystem, again, isn't the bottleneck.  It's the processing subsystem that is.

[Someone]

Yes, clearly the processing subsystem has to be fast enough to properly deal with the data stream, but:

• It can always sample a smaller amount of data than is actually available if it wouldn't otherwise be fast enough.  After all, the presumption is that it would be faster if the amount of memory were smaller, right?  But it cannot sample data that isn't there.  Hence, more memory is always better than less, as long as there aren't other architectural considerations that would arise.

Thats what we have now, scopes that have extremely deep memory that is not used when run in their "realtime" or fastest update configurations, and those fast modes are still much slower than the scopes using hardware (and expensive memory) to do the display driving. We don't have more of the deep memory scopes as the deep memory adds cost and without a fast interface is of limited practical value, people can't/don't/won't pay for a feature they can't/don't/won't use. You can see the memory depth tradeoff in this video:

Right.  But the memory subsystem, again, isn't the bottleneck.  It's the processing subsystem that is.

The processing (more correctly display) problem is embarrassingly parallel, so it can be solved without memory. But thats wildly inefficient so sequential processes (though not necessarily processors) are used to calculate the result incrementally and they require memory, the complex and tricky memory which is very expensive. Instead of saying how this can be done without any specifics you need to sit down and describe a system that achieves your result, because its probably wildly different to whats actually inside a modern scope.

[kcbrown]

Yes, clearly the processing subsystem has to be fast enough to properly deal with the data stream, but:

• It can always sample a smaller amount of data than is actually available if it wouldn't otherwise be fast enough.  After all, the presumption is that it would be faster if the amount of memory were smaller, right?  But it cannot sample data that isn't there.  Hence, more memory is always better than less, as long as there aren't other architectural considerations that would arise.

Thats what we have now, scopes that have extremely deep memory that is not used when run in their "realtime" or fastest update configurations, and those fast modes are still much slower than the scopes using hardware (and expensive memory) to do the display driving.

Nothing beats dedicated hardware specifically designed for the task.   But that doesn't mean you can't create an architecture based on off-the-shelf hardware that comes reasonably close.

Everything here depends on the sample rate.  That drives the requirements for everything downstream of it, including the triggering mechanism.

We don't have more of the deep memory scopes as the deep memory adds cost and without a fast interface is of limited practical value, people can't/don't/won't pay for a feature they can't/don't/won't use. You can see the memory depth tradeoff in this video:

Sure, the tradeoff is there, but that's beside the point.  You can't create a tradeoff if there's no capability to trade against.  You can't trade FFT resolution for speed if you haven't the memory depth to make that trade in the first place.  So again, it's better to have more memory than to not have it.

The processing (more correctly display) problem is embarrassingly parallel, so it can be solved without memory. But thats wildly inefficient so sequential processes (though not necessarily processors) are used to calculate the result incrementally and they require memory, the complex and tricky memory which is very expensive.

Most certainly some of the memory will be of the very fast and/or complex variety.  But the nature of the problem is such that it won't be the main sample memory for which you have to do that.

Look, I get that making main sample memory ultra-fast makes a bunch of other things easier.  You wouldn't have to do double buffering, you'd be able to do triggering directly off the sample memory, etc.  But you don't have to make your main sample memory that fast.  As you say, the display problem is highly parallel in nature (the reason I call it "processing" is that decimation of the data in various ways can be used for more than just creating a nice display, and you might well be logging the processing results rather than being interested in seeing it on the display.  But I think we're nonetheless talking about the same thing).

Instead of saying how this can be done without any specifics you need to sit down and describe a system that achieves your result, because its probably wildly different to whats actually inside a modern scope.

Actually designing an oscilloscope (even if only at the block diagram level) isn't what I originally thought I'd be doing here.  But it could prove interesting and instructive.

[nctnico]

@kcbrown: you and some others are massively overestimating the problems. Streaming data from an ADC into memory is peanuts. Doing this while displaying many waveforms/s is also peanuts (hint: many if not all scopes use double buffering). I create very similar data acquisition systems for one of my customers.

Hmm...double buffering takes care of a lot of problems, I have to admit.  But the double buffering has to exist at the bus level, because you're fighting bus contention.   As long as that's the case, it should be fine.  Even segmented memory capture can bounce between the two banks to avoid contention. 

And I suppose as long as your trigger conditions can be limited to a fairly short amount of memory (enough that you can buffer it separately), you won't have trouble there, either. 

If this problem is as easy to deal with as you say, then why haven't manufacturers stepped up to the plate with massive capture memories based on DDR3 architectures?   The design only has to be done once, right?

Manufacturers are already using double buffering.   So what's the holdup for them?

• There's a really good and non-obvious reason that the obvious solution is not being used.

Just look at GW Instek and Keysight for the answer. GW Instek leverages new technology with lots of processing power to come up with a scope which can deal with the large memory in a useful way. Keysight OTOH rehashes old technology which uses a slow processor so it can't do much with large amounts of data. The hold up is in the engineering costs of creating firmware for a scope. If a manufacturer is too invested into inflexible technology they are unlikely to change even if they are on a dead end road.

Actually, the cost (and time) is in designing and creating a new ASIC, if it was firmware it definitely would have been done already. For the InfiniiVision scopes (which I assume we're talking about, because Infiniium scopes have huge memory depth), all of the acquisition memory is proprietary and built into the ASIC itself.[/list]

IMHO the age of the ASIC based low end scopes is over. Recently I quoted a project where I opted to do a lot of things in software which where traditionally done in an FPGA. Software development is way cheaper than FPGA or ASIC development in both hours and labour costs (the latter if you need to hire a developer) and for low end scopes you can use off-the-shelve parts which provide extremely good performance. What we'll probably see in the near future are low cost scopes with deep memory which use a GPU for math / signal analysis tasks like FFT, measurements, protocol decoding, etc. Basically the PC based oscilloscopes from (less than) a decade ago in a shoe box. A GPU with a bunch of cores can provide an insame amount of processing power for tasks you can do in parallel. A couple of years ago I ventured into crypto currency mining and I was amazed that a relatively cheap video card outperformed a really fast PC by more than 10 times.

[mikeselectricstuff]

IMHO the age of the ASIC based low end scopes is over.

Yes - AFAIK the DSOX1000 is the only ASIC based low-end scope that has come out in a long time, and that's only because Keysight will have recouped their investment on it from the higher end models. I can't imagine it would be viable to do a new ASIC now just for a very crowded low-end market.
 
The requirements for scopes at the low to mid range haven't really changed much in the last ten years, meanwhile processor and memory speeds and costs have continued to move a lot faster.

[David Hess]

Actually designing an oscilloscope (even if only at the block diagram level) isn't what I originally thought I'd be doing here.  But it could prove interesting and instructive.

I ended up revising my block level diagram after the last discussion we had here about minimizing blind time.  I found an acceptable solution for minimizing blind time and maximizing sample rate for a given acquisition record bandwidth and length by decimating into parallel acquisition records of different types (in the same circular memory buffer).

[Someone]

But that doesn't mean you can't create an architecture based on off-the-shelf hardware that comes reasonably close.

It doesn't come close, thats the point. Being one or two orders of magnitude slower might (and is) acceptable for some applications and it isn't for others, just as some applications need very deep memory and some don't, so there are a diversity of products addressing the specific use cases.

The requirements for scopes at the low to mid range haven't really changed much in the last ten years, meanwhile processor and memory speeds and costs have continued to move a lot faster.

Growth in desktop computing has markedly slowed and switched to parallelism to achieve continuing growth:
https://www.karlrupp.net/2015/06/40-years-of-microprocessor-trend-data/
But the I/O throughput hasn't been growing and memory latency has been static for a decade (or two!). Combined with poor power efficiency compared to FPGAs and ASICs, CPUs aren't going to dominate these applications possibly ever. Especially while FPGAs continue their performance growth and improve in compute/energy metrics.

IMHO the age of the ASIC based low end scopes is over. Recently I quoted a project where I opted to do a lot of things in software which where traditionally done in an FPGA. Software development is way cheaper than FPGA or ASIC development in both hours and labour costs (the latter if you need to hire a developer) and for low end scopes you can use off-the-shelve parts which provide extremely good performance. What we'll probably see in the near future are low cost scopes with deep memory which use a GPU for math / signal analysis tasks like FFT, measurements, protocol decoding, etc. Basically the PC based oscilloscopes from (less than) a decade ago in a shoe box. A GPU with a bunch of cores can provide an insame amount of processing power for tasks you can do in parallel. A couple of years ago I ventured into crypto currency mining and I was amazed that a relatively cheap video card outperformed a really fast PC by more than 10 times.

And then coin mining went to ASICs, purely for economic reasons of compute/unit energy. So they still have a place and in desktop appliances it makes sense to keep the power budget down to a practical level rather than trying to cram an off the shelf CPU into it.

[Someone]

We don't have more of the deep memory scopes as the deep memory adds cost and without a fast interface is of limited practical value, people can't/don't/won't pay for a feature they can't/don't/won't use. You can see the memory depth tradeoff in this video:

Sure, the tradeoff is there, but that's beside the point.  You can't create a tradeoff if there's no capability to trade against.  You can't trade FFT resolution for speed if you haven't the memory depth to make that trade in the first place.  So again, it's better to have more memory than to not have it.

If you like that approach then Tektronix do that with their Fastacq mode, you forgo the deep memory to reduce the dead time. But you can't have both unless you throw a lot of expensive hardware at it.

If it was so simple to make deep memory fast, responsive, and low dead time, why aren't there any scopes on the market that offer it? You're welcome to wow us all with you special idea, but continuing to question the current market/products without actually understanding them is running very thin.

[kcbrown]

If you like that approach then Tektronix do that with their Fastacq mode, you forgo the deep memory to reduce the dead time. But you can't have both unless you throw a lot of expensive hardware at it.

Nor am I arguing that you can.  I figured one could get reasonably close (within an order of magnitude) to what you could do with ASICs, but I could easily be wrong about that, and will assume that I am wrong about that from this point forward until I see evidence showing that off-the-shelf is much closer to ASICs for these applications than an order of magnitude or so.  Certainly, the dearth of scopes that get you both without ASICs is a pretty strong argument that ASICs are needed for that.  But which scopes simultaneously get you hundreds of millions of samples and FFT (or other operations that require processing the entire memory buffer) display refresh rates of around 30ms?   That's the kind of scope that would be two orders of magnitude faster than the low end stuff.   You say that ASICs are 1 to 2 orders of magnitude faster than off-the-shelf parts.

But there's a nuance that I didn't explicitly mention that, based on my prior messages, I didn't think I needed to mention.  But it's apparent I do.  What I'm arguing for is the combination of off-the-shelf parts and ASICs (even the lowest end scopes use FPGAs these days), with the latter being used for the parts of the pipeline that need the speed/latency/etc that the former cannot provide.

Now, I'm not arguing that such a combination can get the job done for high-end applications (well, one can expect to see off-the-shelf processors in use for the mundane stuff like the UI, of course, but we're talking about the data acquisition and processing pipeline here), but the original question is why we don't see stuff like DDR3 memory being used at all.

If it was so simple to make deep memory fast, responsive, and low dead time, why aren't there any scopes on the market that offer it? You're welcome to wow us all with you special idea, but continuing to question the current market/products without actually understanding them is running very thin.

Yes, but remember the context of the discussion: low end scopes.   See the first comment in this thread for that context.

We're talking about sample rates of 1-2GS/s.  Not 10GS/s, or 100GS/s.

You'll get no argument from me that if you need sample rates of 10GS/s or higher, and need to maintain 30FPS or so display refresh rates for FFTs and other similar operations that have to use the entirety of the buffer (higher sample rates need more memory just to store the same amount of capture time, so that implicitly means we're talking about a larger buffer to process), then you're going to be using ASICs to achieve that, and the prices will reflect that.


Remember, too, that what I'm looking for is why the use of off-the-shelf DRAM for use as acquisition memory can't work, or isn't used.  Because, as you may recall, I said earlier:

I've learned that if a solution seems blindingly obvious to you (as this seems to be), and people aren't using it, it's due to one of three things:

• You're a genius, and things that are simple and obvious to you just aren't to anyone else (this explanation is the least likely)
• Those who are capable of implementing the solution are so stuck in their ways that they don't want to bother (this seems unlikely also, but possible)
• There's a really good and non-obvious reason that the obvious solution is not being used.

Why haven't designs incorporated off-the-shelf DRAM like what we're talking about, even in the low end scopes?   There has to be a reason, but nobody here has yet explained why it can't be done, or at least why it can't be done economically, even for low-end scopes.   Vague claims about insufficient speed aren't enough here, given that the sequential data transfer rate for modern DDR memory was in the 5+GB/s range as much as 5 years ago, and my raspberry pi is able to do over 1GB/s sequential with a single CPU core.

[nctnico]

The requirements for scopes at the low to mid range haven't really changed much in the last ten years, meanwhile processor and memory speeds and costs have continued to move a lot faster.

Growth in desktop computing has markedly slowed and switched to parallelism to achieve continuing growth:
https://www.karlrupp.net/2015/06/40-years-of-microprocessor-trend-data/
But the I/O throughput hasn't been growing and memory latency has been static for a decade (or two!). Combined with poor power efficiency compared to FPGAs and ASICs, CPUs aren't going to dominate these applications possibly ever. Especially while FPGAs continue their performance growth and improve in compute/energy metrics.

Look at any high end scope: all of them a driven by a PC and let the PC's CPU do the number crunching. It isn't economically viable to convert the algorithms into an ASIC of FPGA because the volumes are too low and modern day CPUs / GPUs offer much better performance per Watt for plain number crunching.

[Someone]

Now, I'm not arguing that such a combination can get the job done for high-end applications (well, one can expect to see off-the-shelf processors in use for the mundane stuff like the UI, of course, but we're talking about the data acquisition and processing pipeline here), but the original question is why we don't see stuff like DDR3 memory being used at all.

There are scopes using DDR for their acquisition memory, across the entire range. But it comes with limitations and they can't just add more capture depth without compromising the user interface (its been done before when Mpts of memory was large and zooming in and out was painfully slow).

Yes, but remember the context of the discussion: low end scopes.   See the first comment in this thread for that context.

We're talking about sample rates of 1-2GS/s.  Not 10GS/s, or 100GS/s.

Low end scopes are the 5-10Gs/s market, if you want to talk entry level then yes, several hundred dollar scopes with 1Gs/s are there but they can't support expensive acquisition systems. Chips of DDR are just a tiny part of what makes deep acquisition memory work, rigol have pushed that out further each year:
https://www.flickr.com/photos/eevblog/8022112817/in/album-72157631618295437/
https://www.skhynix.com/eolproducts.view.do?pronm=DDR2+SDRAM&srnm=H5PS5162GFR&rk=03&rc=consumer
and they work and feel like old tek scopes in many respects with the slow updates and difficulty in accessing the information in the deep memory.

You'll get no argument from me that if you need sample rates of 10GS/s or higher, and need to maintain 30FPS or so display refresh rates for FFTs and other similar operations that have to use the entirety of the buffer (higher sample rates need more memory just to store the same amount of capture time, so that implicitly means we're talking about a larger buffer to process), then you're going to be using ASICs to achieve that, and the prices will reflect that.

You need to get the idea of FPS out of your head, its not a good measure of how well the scope is performing. What is more important is how much data actually gets to the screen, when you're on a long timebase then the update can only be as fast as the acquisition a 1s sweep can only provide a 1s update. What you miss is that most scopes to get the information to the screen stop the acquisition and have a substantial blind time where although the screen may be flickering away quickly at high FPS much of the information captured by the ADC is completely ignored.

FFT is a good example as the computational cost grows with N log N, so its not just a matter of linear scaling to support deeper memory. The processing demands blow up quickly , unlike the display plotting which is proportional to the memory depth multiplied by the output display resolution (where the low latency ram is the limiting factor).

[Someone]

The requirements for scopes at the low to mid range haven't really changed much in the last ten years, meanwhile processor and memory speeds and costs have continued to move a lot faster.

Growth in desktop computing has markedly slowed and switched to parallelism to achieve continuing growth:
https://www.karlrupp.net/2015/06/40-years-of-microprocessor-trend-data/
But the I/O throughput hasn't been growing and memory latency has been static for a decade (or two!). Combined with poor power efficiency compared to FPGAs and ASICs, CPUs aren't going to dominate these applications possibly ever. Especially while FPGAs continue their performance growth and improve in compute/energy metrics.

Look at any high end scope: all of them a driven by a PC and let the PC's CPU do the number crunching. It isn't economically viable to convert the algorithms into an ASIC of FPGA because the volumes are too low and modern day CPUs / GPUs offer much better performance per Watt for plain number crunching.

You don't need to move all the computation to hardware, but scopes are predominantly using FPGAs or ASICs for their front end acquisition memory. Its then a small step to put plotting and/or histogramming and/or measurement/fft acceleration. All the big names are doing it:
http://jaunty-electronics.com/blog/2013/10/look-inside-teledyne-lecroy-hdo4024/
https://www.eevblog.com/forum/blog/eevblog-342-agilent-90000-oscilloscope-teardown/?all
https://www.eevblog.com/forum/other-blog-specific/video-review-and-teardown-of-a-keysight-mso-s-series-10-bit-20gss-oscilloscope/  (15 minutes in)
http://debugmo.de/2013/03/whats-inside-tektronix-dpo5034/
Because the link from acquisition memory to the CPU is a greatly limiting factor in the PC based designs. So preprocessing as much as possible in hardware is an important part of contemporary scope designs.

It isn't economically viable to convert the algorithms into an ASIC of FPGA because the volumes are too low and modern day CPUs / GPUs offer much better performance per Watt for plain number crunching.

Citation needed for the energy efficiency claims, you're standing out on your own with that one.

[nctnico]

Look at the power requirement for the GDS2000E CPU based scope (30W) versus the Keysight DSO1000X ASIC based scope (50W). Because of the economy of scale the first uses a 28nm CPU+FPGA while the second uses a 90nm ASIC. Note how Dave notices the ASIC gets 'hot as hell' in his DSO1000X hack video.

[kcbrown]

There are scopes using DDR for their acquisition memory, across the entire range. But it comes with limitations and they can't just add more capture depth without compromising the user interface (its been done before when Mpts of memory was large and zooming in and out was painfully slow).

That latter makes me wonder why that is, if the speed of the memory is sufficient that it is not imposing a bottleneck on the rest of the processing pipeline.  I know that certain types of operations that you'll ultimately want to build a display for, like FFT, work better if you scale them to cover as much of the sample memory as possible, but I still don't understand why normal UI operations would be compromised by more memory.

For instance, let's say that you've got your display window set to cover the entire buffer (Siglent's current lineup always behaves like this).  There isn't anything that says that you have to process every point in the buffer under those conditions, right?   Or is there?

Low end scopes are the 5-10Gs/s market,

Oh.  I didn't realize there was that distinction.  I was rather under the impression that 5-10GS/s was "midrange".  No matter.  I don't mind using your terminology for this.   

if you want to talk entry level then yes, several hundred dollar scopes with 1Gs/s are there but they can't support expensive acquisition systems.

Why would the acquisition system have to be more expensive if you're using DDR memory for the acquisition buffer, when DDR memory controller designs for FPGAs are widely available these days?

Chips of DDR are just a tiny part of what makes deep acquisition memory work, rigol have pushed that out further each year:
https://www.flickr.com/photos/eevblog/8022112817/in/album-72157631618295437/
https://www.skhynix.com/eolproducts.view.do?pronm=DDR2+SDRAM&srnm=H5PS5162GFR&rk=03&rc=consumer
and they work and feel like old tek scopes in many respects with the slow updates and difficulty in accessing the information in the deep memory.

Can you be more specific in what you mean by the "difficulty in accessing the information in the deep memory"?

You need to get the idea of FPS out of your head, its not a good measure of how well the scope is performing.

No, but it's a good measure of how fluid the display appears to be.  It's a perception thing.  Humans simply do not benefit from a display change rate that exceeds something like 120 times per second (they might be able to see changes that are faster than that, but that matters little when human reaction times are an order of magnitude longer).   And what we're talking about here is an oscilloscope, not a real-time first-person shooter game where a person's ability to react in real time to changes in the display matter.   For oscilloscope display purposes, a display update rate of 30 times/second (which is the very same thing as saying "30 FPS") is perfectly adequate.   That does not mean that the display should be skipping events which occur more quickly than that.  A fast glitch should still be shown, within the persistence parameters that have been set.

And note that I'm talking about display update rates here.  Obviously, if the rate at which the data gets decimated and processed for display is lower than some reasonable minimum (20 times per second?), it'll begin to impact the user's perception of the display at a minimum, and perhaps even of the UI.

There's a lot more than goes into the UI in terms of making it responsive than just the rate at which data on the display is updated.  With the sole exception of the initial amount of time it takes to perform decimation of the acquired data and to get it to the screen after you make a UI change, all of the rest of the lag in the UI is down to the coding of the UI and the speed of the UI processor itself.

What is more important is how much data actually gets to the screen,

Data on the screen is already in decimated form.  So what exactly do you mean by "how much data actually gets to the screen" here?

More to the point, you can't see on the screen more data than the display itself is actually capable of displaying.   If I have a 1000 pixel wide display, that means I can only display 1000 pixels of horizontal information.  Period.   Even if I have a million points of acquisition within the time frame represented by the screen, I can't display all of them.  I can only display a 1000 pixel representation of them.

And that means that I might be able to take shortcuts, depending on the set of operations that are being performed in order to transform the data in the acquisition buffer into the data that is actually displayed.   For instance, for the FFT, I can sample subsets of the acquisition data, and as long as my technique preserves the frequency domain statistical characteristics of the data in the acquisition buffer, what lands on the screen will be a reasonable representation of the FFT, right?  Moreover, I can't display more than 1000 discrete frequency bins from the FFT at any given time, either, precisely because my display is limited to 1000 pixels of width.

The intensity grading mechanism is one of the mechanisms where you can't get away with subsampling, though, but at the same time, the transforms required for it are very simple and, more importantly, because the target of the transforms is the display, it means the target buffer can be relatively small, which means it can be very fast.

when you're on a long timebase then the update can only be as fast as the acquisition a 1s sweep can only provide a 1s update.

Of course.  Nobody is arguing otherwise here.  But that's not the situation we're talking about, is it?   We're talking about the situation where something in the processing pipeline is a bottleneck, i.e. one where if the processing pipeline were faster, then the display update rate would be faster.

What you miss is that most scopes to get the information to the screen stop the acquisition and have a substantial blind time where although the screen may be flickering away quickly at high FPS much of the information captured by the ADC is completely ignored.

Okay.  And why is that?

Why can't the scope continue to acquire samples unless the triggering mechanism can't keep up?

FFT is a good example as the computational cost grows with N log N, so its not just a matter of linear scaling to support deeper memory. The processing demands blow up quickly , unlike the display plotting which is proportional to the memory depth multiplied by the output display resolution (where the low latency ram is the limiting factor).

So you wind up with a limited number of points in the FFT's baseline.  That doesn't mean you have to sacrifice acquisition memory to get that.   You can just as easily subsample the acquisition memory to build your FFT, right?  The original data remains there to be examined in other ways if you want.

More memory gets you more flexibility.   How is it ever a downside when at worst you can always pretend that you have less memory to work with?



I'm really going to have to work up a block diagram of what I have in mind here.  I can hardly believe someone else hasn't already thought of the overall architecture I have in mind.

[Someone]

Look at the power requirement for the GDS2000E CPU based scope (30W) versus the Keysight DSO1000X ASIC based scope (50W). Because of the economy of scale the first uses a 28nm CPU+FPGA while the second uses a 90nm ASIC. Note how Dave notices the ASIC gets 'hot as hell' in his DSO1000X hack video.

So an ASIC approaching end of life is a fair comparison to a current CPU+FPGA, also note FPGA

It isn't economically viable to convert the algorithms into an ASIC of FPGA because the volumes are too low and modern day CPUs / GPUs offer much better performance per Watt for plain number crunching.

FPGAs are more power efficient than a CPU for most tasks, they might cost more to develop for or manufacture but power efficiency is one of their key benefits along with the high throughput. And an ASIC is more power efficient again.

It isn't economically viable to convert the algorithms into an ASIC of FPGA because the volumes are too low and modern day CPUs / GPUs offer much better performance per Watt for plain number crunching.

Citation needed for the energy efficiency claims, you're standing out on your own with that one.

Citation still needed.

[kcbrown]

Look at the power requirement for the GDS2000E CPU based scope (30W) versus the Keysight DSO1000X ASIC based scope (50W). Because of the economy of scale the first uses a 28nm CPU+FPGA while the second uses a 90nm ASIC. Note how Dave notices the ASIC gets 'hot as hell' in his DSO1000X hack video.

That's not a fair comparison, because it's comparing a modern design and implementation against one which is several years old.  The question is whether an ASIC of the same technology level as a processor-based implementation will have better performance/watt characteristics.