Audio sample rate and latency in live sound.

[Dan Moos]

I'm in a discussion on an audio engineer's group. The majority are stating that audio sampled at 96k will result in lower latency through the system compared to 48k samples. 

The biggest reason they state is that the dsp is waiting for a specific number of samples before it can work,  and thus, 96k audio "fills the buffer faster".

To me, this sounds like a fundamental misunderstanding of what "sample rate means". It also implies that DSPs are LESS efficient with LESS data, which also doesn't sound right to me. 

But they are claiming the specs on their equipment specifically state less latency at 96k vs 48k. I have no reason to not believe them, so now I'm confused.

Anyone with inside knowledge wanna help me understand?

[John B]

Sounds about right. For computer + interface setups you have an input sample buffer + processing time + output sample buffer. Faster sample rate represents a smaller period of time.

[Dan Moos]

We are talking about live, real time stuff.

So DSPs really wait for X number of samples before going to work? Regardless of how fast they are coming? So a native 48k system wouldnt have this latency issue?

[Ed.Kloonk]

The biggest reason they state is that the dsp is waiting for a specific number of samples before it can work,  and thus, 96k audio "fills the buffer faster".

The problem with the story is that you normally set a buffer size for a duration not necessarily a arbitrary number of samples. The reason memory buffers are used at all is (usually) the CPU and the IO speeds don't always align and the buffer helps alleviate the mismatch of timings.

With DSP's however, they do need a sample packet to examine the incoming waveform. Though the newer tech needs much less and can do much more.

My question is: if you can fit enough samples in a 96k buffer to complete the task without problems, why then can't a 48k buffer do the same work in the same amount of time?

Perhaps the software is configured wrong or your ppl aren't fully understanding the point of buffering.

[T3sl4co1l]

More or less an undefined problem as it could simply be whatever.  And likely a concern not rooted in reality, as delays of some ~ms are very difficult to perceive in the first place.

On multitasking OS, typically the buffers will be sized for some ms.  And however many samples fit in there, is whatever fits in there.  It could be 1024 samples, it could be 1 or 10ms...  It could even be buffered in multiple places (input buffers, multiple stages of sound system, output buffers..), impossible to tell without insight into all the drivers involved.  Not to mention format and rate conversion; the internal representation might be a different sample rate, floating or fixed point, etc.

Tim

[SiliconWizard]

We are talking about live, real time stuff.

So DSPs really wait for X number of samples before going to work?

Yes of course, it's always buffered. Both transfering AND processing samples one at a time instead of by packets is highly inefficient.

You basically have a double-buffering scheme. While one buffer is being transfered with data, the other is being processed - same for the output, but in reverse.
The smaller the buffer size, and the higher the overhead, to the point of spending more processing time in overhead than actual processing.

A purely hardware implementation - such as on FPGA - could potentially be designed to be able to process one sample at a time, but for any software-based solution, that's not practical.

[John B]

We are talking about live, real time stuff.

So DSPs really wait for X number of samples before going to work? Regardless of how fast they are coming? So a native 48k system wouldnt have this latency issue?

There is no "real time" in this case, there is always a processing time. Even digital mixers, audio FX etc will list a latency time. Somewhere around 2ms.

My computer and interface setup achieves around 6ms latency, so totally suitable for real time usage.

[Someone]

So DSPs really wait for X number of samples before going to work? Regardless of how fast they are coming? So a native 48k system wouldnt have this latency issue?

Its not inherent to DSPs, but rather the algorithm implemented. A lazy programmer might make it work for 96k (with some fixed buffer/block depth) and then for 48k just rescale the parameters, at which point yes, a specific device might have more latency at 48k than at 96k.

[Someone]

A purely hardware implementation - such as on FPGA - could potentially be designed to be able to process one sample at a time, but for any software-based solution, that's not practical.

Sounds like an answer from someone who hasn't used DSPs, they are entirely suitable for low latency sample-by-sample processing. Short pipelines with  deterministic IO and memory access differentiating them from general purpose processors.

[jonpaul]

Bonjour, was design and consulting on RT SA, digital audio interfaces  in 1970s...1990.

a few precisions on the discussion, as seems some confusion:

Standard FS rates are 44.1, 48, 96, 192 and 384 kHz. The rates have no direct connection to latency.

Real time mean the rate of flow of information is same at the system input and output regardless of latency.

The latency depends on the processing time and DAC/ADC latency, plus any buffers. The time will be proportional to the clock period, so in the same system as you change clock freq of the entire system, the latency will change proportionally. As must DSP operate on a fixed clock and the ADC/DAC on multiple of FS the calculation of latency is a bit more difficult.

Main benefit of FS over 48 kHz is use of gentle LPF (instead of "brick wall") to avoid aliasing, eg oversampling FIR digital filters used in oversampling ADCs.

Hope this is interesting,

Bon Chance,

Jon

[mag_therm]

We have some issues here.
1)
I have a vintage receiver set up with a demodulator I just built for FT8 , feeding via the hardware pch card
 to the wsjt-x app. Latency is no problem for ft8 !

But I  use same set-up for short wave listening phone (ssb voice) on 20 metre etc bands.
The calf audio filter sent to playback outputs is quite effective for the QRN (noise) on 80m and 20 m
The audio is so slow that it impedes the ability to manually tune in a ssb station.
Receiver's hardware audio is off.
here is the linux desktop set up with jackctl:
https://app.box.com/s/j7et0nz1dtqj22jlpsmj1kn1ninyk3xv

2)
My son , 16 , is a budding musician.
We tried to do a multi-track recording to Audacity using an internet rhythm track.
But latency problem, maybe need 2 computers, one feeding the off-internet into the mixer?
I read (search) that musicians can't tolerate latency. One mentions about 8 ms, others mentioned lower threshold.

[Siwastaja]

+1 to "it depends". There is no such fundamental rule at all, but do remember that the guys claiming that might have practical experience about some certain products that behave the way they suggest. "Programming laziness" would be a good description. After all, you can pretty much achieve the same latency regardless of the sample rate, but that might require a tad of extra work compared to just minimizing latency on one (recommended) sample rate and then make the rest more or less "just work somehow".

Buffers are needed to do processing in chunks without audible glitches, and this is all measured in time. And as you say in opening post, doing the same processing at lower fs is actually faster, so one could theoretically reduce the chunk length in time; i.e., quadratically reduce the chunk length in number of samples. But OTOH, some lazy design could use fixed buffer size in number of sampled, not time.

[coppice]

How long does it take a sample to come out of an ADC? In the case of something like a SAR ADC is usually one sample time. However, all the really good audio ADCs are sigma-delta types. Those have considerable latency. The basic conversion process gives you several samples of delay, but that's the tip of the iceberg. If you want a fairly quick result from a sigma delta converter to have to accept a heavily rolled off frequency response. If you want a flat frequency response, as in audio, you need a further stage of filtering, which causes many samples of additional delay in the result. Try looking at a range of sigma delta converters, and you will usually see 2 types - low latency, rolled off response ones, suitable for applications where low latency is essential, and flat response high latency. Whichever type you get, the latency is way more than half a 48ksps sample, so using 96ksps is irrelevant to the overall delay.

I've talked with chip designers who have looked at making a single sigma delta converter chip with both a low latency rolled off response, and a high latency flat response type of output. Lots of applications require that. There are many industrial (typically power) applications where a device need to provide a low latency protection signal, while also making precise wideband measurements. I don't know if any have come to market.

[gf]

My son , 16 , is a budding musician.
We tried to do a multi-track recording to Audacity using an internet rhythm track.
But latency problem, maybe need 2 computers, one feeding the off-internet into the mixer?
I read (search) that musicians can't tolerate latency. One mentions about 8 ms, others mentioned lower threshold.

AFAIK, you can adjust latency in Audacity (and in other multi-track recording programs), in order the the tracks being played back and the newly recorded track are in sync then.

[gf]

...the latency is way more than half a 48ksps sample, so using 96ksps is irrelevant to the overall delay.

I looked at the datasheet of an arbitrarily chosen audio ADC.
It certainly does make a difference whether we get a latency of 17.1 samples @48kSa/s = 356µs, or 7.2 samples @384 kSa/s = 19µs (both with the regular linear phase decimation filter).
These numbers do not confirm that a higher sample rate were irrelevant. At 384 kSa/s, the cut-off frequency of the decimation filter is simply 8x higher, which enables a filter with significantly lower group delay. But the whole real-time processing chain needs to run at the higher sample rate then. Any decimation to a lower sample rate in the procesing chain introduces additional delay due to the required decimation filter, which defeats the latency advantage of the higher ADC sample rate again.

How long does it take a sample to come out of an ADC? In the case of something like a SAR ADC is usually one sample time.

Don't forget the group delay of the analog anti-alising filter which is required in front of the SAR ADC. Again, this group delay depens on the filter cut-off frequency dictated by the sample rate. At the end you have to spend latency either for the analog anti-alising filter, or for the digital decimation filter in the Sigma-Delta ADC. There is no free lunch.

[coppice]

...the latency is way more than half a 48ksps sample, so using 96ksps is irrelevant to the overall delay.

I looked at the datasheet of an arbitrarily chosen audio ADC.
It certainly does make a difference whether we get a latency of 17.1 samples @48kSa/s = 356µs, or 7.2 samples @384 kSa/s = 19µs (both with the regular linear phase decimation filter).
These numbers do not confirm that a higher sample rate were irrelevant. At 384 kSa/s, the cut-off frequency of the decimation filter is simply 8x higher, which enables a filter with significantly lower group delay. But the whole real-time processing chain needs to run at the higher sample rate then. Any decimation to a lower sample rate in the procesing chain introduces additional delay due to the required decimation filter, which defeats the latency advantage of the higher ADC sample rate again.

Even if you don't decimate, and run the processing at 384ksps, most filtering introduces a substantial latency. People love using massive impulse response filters, and those can introduce considerable latency. Causality is a harsh mistress.

[gf]

Even if you don't decimate, and run the processing at 384ksps, most filtering introduces a substantial latency. People love using massive impulse response filters, and those can introduce considerable latency. Causality is a harsh mistress.

Indeed. But then you can blame the particular "processing" steps . A simple mixer (w/o any tone controls) would not harm ¹⁾. But OTOH, people which complain about latency of digital filters do forget that analog filters don't come without group delay either.

Edit: ¹⁾...granted that the clocks of all sources are in sync (which is hopefully granted in a studio environment). ASRC had again some overhead.

[Marco]

It's not like DAWs work on real time operating systems with 1 cycle or less latency ... how is it relevant except for stuff like stomp boxes? Even then, the musician isn't going to notice the difference.

A massive FIR reverb with <1 cycle algorithmic latency for output from the the first sample of the impulse can be implemented by anyone now, with a small FIR filter and FFT based filters increasing geometrically in size, the important Lake DSP patents have all expired.

[coppice]

Even if you don't decimate, and run the processing at 384ksps, most filtering introduces a substantial latency. People love using massive impulse response filters, and those can introduce considerable latency. Causality is a harsh mistress.

Indeed. But then you can blame the particular "processing" steps . A simple mixer (w/o any tone controls) would not harm ¹⁾. But OTOH, people which complain about latency of digital filters do forget that analog filters don't come without group delay either.

Of course. Buffering delays are unique to a digital implementation, but filter delays are baked into the maths. It doesn't matter if the implementation is analogue or digital.

[SiliconWizard]

Of course, delays are inevitable. That's basic physics.

As to digital audio, which is discrete, latency will always be a multiple of the sampling period. So that's the link between latency and sampling rate, the rest depending entirely on the implementation, of course. Obviously, a given latency in a given number of samples will be shorter if the sample rate is higher, but that ends there.

On computer audio, it was very common to have either a fixed buffering in number of samples, or user-selectable, but only among a limited number of options. For instance, old DAW software frequently had minimum buffer sizes of 256 samples, or so. In this case, it's obvious the minimum latency would be reduced if sampling rate was higher. Modern audio software on modern OSs have much better latency usually, due to the audio subsystems in general-purpose OSs being much better than they used to, scheduling being better, data throughput being higher, and so on. So the point holds a lot less these days - the limiting factor will be the inherent latency in the OS scheduling, and data throughput, not the buffer sizes per se.

And of course that's just about the latency of audio without any processing. Further processing can add additional delays, thus "latency".

But in the extreme case for which you can achieve a one-sample latency only, then the sampling rate will definitely dictate the latency. In all other cases... it just depends.

[coppice]

As to digital audio, which is discrete, latency will always be a multiple of the sampling period. So that's the link between latency and sampling rate, the rest depending entirely on the implementation, of course. Obviously, a given latency in a given number of samples will be shorter if the sample rate is higher, but that ends there.

Where did you get that idea? The latency of many ADCs works out to be something and a half samples, due to the way the comb filtering works. Depending on the nature of the response flattening filter, it might be any fraction, but something and a half is quite common.

[Bassman59]

As to digital audio, which is discrete, latency will always be a multiple of the sampling period. So that's the link between latency and sampling rate, the rest depending entirely on the implementation, of course. Obviously, a given latency in a given number of samples will be shorter if the sample rate is higher, but that ends there.

Where did you get that idea? The latency of many ADCs works out to be something and a half samples, due to the way the comb filtering works. Depending on the nature of the response flattening filter, it might be any fraction, but something and a half is quite common.

This is correct! Look at the data sheet for, say, the PCM4202 ADC. It specifies group delay as 9.5/fs seconds. This tells us that the filter delay is fixed and the absolute time through it scales with sample rate. At 48 kHz the delay is 198 microseconds. At 192 kHz the delay is 49.5 microseconds.

This latency is, of course, utterly swamped by acoustic times-of-flight and is certainly minimal when compared with processing latency. Of course it does matter if you have multiple coherent sources, perhaps like two microphones in X/Y capturing "stereo."

[David Hess]

This is correct! Look at the data sheet for, say, the PCM4202 ADC. It specifies group delay as 9.5/fs seconds. This tells us that the filter delay is fixed and the absolute time through it scales with sample rate. At 48 kHz the delay is 198 microseconds. At 192 kHz the delay is 49.5 microseconds.

That is because the fixed length filter tracks the Nyquist frequency for noise shaping and anti-aliasing.  If the filter was tied to a fixed frequency, then doubling the sample rate would require twice as many samples to be processed and the absolute latency would not change.

[SiliconWizard]

Absolutely. But in common sigma-delta ADCs, that's proportional to the sampling period as show above. So, one can argue until the cows come home, but the point was to explain the OP in simple concepts in what ways "latency" and sampling rate are related in typical systems. (And in what ways they wouldn't be.)

(Note that in sigma-delta converters, the sampling rate for the modulators and decimation filters is a multiple of the output sampling rate - oversampling.)

Then there's the point of defining "latency" in a *discrete* system, and equating it to the group delay is an interesting debate. Group delay can be fractional (but still proportional to the sampling period for a *given* filter structure), but you can't get a sample before it's ready to be read. So that opens a can of worms depending on your definition of latency and, from there, what you are going to use this figure for. That point just to open the question of defining "latency" in a discrete system.

[gf]

Group delay can be fractional (but still proportional to the sampling period for a *given* filter structure)

And it can be constant (-> linear phase FIR filter), or frequency-dependent (-> IIR filter or analog filter).