-
I ask this question because Windows/Linux/IOS/Android all have a 10ms or thereabouts latency when it comes to doing real-time audio processing. This 10ms figure goes back years and hasn't improved with successive generations of hardware and operating system upgrades. If anything it's actually got worse in recent times.
When recording live music this 10ms delay makes it difficult or impossible to to do real-time monitoring as the musician gets perturbed by the lag which affects their timing. The only option is to bypass the computer for monitoring, but then you have to listen to the recording afterwards to check for clipping or glitches due to buffer underruns. -
One problem for real time operation on the PC is the virtual memory. Though not really needed anymore the relevant program might worst case be somewhere in swap space.
With windows a 10 ms worst case delay would be nice, when switching programs it sometimes feels like several seconds
-
Beaglebone blacks do and several other arm chipsets have mcu`s or fpga`s integrated.
Mostly pcs expect you to plug such things into the usb or pcie channels when you want them. -
Isn't it that some high end sound cards have DSPs to act as the "real time" system as far as audio is concerned?
-
In the old IBM PC standard, the resolution was about 17ms, from the date/time clock registers system.
Today PCs have many High Resolution Timers
https://docs.microsoft.com/en-us/windows/desktop/sysinfo/acquiring-high-resolution-time-stamps#hardware-timer-info
1 microsecond is not a problem for Windows. For low latency, you may want to look for ASIO drivers (less than 1ms latency). Not all producers provide ASIO drivers, so choose a sound card with ASIO drivers.
-
The original IBM PCs had near non-existent audio, even compared to other computers of that era it's pathetic.
Sound cards were an afterthought, both in hardware and software.
I wonder if the 10ms latency comes from IRQ polling, or is a byproduct of the sample rate. -
You need a Real Time OS (like QNX) on top of the hardware. Windows/Linux are not RT OS.
-
When recording live music this 10ms delay makes it difficult or impossible to to do real-time monitoring as the musician gets perturbed by the lag which affects their timing.
I don't see the problem. 10ms an order of a magnitude is faster than the reaction time of a human. It's also quicker than anyone can physically press a button or key. Take a digital stop watch, reset it to zero and press the start/stop button twice, as quickly as possible. If you can start and stop the clock in under 0.03s, that's 30ms, then you're doing well.QuoteSimple reaction time is the motion required for an observer to respond to the presence of a stimulus. For example, a subject might be asked to press a button as soon as a light or sound appears. Mean RT for college-age individuals is about 160 milliseconds to detect an auditory stimulus, and approximately 190 milliseconds to detect visual stimulus. The mean reaction times for sprinters at the Beijing Olympics were 166 ms for males and 189 ms for females, but in one out of 1,000 starts they can achieve 109 ms and 121 ms, respectively. This study also concluded that longer female reaction times can be an artifact of the measurement method used, suggesting that the starting block sensor system might overlook a female false-start due to insufficient pressure on the pads. The authors suggested compensating for this threshold would improve false-start detection accuracy with female runners.
https://en.wikipedia.org/wiki/Mental_chronometry#Types
-
Your ear can detect a 10ms lag in sound. If you play a note on, say, your guitar and hear it being played 10ms later, that confuses your sense of rhythm.
On my computers I have to set the sound cards to play back the raw input signal directly to the headphones, bypassing all the sound processing, otherwise it gets impossible to play or sing.
-
This is still why I own a dedicated keyboard (ancient Korg Triton). Latency does my brain in.
That’s 10ms through the PC. There is external latency as well which can amount to a few ms like outboard DSP etc. -
I don't see the problem.
It's a real problem. The AES did a study which gave numbers as low as 3ms affecting vocalists.
https://community.bose.com/ziiiw76728/attachments/ziiiw76728/Hearphones/594/1/AES_Latency.pdf
10ms is annoying for percussive or plucked instruments. -
1 microsecond is not a problem for Windows. For low latency, you may want to look for ASIO drivers (less than 1ms latency). Not all producers provide ASIO drivers, so choose a sound card with ASIO drivers.
Thanks, I was wondering what that sound-driver feature was in my older Audacity on my old laptop. It seemed to have no effect on my laptop audio other than the local digital loop-back recording input selection was missing. Now I have something I can measure on my scope to verify.
My new PC only has Host: MME or Direct Sound, or Windows WASASPI, while the 'recording' settings have the feature of 'Software Playthrough Of Input'.
-
I don't see the problem.
It's a real problem. The AES did a study which gave numbers as low as 3ms affecting vocalists.
https://community.bose.com/ziiiw76728/attachments/ziiiw76728/Hearphones/594/1/AES_Latency.pdf
10ms is annoying for percussive or plucked instruments.
I'm surprised that the Saxophone is the most latency sensitive instrument by far, even above vocals. Even 1ms delay with a saxophone is already beginning to effect the musician. When I was young, I once played the Alto Saxophone, and today, thanks to that, I'm super sensitive to lip-sync errors in AV equipment to the point that most divx videos give me a headache. Only direct Bluray movies, played on a PC through a 150hz CRT really give me that synchronous visual to audio life experience I missed from the days of uncompressed all analog studio AV equipment. (Modern big screen TV as a monitor suck shit with a huge video lag.) I'm also super sensitive to frame drops and other minute AV artifacts which seem to blow over everyone else, but dead obvious to me. Don't know why. It's like no one is paying attention to what they are seeing and hearing, especially on the really bad stuff until I point it out to them, tell them to count the syllables in their head as they hear them watching the lips of the on screen actor that they realize the lips are off by half or a full syllable.
Thankfully on the PC, I found video players which allow me to re-sync the audio, but, it is a headache as each video have their offset error unless it is an uncompressed blu-ray original. Youtube's and NetFlix's sound really pisses me off...
-
When I was young, I once played the Alto Saxophone, and today, thanks to that, I'm super sensitive to lip-sync errors in AV equipment to the point that most divx videos give me a headache.
Video-audio sync issues drive me insane as well. That is one of the things I like about watching most of my content through a MythTV frontend, instant fixing of the sync on the fly at the touch of a key.QuoteThankfully on the PC, I found video players which allow me to re-sync the audio, but, it is a headache as each video have their offset error unless it is an uncompressed blu-ray original. Youtube's and NetFlix's sound really pisses me off...
I too, wish that YouTube's video player had an audio sync adjust. Drives me nuts!
-
I feel your pain, multiplied by a 120inch video projected screen.QuoteThankfully on the PC, I found video players which allow me to re-sync the audio, but, it is a headache as each video have their offset error unless it is an uncompressed blu-ray original. Youtube's and NetFlix's sound really pisses me off...
I too, wish that YouTube's video player had an audio sync adjust. Drives me nuts!
-
What would be the relevance of any of that to a musician playing along with something? A musician doesn't simply react to what is being played, for the very reasons you described. They predict, and tune their predictions as the pace of the music changes. Good musicians can match each other's timing to a very fine degree, or twist those timings for effect. A more relevant measure of the significance of a delay is what that represents in terms of the spacing of the musicians. 10ms is like moving about 3m (or 1.5m if you are considering a round trip).When recording live music this 10ms delay makes it difficult or impossible to to do real-time monitoring as the musician gets perturbed by the lag which affects their timing.
I don't see the problem. 10ms an order of a magnitude is faster than the reaction time of a human. It's also quicker than anyone can physically press a button or key. Take a digital stop watch, reset it to zero and press the start/stop button twice, as quickly as possible. If you can start and stop the clock in under 0.03s, that's 30ms, then you're doing well.QuoteSimple reaction time is the motion required for an observer to respond to the presence of a stimulus. For example, a subject might be asked to press a button as soon as a light or sound appears. Mean RT for college-age individuals is about 160 milliseconds to detect an auditory stimulus, and approximately 190 milliseconds to detect visual stimulus. The mean reaction times for sprinters at the Beijing Olympics were 166 ms for males and 189 ms for females, but in one out of 1,000 starts they can achieve 109 ms and 121 ms, respectively. This study also concluded that longer female reaction times can be an artifact of the measurement method used, suggesting that the starting block sensor system might overlook a female false-start due to insufficient pressure on the pads. The authors suggested compensating for this threshold would improve false-start detection accuracy with female runners.
https://en.wikipedia.org/wiki/Mental_chronometry#Types -
Now you understand why orchestras have conductors.
-
Isn't it that some high end sound cards have DSPs to act as the "real time" system as far as audio is concerned?
Apparently yes, although the manufacturers do a poor job of explaining the technical details. With these hybrid devices they don't tell you how much processing is done on the PC vs what's done in the box. Here's a picture of the interior of a $900 audio interface.
-
There is often confusion with regards to latency between simple "reaction times", audio/video sync, single source latency (like the delay between the timer on a music player and the actual sound) and musician latency - where someone creates the sound, and there is latency between that and hearing it, like with guitar effects, or using midi instruments. All of these have different latency tolerances, and then of course there is individual tolerance to latency.
Which one are we talking about in this thread?
In terms of "single source" latency really doesn't matter - which is why MP3's still rule even though they require a fairly large buffer, and therefore high latency. Because there is only one source of audio, there is nothing to compare it to, so latency can be seconds. It is also why MP3 can be used to record, but not for real-time transmission, regardless of how fast a processor one uses.
For audio/video sync that tolerance is different because it involves both visual and audible stimulus. I have no hard figures on this, but I vaguely remember reading about bluetooth audio causing sync problems with its stock codec with ~150ms latency, and that <~40ms is required for imperceptible sync.
For instances where one is *creating* the sound, as in singers, drummers, guitarist, musicians etc.. the figure is generally even lower, being roughly 20ms. Perception of delay here involves actually physical movements to audible sounds, again a different route in the brain. I am totally unsure of this as I have used guitar effects for years, and even in the 90's where the latency was >40ms I didn't notice it. Also, as coppice rightly pointed out, 10ms is roughly 3.5m distance - which is easily how far a guitarist can be away from his amp. I think 10ms isn't unreasonable for a delay. Even with ASIO midi drivers on my PC I get about 15ms.
Also people forget about buffers. Digital processes generally require a certain "block" of audio to work with (like MP3 mentioned above). So it doesn't matter how fast your processor is, if a process requires a chunk of 2k samples, at 48ksps, thats ~42ms buffer latency right there. This is why modern real-time audio compression codecs work on smaller chunks of audio, and also why some resort to higher sample rates even though there isn't much different in perceived quality going beyond 48k - it halves the latency.
Sadly, any topics surrounding perception, or subjective things like audio latency, or quality tend to get quite heated, mostly because what you "hear" is also about expectations
You can adjust how much "latency" someone hears just by telling them numbers.
-
I don't see the problem. 10ms an order of a magnitude is faster than the reaction time of a human.
The problem is real, especially when someone plays an instrument and the sound does not come when the brain expects it.
If the delay is too big, even the speech is impaired severely. Just for the fun of it:
-
I noticed that with the virtual MIDI on USB only keyboards.
Not a set delay but random delays between the key press.
I hadn't noticed with midi on the Yamaha SW1000XG and the Creative EM-U but I have in the past with some software midi synthesizers. -
And THAT is the real trap small amounts of latency are usually not a problem for a keyboard player, but VARIABLE latency is a nightmare.
You really need to be timestamping the arriving MIDI with the sound cards current sample number and then generating the resulting note a defined (and constant) number of samples later.
PCs actually can do reasonably low latency, but as ever it comes at the expense of thru-put and power, basically if you pin your job to one processor and just spin in a polling loop you can get wonderfully low latencies, but nobody likes a process that does that, even if the box is not being used for much else.
Speed of sound is roughly 1ms per foot, so that gives you some idea of where the limits are likely to be. -
I noticed that with the virtual MIDI on USB only keyboards.
Not a set delay but random delays between the key press.
I hadn't noticed with midi on the Yamaha SW1000XG and the Creative EM-U but I have in the past with some software midi synthesizers.
If you're using PC based plugins, then the variable latency is likely from the ASIO output, rather than processing the MIDI commends themselves. Years ago when I was recording, I of course needed real-time feedback from the synth, which had a slight delay that was off-putting (infuriating) but playing back the MIDI sounded fine. You can get custom ASIO drivers and set the latency to something more sensible like 20ms. This is fixed, and as long as your system can cope (it will) it won't vary. -
The trap is that the MIDI arriving from the keyboard is async with respect to the buffer refill callback from the sound card, so if you do the naive thing and have the callback process all the complete midi messages available at the time it runs, then you get anything up to a buffers worth of random timing variation.
It took a while for sound cards that could time stamp the MIDI with their buffer position to arrive, and then longer for software that knew how to use this to give **CONSTANT** latency to hit the scene.
20ms that is constant is MUCH less of a problem then 10-20ms that varies from note to note.
Regards, Dan. -
Back to reasoning as to why audio latency is so poor: because it's not a well known issue, and it's often easier for developers if they don't care about it.
Case in point: pulseaudio. It runs in userland like any other process and shims itself between the kernel's audio system (ALSA) and your programs. The only way it can do this reliably is by introducing more latency. This allows it lots of time to apply effects and features whilst not requiring any special privileges or changes to your kernel. Sidenote: pulseaudio sucks for many reasons, avoid it if you can, latency is only one of its common issues.
A similar situation exists with many laptop touchpad input devices. I've found some OEM windows drivers make using them intolerably laggy. Even under Linux I can "feel" the delay in touchpads. Perhaps it is slow filtering, perhaps it is video delay, but it's definitely something that could be solved if people cared more about it. It's easier not to.
Keyboards are an interesting case. I had to throw away a PS2 keyboard years ago because it had a noticeable delay. Compared to another PS2 board it was horrible. Might have been the physical mechanism instead of the electronics (anti-bounce?), not sure.QuoteWhy don't desktop computers have realtime subsystems?
The closest we can get (without dropping a modern OS) is a kernel that lets us cheat the normal process scheduler.
In the Linux world there are projects like Jack with realtime kernel scheduling support to shave the milliseconds. Best I've ever run this at is 22ms internal latency (the number Jack itself reports, ignoring I/O and peripherals) but I don't run anything particularly modern. Very popular project for composers and other routing needs, I recommend the program qjackctl as a frontend.