EEVblog Electronics Community Forum

General => General Chat => Topic started by: e100 on November 10, 2018, 01:03:04 pm

Title: Why don't desktop computers have realtime subsystems?
Post by: e100 on November 10, 2018, 01:03:04 pm
I ask this question because Windows/Linux/IOS/Android all have a 10ms or thereabouts latency when it comes to doing real-time audio processing. This 10ms figure goes back years and hasn't improved with  successive generations of hardware and operating system upgrades. If anything it's actually got worse in recent times.

When recording live music this 10ms delay makes it difficult or impossible to to do real-time monitoring as the musician gets perturbed by the lag which affects their timing. The only option is to bypass the computer for monitoring, but then you have to listen to the recording afterwards to check for clipping or glitches due to buffer underruns.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: Kleinstein on November 10, 2018, 01:13:07 pm
One problem for real time operation on the PC is the virtual memory. Though not really needed anymore the relevant program might worst case be somewhere in swap space.

With windows a 10 ms worst case delay would be nice, when switching programs it sometimes feels like several seconds  :-- |O
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: rjp on November 10, 2018, 01:21:50 pm
Beaglebone blacks do and several other arm chipsets have mcu`s or fpga`s integrated.

Mostly pcs expect you to plug such things into the usb or pcie channels when you want them.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: NiHaoMike on November 10, 2018, 02:15:12 pm
Isn't it that some high end sound cards have DSPs to act as the "real time" system as far as audio is concerned?
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: RoGeorge on November 10, 2018, 02:29:34 pm
In the old IBM PC standard, the resolution was about 17ms, from the date/time clock registers system.

Today PCs have many High Resolution Timers
https://docs.microsoft.com/en-us/windows/desktop/sysinfo/acquiring-high-resolution-time-stamps#hardware-timer-info

1 microsecond is not a problem for Windows.  For low latency, you may want to look for ASIO drivers (less than 1ms latency).  Not all producers provide ASIO drivers, so choose a sound card with ASIO drivers.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: station240 on November 10, 2018, 02:37:29 pm
The original IBM PCs had near non-existent audio, even compared to other computers of that era it's pathetic.
Sound cards were an afterthought, both in hardware and software.

I wonder if the 10ms latency comes from IRQ polling, or is a byproduct of the sample rate.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: TK on November 10, 2018, 02:51:25 pm
You need a Real Time OS (like QNX) on top of the hardware.  Windows/Linux are not RT OS. 
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: Zero999 on November 10, 2018, 02:55:32 pm
When recording live music this 10ms delay makes it difficult or impossible to to do real-time monitoring as the musician gets perturbed by the lag which affects their timing.
I don't see the problem. 10ms an order of a magnitude is faster than the reaction time of a human. It's also quicker than anyone can physically press a button or key. Take a digital stop watch, reset it to zero and press the start/stop button twice, as quickly as possible. If you can start and stop the clock in under 0.03s, that's 30ms, then you're doing well.
Quote
Simple reaction time is the motion required for an observer to respond to the presence of a stimulus. For example, a subject might be asked to press a button as soon as a light or sound appears. Mean RT for college-age individuals is about 160 milliseconds to detect an auditory stimulus, and approximately 190 milliseconds to detect visual stimulus. The mean reaction times for sprinters at the Beijing Olympics were 166 ms for males and 189 ms for females, but in one out of 1,000 starts they can achieve 109 ms and 121 ms, respectively. This study also concluded that longer female reaction times can be an artifact of the measurement method used, suggesting that the starting block sensor system might overlook a female false-start due to insufficient pressure on the pads. The authors suggested compensating for this threshold would improve false-start detection accuracy with female runners.
https://en.wikipedia.org/wiki/Mental_chronometry#Types
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: bsfeechannel on November 10, 2018, 03:08:39 pm
Your ear can detect a 10ms lag in sound. If you play a note on, say, your guitar and hear it being played 10ms later, that confuses your sense of rhythm.

On my computers I have to set the sound cards to play back the raw input signal directly to the headphones, bypassing all the sound processing, otherwise it gets impossible to play or sing.

Title: Re: Why don't desktop computers have realtime subsystems?
Post by: bd139 on November 10, 2018, 03:10:26 pm
This is still why I own a dedicated keyboard (ancient Korg Triton). Latency does my brain in.

That’s 10ms through the PC. There is external latency as well which can amount to a few ms like outboard DSP etc.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: e100 on November 10, 2018, 03:11:05 pm
I don't see the problem.

It's a real problem. The AES did a study which gave numbers as low as 3ms affecting vocalists.
https://community.bose.com/ziiiw76728/attachments/ziiiw76728/Hearphones/594/1/AES_Latency.pdf (https://community.bose.com/ziiiw76728/attachments/ziiiw76728/Hearphones/594/1/AES_Latency.pdf)
10ms is annoying for percussive or plucked instruments.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: BrianHG on November 10, 2018, 03:20:31 pm
1 microsecond is not a problem for Windows.  For low latency, you may want to look for ASIO drivers (less than 1ms latency).  Not all producers provide ASIO drivers, so choose a sound card with ASIO drivers.
Thanks, I was wondering what that sound-driver feature was in my older Audacity on my old laptop.  It seemed to have no effect on my laptop audio other than the local digital loop-back recording input selection was missing.  Now I have something I can measure on my scope to verify.

My new PC only has Host: MME or Direct Sound, or Windows WASASPI, while the 'recording' settings have the feature of 'Software Playthrough Of Input'.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: BrianHG on November 10, 2018, 03:42:14 pm
I don't see the problem.

It's a real problem. The AES did a study which gave numbers as low as 3ms affecting vocalists.
https://community.bose.com/ziiiw76728/attachments/ziiiw76728/Hearphones/594/1/AES_Latency.pdf (https://community.bose.com/ziiiw76728/attachments/ziiiw76728/Hearphones/594/1/AES_Latency.pdf)
10ms is annoying for percussive or plucked instruments.

I'm surprised that the Saxophone is the most latency sensitive instrument by far, even above vocals.  Even 1ms delay with a saxophone is already beginning to effect the musician.  When I was young, I once played the Alto Saxophone, and today, thanks to that, I'm super sensitive to lip-sync errors in AV equipment to the point that most divx videos give me a headache.  Only direct Bluray movies, played on a PC through a 150hz CRT really give me that synchronous visual to audio life experience I missed from the days of uncompressed all analog studio AV equipment.  (Modern big screen TV as a monitor suck shit with a huge video lag.)  I'm also super sensitive to frame drops and other minute AV artifacts which seem to blow over everyone else, but dead obvious to me.  Don't know why.  It's like no one is paying attention to what they are seeing and hearing, especially on the really bad stuff until I point it out to them, tell them to count the syllables in their head as they hear them watching the lips of the on screen actor that they realize the lips are off by half or a full syllable.

Thankfully on the PC, I found video players which allow me to re-sync the audio, but, it is a headache as each video have their offset error unless it is an uncompressed blu-ray original.  Youtube's and NetFlix's sound really pisses me off...
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: drussell on November 10, 2018, 03:52:39 pm
When I was young, I once played the Alto Saxophone, and today, thanks to that, I'm super sensitive to lip-sync errors in AV equipment to the point that most divx videos give me a headache.

Video-audio sync issues drive me insane as well.  That is one of the things I like about watching most of my content through a MythTV frontend, instant fixing of the sync on the fly at the touch of a key. 

Quote
Thankfully on the PC, I found video players which allow me to re-sync the audio, but, it is a headache as each video have their offset error unless it is an uncompressed blu-ray original.  Youtube's and NetFlix's sound really pisses me off...

I too, wish that YouTube's video player had an audio sync adjust.   Drives me nuts!  :scared:
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: BrianHG on November 10, 2018, 04:02:35 pm

Quote
Thankfully on the PC, I found video players which allow me to re-sync the audio, but, it is a headache as each video have their offset error unless it is an uncompressed blu-ray original.  Youtube's and NetFlix's sound really pisses me off...

I too, wish that YouTube's video player had an audio sync adjust.   Drives me nuts!  :scared:
I feel your pain, multiplied by a 120inch video projected screen.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: coppice on November 10, 2018, 04:10:22 pm
When recording live music this 10ms delay makes it difficult or impossible to to do real-time monitoring as the musician gets perturbed by the lag which affects their timing.
I don't see the problem. 10ms an order of a magnitude is faster than the reaction time of a human. It's also quicker than anyone can physically press a button or key. Take a digital stop watch, reset it to zero and press the start/stop button twice, as quickly as possible. If you can start and stop the clock in under 0.03s, that's 30ms, then you're doing well.
Quote
Simple reaction time is the motion required for an observer to respond to the presence of a stimulus. For example, a subject might be asked to press a button as soon as a light or sound appears. Mean RT for college-age individuals is about 160 milliseconds to detect an auditory stimulus, and approximately 190 milliseconds to detect visual stimulus. The mean reaction times for sprinters at the Beijing Olympics were 166 ms for males and 189 ms for females, but in one out of 1,000 starts they can achieve 109 ms and 121 ms, respectively. This study also concluded that longer female reaction times can be an artifact of the measurement method used, suggesting that the starting block sensor system might overlook a female false-start due to insufficient pressure on the pads. The authors suggested compensating for this threshold would improve false-start detection accuracy with female runners.
https://en.wikipedia.org/wiki/Mental_chronometry#Types
What would be the relevance of any of that to a musician playing along with something? A musician doesn't simply react to what is being played, for the very reasons you described. They predict, and tune their predictions as the pace of the music changes. Good musicians can match each other's timing to a very fine degree, or twist those timings for effect. A more relevant measure of the significance of a delay is what that represents in terms of the spacing of the musicians. 10ms is like moving about 3m (or 1.5m if you are considering a round trip).
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: bsfeechannel on November 10, 2018, 04:22:38 pm
Now you understand why orchestras have conductors.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: e100 on November 10, 2018, 04:34:42 pm
Isn't it that some high end sound cards have DSPs to act as the "real time" system as far as audio is concerned?
Apparently yes, although the manufacturers do a poor job of explaining the technical details. With these hybrid devices they don't tell you how much processing is done on the PC vs what's done in the box. Here's a picture of the interior of a $900 audio interface.

(https://images-na.ssl-images-amazon.com/images/I/91fYrkrlrBL._SL1410_.jpg)
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: Buriedcode on November 10, 2018, 04:59:13 pm
There is often confusion with regards to latency between simple "reaction times", audio/video sync, single source latency (like the delay between the timer on a music player and the actual sound) and musician latency - where someone creates the sound, and there is latency between that and hearing it, like with guitar effects, or using midi instruments.  All of these have different latency tolerances, and then of course there is individual tolerance to latency.

Which one are we talking about in this thread?

In terms of "single source" latency really doesn't matter - which is why MP3's still rule even though they require a fairly large buffer, and therefore high latency.  Because there is only one source of audio, there is nothing to compare it to, so latency can be seconds.  It is also why MP3 can be used to record, but not for real-time transmission, regardless of how fast a processor one uses.   

For audio/video sync that tolerance is different because it involves both visual and audible stimulus.  I have no hard figures on this, but I vaguely remember reading about bluetooth audio causing sync problems with its stock codec with ~150ms latency, and that <~40ms is required for imperceptible sync.

For instances where one is *creating* the sound, as in singers, drummers, guitarist, musicians etc.. the figure is generally even lower, being roughly 20ms. Perception of delay here involves actually physical movements to audible sounds, again a different route in the brain. I am totally unsure of this as I have used guitar effects for years, and even in the 90's where the latency was >40ms I didn't notice it.  Also, as coppice rightly pointed out, 10ms is roughly 3.5m distance - which is easily how far a guitarist can be away from his amp.  I think 10ms isn't unreasonable for a delay.  Even with ASIO midi drivers on my PC I get about 15ms.

Also people forget about buffers.  Digital processes generally require a certain "block" of audio to work with (like MP3 mentioned above).  So it doesn't matter how fast your processor is, if a process requires a chunk of 2k samples, at 48ksps, thats ~42ms buffer latency right there.  This is why modern real-time audio compression codecs work on smaller chunks of audio, and also why some resort to higher sample rates even though there isn't much different in perceived quality going beyond 48k - it halves the latency.

Sadly, any topics surrounding perception, or subjective things like audio latency, or quality tend to get quite heated, mostly because what you "hear" is also about expectations :)  You can adjust how much "latency" someone hears just by telling them numbers.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: RoGeorge on November 10, 2018, 05:18:13 pm
I don't see the problem. 10ms an order of a magnitude is faster than the reaction time of a human.

The problem is real, especially when someone plays an instrument and the sound does not come when the brain expects it.

If the delay is too big, even the speech is impaired severely.  Just for the fun of it:

https://www.youtube.com/watch?v=dK2ylXWn_v4 (https://www.youtube.com/watch?v=dK2ylXWn_v4)

 :-DD
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: MrMobodies on November 10, 2018, 07:05:05 pm
I noticed that with the virtual MIDI on USB only keyboards.
Not a set delay but random delays between the key press.

I hadn't noticed with midi on the Yamaha SW1000XG and the Creative EM-U but I have in the past with some software midi synthesizers.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: dmills on November 10, 2018, 08:45:08 pm
And THAT is the real trap small amounts of latency are usually not a problem for a keyboard player, but VARIABLE latency is a nightmare.

You really need to be timestamping the arriving MIDI with the sound cards current sample number and then generating the resulting note a defined (and constant) number of samples later.

PCs actually can do reasonably low latency, but as ever it comes at the expense of thru-put and power, basically if you pin your job to one processor and just spin in a polling loop you can get wonderfully low latencies, but nobody likes a process that does that, even if the box is not being used for much else.

Speed of sound is roughly 1ms per foot, so that gives you some idea of where the limits are likely to be.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: Buriedcode on November 10, 2018, 08:57:11 pm
I noticed that with the virtual MIDI on USB only keyboards.
Not a set delay but random delays between the key press.

I hadn't noticed with midi on the Yamaha SW1000XG and the Creative EM-U but I have in the past with some software midi synthesizers.

If you're using PC based plugins, then the variable latency is likely from the ASIO output, rather than processing the MIDI commends themselves.  Years ago when I was recording, I of course needed real-time feedback from the synth, which had a slight delay that was off-putting (infuriating) but playing back the MIDI sounded fine.  You can get custom ASIO drivers and set the latency to something more sensible like 20ms.  This is fixed, and as long as your system can cope (it will) it won't vary.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: dmills on November 10, 2018, 09:03:03 pm
The trap is that the MIDI arriving from the keyboard is async with respect to the buffer refill callback from the sound card, so if you do the naive thing and have the callback process all the complete midi messages available at the time it runs, then you get anything up to a buffers worth of random timing variation.

It took a while for sound cards that could time stamp the MIDI with their buffer position to arrive, and then longer for software that knew how to use this to give **CONSTANT** latency to hit the scene.

20ms that is constant is MUCH less of a problem then 10-20ms  that varies from note to note.

Regards, Dan.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: Whales on November 10, 2018, 09:05:56 pm
Back to reasoning as to why audio latency is so poor: because it's not a well known issue, and it's often easier for developers if they don't care about it.

Case in point: pulseaudio.  It runs in userland like any other process and shims itself between the kernel's audio system (ALSA) and your programs.  The only way it can do this reliably is by introducing more latency.  This allows it lots of time to apply effects and features whilst not requiring any special privileges or changes to your kernel.  Sidenote: pulseaudio sucks for many reasons, avoid it if you can, latency is only one of its common issues.

A similar situation exists with many laptop touchpad input devices.  I've found some OEM windows drivers make using them intolerably laggy.  Even under Linux I can "feel" the delay in touchpads.  Perhaps it is slow filtering, perhaps it is video delay, but it's definitely something that could be solved if people cared more about it.  It's easier not to.

Keyboards are an interesting case.  I had to throw away a PS2 keyboard years ago because it had a noticeable delay.  Compared to another PS2 board it was horrible.  Might have been the physical mechanism instead of the electronics (anti-bounce?), not sure. 

Quote
Why don't desktop computers have realtime subsystems?

The closest we can get (without dropping a modern OS) is a kernel that lets us cheat the normal process scheduler.

In the Linux world there are projects like Jack (http://www.jackaudio.org/) with realtime kernel scheduling support to shave the milliseconds.  Best I've ever run this at is 22ms internal latency (the number Jack itself reports, ignoring I/O and peripherals) but I don't run anything particularly modern.  Very popular project for composers and other routing needs, I recommend the program qjackctl as a frontend.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: joeqsmith on November 10, 2018, 09:25:42 pm
You need a Real Time OS (like QNX) on top of the hardware.  Windows/Linux are not RT OS.

QNX was the first thing I thought of as well.  Back then I remember a company (VentrureComm (SP)) who obtained the source for parts of Windows NT in order to create a RT version of the OS.  In return MS would get their source code.   Then again, that has been a VERY long time ago.   My old LeCroy uses an Intel 486DX and they ran PSOS on it.   

I've used Windows for various synchronous control systems.   At the time I made this video, still had a P4 running Windows XP.  I hate upgrading PCs. I broke down and bought a new PC a couple of years ago.  The amount of computing power is incredible.  I ended up changing the whole software architecture.     

https://youtu.be/Lqa2lnxoIsU?list=PLZSS2ajxhiQBvWvqMVLdRQMjGofKpQUJr
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: james_s on November 10, 2018, 09:36:43 pm
The original IBM PCs had near non-existent audio, even compared to other computers of that era it's pathetic.
Sound cards were an afterthought, both in hardware and software.

I wonder if the 10ms latency comes from IRQ polling, or is a byproduct of the sample rate.

While that's true, the IBM PC was designed as a business machine, not a toy. I can hardly fault the engineers for not bothering to include better sound capabilities. The mainframes and terminals that dominated businesses of the day didn't have fancy audio capabilities either, that was only in the home computers and was used almost exclusively for games. The Macintosh was one of the first PCs aimed at professionals to have decent audio playback abilities but even then it was some time before they offered audio input to get sounds onto the computer.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: Marco on November 10, 2018, 09:56:37 pm
Because they never considered real-time applications worthy of dedicating resources to at the start. So both OS and driver writers have tons of tools which can just lock the CPU for aeons with the absolute knowledge they won't be interrupted and they have made a lot of use of those tools. Fixing this after the fact means fixing a whole lot of code and deprecating a lot of old drivers, for a small niche application. Microsoft has never considered it important enough.

That said, Linux is getting there with preempt-rt as long as you stick with mainline (https://www.linux.com/blog/elc/2018/4/developers-prepare-your-drivers-real-time-linux) (NVIDIA for all their supposed embedded focus frequently makes a mess of it). There's also Xenomai Cobalt which is a realtime co-kernel for Linux, which just takes control away from the kernel and drivers whenever it wants, which needs a lot of patching of the kernel (and some prayers for third party drivers). Also it's a tough environment to develop for and doesn't do much good for existing Linux applications ... Bela (http://bela.io/) did it for audio, but you'll have to write your own effects for their framework.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: NiHaoMike on November 10, 2018, 10:25:19 pm
Case in point: pulseaudio.  It runs in userland like any other process and shims itself between the kernel's audio system (ALSA) and your programs.  The only way it can do this reliably is by introducing more latency.  This allows it lots of time to apply effects and features whilst not requiring any special privileges or changes to your kernel.  Sidenote: pulseaudio sucks for many reasons, avoid it if you can, latency is only one of its common issues.
I have had just plain ALSA on my PC since it's easier to get bit perfect playback that way.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: SiliconWizard on November 10, 2018, 11:42:04 pm
Real-time kernels/real-time scheduling are not adapted to general-purpose desktop computing. That's the essential and main reason.
Another reason is that the use cases where low latency is really important are still kind of secondary on desktop computers: you'll need that for real-time playing of virtual instruments or audio effects. If you're just recording/playing, latency doesn't matter as it gets compensated by software. Whereas real-time playing of virtual instruments and effects is becoming a thing, we can still largely do without it by using dedicated hardware.

That said, both Windows and Linux (I don't know MacOS well, but I'd guess that it's the same) have improved a lot regarding average latency. Getting a reliable 10 ms audio latency is usually no problem with decent hardware. Software, beyond the OS, also plays an important role: you'll get reliable performance only with well written drivers and well written application software. Not all audio software is created equal!

As for me, I find that a 10 ms latency is acceptable for real-time playing. Anything above this starts becoming noticeable. Of course this will depend on each individual to a degree.

If you want to get latencies under that in a reliable manner, you can use Linux and a Linux real-time kernel such as linux-rt. I've done real-time audio processing on Linux with linux-rt and you can get latencies as low as 2-3 ms RELIABLY (no drop-out) even with USB audio interfaces (as long as they are not crap).

There are other real-time oriented OSs out there, but frankly Linux with a linux-rt kernel does the job and is a good start.

Title: Re: Why don't desktop computers have realtime subsystems?
Post by: coppice on November 10, 2018, 11:46:19 pm
When people say that desktop machines should have hard real time capabilities as standard, they aren't really allowing for the huge impact cache has on the performance of modern machines. Turn off the caches and its easy to make something hard real time, but rather slow. Turn on the caches, and you turn on a bunch of difficulties.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: dmills on November 11, 2018, 12:21:37 am
Cache, TLB, thermal throttling (Including weird things like thermal throttling the dram, yes it is a thing!), SMM mode, PCI bus contention (Graphics card driver authors, looking at YOU), cackhanded driver authors generally... The pain goes on.

Fortunately, for audio we don't really need  formal hard realtime, a missed deadline once every few weeks will not kill anyone.

Regards, Dan.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: langwadt on November 11, 2018, 12:48:08 am
I don't see the problem. 10ms an order of a magnitude is faster than the reaction time of a human.

The problem is real, especially when someone plays an instrument and the sound does not come when the brain expects it.

If the delay is too big, even the speech is impaired severely.  Just for the fun of it:

https://www.youtube.com/watch?v=dK2ylXWn_v4 (https://www.youtube.com/watch?v=dK2ylXWn_v4)

 :-DD

I've heard that people who stutter don't when they hear themselves delayed enough, something about it "breaking"
the feedback loop between speech and ear 


Title: Re: Why don't desktop computers have realtime subsystems?
Post by: BrianHG on November 11, 2018, 02:37:21 am
LITTLE TRICK FOR MIDI LATENCY WHEN USING AN FTDI RS232 converter with generic midi-RS232 interface:
See screenshot photo below:
Go into hardware 'Device manager', 'USB serial comport', 'Port Settings', 'Advance' and select '1 ms' for 'Latency Timer' and your RS232 port will respond 16x as fast!

However, it does eat up around 1% of a cpu core whenever the port is in use, regardless of the software, it can make other tasks jitter or slow down since the RS232 port will take priority.  2ms delay solves this problem, but then it is only 8x as responsive as the default 16.

Also, your RS232 port will now transmit and receive even single byte fast enough to create a motor speed control loop regulator by PC controlled loop.  (I know this is a dumb method, but, I was once in a spot where I needed a quick fix for testing before re-programming an external microcontroller and with a sub 1ms loop for a treadmill size motor with flywheel, it worked)
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: BrianHG on November 11, 2018, 02:47:41 am
When recording live music this 10ms delay makes it difficult or impossible to to do real-time monitoring as the musician gets perturbed by the lag which affects their timing.
I don't see the problem. 10ms an order of a magnitude is faster than the reaction time of a human. It's also quicker than anyone can physically press a button or key. Take a digital stop watch, reset it to zero and press the start/stop button twice, as quickly as possible. If you can start and stop the clock in under 0.03s, that's 30ms, then you're doing well.

Don't be too sure.  In my father's office, we were forced to keep his secretary's computer running 'today' on Win98 with a PS2 keyboard, because, when she entered invoice numerical prices while typing invoice values, she typed too fast for the keyboard to respond in time and even display the figures on modern 32 bit or 64 bit OSs.  When I say fast, I mean you could not see her finger key contacts with your eyes and the audible sound was nothing but a rattle of noise.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: David Hess on November 12, 2018, 01:08:13 am
I don't see the problem. 10ms an order of a magnitude is faster than the reaction time of a human. It's also quicker than anyone can physically press a button or key. Take a digital stop watch, reset it to zero and press the start/stop button twice, as quickly as possible. If you can start and stop the clock in under 0.03s, that's 30ms, then you're doing well.

10 milliseconds may be faster than the reaction time but it is 2 orders of magnitude larger than the difference in time that the human nervous system can anticipate which is what a musician does when keeping time.

https://www.electronicdesign.com/defense/whats-all-why-stuff-anyhow (https://www.electronicdesign.com/defense/whats-all-why-stuff-anyhow)

If a ball at the end of your arm, at a radius of 3 ft., is moving at 90 mph or 132 ft./s, it will change its ANGLE by 0.0044 radians, or 0.252°, every 100 µs. And an angle of 0.0044 radians corresponds with an uncertainty of 6.4 in. (±3.2 in.), after the ball—or rock—has gone 60 ft.

Don't be too sure.  In my father's office, we were forced to keep his secretary's computer running 'today' on Win98 with a PS2 keyboard, because, when she entered invoice numerical prices while typing invoice values, she typed too fast for the keyboard to respond in time and even display the figures on modern 32 bit or 64 bit OSs.  When I say fast, I mean you could not see her finger key contacts with your eyes and the audible sound was nothing but a rattle of noise.

I touch type fast enough for this to be a problem including on the forums here which trash the performance of the browser with scripts and animations.  Modern computers are *slower* when interface latency is measured.  The 32-bit 50MHz ARM based HP50g is *slower* than the ancient 4-bit 2MHz Saturn based HP48g.  Tell me again how ARM processors are power competitive with the 8-bit processors they replace; they are not.



A real time operating system is not incompatible with desktop or workstation usage; check out BeOS.  But relaxed timing and increased buffering has minor performance advantages and makes things easier.  Some delays like going through an FIR filter are unavoidable.

Title: Re: Why don't desktop computers have realtime subsystems?
Post by: james_s on November 12, 2018, 02:26:26 am
I'm not so sure that's due to any fault with the processor. More likely it's a matter of the newer processor being so powerful that developers don't think they need to bother optimizing anything. The software industry as a whole has been in a race to the bottom, the goal now being to make things just good enough that people will still pay for them.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: Marco on November 12, 2018, 10:19:16 am
People get deep deep sub millisecond latency with preemp-rt on SBCs ... it's not the processor, it's a problem of the complexity of the OS and the tools which have been given to driver and OS subsystem writers.

Basically real time inverts the way things are done normally. Users get real-time, driver and OS developers get screwed with unbounded latencies for any section of their code ... while they have been used to be able to take control for milliseconds at a time. They don't like that, they are lazy bums like the rest of us.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: hans on November 12, 2018, 01:28:28 pm
Computers can do sub-ms timing, but not predictably and are tuned for the average-case performance.

In Windows/C#.NET you can call Thread.Sleep(1), but have Windows return to your thread after 1ms, but also after 16ms. I think 16ms was the lower bound for which behaviour is defined (anything above works pretty well).

In real-time you want to bound the worst-case, which is a whole different task. Most RTOS are capable of more accurate timing than PCs, but may still get stuck on shared resource problems and priority inversion (which is more of a software design problem than something an RTOS will fix for you).


When recording live music this 10ms delay makes it difficult or impossible to to do real-time monitoring as the musician gets perturbed by the lag which affects their timing.
I don't see the problem. 10ms an order of a magnitude is faster than the reaction time of a human. It's also quicker than anyone can physically press a button or key. Take a digital stop watch, reset it to zero and press the start/stop button twice, as quickly as possible. If you can start and stop the clock in under 0.03s, that's 30ms, then you're doing well.

Don't be too sure.  In my father's office, we were forced to keep his secretary's computer running 'today' on Win98 with a PS2 keyboard, because, when she entered invoice numerical prices while typing invoice values, she typed too fast for the keyboard to respond in time and even display the figures on modern 32 bit or 64 bit OSs.  When I say fast, I mean you could not see her finger key contacts with your eyes and the audible sound was nothing but a rattle of noise.


USB devices need buffers to bridge the gap between the USB host polling the device. A PS2 keyboard doesn't have a buffer and will output a serial code straight to the machine, which will need to handle an IRQ and process the key. Maybe thats the problem.. too much software for the hardware to handle. Gamers still love PS2 keyboards for this IRQ-like response time, because even configuring the USB host to 1000Hz / 1ms poll intervals is not good enough for them.

I've seen sub-10ms timing between keystrokes (muscle memory) while typing myself, and I can reach ~600 - 700 keystrokes per minute on a good day.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: Marco on November 12, 2018, 02:33:00 pm
Computers can do sub-ms timing, but not predictably and are tuned for the average-case performance.
I thought it was obvious from context, but when I said people get deep deep sub millisecond latency with preemp-rt on SBCs I meant they get that as max latency.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: glarsson on November 12, 2018, 02:51:01 pm
Computers can do sub-ms timing, but not predictably and are tuned for the average-case performance.
I thought it was obvious from context, but when I said people get deep deep sub millisecond latency with preemp-rt on SBCs I meant they get that as max latency.
I have been doing real time systems for many years. For me one millisecond is a long time.
Example of an application that has been in use for over ten years: Hardware is a SPARC T2 server with Solaris operating system. Application runs ten threads and each thread needs to react to events, and do I/O, with a latency of max 64us. All ten threads runs in one CPU core. Note that all ten threads might need to do I/O in the same 64us "slot".
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: hans on November 12, 2018, 04:58:21 pm
Computers can do sub-ms timing, but not predictably and are tuned for the average-case performance.
I thought it was obvious from context, but when I said people get deep deep sub millisecond latency with preemp-rt on SBCs I meant they get that as max latency.
I wasn't commenting to preemp-rt in particular, but yes agreed. A "computer" (as in desktops, laptops, tablets, smartphones) and SBC with RT kernels are totally different worlds.

Even then, a RTOS can't provide miracles neither to any application. In desktops we're used (or like to see!) full utilization of CPU and hardware resources when we strive the optimal average case performance. For real-time that would be a very bad thing to do.

Most schedulers used today in real-time environments need "spare" CPU time (https://en.wikipedia.org/wiki/Rate-monotonic_scheduling) to meet their deadlines. Optimal schedulers (https://en.wikipedia.org/wiki/Earliest_deadline_first_scheduling) exist.. but unfortunately they aren't available in preempt-rt (by default). Or in fact many of the industry RTOS, because we're not used to programming with "deadlines" in mind (which, admittedly, would be relatively easy to implement for periodic tasks).
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: Marco on November 12, 2018, 06:33:28 pm
It's just a question of being cooperative, almost all of the code in OS's which doesn't allow itself to be interrupted does so unnecessarily ... you don't really lose anything by designing it to be realtime compatible, except needing to pay better developers and bullying the laziest ones.

As glarsson reminded us, Solaris did realtime on the desktop a long time ago.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: John B on November 12, 2018, 08:42:02 pm
I usually run a buffer of 128 samples at 48kHz. Its a good compromise for real time performing, while still allowing for non-predictable tasks that suddenly take CPU run time, causing sample drop outs.

My software gives me the following info:

Input latency: 3.6ms
Processing latency: 2.7ms (128 samples @ 48kHz)
Output latency: 0.9ms

Total: 7.2ms

Ill echo what's said previously. Even 7.2ms is borderline acceptable, depending on the instrument. Percussive sounds are the most demanding on latency.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: ebastler on November 12, 2018, 08:53:36 pm
Quote
Why don't desktop computers have realtime subsystems?
The closest we can get (without dropping a modern OS) is a kernel that lets us cheat the normal process scheduler.

Didn't the NeXT computers have 56000 DSP subsystems for audio processing, back in the early 90s? I have no idea what software the 56000 was running, however. (The NeXT itself obviously was running NeXtStep, a Unix derivative.)
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: coppice on November 12, 2018, 09:02:22 pm
Didn't the NeXT computers have 56000 DSP subsystems for audio processing, back in the early 90s? I have no idea what software the 56000 was running, however. (The NeXT itself obviously was running NeXtStep, a Unix derivative.)
The NeXT machines did have a 56k in them, but I don't think they ever got it doing very much. There was a surge of activity in about 1993, where DSP makers, including Motorola with the 56k, worked closely with MS to make a hardware DSP block a normal adjunct function in Windows machines. It never went very far, and the DSP functions, like soft modems, ended up running on the main CPU.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: jmelson on November 12, 2018, 11:23:19 pm
I ask this question because Windows/Linux/IOS/Android all have a 10ms or thereabouts latency when it comes to doing real-time audio processing.
There are at least THREE real time options on some Linux kernels.  RTAI, Xenomai and RT-Preempt.  At one time, RTAI was far superior, but now at least RT_Preempt has caught up.  Generally, latency can be down to a couple microseconds, with +/- 10 us jitter using RT_Preempt.  On SOME X86 hardware, the jitter can be down to +/- 5 us with RTAI.

I use RTAI and RT_Preempt with the LinuxCNC motion control (CNC) system.

The results on the Beagle Board Black show about twice the jitter figures given above, which is still pretty good.

Note that due to BIOS features, some specific hardware models (motherboards) perform horribly in this regard, and others that seem quite similar do very well.  The above RT packages have latency/jitter tests that can tell you very quickly how well a specific combo of OS/RT package/CPU/BIOS perform.

Jon
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: David Hess on November 13, 2018, 12:40:56 am
There is something which I forgot to mention which is a major problem with x86 systems as far as latency and real time performance.

SMM (System Management Mode (https://en.wikipedia.org/wiki/System_Management_Mode)) completely trashes real time performance independent of what the operation system and hardware is capable of.  There are special BIOSes which keep this under control for applications where real time performance matters but the performance of consumer systems is random leaning toward as bad as they can get away with.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: Marco on November 13, 2018, 02:55:15 am
There are special BIOSes which keep this under control for applications where real time performance matters but the performance of consumer systems is random leaning toward as bad as they can get away with.
Asus Chromebox 3 should be pretty good, comes with Google's version of coreboot and can be reflashed with normal coreboot.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: BrianHG on November 13, 2018, 05:02:38 am
With ISA based sound cards, or even the appropriate USB ones on a dedicated channel, it should be possible to trick Windows OS and get real-time microsecond performance, however, it would only be in your own app, there would be no windows system audio without extra programming and 1 cpu core would be pretty much eaten alive stuck at around 10% usage all the time.  Though, you would never need to worry about buffer under-run errors if programmed properly.


Note, 5 years ago, In Win7 64 bit, I had a test display rendering tool where I manually swapped the video-cards full graphics screen buffer timed to one of the system's precision timers which operate in the microsecond range.  I was able to swap page buffers on specific lines of video on the screen with other tasks running.  Meaning, a valid Window  app can time and interrupt to call system functions and control connected hardware with accuracy down to a 65Khz line of video without fail.  With this, you can make an audio driver which responds within 2 to 4 audio samples if you care to write your own sound driver.

So, there is no need for a real-time OS to make a super fast/short sound buffer sound driver.
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: Circlotron on November 13, 2018, 05:26:24 am
When I was young, I once played the Alto Saxophone, and today, thanks to that, I'm super sensitive to lip-sync errors in AV equipment to the point that most divx videos give me a headache.
I only recently learned what a clapperboard is used for in the motion picture industry. Visual and audio sync reference.
https://en.wikipedia.org/wiki/Clapperboard (https://en.wikipedia.org/wiki/Clapperboard)
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: drussell on November 13, 2018, 06:55:41 am
I only recently learned what a clapperboard is used for in the motion picture industry. Visual and audio sync reference.

Dare I ask what you thought it was for?  :)
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: ebastler on November 13, 2018, 07:15:02 am
I only recently learned what a clapperboard is used for in the motion picture industry. Visual and audio sync reference.

Dare I ask what you thought it was for?  :)

Get the actors' attention?  ;)
Title: Re: Why don't desktop computers have realtime subsystems?
Post by: Circlotron on November 13, 2018, 11:01:27 am
I only recently learned what a clapperboard is used for in the motion picture industry. Visual and audio sync reference.

Dare I ask what you thought it was for?  :)
I thought it was a meaningless, anachronistic tradition, like when they ring a bell and everyone claps at the closing of the NYSE.