EEVblog Electronics Community Forum

Products => Computers => Programming => Topic started by: Phaseseeker on November 30, 2021, 02:36:05 pm

Title: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: Phaseseeker on November 30, 2021, 02:36:05 pm
Like the title says, I need to resample PCM audio in the S16LE format (signed 16-bit integer, little endian) sampled at 9.6KHz ideally up to 48  kHz on the fly. This is because I'm trying to interface to some old software that is partially closed source; I have the code that interfaces with the audio card (it just uses standard ALSA functions like snd_pcm_writei() and snd_pcm_readi() to write the data to the card) but the DSP code is proprietary and refuses to work at any sample rate that is not 9.6 KHz and I have no audio card that natively supports such rate.  Ideally I'd want to resample it to 48 kHz (since every card I have seems to support it) but both the nearest AC97 standard rates of 8 kHz and 11.025 kHz would be fine. I tried using libsamplerate but the problem is that I need to convert the audio coming from the DSP software on the fly and if the resampling introduces too much delay I start getting overruns on the soundcard buffers; even the fastest converter mode in the library is still too slow. I suspect some of it has to do with the fact that the library only works with PCM data in float format and the back and forth conversion adds a lot of delay but I don't have a lot of experience with PCM audio so I don't have any ideas on how to fix this (my first idea was to simply delete a sample every n to downsample from 9.6 kHz to 8 kHz but I've learnt that it's a bad idea...), so any suggestion is welcome.
Thanks in advance
Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: Kalvin on November 30, 2021, 03:08:29 pm
Conversion from 9.6kHz to 48kHz should be pretty straight forward, depending of how good results you want. The ratio of 48kHz / 9.6kHz = 5, which is a nice integer value. A brute-force method would be using just a simple linear interpolation, but the result may not be as good as you might want. Using zero stuffing and an interpolating FIR filter may be something that you want to try out: https://en.wikipedia.org/wiki/Upsampling#Upsampling_by_an_integer_factor
Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: Ed.Kloonk on November 30, 2021, 03:32:36 pm
What card is it?
Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: Phaseseeker on November 30, 2021, 04:02:03 pm
Quote from: Ed.Kloonk on November 30, 2021, 03:32:36 pm
What card is it?
My soundcard? I'm simply using my laptop's integrated sound card (ALSA calls it "CX8070 analog").

@Kalvin
Thanks for the link, I'll look into it. Though, thinking about it, the problem with 48 kHz might be that there's no way the sound can buffer can take all the data in one transfer (since I also need to transfer 2x the data because the sound card doesn't support mono output).
Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: SiliconWizard on November 30, 2021, 05:57:04 pm
Quote from: Kalvin on November 30, 2021, 03:08:29 pm
Conversion from 9.6kHz to 48kHz should be pretty straight forward, depending of how good results you want. The ratio of 48kHz / 9.6kHz = 5, which is a nice integer value. A brute-force method would be using just a simple linear interpolation, but the result may not be as good as you might want.

Well. We dont know much about the OP's requirements and the kind of "audio" they deal with, but as a general thought, if you're using audio sampled at 9.6 kHz, you're probably not expecting Hi-Fi.

Anyway, a common and nice alternative to linear interpolation is using an higher-order interpolation. Cubic interpolation works quite well for audio - take a look at Catmull-Rom splines. I've used that almost exclusively for real-time sample rate conversion. Still more efficient than using a large FIR filter.

And that said, I'm a bit surprised that libsamplerate would be too "slow" here. But we may also need to know what kind of hardware this all runs on. Maybe it's a pretty old machine. And libsamplerate is not the most efficient thing on earth either (and I think it has various modes, so maybe the OP is using one of the "highest quality" but most taxing mode.)


Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: ejeffrey on November 30, 2021, 06:55:00 pm
I struggle to imagine any computer capable of running Linux produced in the past 20 years that can't handle upconverstion from 9.6 kHz to 48 KHz in real time with modest effort.  A pentium-60 should be able to do this.  Of course an inefficient implementation written in python using excessively large and un-optimized FIR window for such a relatively low-fi source could overwhelm any CPU, but otherwise this should be easy.

I'm also not sure what the buffering problem is.  Again, the size buffers needed for 48 kHz stereo audio are quite minimal by any modern standards.  It sounds like we need more detail. 
Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: Kalvin on November 30, 2021, 07:38:02 pm
Has OP already considered using Virtual Audio Cable [for Windows]: https://vac.muzychenko.net/en/

It seems to have short latency and contains sample rate conversion built-in.

- Almost any of fixed point PCM audio formats (1000..384000 samples per second, 8..32 bits per sample, 1..32 channels).
- PCM format conversion (sampling rate, bits per sample, number of channels).

Personally I have not used that, so I do not know how well that performs etc. but this may worth taking a look.
Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: Phaseseeker on November 30, 2021, 09:18:31 pm
First of all, thank you for the various suggestions. I should've explained in more detail what I'm trying to achieve.
The software I'm trying to interface with is slmodem (https://github.com/leggewie/pkg-sl-modem), a software modem driver; I'm trying (admittedly with no practical reason other than curiosity) to adapt it to use a sound card, instead of the actual modem circuitry as the "analog interface" (for lack of a better term) so that I can make two computers talk to each other using audio signals.
The software is made of two parts, a kernel mode driver and an user mode daemon (slmodemd) which has all the DSP code; the thing that makes the software ideal for my purposes is that slmodemd can be compiled to use ALSA to interface with the hardware (since the MC97 spec treats all the audio to and from the actual modem circuitry as standard PCM audio streams like the ones to and from an AC97-compliant soundcard). I can tell it to use my soundcard (alsa hw:01,00 in my case) as the modem device, and it will happily use it, BUT the DSP code is proprietary and provided as a compiled 32-bit x86 binary blob (dsplibs.o) that is hardcoded to work at 9.6 kHz sample rate; changing the "srate" parameter in the modem structure seems to work at first, but it will refuse to generate the appropriate signals without throwing any meaningful error; I disassembled dsplibs.o and, sure enough, it checks that the rate is set to 9600 and quits if it's not.
My first thought at this point was to simply modify the code used to read and write data from the "modem" to resample the data before sending it. The relevant code is in "modem_main.c", but basically it works as follows:
As said before, I tried using libsamplerate to resample the data: on startup, I allocated all the needed buffers (1 for data converted to float, 1 for the resampled data, 1 for the resampled data converted back to s16_le) and initialized two SRC_DATA structures pointing to the appropriate buffers. Then, when the code calls alsa_device_write, I would just call src_short_to_float_array to convert the data to float, src_process to resample, then convert the data back to short, and interleave it with zeroes to turn it into a stereo stream (again, the final buffer was already allocated, I just wrote the resampled data in buffer[0], buffer[2] ecc) and finally feed it to snd_pcm_writei. I haven't modified the read function yet, but the process would simply be the reverse.
I tried with 11025 Hz and 48000 Hz sample rates, but the problem is that I just could not push data to the sound card fast enough and ALSA would report an overrun. Part of the problem is that the modem_process function doesn't always return a full 2048 samples (output data size can vary quite a bit depending on what the code is doing) and I obviously have to wait for it to finish processing before I can write or read new data. The function itself SHOULD be quite fast (the readme states the whole thing can run on a P1 MMX @ 233 MHz and I'm running this on a Ryzen 5 3500U) but I can't seem to get it to work.
Edit: I thought that maybe the overrun was on the read side (since I didn't modify that function and it's only reading 2048 samples maximum instead of 2048*5*2) but quickly turning on debug mode it appears that it gets an xrun only on write...
Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: Foxxz on December 01, 2021, 05:30:28 am
I use a linux program called "sox" to do audio conversion the fly by piping audio into STDIN and STDOUT for further processing. I have a program that generates 16 bit signed integer PCM and another program that consumes 32 bit floating point audio

Code: [Select]
audio_source |sox -r 44100 -t raw -e signed -b 16  -c 1 - -r 44100 -t raw -e float -b 32  -c 1 - |audio_consumer

Perhaps looking further into how this program operates would help you.
Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: newbrain on December 01, 2021, 12:42:20 pm
Quote
and I'm running this on a Ryzen 5 3500U
I'm resampling with linear interpolation from 48 kS/s stereo 16 bit to 1.536 MS/s stereo 16 bit (32 ×).
In real time. On a Raspberry Pi Pico.

And on a RPi3 I'm using Python and scypy resampler lib to resample, still in real time, from 12 kHz to 48 kHz - float stereo to 16 bit stereo.

Definitely something else is amiss, for an R5 3500 this should be load background noise in terms of loading.
Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: Phaseseeker on December 01, 2021, 03:34:26 pm
Quote from: newbrain on December 01, 2021, 12:42:20 pm
Quote
and I'm running this on a Ryzen 5 3500U
I'm resampling with linear interpolation from 48 kS/s stereo 16 bit to 1.536 MS/s stereo 16 bit (32 ×).
In real time. On a Raspberry Pi Pico.

And on a RPi3 I'm using Python and scypy resampler lib to resample, still in real time, from 12 kHz to 48 kHz - float stereo to 16 bit stereo.

Definitely something else is amiss, for an R5 3500 this should be load background noise in terms of loading.
I'm beginning to think that my problem here is that my code needs to convert short "bursts" of data (whatever number of samples the modem_process is spitting out) and then sit idle until the next block of data comes, so my soundcard just runs out of samples before the new block of data is generated and converted. It's more of a latency problem than an issue with speed per se, if that makes sense.
Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: SiliconWizard on December 01, 2021, 06:58:04 pm
Quote from: Phaseseeker on November 30, 2021, 09:18:31 pm
Part of the problem is that the modem_process function doesn't always return a full 2048 samples (output data size can vary quite a bit depending on what the code is doing) and I obviously have to wait for it to finish processing before I can write or read new data.

Uh. Is that not just "part" of the problem, but your whole problem precisely?
Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: gf on December 01, 2021, 08:49:34 pm
Aren't there some ALSA plugins which can convert sample rate on the fly, so that you can send PCM data at (say) 8kSa/s while the sound card still runs at (say) 48kSa/s?
[ This still does not solve any synchronization/flow-control problems, of course - they are a different issue. ]
Title: Re: Resampling PCM S16LE @ 9.6 kHz to 11025 Hz or 48000 Hz on the fly
Post by: Phaseseeker on December 01, 2021, 09:27:05 pm
Quote from: SiliconWizard on December 01, 2021, 06:58:04 pm

Uh. Is that not just "part" of the problem, but your whole problem precisely?
Yeah, but if the modem_process code works on "sound" hardware that supports 9.6 kHz sample rate, and does so on slow processors, it should be more than capable of pushing enough data to keep the soundcard buffer full while it generates newer data, right? I ran another test, and the resampling code does work if the size of the data generated by modem_process is less that a couple hundred samples, so I would say that it's my resampling code that's too slow. I used libsamplerate with the "SRC_ZERO_ORDER_HOLD" mode, which according to the documentation should be "blazingly fast"...
Quote from: gf on December 01, 2021, 08:49:34 pm
Aren't there some ALSA plugins which can convert sample rate on the fly, so that you can send PCM data at (say) 8kSa/s while the sound card still runs at (say) 48kSa/s?
[ This still does not solve any synchronization/flow-control problems, of course - they are a different issue. ]

I tried plughw, but it does not support snd_pcm_link, so the code does not run