Microcontroller with USB audio stack

[ivan747]

Hi everyone,

I’m looking for a microcontroller with a USB peripheral and software libraries for implementing a sound card (audio device class). The output is going to be I2S.

No DSP is necessary (audio amp has a DSP controlled by SPI), but if the chip is exceptionally cheap and has this capability I guess it’s good to have.

Let me know if you have other recommendations, specially if they have an example. My priority is the ease of use of the USB libraries.

Thank you



P.S. I saw a Microchip app note (AN1422) saying their dsPIC32 can do it, but to be aware of buffer under runs.

[Doctorandus_P]

Audio is not very demanding for USB. A standard 48kHz 16 bit stereo stream is

>>> 48e3 * 16 * 2 = 1.536 Mbps

And this is less then 15% of a 12Mbps USB connection. My first guess is that any uC with an USB interface can do audio, and it would mostly depend on what other stuff you want to do. Do you need I2S? Do you need to do calculations on the audio?

A quick search:
https://html.duckduckgo.com/html?q=audio+device+class+stm32

Coughs up links like:
https://www.st.com/en/embedded-software/x-cube-usb-audio.html

dm00383815-usb-device-audio-streaming-expansion-package-for-stm32cube-stmicroelectronics.pdf

UM2195 Frequently asked questions
Q: How can I get the USB audio streaming Expansion Package for STM32Cube™?
A: The library sources are provided for free download from the STMicroelectronics web site
at www.st.com.

And:
https://github.com/dragonman225/stm32f469-usbaudio

[Njk]

I can confirm. Actually, the only requirement is a USB controller. The lib is not necessary. Audio+HID+DFU+SCSI server for flash storage emulation, 3-way audio mixer for conference calls, and the debug console for my convenience. And yes, I2S codec on the other end. Everything was done in early 2K, entirely from scratch, for a Silabs 8051 MCU (modem and VOIP applications). Took about 20K in ROM, AFAIR. I'm afraid using a lib here will bring more harm to a design guy primarily because that creates a temptation to skip reading all the USB specs.

[SiliconWizard]

UAC 1 or UAC 2?

[tggzzz]

XMOS xCORE devices are widely used in USB+audio applications.

Unlike all other MCUs, they have deterministic hard realtime behaviour[1]: the IDE examines the (optimised) compiler output, and specified the exact min and max cycle counts between here and there. None of this "run and measure the timing, and hope you've stumbled across the worst case" crap

Up to 32cores/chip, up to 4000MIPS/chip, buy them at DigiKey.

Fun and easy to use, it took me <1day to be ingesting multiple 62Mb/s bitstreams, while simultaneously processing them and uploading results across USB to a PC. I was very pleasantly surprised at that! All guaranteed timings

[1] apparently important for audio work

[tszaboo]

Depends on your speed. If 48KHz 2ch 16 bit is enough, there are plenty of choices. I choose the STM32F411 for personal projects because the boards for it are available for like 4 dolalrs.
If you need higher speed, USB audio class 2, there is AVR32 and XMOS that you can choose. Both of them are weird architectures for me. Maybe others.
I'm actually hoping USB audio class 3 will have a much higher adaptation becaseu the headphone jack disappearing from phones.
There is also the route of just using an ASIC to do the audio part, and have the I2S signal going through a micro or whatever else if needed.

[tggzzz]

Depends on your speed. If 48KHz 2ch 16 bit is enough, there are plenty of choices. I choose the STM32F411 for personal projects because the boards for it are available for like 4 dolalrs.
If you need higher speed, USB audio class 2, there is AVR32 and XMOS that you can choose. Both of them are weird architectures for me. Maybe others.
I'm actually hoping USB audio class 3 will have a much higher adaptation becaseu the headphone jack disappearing from phones.
There is also the route of just using an ASIC to do the audio part, and have the I2S signal going through a micro or whatever else if needed.

Familiarity is important, of course.

Does the STM32F411 and its toolset make any latency guarantees while USB traffic is being transmitted or received? I'm told jitter is a killer for some audio applications.

[tszaboo]

You are mixing two concepts. Jitter is on the nanosecond level, less than 1 clock. Latency is several clock long.
It doesn't skip, the latency is stable, plus there is some minimal buffering for the USB stream. This increases latency.
The jitter, it depends on the crystal used, and the PLL. I didn't investigate jitter.

[tggzzz]

You are mixing two concepts. Jitter is on the nanosecond level, less than 1 clock. Latency is several clock long.
It doesn't skip, the latency is stable, plus there is some minimal buffering for the USB stream. This increases latency.
The jitter, it depends on the crystal used, and the PLL. I didn't investigate jitter.

I am well aware that latencies that vary cause jitter in the output timing.

Where software is involved, any path differences in the executed code can cause jitter. If you are "multiplexing" many activities/processes (in the CSP sense) onto a single processor, that inherently changes the executed code and hence changes the latency - potentially by many many many clock cycles. No surprises there. The extent and significance of such jitter is, of course, dependent on the application's requirements.

Where you are using a library for a major chunk of code (e.g. a USB library),  the time the processor is unavailable for "your" hard real-time functions can be large - and effectively impossible to predict.

One way of avoiding those problems is to grossly underuse the processor. I have never had that luxury; all my designs have been pushing the limits. Then you still have to measure and hope.

Another way of avoiding those problems is to have no caches, no interrupts, and multiple slightly simpler processors, one for each function[1]. Given that base and a decently parallel language and concepts, it is practical to have predictable performance and latency and jitter.

[1] N.B. Sun used similar techniques with their Niagara processors, and gained startling speedups on commercially important classes of "embarrassingly parallel" computation. Unfortunately Oracle Happened.

[SiliconWizard]

That's a lot of nice talk, but the real question here is IMO again, UAC 1 or UAC2?

Implementing UAC 1 on even a modest MCU is simple enough and doesn't require a lot of care if you half know how to use a MCU properly. Existing UAC 1 implementations are usually "good enough" too.

UAC 2, OTOH, is another matter entirely. It's tough shit to implement properly and there are very few existing implementations at all, and even fewer that are free to use (or at least without exorbitant licenses). Don't get fooled by what could appear as an UAC 2 example in some MCU's SDK: most are utter crap that don't implement UAC 2 as it should be and will fail to work without endless issues.

It is absolutely possible to do it, but it's a complex matter.

Which is why, indeed, many (if not most) professional audio interfaces these days use XMOS chips - as tggzzz suggested - because, among other things, they have a very good UAC 2 implementation that just works. There's very little to actually do if you choose this path: just configuring the audio device you want to expose, the rest is already done for you by XMOS.

But if all you want to do is a simple audio interface, stereo, that will fit UAC 1 specs, implementing UAC 1 is a lot easier and is a much more realistic endeavor on almost any small 32-bit MCU with a USB FS interface.

[tggzzz]

SiliconWizard clearly knows a lot more about USB than I do

The general points I make (about lack of predictability and latency jitter) are applicable to many (most?) hard realtime applications where the processor is actively processing for more than 1% the time.

Here's an XMOS view of UAC1 vs UAC2 https://www.xmos.com/download/Why-do-you-need-USB-Audio-Class-2(1.0).pdf

[Nominal Animal]

I’m looking for a microcontroller with a USB peripheral and software libraries for implementing a sound card (audio device class). The output is going to be I2S.  [...] My priority is the ease of use of the USB libraries.

Well, Teensy 4.0 has two I²S outputs (one with 1-4 outputs AKA 2/4/6/8 channels, the other with 1 output), and a dedicated library with a graphical design tool (albeit designed for 44.1kHz sample rates).  Basically, you can create a 44.1kHz stereo 16bit USB sound card quite easily.  For a straight USB → I2S, the audio library handles everything, so you only need

Code: [Select]

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

AudioInputUSB            usb1;
AudioOutputI2S           i2s1;
AudioConnection          patchCord1(usb1, 0, i2s1, 0);
AudioConnection          patchCord2(usb1, 1, i2s1, 1);

void setup(void) {
}

void loop() {
    // Host volume setting is described by usb1.volume(), returning a value between 0.0f and 1.0f
}
The development is done in the Arduino environment, using the Teensyduino add-on.
Even the USB device type is selected in the IDE from a menu (in this case, either Audio or Serial+MIDI+Audio), so it is easy to get going.

The sources for the Teensy Audio library are here, so if you want multichannel audio support (4-8 channels, each 16bit 44.1kHz/48kHz), with output to two I²S ports (one with 1-4 data outputs and 1 input, the other with 1 output and 1 input), you may need to edit the libraries; but there is also a Teensy-only forum at forum.pjrc.com where you can get help if you get stuck (but be prepared to show your own efforts first; even there, nobody will write the code for you).  There is also support for SPDIF input and output, and several codecs (ADCs and DACs).  Note that for I²S, you do not need any additional boards (audio shields or anything).

Depending on exactly how you intend to connect the I²S device to the USB microcontroller, I would be very careful about ground loops.  I assume your mixer does not derive its power from your PC, so signal isolation is something you could consider.  For SPI, I'd recommend TI ISO7741 (2.25V to 5.5V on either side, so also works as a level shifter); for one output I²S, ISO7740; for three-output I²S (six channels), ISO7760.  These are very easy to use, cheap (2-3 € in singles at Mouser) and only require a 100nF = 0.1µF bypass capacitor between VCC and ground on each side of the chip.  (Both sides need their own supply and ground, of course.)
(For the MCLK (11-16 MHz audio clock), a 100 Ohm resistor in series is a good idea, to keep out noise.)

I would design a separate carrier board for this, on top of which you solder a Teensy 4.0 on pins.  If you want to design your own Teensy 4.x -compatible board, PJRC will even sell the proprietary bootloader chip for $6.25.  It is not open hardware, but Paul and Robin are good, friendly people; and the library code is quite excellent (especially compared to typical Arduino libraries).

[tszaboo]

You are mixing two concepts. Jitter is on the nanosecond level, less than 1 clock. Latency is several clock long.
It doesn't skip, the latency is stable, plus there is some minimal buffering for the USB stream. This increases latency.
The jitter, it depends on the crystal used, and the PLL. I didn't investigate jitter.

I am well aware that latencies that vary cause jitter in the output timing.

Where software is involved, any path differences in the executed code can cause jitter. If you are "multiplexing" many activities/processes (in the CSP sense) onto a single processor, that inherently changes the executed code and hence changes the latency - potentially by many many many clock cycles. No surprises there. The extent and significance of such jitter is, of course, dependent on the application's requirements.

That's not jitter. Jitter is tiny changes in your audio clock. Let's say your clock is 48KHz * 16 bit *2 = 1536KHz. the MCU has hardware that will push out 1 bit every 1us or so. If your buffer is empty, it will either raise an exception, or send zeros or something. The change of this frequency would be jitter. So your crystal oscillator changing 10ppm, or your PLL is working on a fractional division or something like that. The former is not called jitter, so please stop using it on a technical forum, especially since there is so much rubbish information out there for audiofools.
And yes, you need to make sure that any calculation on the signal is done in worst case conditions, like any other real time system.

[tggzzz]

You are mixing two concepts. Jitter is on the nanosecond level, less than 1 clock. Latency is several clock long.
It doesn't skip, the latency is stable, plus there is some minimal buffering for the USB stream. This increases latency.
The jitter, it depends on the crystal used, and the PLL. I didn't investigate jitter.

I am well aware that latencies that vary cause jitter in the output timing.

Where software is involved, any path differences in the executed code can cause jitter. If you are "multiplexing" many activities/processes (in the CSP sense) onto a single processor, that inherently changes the executed code and hence changes the latency - potentially by many many many clock cycles. No surprises there. The extent and significance of such jitter is, of course, dependent on the application's requirements.

That's not jitter. Jitter is tiny changes in your audio clock. Let's say your clock is 48KHz * 16 bit *2 = 1536KHz. the MCU has hardware that will push out 1 bit every 1us or so. If your buffer is empty, it will either raise an exception, or send zeros or something. The change of this frequency would be jitter. So your crystal oscillator changing 10ppm, or your PLL is working on a fractional division or something like that. The former is not called jitter, so please stop using it on a technical forum, especially since there is so much rubbish information out there for audiofools.

While you give one example of jitter, there are many many other valid examples of jitter. Audio systems are a vanishingly small subset of hard real-time systems.

For a less blinkered definition see, for example https://en.wikipedia.org/wiki/Jitter which has no mention of magnitude of the timing variation.

I/O jitter can occur when a buffer  becomes empty (output) or full (input) - for whatever reason. Anybody that has worked comms systems containing software will know that scheduling problems are a serious concern in that respect. One significant source of those is code developed elsewhere which doesn't have specified time-domain performance. Another is the performance of an external system connected to yours.

To reuse your tone of voice, if you are going to comment on a technical forum, please realise that your technical knowledge and understanding cannot encompass everything.

And yes, you need to make sure that any calculation on the signal is done in worst case conditions, like any other real time system.

Calculations are good. Measurements are not, because you can (and probably have) missed the worst case.

What calculations can you do on most systems, especially those containing caches, using interrupts, using third party libraries? Detalls please.

[tszaboo]

You are mixing two concepts. Jitter is on the nanosecond level, less than 1 clock. Latency is several clock long.
It doesn't skip, the latency is stable, plus there is some minimal buffering for the USB stream. This increases latency.
The jitter, it depends on the crystal used, and the PLL. I didn't investigate jitter.

I am well aware that latencies that vary cause jitter in the output timing.

Where software is involved, any path differences in the executed code can cause jitter. If you are "multiplexing" many activities/processes (in the CSP sense) onto a single processor, that inherently changes the executed code and hence changes the latency - potentially by many many many clock cycles. No surprises there. The extent and significance of such jitter is, of course, dependent on the application's requirements.

That's not jitter. Jitter is tiny changes in your audio clock. Let's say your clock is 48KHz * 16 bit *2 = 1536KHz. the MCU has hardware that will push out 1 bit every 1us or so. If your buffer is empty, it will either raise an exception, or send zeros or something. The change of this frequency would be jitter. So your crystal oscillator changing 10ppm, or your PLL is working on a fractional division or something like that. The former is not called jitter, so please stop using it on a technical forum, especially since there is so much rubbish information out there for audiofools.

While you give one example of jitter, there are many many other valid examples of jitter. Audio systems are a vanishingly small subset of hard real-time systems.

For a less blinkered definition see, for example https://en.wikipedia.org/wiki/Jitter which has no mention of magnitude of the timing variation.

I/O jitter can occur when a buffer  becomes empty (output) or full (input) - for whatever reason. Anybody that has worked comms systems containing software will know that scheduling problems are a serious concern in that respect. One significant source of those is code developed elsewhere which doesn't have specified time-domain performance. Another is the performance of an external system connected to yours.

To reuse your tone of voice, if you are going to comment on a technical forum, please realise that your technical knowledge and understanding cannot encompass everything.

And yes, you need to make sure that any calculation on the signal is done in worst case conditions, like any other real time system.

Calculations are good. Measurements are not, because you can (and probably have) missed the worst case.

What calculations can you do on most systems, especially those containing caches, using interrupts, using third party libraries? Detalls please.

There is one definition for jitter in terms of digital audio. Definitions for other systems don't apply.
https://en.wikipedia.org/wiki/Audio_system_measurements#Digital

A measurement of the variation in period (periodic jitter) and absolute timing (random jitter) between measured clock timing versus an ideal clock. Less jitter is generally better for sampling systems.

Also you are on your own in this discussion, I'm done.

[tggzzz]

You are mixing two concepts. Jitter is on the nanosecond level, less than 1 clock. Latency is several clock long.
It doesn't skip, the latency is stable, plus there is some minimal buffering for the USB stream. This increases latency.
The jitter, it depends on the crystal used, and the PLL. I didn't investigate jitter.

I am well aware that latencies that vary cause jitter in the output timing.

Where software is involved, any path differences in the executed code can cause jitter. If you are "multiplexing" many activities/processes (in the CSP sense) onto a single processor, that inherently changes the executed code and hence changes the latency - potentially by many many many clock cycles. No surprises there. The extent and significance of such jitter is, of course, dependent on the application's requirements.

That's not jitter. Jitter is tiny changes in your audio clock. Let's say your clock is 48KHz * 16 bit *2 = 1536KHz. the MCU has hardware that will push out 1 bit every 1us or so. If your buffer is empty, it will either raise an exception, or send zeros or something. The change of this frequency would be jitter. So your crystal oscillator changing 10ppm, or your PLL is working on a fractional division or something like that. The former is not called jitter, so please stop using it on a technical forum, especially since there is so much rubbish information out there for audiofools.

While you give one example of jitter, there are many many other valid examples of jitter. Audio systems are a vanishingly small subset of hard real-time systems.

For a less blinkered definition see, for example https://en.wikipedia.org/wiki/Jitter which has no mention of magnitude of the timing variation.

I/O jitter can occur when a buffer  becomes empty (output) or full (input) - for whatever reason. Anybody that has worked comms systems containing software will know that scheduling problems are a serious concern in that respect. One significant source of those is code developed elsewhere which doesn't have specified time-domain performance. Another is the performance of an external system connected to yours.

To reuse your tone of voice, if you are going to comment on a technical forum, please realise that your technical knowledge and understanding cannot encompass everything.

And yes, you need to make sure that any calculation on the signal is done in worst case conditions, like any other real time system.

Calculations are good. Measurements are not, because you can (and probably have) missed the worst case.

What calculations can you do on most systems, especially those containing caches, using interrupts, using third party libraries? Detalls please.

There is one definition for jitter in terms of digital audio. Definitions for other systems don't apply.
https://en.wikipedia.org/wiki/Audio_system_measurements#Digital

A measurement of the variation in period (periodic jitter) and absolute timing (random jitter) between measured clock timing versus an ideal clock. Less jitter is generally better for sampling systems.

Also you are on your own in this discussion, I'm done.

This isn't an audio system. It is an audio+MCU+USB(+host) system.

There are different definitions of jitter for USB systems. The USB system could have a deleterious consequenses, as could other processing inside the OP's MCU.

I don't know what you mean by "on your own in this discussion", but SiliconWizard's post https://www.eevblog.com/forum/microcontrollers/microcontroller-with-usb-audio-stack/msg5342681/#msg5342681 contains useful corroboration.

[tszaboo]

I don't know what you mean by "on your own in this discussion", but SiliconWizard's post

As in: I don't care anymore what you have to say about the topic.

[tggzzz]

I don't know what you mean by "on your own in this discussion", but SiliconWizard's post

As in: I don't care anymore what you have to say about the topic.

Ah. I thought it was as in "I realise I'm wrong and so will bow out".

[alex_]

Implementing this on XMOS devices was pretty straightforward and quality was excellent, especially if you have I2S and a masterclock from your amp.
Be very careful with buffer under/over flows, sending DC to the amp is generally a bad idea always keep a high pass or some failsafe in your DSP if you can.