Project: Digital audio router

[paulca]

I'm not calling it a mixer for a reason.

Problem:  I get fed up having multiple audio sources and multiple audio outputs and most often, no way really to switch/mix between them.

Current setup and example of inflexibility:  I have one feed from 2 PCs, which I manually swap, which are analogue 3.5mm stereo jack outputs.  I have a multi-output headphone amplifier such that I can direct that output to either speakers, headphones or both.  Speakers are self powered Newark monitors.  Headphones are powered by my custom OP551 monster amp.  The beringher headphone amp is just as an output splitter with level control. The inflexibility comes when you consider the wireless headphones and wireless headset.  When these are used, only one of the two PCs supports it and it is software control panel level selection of that single output device. (Speakers and headphones outputs mute).

Ideas towards solution:  A centralized routing box which can take, say,

* USB audio from several sources, (2 say),
* Provide an AUX 3.5mm jack ADC input  and/or RCA Phono, maybe SPDIF.
* Provide both a Bluetooth classic source, but also provide a sink (such that the phone et.al becomes a valid source).
* Provide a way to switch between inputs and outputs and ideally provide a way to mix them together, (example: I might be listening to Spotify on my phone with my wireless headphones as output, both bluetooth but would still like to hear desktop notifications from either PC.)
* Some way to control the basics with tactile, mechanical controls - tactile switches and rotary encoders with a master output pot on the analouge output to speakers.  Maybe a "Mute all" button.
* Lots of nice to haves.

Without refining those down into MoSCoW priorities, I started looking into the practicalities of all of the above!.  Seems daunting.  It is daunting!  I'm looking at a year maybe here.

Bluetooth.  ESP32.  Can be both source and sink, although having worked with it, I would suggest the ESP32 would make a good BT source OR sink and if it is that it does nothing else.  Task management and memory management with the BT stack is dubiously unstable.  I can have 2 of them, they are cheap and can run fairly "encapsulated" and not require a lot of PCB work.

ADCs.  I'm not an audio-phool.  If it provides me with clean 16bit 48K with no processor or HDMI buzz/hiss I would be satisfied.  If it can be pushed, even under restrictions to 24bit 96k I'd be happy.  I don't think my ears would care about 192k, I'll leave that to the performance tests later.  Thus, any basic common or garden variety ADC that outputs I2S will do.  If said ADC supports all three inputs, such as 3.5mm jack, rca phono, SPIDF (even if mechanically switched or just static mixed) all the better.  I'd even go as far as excepting the input front ends to be off the shelf ADC modules from AliE.

DACs for analog line level 10k speaker output, same again, even a PCM102, maybe a higher end one, (thats not hard to find!).

Controls, rotary encoder(s), tactile switches and a TFT touch screen, maybe a WS2812b output for levels etc.

All of the above is pretty much given, even as I said to using off-the-shelf modules for most of it.  I'm not making a pocket watch!

The hard part will be tying it all together.

One option is a high end DSP I2S controller chip like a Cypress.  Something that will have half a dozen or more half duplex I2S, a few duplex I2Ss and enough power to run half a dozen IIR/FIR filters, a few resamples and a few down mixes.  To me this sounds more like a career with high costs in time and money to get started.  Probably looking at £50-100 DSP chips on 100+ pin BGAs.

The other option that is far more appealing is to use some high performance MCUs.  There are a few of the SMT32H7 and STM32F7 models aligned for I2S, DSP and SAI.  I think the highest I seen was 8 in total of half duplex SAI or I2S.  Would they have the power, don't know, need to test.  Would they have the power without exceedingly well written code?  Again needs to be tested.  As this is a one/two off design, I can afford to add a few STM32s and "tier" or "tree" them.

As an example.  An STM32F411 is perfectly capable of running a USB Audio Class device front end, resampling it, normalizing it and sending it out half duplex I2S or SAI.  It might even have the power for a basic 3 band EQ.

Similarly STM32F411's can run one or more ADC front ends and provide similar pre-processing.

It still does not reduce the amount of I2Ss I have as Ins and Outs to be routed, but I can simplify the total routing paths into a sensible set and downmix into groups before downmixing and upmixing / EQ'ing the outputs.  I still need research as to what horse power I actually need, I might be able to combine a lot of things onto two large MCUs or ... probably easier to spread it out across 4 smaller componentized ones.  Like a pipeline.

If I manage to do this I expect I will never, ever want to see another I2S clock issue in my life.

[paulca]

Research done: 
ESP32s as BT source and as BT sink work fine. 
ESP32-A1S module w/ ES8388 codec.  Both with I2S sink/source.  Duplex, single stereo stream.
I'm comfortable the ESP32 will handle the BT source/sink, but as mentioned, when you try and multi-task it, say with a BT input, an I2S output AND then try and run an FFT on the user core it starts to come apart.
I have tested the STMF411CE will register fine as an USB Audio device.  Not tried to do much with the incoming data though, that's next.

Next up:
ESP32 BT sink + external DAC to headphones.
ESP32 BT source + external ADC from PC
ESP32 DAC -> ADC with or without filters.

STM32 I2S IIR filters, FIR filters, downmix, resample and upmix tests for horse power and peripheral count.

STM32 <-> ESP32 I2S full duplex
STM32 <-> STM32 I2S full duplex (multi)

Lots of breadboard tinkering and low level buffering C code, yummy.

[tooki]

Take a look at the ESP32 LyraT series of boards, designed specifically for audio. I’ve been meaning to pick one up for myself.

[paulca]

Take a look at the ESP32 LyraT series of boards, designed specifically for audio. I’ve been meaning to pick one up for myself.

Thanks.  I did buy an AITHinker Devkit with the ESP32-A1S module (which includes 64MBit PSRAM and an ES8388 Audio codec), but ... it gets complicated and it seems there is controversy with the AIThinker/Espressif modules.  It's a mess of pin out ambiguity, the ADF/IDF lacking the I2S master clock attribute of the interface etc.

I have also order an actual Lyra V4.3 which should open up the prototyping flexibility with the ESP32 Bluetooth front ends.

In the original post I left out what I intend to do with the headphones.  I searched and searched but have yet to find a i2S headphone amp which will produce more than 20-40mA output current at 5V p2p.  OPA551s aren't digital.  Whilst tired I did order to MAX amps, hoping to make a stereo pair for head phones, but.... nope, they are 2 channels of differential amps, push and pull.  So to noramlise them into single ended I would need op amps capable of doing so with a MAX of 1.5 watt output with +/- 12V rails. 

So I expect it will be a board trace analogue section from a DAC direct to a pair of OPA551s for the headphone output. 

Unless someone can recommend a non "Sony nanny state pac" I2S amp with 2 single ended outputs, aka headphone.

[moffy]

Don't know all the detail but the Raspberry Pi 4 has wifi and bluetooth, as well as a number of HATs that are stereo ADC in and stereo DAC out as well as a SPDIF output HAT:https://www.hifiberry.com/shop/#boards
Not sure if it would be suitable but worth a look.

[paulca]

Updates:

Finally cleared a breadboard and made a start on this learning, testing, researching phase.

I started with an STM32 Blackpill as a USB Audio device, using a simple PCM5201 as the headphone output device over I2s.

The good news:  I could get lovely clear digital audio over I2S into the PCM module and out of the headphones.
The bad news:  I ran into buffer sync issues with the STM32 Audio device library.  https://www.eevblog.com/forum/microcontrollers/stm32-usb-audio-(uac)-gt-i2s-buggy-drivers/

At least I produced a working path, except for the odd buffer over/underun.  I think starting with USB was a bad idea and I should start with an I2S ADC, a simple single master clock and test performance on how much can I do between buffer flips in terms of processing. Samples and if I need to go up to an F7 or H7 to get anything useful done.  I'm hoping the BlackPill F411 will be capable of at least running a 3 band EQ on a stream, and/or mix two streams together to one output.... maybe switch the I2S between two output PCM devices.  Things like that.

[dmills]

The elephant in the room is almost always clocking, and it is well worth giving this a lot of thought early on.

I would be reaching for a modest FPGA (Probably a small Zynq so I get USB with just a PHY), can get you resampling, clock dividers, EXTENSIVE mixing (Surprisingly processor intensive if you want to make sure there is no possibility of clicks when muting or unmuting or zipper noise when changing digital gain, but trivial in fabric), and enough processing to make for a nice user interface.

Got a box here doing 132:2 mixing with pretty much that architecture, mix of MADI, AES3, Analog and SDI. 

[Buriedcode]

IIRC Most USB audio implementations consuming data (PC -> DAC) use isosynchronous transfers by default, that either dither the devices clock to match the hosts clock - or double/drop samples to prevent buffer under/overrun.

I had a PIC24 project some years back that use asynchronous transfers, where the USB device uses a timer and the buffer level to send feedback to the host, so the host can send a different number of samples per packet (USB packet every 1ms, 48kHz sample rate, noramlly sends 48 samples, but can send 47, or 49).  This means the DAC can run off the USB devices clock rather than be at the mercy of the host.  I remember it being a bit of a faff though as the USB spec demanded the feedback be in a specific number format.

If you're doing anything more than a simple DAC (eg: DSP), an FPGA or XMOS device would probably serve you better.

[paulca]

For now I'm placing FPGAs on the "Won't this time" list.

This isn't a commercial product.  BOM costs are not that much of a factor I'm only going to produce one or two.

So that means I don't need to pack as much functionality as I can into devices.  I can spread the load out.  I am even considering using individual MCUs or hardware ICs for each digital front end.  eg:

2x BT Endpoints - 1 Source, 1 Sink.  These can be 2 ESP32s OR 2 BT ICs with I2S out.
3x USB Endpoints - All 3 sources, if I want LineIn/MicIn/Headset support that can go on the "Could do" list for later revisions. - These can be STM32s or hardware ICs like PCM2707 if I can get one this year.
2x Analogue line in - ADC ICs -> I2S

Outputs, we already have 1 BT Sink.  But I feel a single USB digital out, 2x Analogue Line Outs,  1x Boosted headphone amp would be okay.  Maybe a space I2S header or two i/o if available.

There is a lot there, but I might be wrong, I am thinking giving many devices small simple tasks that do not tax them and can survive with less than optimised coding, then integrating them under MCU control will be easier than buying a "mothership" IC like a large FPGA or a Cypress DSP and doing it all in ONE chip.

[paulca]

After a bit of a head banging session with the STM32 HAL Libraries: https://www.eevblog.com/forum/microcontrollers/stm32-i2s-w-external-master-clock/

I managed to get a "a-typical" signal path up and running with the topology:

WM8786 ADC (Slave 16bit 48K) -i2s-> STM32 (Master/Master) -i2s-> PCM5102 (Slave 16bit 48K)

Additionally I added other tasks, just to test the water on how picky things are on timing.  It looks like about 25% load duty cycle just copying the buffers int, by int.  Which sounds bad, but that's 250uS on a single frame buffer.

See:
https://www.eevblog.com/forum/beginners/breadboards-and-mhz/

Giving it a much more beefy task like, calculating and rendering a peak level indicator and I had to get much more careful and spread the task out around the "super loop" and not do it all in one cycle.  I even ended up making the "super loop" "ordered" and the audio buffer processing mutually exclusive so it always gets priority.  Even with that, the rendering of the single channel meter on a TFT screen (a bar 20 pixels by 320 pixels), took longer than a single or double frame buffer.  I had to go to a whole 4ms buffer time.

Firstly though, the TFT driver is not using DMA, just interrupts, so it's pretty invasive, a DMA driver might be better, but ... It's not a valid use case really.  No audio processing MCU is going to been made deal with anything more than a tiny oLED in this project.  There will hopefully be a 4" TFT touch screen, but, the MCU 'owning it' will not have audio time critical tasks.  UI / HID inputs are incredibly slow in comparison and can be queued, wait and updates sent/received when time allows.

I'm studying up to see if I can pick up enough terminology to use/abuse and tune a filter library for the EQ stage.  In parallel, other than continuing the breadboard prototypes I am exploring the architecture design.

There are still questions surrounding the USB "front ends", aka inputs.  I hope to use high speed SPI for the internal streams.  I only need to couple to the audio clocks at the front-ends and rear-ends.  Where an I2S ADC, DAC or USB interface will have to actually clock out the audio.  For everything in the middle I do not need to slave to the audio clock and can send many I2S streams over a synchronous time sliced SPI/DMA bus at 48Mbit/s+.

I have started a google sheet with timings for various packet/buffer sizes.  I need to form a complete end-to-end path analysis and workout total latency end to end.

Ideally, unless I implement endpoint feedback on the USB - which will primarily be the interface live audio comes in - I have set the optimistic latency budget to under 20ms.  If I intended to connect an instrument to it, it would be less than 10ms.  Luckily I don't.  However By 50ms (2 HD video frames) you might just start to notice A/V sync issues, by 200ms they will be obvious and annoying.

For a "worst" case scenario, assuming 16@48k, it works out at something like a 0.25ms latency to write 4 channels of I2S data to a 48Mbits SPI.  ie, the longest a channel has to wait on its next data is about a quarter of a milisecond.  Obviously, however, that goes up if you need to buffer for any reason.  Also, if a stream misses it's CS slot and has to wait on the next slot, it could be over twice that long.  Double the buffer size, double the latency.  However it tells me that SPI will not be the problem.  As long as I implement a "fair" contention policy or stick to the fixed "round robin" time slicing on the bus.

So... if I have a 4ms input buffer to allow for basic input processing like:  calculate the peak audio level, left/right, update an oLED periodically with the input level in %, periodically respond to a rotary encoder.  Also things like dead time detection, buffer under/overruns, anti-pop/zipper/buffer crash detection etc.  It's I2S stream gets sent on the SPI bus when it's CS line is pulled.

All 4 channels dump their streams on to a single SPI bus.  The bus mastering will be handled by one of a pair of STM32 bus processors.  Primary and secondary.  These will be where the mixing, routing and EQing happens.  So all inputs are SPI slaves, the primary bus processor (SPI Master) gives them permission, round robin, to off load I2S frames onto the bus for an agreed time.  However the other bus processor is a slave and is free to "spy on the spi".  Such that if it's been set up to process channel 3 it will watch the bus CS lines for channel 3 to start and buffer off it's I2S frames for processing onward.  Similarly the primary bus processor, the master, can buffer off whatever streams it is setup to process.

Hence the architecture thus far is sort of:

n Inputs -> SPIBus-> 2 processors -> SPIBus -> nOutputs.

Maybe, 4->1->2->1->4.

Other "channels" to consider are the HID controls for the processors.  I haven't really spent much time thinking about this other than the realisation the realtime nature of audio will mean that HID part will need to be decoupled out of the critical path... that's going to involve at least one more "admin/status/control" bus.  Although that would be a low speed, bursty, low priority bus.  Messages like errors, "Output 4 BUF_UR" causing a warning indicator LED or TFT panel to light up momentarily to communicate this.  Also more routine updates for channel levels, CPU load on processors, that kind of thing.  Obviously the ability to see the input channels, select them, select the mix levels, eq settings and the output channel to assign it to.  All have to be designed into a UI.

A a programmer though, I know what I'm going to write a command line UART interface first, just so I don't have to get bogged down into painstaking UI design.  As a programmer I can see that UART command interface being far too permanently temporary too!

Open Issues: 
* Fix/Implement endpoint feedback and Isochronous sync for the USB audio on STM32.... or explore the hardware availability for using hardware USB Audio bridges like PCM270x.
* HAL Driver bugs are irritating, every time I regen my code from MX I have to go and fix their bugs again.  Need a more workable solution without leaving MX behind completely.
* Implement a working stream drop/pad for reclocking broken streams or fixing buffer over/under runs.
* IC Availability issues and exploring AliExpress sources to see if they work.  If it works as it should I might not care if it's a fake at this stage.

Questions:
* is going to 24bit internal audio format and using the extra 8 bits as pure digital headroom, before limiter/compressor it back to 16bit at the output stage.....  going to be worth the extra processing overheads.
* How will SPI Slave timing work in terms of the critical CS lines.  If a slave is not immediately ready to send when it's line is pulled this has to be detected somehow.  I will read up, but I expect this issue can be solved by allowing either end control over the CS line.  The bus master pulls the line low for a very short amount of time at the start of the slaves window.  The slave has to immediately respond pulling the CS line low again, then transmitting.  The issue I see here is the bus master is handing control to the slave, if the slave gets "naughty" and hogs the bus, its a bug, but a nasty to fix one.  It also has electrical considerations when asking can the master pull the line harder than the slave to "cut it off"... I don't think that would matter, if it's stomping on the MOSI/MISO it's irrelevant what the CS lines are saying. So the bus has to be "consensual" and "fair" on both master and slave sides.  If that can be maintained ... and I am the single programming writing both slave and master, then I can implement a dynamic bus mastering protocol which offers space to a slave, but revokes it if the slave does not immediately respond with interest by holding it's CS line low.  The master will then trust the slave to not hog the bus, allow it to hold the line low until it's finished, then round-robin to the next "fairest" slave.

[tszaboo]

The elephant in the room is almost always clocking, and it is well worth giving this a lot of thought early on.

I would be reaching for a modest FPGA (Probably a small Zynq so I get USB with just a PHY), can get you resampling, clock dividers, EXTENSIVE mixing (Surprisingly processor intensive if you want to make sure there is no possibility of clicks when muting or unmuting or zipper noise when changing digital gain, but trivial in fabric), and enough processing to make for a nice user interface.

Got a box here doing 132:2 mixing with pretty much that architecture, mix of MADI, AES3, Analog and SDI.

You can use a tiny FPGA or even a CPLD, if you use a resampler from TI (I'm sure there are others). Just connect the input and output I2S to the FPGA and route the signal through it if necessary
I was thinking about something similar some time ago, but ultimately gave up. When I sat down and wrote all the requirements, it ended up too long, requiring a lot of voltage rails, design work, pitfalls, and a lot of work to make this possible. It would be interesting to do, if someone would pay me to do it.

[Swake]

Sounds complicated   That one year project implementation estimate is very optimistic if you ask me. And unless it is your dream to build something like this it would be a show stopper for me as there are many other things I would do with that time.

Check a musicians site such as Thomann for 'audio interface'. Hundreds of devices to choose from all price categories (even the most expensive will be cheaper than your time is worth).

Going the old-school analog route I'd make a input / output matrix based on CMOS 4066 analog switches. ESP32 can do control them no problem. If you want to go the audiophile route replace the 4066 with gold-contact-relays.

This thing is a 8 input x 8 outputs router / on each of the input and outputs there is a 7 band parametric equalizer with delay, filters, etc... / fancy software / Wifi / etc... for 300 bucks: https://www.aliexpress.com/item/1005002668417414.html
I have never heard about that brand but maybe it is the best thing in the world.

Or score a second hand analog mixer for cheap.

[paulca]

I'm no audio phool, but I can tell the difference between a digital TV picture and an analogue one.  I can also, even with my old ears, tell the difference between a fully digital signal path over one that's been dragged up and down a few analogue paths.

It does often amuse  me when you hear audiophiles bleating about quality and then listening to classic rock and jazz recorded on wax cylinders (exaggeration).

Most of my music currently has a digital source to-me at least.  Yes a lot of the, say, Spotify or YouTube music I listen to is actually a rather poor recording of a vinyl record!  (I dont joke).  However a lot of it is purely digitally mastered, never been heard, content.  When you play the analogue output of the PC into an ADC path and out again, you can tell the difference straight away when compared with the same bitrate over a digital stream like USB.  Each analogue filter stage has a tolerance in it's components and response.  So the ADC and DAC filters will never perfectly overlap and thus repeated digitisation and analogue representation degrades the signal... especially when it's not always the same equipment doing it.

I was even debating using a hardware I2S headphone amp.  However, while they made a load of I2S digital amp stages, even fully digital class Ds ... they only make differential ones.  Nobody makes an I2S headphone amp with more than like 50-70mW output.  I can certainly use 2 differential amps that each put out 1W of power and then use a few high current opamps to single end them.  Or I could just use a standard DAC and 2xOPA551s.

I did start a project about 2 years ago, to do it the analogue route, following along with a book.  You quickly find that to do things "properly" and not run into issues, you need a LOT of op-amps to buffer, mix, filter, etc.  At EVERY stage.  A single 1 channel fully paramteric EQ stage was about 3 OpAmps.  For a single MONO channel with input, 3 band EQ, balance, bus level, send/return bus, totalled about 16 op amps.  For 4 input, 4 output stereo I needed 8 input rails, 4 bus rails, 4 output rails and a bus intermix stage.  I did design and even prototype a few of those stages and I did set about planning out the schematic, but it became far, far too much work.

In contrast in this project I am making heavy use of either MCUs or precanned digital components.  The ADCs and DACs I have selected have no I2C config as they have no real config.  I'm standardizing on bitrates/bits per samples.  I'm liiting my use cases.  I think it's going to be soldering a half dozen ICs, decoupling caps and such to a board and the rest will be software.

[paulca]

To recap on my "power hungry - sounds dangerous" headphone amp specs...

Typically when listening to music I run an EQ which has a large bass shelf up to +15db at 50Hz. +3db at 100Hz.   -5db around 1-2kHz and fading out to a 0db shelf for the high end.  They move around.  (this is a software eq in a browser).

So there is at least 18db across the EQ, such that if I am actually able to generate enough current to get, say 95db of bottom end bass, I'm only getting less than 80db on the mid range.  Exactly as the EQ is intended for.  With the mid range being where you ear is most sensitive it is where loud music does the damage.  Organised bass frequencies are much less damaging.

The high bass needs current.  Standard OpAmps for headphones and phoneamps themselves tend to use current limiting of around 20mA.  A good one might do multi-band limiting first, which will invariably filter OUT the bass first as it is the current hog.  I want to avoid all of them just by using opamps that don't liimt at 20mA, but will take 100mA.

Yes... If I hit "PLAY" and didn't check volume levels and there is no EQ engaged, it's FAR too loud.  I don't do this.

[SiliconWizard]

Note that it's impossible to size your headphones amp without specifying the kind of headphones you want to use with it.

[paulca]

Note that it's impossible to size your headphones amp without specifying the kind of headphones you want to use with it.

True.  My OPA551 amp was actually calibrated with simulated output bode plot to set the final resistor and AC coupling cap values based on my specific headphones.   AudioTechnica MX50.  I believe they are 38Ohm.  Vague memories of 100Ohms per channel.  Which balanced the 220uF output capacitor allowing an acceptible frequency response.  I even think I calculated it to the db with the sensitivity with impedance etc. etc.

What I failed to do in the first version was increase the impedance of the virtual ground reference voltage divider so it mimicked high frequency power supply noise.  Fixed by a helpful person on here.  The downside of increasing the impedence on the voltage divider for VGND is it takes nearly 10 seconds for the VGND to charge it's caps and stabilise.  Smaller caps and a postponed "enable" pin... or a by pass circuit to charge those caps quickly at first.

The other other "Ohh..."  I had recently is that while I have OPA551s doing the main audio amplification stage.  I believe I only have an NE5532 buffering the VGND.  My lack of understanding of OPAmps may have led me to not provide enough current path for the return current on that virtual ground which need to be handled by the VGND buffer amp.  If I did indeed do this, it will mean when the OPAs exceed the current limit of the NE5532 it will current limit and the VGND will become unstable.

Neither hear nor there, that circuit needs to be pulled out of the archives and reimplemented.  So those issues can be addressed.

[paulca]

... and importantly that headphone amp has been faultless in service with said faults/short comings for 2 years!  It works.

If anything I think I went to conservative on the final output stage.  I think increasing the cap size and lowering the resistors and not even attempting to limit the output, right up to the absolutely power limit of the headphones or beyond, might be the next approach.  A 50Ohm and a 440uF.  (playing).

EDIT:
The other consideration is the input voltage range.  It will be a normal headphone amp with 6V input.  It will take up to 16V input, limited by the input bulk capacitor.  It's been calibrated for that worst case of potentially 12-14Vpp.  That driving the final RC output circuit.  Thus, probably, when running on a normal day-2-day voltage of 12V like I run it a lot.  It is not capable of outputting it's full potential and the output "curbing" is over restrictive.

In short, the result of this is... I have tolerated full volume output of the amp with no gain left at any stage from source through to headphones.  Thus it's too quiet. 

The volume control I want is like the 1970s TRIO amp I used to have.  It had switchable speaker and headphone outs.  When you switched the speakers off and left just the headphone, the volume knob anything beyond about 60% would distort even the best headphones and pushing it would melt their cores.  I destroyed quite a few pairs of headphones with that amp.  Cheap ones mind.  I want that responsibility.  I WANT an amp that can melt my headphones.  Such that the output is fully under my control.

[SiliconWizard]

IME, if you want to be able to cover a wide range of headphones from the small, relatively high-impedance earbuds to the big high-end 8-ohm ones, you'll need up to 1W RMS. That's usually what I've been targetting when designing headphones amps. Anything less than that, and for some headphones (such as AKG or the like), the volume will be very faint. Something that is likely to be a problem if you design pro gear using pro headphones. Now if you target only entry-level headphones/earbuds (which are usually higher impedance and with higher sensitivity) such as what is used with phones, laptops, small music players, etc, then a few 10s of mW is enough, and a simple opamp-based amp may be adequate.

[paulca]

The tests I have done with standard "line" outputs driving headphones from various ICs tells me their output level is almost fine.  They just lack that little extra, especially when you roll in the bass boost.

The PCM5102 output at full volume with headphones connected directly is pretty good.  It just lacks that extra kick to push higher end headphones with high bass bias.  The max level output is very well "managed" and "peak free", such that it's easy to deal with for downstream components like headphone outputs....

It just lacks any balls.  If I can say that.  It takes current to drive headphones at low frequencies at large defection.  Most "standard consumer audio ics" don't delivery that.  By design.   Sometimes it's a bit of a leap to side step them and re-implement them with high current amps.

[paulca]

At least.... in the current case of the project.. I can fix my output levels digitally.  Then I can amplify a known input signal at the output.  I can know it's exact properties and do not need to accommodate a wide range of possibilities.  The headphone output section has only one input and it's digital.

[DrGeoff]

Why not just build/buy a monitor switch (used in studios) to switch the audio signal input connectors to various audio signal output connectors? These are commonly used for multiple input sources (CD players, consoles, iDevices etc) to switch to one of many monitors/speakers.
I built one that does 2 balanced stereo inputs and 4 balanced stereo outputs for switching between monitor speaker sets when checking mixes.

[tszaboo]

Yeah, the headphone amp get's tricky, if you want it to drive anything from 600 Ohm to IEMs.
My answer would be to have two outputs, one with a TPA6120A2 and one with a higher current driver.
Read the blog of the NwAVGuy, he wrote a bunch of text about headphone driving, and practical measurements.
And then you start thinking about differential output with that Sony penta connector, and design half a dozen headphone amplifiers without building a single one. Not speaking out of experience. 

[paulca]

2 outputs is the best way to solve that, you are right.

I can have a clean, PCM5201 output which is good enough to drive headphones for most everything except high volume and will also feed a 10K line in perfectly.

Then I can add the digital headphone amp  experiments as a swappable module.

I'm still chuckling that back then when I was planning the analogue version of this lots of people kept telling me that analogue op amp mixers died in the 90s and reminded me I was following a book written in the 80s!

Now I have figured out it will be easier (for me) to do digitally and lots of people are telling me I'm mad and I should do it analogue. 

Anyway.  I shant get too confident yet...  there are several PoCs coming up that could make or break it.

1.  A downmix.  Get an MCU to receive more than one audio stream and correctly mix and realign them in the output buffer as one. 
2.  EQ processing.  What kind of MCU power do I need for an effective EQ.  Will an M4@100Mhz do?  Do I need an H7@550Mhz for 5 band parametric stereo?  I just don't know yet.

[paulca]

Update.

Prototype 2 channel down mix successful.  No cross fade or levels, that will be next.

Funny thing.  You can treat audio samples as 16 bit unsigned ints.  (not that treating them as series of 8 or 32 bits makes lots of noise).  This work right up until you try and add two of them and divide by 2.  Then you get lots of interesting special noise effects.  Doh!  Works fine as signed 16bit ints as long as you remember to cast part of the expression to 32bit or it may clock over.

The mechanics are fairly simple in this case.  Both streams are the same paramters. One comes from USB, one from an i2s ADC.  Each gives me 96 samples or rather that is the way I set up the DMA buffers to align with the USB 1000Hz SOF packet delivery...

Each does it's signalling in it's call backs and main() spins on if( usb_available && i2s_availble ) flags.  When both are ready, it for loops them, x+y/2 and writes to the output buffer.

Other than adding a POT for cross fading the ratio, I have two other things to address:

* Detecting when a stream has stopped and stop/pause the DMA or better, switch it to an alt buffer of silence.
* clock skew will eventually cause the pointers to catch / get caught by the DMA pointer.  As everything is running off the same clock except for USB, it's that stream which will slip... means I still need to detect this and byte dup/drop to realign the buffers.
* Load testing.  These F411s are not massively powerful, but they do come with a large array of SPI/I2S with a total of 5 half duplex I2S and I think 2 of them can be full duplex.  It might make more sense to do the down mixing/up mixing on the F411s and leave the (on order) H750 to just do the EQ.  It's hard to know what these little things "can" do.  I suppose I can set GPIOs and scope it to see how long it's waiting (for buffer alignment), how long it's processing.

I know when I added the TFT to the previous proto it had to be done really carefully, but there was quite a margin with a single stream copy byte by byte in a for loop.  Could an F411 sustain 4 stream I2s streams averaged down to a single I2S?

[paulca]

Setup.

[paulca]

On audio quality.  When I got a clean USB to PCM5102 stream and the headphones connected to a proper amp with an EQ on it, I could tangibly notice the far higher dynamic range and stereo separation compare to the analogue output from the PC sound card.

It was also very apparent NOT to exist on most of the "straight off the vinyl" on Spotify, but pure digital recordings stood out a mile.

Note that the sound card in my PC is not a cheapo.  It's about as baller as they come on a CrossHair Hero VIII.  (It has gold caps LOL True... though I jest.)  It is quiet compared to most I've used.  The PCB layout has had some money spent on it with "via stitch guards", board cut outs, and the codec itself is under a can.

Everything is 48K/16Bit.

[paulca]

Like most things it looks simple once you figured it out.

Code: [Select]

void init() {
HAL_I2S_Receive_DMA(&hi2s1, (uint16_t*) i2s_channel_buffers, BUFFER_16B_SIZE*2);
}

void USB_Data_Callback(uint8_t *pbuf, uint32_t size) {
usb_channel_read_ptr = pbuf;
usb_available = true;
}

void HAL_I2S_RxHalfCpltCallback(I2S_HandleTypeDef *hi2s) {
i2s_channel_read_buffer = 0;
i2s_available = true;
}
void HAL_I2S_RxCpltCallback(I2S_HandleTypeDef *hi2s) {
i2s_channel_read_buffer = 1;
i2s_available = true;
}

void process() {
if(usb_available  && i2s_available) {
usb_available = false;
i2s_available = false;
for(int i=0;i<96; i++) {
int32_t ch1_spl = ((int16_t*)usb_channel_read_ptr)[i];
int32_t ch2_spl = i2s_channel_buffers[i2s_channel_read_buffer][i];
out_channel_buffers[out_channel_write_buffer][i] = ((int32_t)ch1_spl+ch2_spl)/2;
}
out_channel_read_buffer = out_channel_write_buffer;
out_channel_write_buffer = (out_channel_write_buffer+1)&1U;
out_available = true;
}

if( out_available && first ) {
first = false;
HAL_I2S_Transmit_DMA(&hi2s2, (uint16_t*)out_channel_buffers, BUFFER_16B_SIZE*2);
}
}


[paulca]

Today I took the previous prototype, added 2 ADC channels connected to 100K pots.  Ran them on DMA as slow as they will go writing a 64/2 buffer.  Then I take the average of the 32 values for each channel... each time there is no audio to process.  Converted to floats of 0.0 to 1.0 they are used to gain each stream.

Toggling a GPIO low when it begins processing a buffer and high when it's becomes idle again and ...  For 2 48K stereo 16bit streams it can gain each and add them in 10-11% duty cycle at 1kHz. 

So that's only 10% load capacity on an STMF411.  If I target no more than 50% to leave timing room for all the "need to haves" like muting stopped streams, buffer overrun/underrun detection, etc.  I could still downmix all 4 input channels on a single STM32F411.

There is still more analysis to do with the scope.  I need to set it up so that I can see the buffer callbacks/handovers and determine the total latency caused by the buffer alignment from the persecptive of each stream.

I also need to explore positive gain and overflow detection (clipping) and at least lock it off at he top and not let it overflow to the other end of the range.  Possibly blink a clip light.  I suppose if I just do calculations in 32bit and I can choose if I shift or clip samples that don't fit back into a 16bit int.

[paulca]

Been ill, then it's been christmas, not had the will or the ability.

I did manage to draw a first high level block diagram.

I don't need to implement it all at once, it's modular.  You should get the idea.  F411s act as end point for 1 or 2 I2S sources.  They offload to one of 2 SPI buses.  They allow each I2S to be assigned to one bus.  2 Bus processors mix and EQ whatever audio arrives on their bus, round robin-ing the slaves.  Again 2 output buses are available and each output can be selected between Bus A, Bus B or Mute, the same as the inputs.

Click, then right click "Open in new tab" to see full size.

Big questions.  Clocks.  Some things available in easy to prototype form (modules) have their own clocks and others just have no master clock at all.  Depending on a few things I might be able to arrange it such that clocks carry through, although this would limit the N2N path matrix ability if, say, only sources from BusA can be output on BusA and thus only to outputs which pick up the input clock from the Bus A sources.

Actually, if I just increase the number of buses to the number of clocks that might work.  A Bus is the I2S + the MCLCK that came with it.  Adding an ability to switch clocks on the outputs would reopen the N2N matrix ability.

I need to fully test the Atmel USB interfaces.  I have tested the BT sink, but need to find and test a BT source (or use an ESP32).

[paulca]

Use cases in terms of the diagram.

I can see in "normal" usage patterns that one bus will be "General desktop", have a mild EQ, low gain and mostly output on the "Optical" to the desktop speaker (or Line Out until I upgrade them).  Bus B will be set up for music with an aggressive EQ and spend most of it's time connected to the headphone output.

The unique (I believe) in this setup however is that technically any input can be routed to ALL outputs.  In fact ALL inputs can be routed to ALL outputs simultaneously.  However that is not a practical use case, clock slew and variable output latency is bound to make a mess of it.   It is highly likely though that desktop sounds and possibly even the phone audio will be connected to both the speakers (line or optical) and the BT source and the headphones at a low volume.  So all general inputs from USB for instance will go to all outputs at low level, even while high gain music is playing on the headphones and the desktop is mostly silent.  If/when that becomes annoying (Messenger chimes on your favourite album) a click on an encoder changes that USB input to MUTE or speakers only.

[paulca]

Tested the Atmel USB interface board tonight.  Works fine.  I cycled through everything the DAC would support form 48K@16 to 192K@32.  Couldn't hear a single difference.  I did not enable the DSD support, so no DTS/Dolby etc.  I'm not really sure what I would do with DTS or Dolby on I2S.  Does that work?  Is that a thing?  Is it multiplexed or just the LR-CLCK becomes a TDM clock?