How do ADCs work?

[Fungus]

This is a thing that''s been keeping me awake at night for years. I can't find a good answer (one that satisfies me) so I wonder if any of you lot have any insight.

How do ADCs work? I don't mean at a technical level, I get that part, I mean how do they work in practice.

Here's an example to explain what I mean:

Suppose a musician goes into a recording studio and plays a 22050Hz sine wave into the recording equipment. The ADC has to sample that wave and the mixing desk has to reproduce it. Let's assume for a moment the mixing desk and speakers are perfect.

The ADC works at 44100Hz so if it samples the sine wave using a narrow window at exactly the right phase it will sample the troughs and peaks of the sine wave and everything will work.

But ... what happens if it isn't in phase with the sine wave? Surely the wave will be attenuated, or even vanish completely (eg. you might sample the zero-crossing points using a narrow sample-and-hold window and it will vanish).

Even if the musician plays an 11025Hz sine wave the problem is still there. If you sample exactly in phase using a narrow window then you'll get the peaks and zero-crossing points and everything is perfect. OTOH you might sample 45 degrees out of phase and the signal will be attenuated by 1/sqrt(2). For any sampling method I think there will be a phase angle where this much attenuation will happen.

How is this problem solved? Am I wrong in assuming they do anything solve it? Do they just sample away and say "She'll be right!" when people like me ask stupid questions?

(Remember: This is the industry where you could mail the CD duplicators a carefully produced CD. They'd play the CD into a DAT recorder (with manual level control) and mail a DAT tape to the CD pressing plant. The plant would play the DAT tape into the line-in socket of a CD mastering machine and make the CDs from that. Yes, this actually used to happen...)

Bonus question: Would we get better music if we sampled at 192kHz, moved everything into the frequency domain then reconstructed a 44.1kHz PCM signal using that information?

[Richard Crowley]

The whole point of having a very high sample-rate is that the sample rate clock IS NOT "in phase" with the signal being sampled. In fact if you think about it, since music (or speech or any other natural sound) is NOT composed of nice sine-waves, the concept isn't really viable.  Natural sound is a complex and unpredictable waveform, and the ADC simply samples the instantaneous voltage every couple of microseconds and generates a number that represents the voltage at that time.

Now, you are correct that when you start getting to very high frequencies, approaching the sample-rate, you will get a roll-off artifact because the sample rate can't keep up.  That is why the sample rate is selected to be at least 2x higher than the highest frequency of interest.  Because of the filter technology back at the time, the consortium that devised the audio CD selected 44.1 KHz as the sample rate assuming that the highest frequency of interest is 20 KHz.

[amyk]

Yes, that is the well-known theorem. Sampling is always lossy. If you need a better representation of the signal, then increase the sampling frequency.

In audio, the majority of signals are mostly far below the Nyquist frequency enough that the loss of high-frequency components doesn't matter in the end result.

[Richard Crowley]

Bonus response: People have made a case for sampling audio at higher rates (88.2 KHz, 192.4 KHz). When we expect to do significant digital modification to the recording, it is better to have a higher sample rate to avoid processing artifacts. And higher sample rates are used to simplify the low-pass filters that must be used at the other end of the process (the DAC) where the audio-frequency waveform is re-constructed.  In many cases even the standard 44.1KHz sample stream is extrapolated and "oversampled" to make the output LP filter simpler (fewer large and expensive capacitors) and better (less phase distortion from heavy filtering, etc.)

[AndyC_772]

What a great many people don't really understand, is that reconstructing a sampled waveform is not the same as simply joining the dots. You have to carefully consider the frequency content of the curve you're fitting through those points.

If you sample a sine wave at 4x its frequency, then you're quite right that the samples you get depend on the relative phases of the sine wave and the sampling clock. You might get 0, +peak, 0, -peak, or you might get two samples at +peak/sqrt(2) followed by two samples at -peak/sqrt(2), or any other set of sample points 90 degrees apart.

In all these cases, there's exactly one possible pure sine wave that fits through all those points. The sampled points don't necessarily have to lie at the peaks; the reconstructed wave ends up having the correct amplitude regardless. If you 'join the dots' with straight lines, you'll end up with a waveform that has sharp kinks in it, and the frequency spectrum of that waveform contains high frequency components that simply aren't there in the sampled data. In other words, the join-the-dots process has added high frequency noise.

Even if your DAC introduces sharp kinks or steps, you'd still recover the original waveform if you were able to pass the reconstructed data through a 'perfect' low pass filter with a cut-off at fs/2.

If you want to achieve an audio bandwidth of 20kHz, then theoretically this can be done perfectly by sampling at 40kHz. Sampling at higher frequencies is useful, but only because real filters aren't perfect.

[dom0]

Besides: audio ADCs and DACs are not simple zero-order hold converters, but usually delta/sigma converters with additional interpolation and decimation filters. That's why they can be flat to 20 kHz with 48 kHz "apparent" fs.

[cyr]

The answer to your specific example is that a 22050Hz signal can *not* be sampled correctly by a 44100Hz sample clock, but everything below that can.
The actual sampled values of the 11025Hz signal may not reach the peaks of that sinusoid, but there is only one possible sinusoid that fits through all the points and is below the nyquist frequency, and therefore the original signal can be reconstructed.

This is in theory, in practice it won't be quite true because we don't have perfect filters or perfect ADCs with infinite resolution and no jitter etc.

[dmills]

This is one of the most common misunderstandings about sampling, the sample rate MUST be strictly greater then twice the bandwidth (Not equal to, greater then), it makes a huge difference to the maths.

The other common misunderstanding stems from those silly stairstep diagrams of quantisers you see in books about digital audio for the hard of thinking, yes, a given sample will be quantised to one of those levels, but when you add dither before quantisation output of the quantiser switches between levels so as to give the correct average output, the non linearity goes away and is replaced by noise ~3dB (for TPD dither) above the LSB, you can then hear well down below the noise floor, http://drewdaniels.com/dither.pdf is a copy of Vanderkooy & Lipshitz paper on that subject, for a paper from that pair is is surprisingly approachable.

There is a lovely demonstration of this with a 4 bit quantised piano on this web site that demonstrates the effect of dither rather well as well as a good demo of aliasing.  https://www.soundonsound.com/sos/feb08/articles/digitalaudio.htm (The whole article is mostly pretty good on the subject).

Regards, Dan.

[Fungus]

In all these cases, there's exactly one possible pure sine wave that fits through all those points. The sampled points don't necessarily have to lie at the peaks; the reconstructed wave ends up having the correct amplitude regardless. If you 'join the dots' with straight lines, you'll end up with a waveform that has sharp kinks in it, and the frequency spectrum of that waveform contains high frequency components that simply aren't there in the sampled data. In other words, the join-the-dots process has added high frequency noise.

Yes, I get that part. The part that bothers me is the attenuation of the signal. It seems unavoidable, so why isn't audio recording a hit/miss affair? Why don't musicians have record things several times until they get a recording they like because it synced well with the ADC? (OK, it's very unlikely they'd play a lot of notes at 11025Hz but you know what I mean...)

Also: If the input wave is at (eg.) 11024Hz you'll go in and out of phase with the ADC producing beat frequencies and all sorts of horrible artifacts.

[Fungus]

Why don't musicians have record things several times until they get a recording they like because it synced well...

Actually, sometimes they do 

[dmills]

Also: If the input wave is at (eg.) 11024Hz you'll go in and out of phase with the ADC producing beat frequencies and all sorts of horrible artifacts.

Nope, you are thinking curve fitting, not filtering.

11024 in at 44.1K system will have (abscent reconstruction filters) a component present at 11024, and one at n * (44100 +- 11024) for all positive integer n, when you filter off all the higher components you get just the 11024 AT the level you started with irrespective of the sampling phase.

Consider that Fs/4 example (because the math is easy) 0,1,0,-1 Vs 0.707, 0.707,-0.707,-0.707, now both uniquely specify the same periodic band limited signal, as they must  given that if there were any harmonics present they would fail to fit within the band limit (Second harmonic would be Fs/2 which fails to fit), so the only waveform that data can represent is a sine wave, which just leaves the question of amplitude.

Well, 0,1,0,-1 as a sine wave clearly has an RMS power of 1/Sqrt 2, as clearly does 0.707,0.707,-0.707,-0.707, so the amplitude of the only component that fits into the band limit is the same, irrespective of the sampling phase.

Regards, Dan.

[DanielS]

Yes, I get that part. The part that bothers me is the attenuation of the signal. It seems unavoidable, so why isn't audio recording a hit/miss affair? Why don't musicians have record things several times until they get a recording they like because it synced well with the ADC? (OK, it's very unlikely they'd play a lot of notes at 11025Hz but you know what I mean...)

How much real-world sound is any nearly exact fraction of the ADC/DAC's sampling frequency for any length of time? Practically never - no "natural" musical instrument with their associated musician or singer is capable of producing perfectly steady tones (within a few Hz) long enough for beats with sampling frequency to become noticeable.

With a phase drift rate of 0.0082 degrees per sample, the beats this generates are at -77dB (0.014%) assuming a full-scale input for the worst possible result, which is already well beyond the THD+N of most affordable power amplifiers.

[Lukas]

Yes, that is the well-known theorem. Sampling is always lossy.

But only technically. When sampling a perfectly band-limited signal, the sampled signal can be reconstructed without any loss.

Generally, I can really recommend this video: https://xiph.org/video/vid2.shtml

[coppice]

Yes, that is the well-known theorem. Sampling is always lossy.

But only technically. When sampling a perfectly band-limited signal, the sampled signal can be reconstructed without any loss.

To be more precise, sampling is lossless when sampling a perfectly band limited signal, using a Dirac delta function sampler, and an ADC with limitless amplitude accuracy. The sources of loss are by how much you fail to fulfil those 3 requirements.

Generally, I can really recommend this video: https://xiph.org/video/vid2.shtml

That is definitely the current go to source for information for those confused.

[Fungus]

Well, 0,1,0,-1 as a sine wave clearly has an RMS power of 1/Sqrt 2, as clearly does 0.707,0.707,-0.707,-0.707, so the amplitude of the only component that fits into the band limit is the same, irrespective of the sampling phase.

Yes, but when a DAC reconstructs the signal using those numbers it will generate a square wave of amplitude 0.707. The harmonics will be lost because they're beyond the range of hearing. That leaves us with a sine wave of amplitude 0.707.

[DanielS]

Yes, but when a DAC reconstructs the signal using those numbers it will generate a square wave of amplitude 0.707. The harmonics will be lost because they're beyond the range of hearing. That leaves us with a sine wave of amplitude 0.707.

What band-limited signal passes through +0.707V on both its way up and down from a peak, then -0.707V twice during the negative? A sine wave of frequency Fs/4 with a 45 degree phase relative to the sampling clock and 1V peak amplitude. That's the "reconstruction" part of the "reconstruction filter" - imagine a critically damped LC filter on the DAC's output. If you send it a square wave, it will overshoot and you end up with about 1V peak.

[dmills]

Yes, but when a DAC reconstructs the signal using those numbers it will generate a square wave of amplitude 0.707. The harmonics will be lost because they're beyond the range of hearing. That leaves us with a sine wave of amplitude 0.707.

Not if the reconstruction filter (Which is an essential part of a DAC) is even halfway sane...

If you take the reconstruction filter to have a sinc function impulse response (fairly typical) then you will find that both cases give the same amplitude at the output of the filter, again this is NOT curve fitting.

Reconstruction filters are NOT an optional part of a DAC.

Regards, Dan.

[dom0]

Reconstruction filters are NOT an optional part of a DAC.

At least if you want AC signals :-)

[Fungus]

That's the "reconstruction" part of the "reconstruction filter" - imagine a critically damped LC filter on the DAC's output. If you send it a square wave, it will overshoot and you end up with about 1V peak.

Reconstruction filters are NOT an optional part of a DAC.

OK, I think I'm starting to grok it...

I'm correct in thinking that the ADC is imperfect (and can never be fixed!). The bit I was missing is that the reconstruction is more complex.

At a theoretical level you use a sinc function to join the dots.

At a practical level you can either use either a closely-matched LC circuit on the output or oversample and do the sinc math in software.

Thanks for all the input!

[dmills]

Nothing is ever perfect, there is ALWAYS nose (thermodynamics dictates that), but a properly implemented ADC with properly designed anti aliasing filters and correctly done dithering is about as good as anything else is (and orders of magnitude better then tape or vinyl).
In fact, making the analogue front end as good as the ADC chip itself is a major engineering challenge, they are that good.

Most DACs actually use half band filters rather then sinc because it is computationally very much cheaper, and nobody today uses LC for the reconstruction (Well I am sure somebody does, but it is painful), multiple feedback filters around opamps are the normal way, together with oversampling.

The DAC is a system that includes the reconstruction filter, and once you view it that way you come to the conclusion that in a theoretical sense an ADC -> DAC chain is linear and just adds a little noise at a well defined level below full scale (Much as any circuit does).

Regards, Dan.

[hamster_nz]

I'm correct in thinking that the ADC is imperfect (and can never be fixed!). The bit I was missing is that the reconstruction is more complex.

It is not so much that it is 'imperfect', even an ideal ADC will have that behavior.

At the DAC end of the chain, the problem is that rather than it being a stream of infinitely narrow pulses the output of the raw DAC is a stream of flat-topped sample-wide rectangles. This is the equivalent to the ideal stream of impulses multiplied by a 'unit rectangle' (i.e. one sample period wide, with a value of 1), and in doing so it alters the frequency content. It's the frequency spectrum of the unit rectangle that is the problem.

To compensate for this, after the DAC the high frequency components ( > F/2) needs to be removed, and the fall-off that occurs in the frequencies < F/2 needs to be compensated for.