Low frequencies Vector Network Analyzer, arduino based.

[MasterT]

https://www.youtube.com/watch?v=83TpYkJQTyU&feature=youtu.be

Feedback is welcome on a project, I'm having fun with. 

Frequency range: 1k - 6.8M
  DDS synthesizer AD9833
Accuracy of the phase measurement better than:
 0.01 degree for frequencies < 100k.
 0.3   < 1M
 TBD > 6.8M
Modest results at high frequencies due to limited GBW of the MCP6S91.(Needs to be replaced. PGA feature is not active at the moment, both channels in buffer G=1 mode.)

Arduino adc sampling rate set 1 MSPS, 12-bits, 2 channels at 500 ksps. Sampling above 250 kHz is done in "undersampling" mode, aliasing number goes up to 28(!) at 6.8M. FFT size is 2048, 244 Hz/bin.

22-09-2018
/////  Updates, after some debates we have on two pages...


Project is inspired by AD5933 IC, to create something even better and lower cost.

Objective of this project - network analyzer - is to measure magnitude and phase.
What next is not completely  defined yet,  using precise phase data impedance could be calculated, capacitance, inductance, phase and gain of the OPA or any other amplifiers could be traced with one click on a knob.

Arduino DUE as a base platform was chosen, because it's cheap and well supported on the software side.

Primary goal is accuracy of this two parameters: magnitude and phase.
To get 0.01 degree or better, w/o paying to much for gold plated IC, extremely overpriced high speed ADC, etc. 

What I have is 12-bits adc, that is actually 9.6 (ENOB at 1 MSPS). So, SNR is less than 60 dB, not accounting for buffer amps, interference and all noise from digital circuits around.
60dB is Very low to reach the goal. To increase SNR ( and to calculate a magnitude & phase, of course) FFT-2048 was selected.
And here is the magic, fft provides 30 dB improvement in SNR, bringing it's up to 90 dB. That is about the same that someone could get out of expensive 16-bits ADC.
 If I make fft size bigger, sacrificing speed, even better SNR ratio could be reached, and consequently, more accurate measurement results for magnitude & phase.

Logic diagram (crystalized on page 2):  Phase Error - > SNR - > Bandwidth - > FFT

Now more software tricks...

I'd underline, highest priority is for phase accuracy. 
Phase is easily calculated out of the fft results:

Code: [Select]

double get_Phase(int *fr, int n_bin)
{
  int real = fr[n_bin];
  int imag = fr[FFT_SIZE -n_bin];
  double Phase = RAD_CONV *atan2((double)imag, (double)real);
  return Phase;
}
Because there is a ratio: imag/real (arctang ( sin /cosine)), the most accurate results
is when ratio is close to 1. I don't think it has to be proofed.
There are couple steps implemented in software to keep an error smalest

To minimize phase noise:

1. DDS frequencies is always set to exact FFT bin, so it's corrected
   +- 122 Hz up or down to nearest fft bin.

2. DDS synthesizer is preset to phase zero each time new freq. value loaded,
   hold in reset mode, and start simultaneously with arduino adc sampling.
   It's helpful at lower frequencies <1 MHz.
   Since ADC is running at its own clock derived from arduino DUE prescaller
   settings, that different from on-board DDS module's 25 MHz crystal,
   phase jitter still exist. AD9833 is also not very stable on this matter:

LATENCY PERIOD
A latency period is associated with each asynchronous write
operation in the AD9833. If a selected frequency or phase
register is loaded with a new word, there is a delay of seven
or eight MCLK cycles before the analog output changes. The
delay can be seven or eight cycles, depending on the position
of the MCLK rising edge when the data is loaded into the
destination register.

3. I'm thinking to add some quality factor, when phase data close to  0 or close to 90
- results should be discarded and measurements repeated at this point.

I attached a "working copy" of the software. Be careful, there are many debug messages, and may be some harsh comments, don't take it personally.
King of the dsp engine - fft library - was posted here, message #4 https://www.eevblog.com/forum/microcontrollers/fft-processing-using-ucpu/
 Though, any other will make no difference, DSP pack for cortex-3 for example (not tested).

[metrologist]

Looks like I'm gonna have to get a due.

[MasterT]

Looks like I'm gonna have to get a due.

Not sure, if there is a question why I'm using arduino? No affiliation. Simply, I know sam3x better than stm32f3/4xx, and digging up 1500+ pages DS is not funny, even I have to do it once in a while.
For person, who completes at least one project with uCPU, the choice of architecture /hardware would be greatly biased to "pre-compiled" software pieces he has, availability of libraries and on-line examples. Certainly, it 'd be possible to write your own graphics  library, DDS/ TFT drivers etc, but  it's  would takes a years for amateur /hobby programmers.
Most important things for this device is ENOB / ADC performance, and I know sam3x has >9.6bits at 1msps rate. Blue pill, stm32f303 for example, has less than 8-bits, so board may be used with lower phase accuracy output

[metrologist]

No question, just admiration. I only have the 328p based Arduinos. Did I miss the project details, or are they not ready yet?

[DaJMasta]

Maybe it's more of a software issue, but I would have thought than an external ADC would be the obvious choice.  Sure, the integrated ones can be alright, but you could easily get a couple more bits worth of resolution and enough sample rate to not have to deal with aliasing in this bandwidth... and you're already using an external DAC.

Maybe an R2R DAC using arduino output pins and the internal ADC could be used for a lower frequency cap option, but if you're going to go with external chips, I figure an external ADC as well would give you a fair bit better performance for a few extra bucks and a little board real estate.  Not actually doing the sampling on the main MCU also would probably give you some responsiveness in the UI.


Certainly a neat project, though, can you do SOLT calibration or is this just raw data?

[MasterT]

No question, just admiration. I only have the 328p based Arduinos. Did I miss the project details, or are they not ready yet?

Project is not finished yet, may be never would,  I'm still testing all possibilities.  I was thinking of using UNO board, but than reject. Technically it possible to run 512-fft on UNO, and sampling at 64-76 ksps with modest 9-bits achievable, but: not enough pins - TFT display shown on a video eats up 13-pins out of 18 (2 no use - serial link). Plus, scan mode instead of 5-8 seconds on DUE board would takes 50-90 seconds - unacceptable slow for tunning something real-time.

Maybe it's more of a software issue, but I would have thought than an external ADC would be the obvious choice.  Sure, the integrated ones can be alright, but you could easily get a couple more bits worth of resolution and enough sample rate to not have to deal with aliasing in this bandwidth... and you're already using an external DAC.

Maybe an R2R DAC using arduino output pins and the internal ADC could be used for a lower frequency cap option, but if you're going to go with external chips, I figure an external ADC as well would give you a fair bit better performance for a few extra bucks and a little board real estate.  Not actually doing the sampling on the main MCU also would probably give you some responsiveness in the UI.


Certainly a neat project, though, can you do SOLT calibration or is this just raw data?

External ADC - is a way on FPGA, but there are no any benefits of using it with uCPU. Maximum you can get about 2.5 MSPS over 42 MHz SPI (16-bits w/o software overhead).
SAM3X doesn't have parallel port DMA, mapping pins is out off order and parallel ADC bus I'm sure would be even slower. Plus cost of external ADC, loosy 500 ksps 16-bits more than 30$.  Doesn't make any sense.
Aliasing is not an issue, I already tested sam3x adc - it has not any issue to undersample up to 30 (!) MHz. For continuous sine wave like in this project internal adc is perfect.
I do not have DAC in a project, $3 DDS module from ali, AD9833 has internal DAC, but since it's not accessible, there is no point to talk that we have a DAC. Just DDS.
 What is required, a good PGA, I'm lazy to build one myself. PGA with gain as little as 16-32 equals to 4-5 additional bits, it easily turns sloppy 8-bitter into luxurious 12-13 bits, arduino's 12-bitter into 16-17 !
Or if double stages 32+32 - into unimaginable 22-bits with 1 msps.

[coppercone2]

you know what would be nice, a DSA

[MasterT]

you know what would be nice, a DSA

google says: Democratic Socialists of America, is it correct?

[coppercone2]

no think of it like a dynamic signal analyzer not some kind of commie plot

[xaxaxa]

I would use an audio ADC, e.g. pcm1802. Cheap and has excellent linearity and dynamic range that no MCU ADC would ever hope to match. For higher frequencies (>1MHz) a mixer is a must.

What directional coupler are you using for low frequency?

[MasterT]

What directional coupler are you using for low frequency?

Not relevant, there is no any directional coupler.  I used non RF approach, similar to analog device AD5933

[RoGeorge]

Nice, well done! 
Do you have any plans to publish the software, too?

[MasterT]

Nice, well done! 
Do you have any plans to publish the software, too?

Difficult to say, have to clean it up first. It's like a big pizza, assembled out of "customized" libraries. 
For example, TFT, GFX - Adafruit libraries, tweaked internally to make it compatible with mcufiends ILI9341 shield.
For rotary encoder same things, was not able to find "plug-in and use"  arduino DUE version, have to re-write interrupt driven UNO version.
DDS AD9833 again, tweaked to synchronize with adc start dma conversion.
Customized I mean on direct port manipulation level, so pieces of code would not run on any other uCPU except sam3x.

[ogden]

I would use an audio ADC, e.g. pcm1802. Cheap and has excellent linearity and dynamic range that no MCU ADC would ever hope to match. For higher frequencies (>1MHz) a mixer is a must.

You can't "undersample" with most SD ADC's so w/o mixer you are stuck with around 1/10 of sampling frequency - to get +/- meaningful phase/amplitude measurements. In general it means with such a slow ADC's you shall implement mixers, this also means added complexity. This basically means N2PK VNA or it's simplified variant with two AD9833's.

Feedback is welcome on a project, I'm having fun with. 

Nice project indeed, but I would not name it VNA. It's more like RF impedance meter "with a twist". Also you shall write "theory of operation" or provide pointer to existing one - because many who read this thread possibly do not understand operation of instrument you are building.

I would use DCT DFT instead of FFT. Your DDS is generating just single frequency, so there's no point to calculate all the bins of FFT that does not relate to frequency of DDS, you basically need to calculate just one which is what DFT does. With DFT you don't need so much memory and CPU resources as well.

[MasterT]

Feedback is welcome on a project, I'm having fun with. 

Nice project indeed, but I would not name it VNA. It's more like RF impedance meter "with a twist". Also you shall write "theory of operation" or provide pointer to existing one - because many who reads this thread possibly do not understand operation of instrument you are building.

I would use DCT instead of FFT. Your DDS is generating just single frequency, so there's no point to calculate all the bins of FFT that does not relate to frequency of DDS, you basically need to calculate just one which is what DCT does. With DCT you don't need so much memory and CPU resources as well.

 Thanks. I should put a reference link to AD5933 in the first message, and since AD used "1 MSPS, 12-Bit Impedance Converter, Network Analyzer" as a title for theirs chip, I would keep network analyzer as a name of my project.

 I intended to get comments from the audience that already posses a knowledge  "theory of operation."
 
 Manny people don't understand the absolute supremacy of the FFT on any other method of data processing. The question of "why using FFT instead of DCT?" is the same as "why to sample data by ADC and than process in DSP core, if we can drive a couple of MUXes by quadrature shifted clock?"  Because FFT provides processing gain, increasing SNR ratio by >30 dB(!). Using hardware analogy with two MUXes, someone has to put a hardware preselector - 244 Hz wide band pass filter in front of the  MUXes. Practically impossible to build  such BPF with 40-80 dB roll off, adjustable over 1 kHz - 6.8 MHz. Not now, not ever in the future.

[nihtila]

I am not a mathematician and do not understand DCT or those formulas really, but I think the point was that why do you calculate FFT when you are only after one frequency which you know (or are you, you do sweep right?). FFT calculates correlation for all frequencies between zero and Nyquist frequency, and although it is efficient computation, it is probably better just calculate correlation for that one frequency you are interested in. It does not change your processing gain if you do not change the length of your signal.

Of course if you have the computation power it probably does not matter. Or this comment may not even be relevant as I do not know exactly how you compute the impedance.

But your project looks cool! Really looking forward to more information.

[MasterT]

I am not a mathematician and do not understand DCT or those formulas really, but I think the point was that why do you calculate FFT when you are only after one frequency which you know (or are you, you do sweep right?). FFT calculates correlation for all frequencies between zero and Nyquist frequency, and although it is efficient computation, it is probably better just calculate correlation for that one frequency you are interested in. It does not change your processing gain if you do not change the length of your signal.

Of course if you have the computation power it probably does not matter. Or this comment may not even be relevant as I do not know exactly how you compute the impedance.

But your project looks cool! Really looking forward to more information.

The question I see, why someone would calculate all those 1023 bins if results trashed except only one bin that is goes into arctangent formula for phase extraction? Seems like a nonsense.

Lets me be more specific for this project, so it would be easy for me to explain. I have 250 kHz (putting aside aliasing for now) band splitted into 1024 stripes 244 Hz wide each.  My signal is falling into 125 kHz, bin = 512.

 The problem is that ADC data (and any data) hold not only pure signal, but a noise as well. And all these noise present in each stripes, more or less equally (gaussian somekind off).  Noise level is expressed in Volt / sqrt(Hz), so wide bandwidth - higher noise level.  For person who works with RF it's not a news, wider bandwidth - is always band things, and must be avoided.
I already gave a hardware example with MUX (or mixers). So, if I use a hardware phase discriminator, all 250 kHz wide band with all noise present in it goes into output. The only way to stop it, it's a preselector - or filter.

Now, if I do run  a FFT,  its sorting out everything - useful signal , noise - whatever. FFT is a separator, refinery machine, it would filter out /extract noise for example 128 kHz  out of bin of interest 512, 125 kHz.  Same apply to all 1023 bins, they are all cross-referenced with each other. Noise that presents in each stripe is  extracted from 125k, doing so FFT is "cleaning up" "refines" "polishing" my the only one bin of interest. So at the end, even most results are trashed, but it's doesn't matter since each result is already Accounted and extracted from bin 512.  Numerical data in bin 512 becomes sqrt(1023) time LESS noisy. Because an FFT magically pumped out any dust that otherways will go directly into output, into arctangent formula and generate a phase error.

[nihtila]

I'm not sure what you mean by "cross-referencing" between bins. Isn't FFT effectively just a bank of narrow bandpass filters?

Try to generate 125kHz digital reference signal (sine) of the same length as your FFT, and use the same windowing. Correlate that with your wideband signal. You will get one number. If I am not completely wrong, that is the same number as your FFT gives for that bin.

[ogden]

I would use DCT instead of FFT.

Manny people don't understand the absolute supremacy of the FFT on any other method of data processing. The question of "why using FFT instead of DCT?" is the same as "why to sample data by ADC and than process in DSP core, if we can drive a couple of MUXes by quadrature shifted clock?"  Because FFT provides processing gain, increasing SNR ratio by >30 dB(!).

Well, I have to admit two things: 1) I was wrong suggesting DCT (discrete cosine transfer) when actually I had DFT (discrete fourier transfer) in mind 2) you are mistaken about "FFT supremacy on any other method of data processing".

Why? - Check datasheet of AD5933, chip you are referring to. It does not do FFT but DFT instead.

[oPossum]

It does not do FFT but DFT instead.

FFT is just a fast way to do a DFT.

[ogden]

It does not do FFT but DFT instead.

FFT is just a fast way to do a DFT.

For single frequency bin?

Good resource to read: https://literature.cdn.keysight.com/litweb/pdf/5950-3000.pdf

[oPossum]

The results are the same for any number of bins.

[ogden]

The results are the same for any number of bins.

Not if you care about memory and CPU resources used.

[oPossum]

You really aren't making much sense.

[ogden]

You really aren't making much sense.

If you do not understand what is FFT compared to discrete sine and cosine transform (DFT) then obviously it does not make much sense to you.