Author Topic: Idea for improving LTDZ Spectrum Analyzer & Tracking Generator 35MHz-4.4GHz  (Read 24278 times)

0 Members and 1 Guest are viewing this topic.

Offline KalvinTopic starter

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
Good points. These devices do not have a quadrature mixer, so your observations are valid indeed. The DC-path from the mixer output to the input of the low-pass filter (120 kHz RBW-filter) is dc-blocked, so there will be a problem with spectral components near 0Hz. Since the tracking generator and the receiver mixer are fed with two separate, independent oscillators, it may be possible to offset the receiver LO a bit higher or lower frequency*, sample the output of the 120 kHz RBW-filter with the SMT32F103 ADC, and then perform a complex spectral shift down to 0Hz in order to get a proper analytical quadrature signal, which should resolve this 0Hz problem. Please correct me I have have overlooked something.

For virtually any {desired input frequency, LO frequency} pair there exist an image input frequency, so that both (desired and image) input frequencies are down-converted by the mixer to the same IF frequency. Once the IF contains contributions from both input frequencies, they can't be separated any more unambiguously. So no, a digital down-conversion step after samling the IF cannot fix it either. Traditional superheterodyne radios solve the problem by ensuring that the image input frequencies do not even enter the mixer (-> band pass filter, tuned to the desired input frequency range, excluding image frequencies). If this is not feasible, then an image reject mixer or quadrature mixer is required, in order to either reject the contribution of the image frequency, or to retain the contributions of both frequencies in a separable way.

The tinySA avoids these issues by using a full-featured tranceiver chip, which includes LNA, VCO, quadrature mixer, IF filter, 2x ADC, some DSP processing, and which outputs the down-converted and decimated digital signal to the MCU.

Yes, you are absolutely right that my idea of spectrum shift was mathematically flawed. That's why I was thinking would it be possible to remove the DC-blocking capacitor from the mixer output in order to improve low-frequency performance near DC. The output of the 120 kHz low-pass filter will then have a DC offset, but it can be removed after sampling the signal by computing the average of the buffered samples and then subtracting this average from the samples.

Are you rather after SA functionality, or Network Analyzer functionality? While a SA has to deal with arbitrary input spectra, a NA only needs to receive/detect only a single frequency at a time (namely the current frequency transmitted by its tracking generator). The latter makes life easier.

Scalar network analyzer with adjustable RBW is the major goal, so that it would be possible to use this simple, inexpensive and readily available hardware for measuring RF filters, use it as a simple antenna analyzer, and make it usable for other RF measurements for amateur radio purposes.

I know that there are different versions of very inexpensive nanoVNA vector network analyzers available, which are handy devices to be used for RF network measurements. The idea of this project is to find out whether the LTDZ could be improved and made more useful by only very minimal hardware modifications and some digital signal processing in the firmware.

Spectrum analyzer would a nice by-product for this project, but with the existing LTDZ's hardware design/architecture the spectrum analyzer will suffer the same problem that you addressed above ie. lack of image-rejection without proper filtering at the input. However, the device can still be used as a rudimentary spectrum analyzer as long as the user knows its limitations, and the image frequency components are well below the actual signal of interest. Of course, tinySA is definitely more suitable as a spectrum analyzer, but as far as I know, tinySA cannot be used as a network analyzer.

It would be possible to add a low-pass filter, an upconverter and a bandpass filter before the LTDZ input for a proper spectrum analyzer. The tracking generator could be used as LO for the upconverter, and the RX LO would mix the bandpass signal down to Zero-IF. I guess the blocks needed can be purchased from eBay or Aliexpress. But this would be a bit outside of the project's scope.
 

Offline KalvinTopic starter

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
here is some firmware source which I found. I don't remember if this source code works ok with LTDZ, but at least it compatible with LTDZ hardware and as I remember it even works. Just too lazy to flash it to test.

As I remember I found it on some site related to D6 device. This is not the same firmware as shipped with LTDZ, and it uses a little different device ID and use a little different protocol, but as I remember it works ok with LTDZ.
All things mixed together into single C file, but code is not too large so it is readable.

I didn't dig into source code deep, just tried to flash different versions and see how it works with LTDZ. As I remember, my NWTSHARP app implements both protocols, so it should works with original firmware and with this one. The difference with original is that one firmware requires to send sweep command with pause, otherwise it doesn't works properly. I don't remember exactly if it happens with original firmware or with this one.

Thanks! I found the same firmware source code from Github. Great to hear that the code is working with the existing PC applications. The code is not too complex to be modified, which encouraged me to start thinking about modifying LTDZ if the first place.

The idea is to create a superset of commands so that the firmware will remain backwards compatible with the existing implementation as much as possible, but will add new commands for configuring the RBW, capturing the samples into sample buffer, reading the captured samples from the sample buffer etc.
 

Offline Bicurico

  • Super Contributor
  • ***
  • Posts: 1707
  • Country: pt
    • VMA's Satellite Blog
That has been done before, for example by a user called "Domenico". Her provided me with a feature-increased firmware for the LTDZ, as well as, some modifications of the original hardware design (like the filter).

This is documented in my blog and the extra commands (which allow for example to control the output power of the signal generator in 4 steps) have been implemented in my "VMA Simple Spectrum Analyser" software.

However, in my opinion, it is a waste of time trying to modifiy and/or improve the LTDZ design. If you guys want to obtain a better/good design with variable RWB and what-not, I recommend to basically start from scratch and develop a proper design. You can (and probably should) use the same main components (ADF4351, M810, etc.). But the filter section needs to be redesigned and I would honestly break with backwards protocol compatibility and write a completely new and more efficient firmware.

The current one for example does not support the huge frequency range and considers frequencies to be 10x smaller (which is why you need the 10x multiplicator in WinNWT/LinNWT). Also, the protocol sends each sample twice, which is a waste of time. Also, there is no calibration and the 10 bits sent in two bytes do not represent a dBm value - the final sample power needs to be calculated from a weird formula, that I copied from the LinNWT source code. While my measurements where acceptably accurate (considering the price and quality of these devices), I do have my doubts if this couldn't be implemented in a better way.

Anyway, I am following this with interest and count on me to adjust my software for any new protocol that might be developed.

Also, as a note, I recently tested the Arinst SSA-R2 TG in my blog, which is a much nicer piece of equipment than the SMA/NWT/D6/LTDZ series of devices.

Regards,
Vitor
 
The following users thanked this post: mahdiks

Offline radiolistener

  • Super Contributor
  • ***
  • Posts: 3279
  • Country: ua
Also, there is no calibration and the 10 bits sent in two bytes do not represent a dBm value - the final sample power needs to be calculated from a weird formula, that I copied from the LinNWT source code.

This is very good, because device sends RAW data and allow to perform much better and more precise calibration on PC side where you can store megabytes of calibration data and perform different calibration calculations.

For example, NWTSHARP-LTDZ software that I posted above store measurement points during calibration process for every frequency point. And then, when you perform measurement it looking for nearest reference point for the measured frequency and use it to perform correction. As result I can get 0.003 dB resolution on NWT7 device with perfectly flat response (about ±0.2 dB across full frequency range).

Actually NWTSHARP-LTDZ store calibration file as an XML with raw adc codes in hex for every frequency point. There are two lists ref_zo and ref_xx for two attenuation levels used during calibration. The value of attenuation level is specified in value_zo and value_xx in dB. Calibration file name contains hardware device ID, so you can use it for two different NWT devices with different ID and calibration file will not be overwritten.

For comparison here is RAW ADC and calibration result on my NWT7 for entire frequency range from 1 kHz to 90 MHz (linear detector).
 
« Last Edit: May 13, 2021, 01:09:50 pm by radiolistener »
 

Offline Bicurico

  • Super Contributor
  • ***
  • Posts: 1707
  • Country: pt
    • VMA's Satellite Blog
My software allows to generate a whole compensation map!

https://vma-satellite.blogspot.com/2019/08/error-mapping-smanwtd6ltdz-devices.html
https://vma-satellite.blogspot.com/2019/09/checking-out-performance-of-siglent.html

I understand what you mean with raw output. However, note that the same PCB could and should (and is) used by some MCU with TFT display for mobile use - for example this one: https://www.aliexpress.com/item/1005001762421373.html

It would, in my opinion, make sense that he firmware could output a sweep as a series of samples in dBm output for quick implementations.

A second, different, command should/could output the same data as raw data (without repetition).

Note that nothing prevents calibration/compensation of the streamed samples in correct dBm format.

My main observation is that in my opinion it does not make much sense trying to improve the performance of the LTDZ, as it was ENGINEERED to be AS CHEAP AS POSSIBLE. It is difficult to afterwards modify it for best performance. It is simpler and better to design a new (correct) layout. And once this is being done, one might as well improve the protocol.

Note that WinNWT/LinNWT was never meant to be open source or freeware! It was developed as a custom software for the original NWT-7, NWT-9, NWT-500 designs published by German magazin "AMATEURFUNK". When BG7TBL designed the original SMA/NWT4000 devices he just used this software, making use of the possibility to set the frequency multiplicator, thus supporting the much increased frequency range. However, the software was illegally shipped with the devices, without the consent of the author or AMATEURFUNK.

Worse than that, the author started to be contacted for support and complains of the not fully adequate software! This is why the relevant download site was taken down by the author.

If one is to develop a better than SMA/NWT/D6/LTDZ design, then one should design a brand new firmware with a new protocol.

Just my opinion, though.

Regards,
Vitor



Online gf

  • Super Contributor
  • ***
  • Posts: 1132
  • Country: de
Yes, you are absolutely right that my idea of spectrum shift was mathematically flawed. That's why I was thinking would it be possible to remove the DC-blocking capacitor from the mixer output in order to improve low-frequency performance near DC. The output of the 120 kHz low-pass filter will then have a DC offset, but it can be removed after sampling the signal by computing the average of the buffered samples and then subtracting this average from the samples.

Spectrum analyzer would a nice by-product for this project, but with the existing LTDZ's hardware design/architecture the spectrum analyzer will suffer the same problem that you addressed above ie. lack of image-rejection without proper filtering at the input.

The existing design does actually not suffer from images. Since it is a direct conversion (zero-IF) receiver, all images which are mapped by the mixer into the RBW filters' passband happen to lie inside the passband anyway (just at a different position), therefore they are supposed to be included in the power calculation and don't need to be excluded. For measuring the power at the low-pass filter output, the design is OK in this regard.

What I wanted to say is that this IF signal is not suitable for sub-dividing the sampled RBW passband into smaller sub-bands, e.g. via FFT.
That's a different use case for which the images inside the RBW passband do matter.
[ Asking Google, I did find approaches here and here to detect potential images post fact in the spectrum and to cut them out. ]

The AC coupling unfortunately splits the frequency response of the RBW bandpass in the midde, so that it has two lobes eventually. It is however difficult to deal with DC offsets. At (say) 1V full scale, one could only tolerate 10µV offset error, if the aim is an error of -100dB. That's not trivial. Adding a capacitor is much easier.

Quote
Scalar network analyzer with adjustable RBW is the major goal, so that it would be possible to use this simple, inexpensive and readily available hardware for measuring RF filters, use it as a simple antenna analyzer, and make it usable for other RF measurements for amateur radio purposes.

Why adjustable bandwidth for NA? The basic functionality of a simple NA is that you send a single frequency (i.e. more ore less zero bandwidth) to the DUT, and you measure the signal coming out out the DUT, which is the same single frequency which you did send to the DUT (only amplitude and phase are different). Then you step to the next freyency and repeat the procedure again and again. Ideally, images should not be a problem either. Why should a potential image frequency come out from the DUT if you don't send it into the DUT? ADC aliasing is mitigated too - for the same reason - granted that the frequency plan is carefully chosen. Harmonics still need to be considered in the frequency plan, since not all DUTs are perfectly linear. And the NA's signal generator does not necessarily generate a pure sine wave either.

One thing which does indeed not fit well is the combination of the AC coupled filter with the zero-bandwidth CW signal and a zero-IF receiver. Since the filter has a null at DC it can't detect the zero bandwidth signal which is down-mixed to exactly DC. Therefore I'd rather pick a non-zero IF for the NA use case (i.e. add an offset to the receiver LO). NanoVNA V2.2 for instance uses 12kHz IF, which is sampled @300kSa/s by the MCU (it is also STM32F103 based, IIRC), and 25 50 samples (one two 12kHz periods) are correlated with a windowed complex sine wave 1) to obtain one measurement (btw, since the NanoVNA is a VNA, the ADC needs to run continuously in order not to lose phase in the pause when the receiver is switched to a different signal source).

EDIT: Regarding "it is also STM32F103 based, IIRC":
I checked again. The schematic mentions both, STM32F103C8 and GD32F303CCT6, but it is obviously the latter. Sorry for the mistake.

EDIT: 1) That's basically a Goertzel dedector, i.e. a DFT for a single frequency. The chosen window function eventually acts as bandpass filter.

EDIT: Regarding "Scalar network analyzer with adjustable RBW is the major goal":
If you intend anyway to sample the pre-filtered IF with the ADC, then why limit to "scalar", and not do "vector"?
External directional bridge and RF switches to switch the (single) receiver input between reference signal, refelected singal, DUT output,... are required for both.
As long as TX oscillator, RX oscillator and ADC clock are frequency-locked to the same xtal by rational factors, no frequency drift between them is to be expected either, and phase can be retained.
Eventually I don't see any difference in the required hardware then - it's just software. Do i miss anything?
« Last Edit: May 14, 2021, 07:44:31 am by gf »
 
The following users thanked this post: Kalvin

Offline KalvinTopic starter

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
Yes, you are absolutely right that my idea of spectrum shift was mathematically flawed. That's why I was thinking would it be possible to remove the DC-blocking capacitor from the mixer output in order to improve low-frequency performance near DC. The output of the 120 kHz low-pass filter will then have a DC offset, but it can be removed after sampling the signal by computing the average of the buffered samples and then subtracting this average from the samples.

Spectrum analyzer would a nice by-product for this project, but with the existing LTDZ's hardware design/architecture the spectrum analyzer will suffer the same problem that you addressed above ie. lack of image-rejection without proper filtering at the input.

The existing design does actually not suffer from images. Since it is a direct conversion (zero-IF) receiver, all images which are mapped by the mixer into the RBW filters' passband happen to lie inside the passband anyway (just at a different position), therefore they are supposed to be included in the power calculation and don't need to be excluded. For measuring the power at the low-pass filter output, the design is OK in this regard.

Thank you for correcting me again, very much appreciated. This is a subject that I know a little, but especially this zero-IF architecture with a scalar mixer is something that I really haven't had to think before at this detail. I will take this as an excellent learning experience/exercise. Quadrature zero-IF architecture (for example Tayloe mixer used in SDR transceivers) and quadrature mixers in general are somewhat more familiar.

Quote
What I wanted to say is that this IF signal is not suitable for sub-dividing the sampled RBW passband into smaller sub-bands, e.g. via FFT.
That's a different use case for which the images inside the RBW passband do matter.

Ok, this is something that was confusing me before. Let's see if I manage to grasp the idea better this time: Since this hardware architecture is essentially a direct conversion zero-IF receiver with a scalar mixer, the frequency components around the LO are mixed, folded and summed so that the resulting spectrum at the output of the mixer is centered symmetrically around DC so that the original "negative" frequency components and the "positive" frequency components relative to LO are now combined. If we now perform spectrum analysis for this signal at the mixer output, we really cannot tell whether the spectrum components we now see were originally below LO or above LO before mixing, thus we would see a distorted signal spectrum. However, the power of the signal can be computed correctly nevertheless. Please correct me if I have missed something.

Quote
[ Asking Google, I did find approaches here and here to detect potential images post fact in the spectrum and to cut them out. ]

Thank you for the links. Really clever ways for suppressing the images and spurs after the "spectral damage has already been done" in the mixer. I need more time to digest those papers.

Quote
The AC coupling unfortunately splits the frequency response of the RBW bandpass in the midde, so that it has two lobes eventually. It is however difficult to deal with DC offsets. At (say) 1V full scale, one could only tolerate 10µV offset error, if the aim is an error of -100dB. That's not trivial. Adding a capacitor is much easier.

The input of the ADC needs to be biased somewhere near VCC/2, so this bias voltage needs to be removed from the ADC samples anyway before actual power calculation. I was thinking that it would not really matter whether we get this bias voltage from the mixer output or from a resistor divider after a capacitor. If the ADC is coupled directly to the mixer output without a capacitor, there would not be a need for any ADC bias resistors. However, as a seconds thought, using capacitor coupling with the resistor divider, the bias voltage may remain more stable compared to mixer output, especially if the DC-voltage at the mixer output varies with the mixer input signal level.

Quote
Quote
Scalar network analyzer with adjustable RBW is the major goal, so that it would be possible to use this simple, inexpensive and readily available hardware for measuring RF filters, use it as a simple antenna analyzer, and make it usable for other RF measurements for amateur radio purposes.

Why adjustable bandwidth for NA? The basic functionality of a simple NA is that you send a single frequency (i.e. more ore less zero bandwidth) to the DUT, and you measure the signal coming out out the DUT, which is the same single frequency which you did send to the DUT (only amplitude and phase are different). Then you step to the next freyency and repeat the procedure again and again. Ideally, images should not be a problem either.

Yes, so true. The tracking generator is essentially injecting only a sinusoidal signal into DUT, so the DUT will also output a sinusoidal signal at the same frequency (with amplitude/phase change but this architecture will ignore the phase information). If the DUT is a non-linear circuit there will be other spectral components, though.

Quote
Why should a potential image frequency come out from the DUT if you don't send it into the DUT? ADC aliasing is mitigated too - for the same reason - granted that the frequency plan is carefully chosen. Harmonics still need to be considered in the frequency plan, since not all DUTs are perfectly linear. And the NA's signal generator does not necessarily generate a pure sine wave either.

Tracking generator (ADF4351) is generating output signal with some harmonics, but those harmonics should not cause problems in general.

Quote
One thing which does indeed not fit well is the combination of the AC coupled filter with the zero-bandwidth CW signal and a zero-IF receiver. Since the filter has a null at DC it can't detect the zero bandwidth signal which is down-mixed to exactly DC. Therefore I'd rather pick a non-zero IF for the NA use case (i.e. add an offset to the receiver LO). NanoVNA V2.2 for instance uses 12kHz IF, which is sampled @300kSa/s by the MCU, and 25 50 samples (one two 12kHz periods) are correlated with a windowed complex sine wave 1) to obtain one measurement (btw, since the NanoVNA is a VNA, the ADC needs to run continuously in order not to lose phase in the pause when the receiver is switched to a different signal source).

This LO frequency offset and Goertzel detector is quite simple to do, as you have described.

Quote
EDIT: Regarding "Scalar network analyzer with adjustable RBW is the major goal":
If you intend anyway to sample the pre-filtered IF with the ADC, then why limit to "scalar", and not do "vector"?
External directional bridge and RF switches to switch the (single) receiver input between reference signal, refelected singal, DUT output,... are required for both.
As long as TX oscillator, RX oscillator and ADC clock are frequency-locked to the same xtal by rational factors, no frequency drift between them is to be expected either, and phase can be retained.
Eventually I don't see any difference in the required hardware then - it's just software. Do i miss anything?

The ADC is sampling low-pass filtered zero-IF baseband (ie. sampling the signal at the output of the 120 kHz RBW filter connected to the output of the scalar mixer). Too many hardware modifications are a bit outside of the original project scope, although it is very hard to resist feature creep. However, if this hardware would be better, those extra features might be more attractive to be implemented. Now the intention is just to improve the existing design with only few hardware changes.
« Last Edit: May 14, 2021, 12:22:23 pm by Kalvin »
 

Online gf

  • Super Contributor
  • ***
  • Posts: 1132
  • Country: de
Ok, this is something that was confusing me before. Let's see if I manage to grasp the idea better this time: Since this hardware architecture is essentially a direct conversion zero-IF receiver with a scalar mixer, the frequency components around the LO are mixed, folded and summed so that the resulting spectrum at the output of the mixer is centered symmetrically around DC so that the original "negative" frequency components and the "positive" frequency components relative to LO are now combined. If we now perform spectrum analysis for this signal at the mixer output, we really cannot tell whether the spectrum components we now see were originally below LO or above LO before mixing, thus we would see a distorted signal spectrum. However, the power of the signal can be computed correctly nevertheless. Please correct me if I have missed something.

Exactly :-+
Regarding "we really cannot tell whether the spectrum components we now see were originally below LO or above LO before mixing":
You only see the sum of the original upper sideband and the mirrored lower sideband. The negative frequencies in the IF spectrum are just a conjucate complex mirror of the positive IF frequencies, which applies to the spectrum of any real-valued signal.

Quote
Yes, so true. The tracking generator is essentially injecting only a sinusoidal signal into DUT, so the DUT will also output a sinusoidal signal at the same frequency (with amplitude/phase change but this architecture will ignore the phase information). If the DUT is a non-linear circuit there will be other spectral components, though.

If you only feed only a single frequency into the DUT, then the "other spectral components" are supposed to be only harmonics (unless the DUT would contain oscillators, mixers, etc. so that IM products can arise).

Quote
The ADC is sampling low-pass filtered zero-IF baseband (ie. sampling the signal at the output of the 120 kHz RBW filter connected to the output of the scalar mixer).

Sure. But for the NA use case you are still free to choose a non-zero IF (by adding an offset to the receiver's LO frequency). You just need to ensure that the chosen IF frequency is inside the passband of the filter (zero is actually not inside the passband, due to AC coupling).

Quote
Too many hardware modifications are a bit outside of the original project scope, although it is very hard to resist feature creep. However, if this hardware would be better, those extra features might be more attractive to be implemented. Now the intention is just to improve the existing design with only few hardware changes.

Which hardware changes in particular (besides IF being sampled by the ADC) were required for a VNA, but not for a scalar NA? My understanding of a scalar NA architecture is still something like figure 1 in this document. In your case, tracking generator and receiver/detector are built-in, so you don't need them externally. But since you have only a single receiver, there is still the need to multiplex the three inputs with external RF switches. This applies to both, scalar and vector NA. Directional bridge needs to be external too, since you don't have a built-in bridge.

[ If I consider the additinally required external coponents, I wonder, whether buying a NanoVNA V2 for 50 bucks isn't the cheaper solution. ]
 

Offline KalvinTopic starter

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
Which hardware changes in particular (besides IF being sampled by the ADC) were required for a VNA, but not for a scalar NA? My understanding of a scalar NA architecture is still something like figure 1 in this document. In your case, tracking generator and receiver/detector are built-in, so you don't need them externally. But since you have only a single receiver, there is still the need to multiplex the three inputs with external RF switches. This applies to both, scalar and vector NA. Directional bridge needs to be external too, since you don't have a built-in bridge.

[ If I consider the additinally required external coponents, I wonder, whether buying a NanoVNA V2 for 50 bucks isn't the cheaper solution. ]

You are right that any modern NA has an architecture that is shown in figure 1. However, this hardware with an external, inexpensive SWR/reflection bridge can be used as a simple, manually operated NA already. This requires extra work from the user as the user needs to perform required measurement steps manually. It is very trivial to add a new command that can be used to set/clear some GPIO pins which can be used for relay control, and the PC software can then perform the necessary steps required by a modern NA. I just want to avoid feature creep and want to proceed one step at a time. This is my first STM32F103 project, so this is quite challenging project already as there are lots of details to learn. Luckily the original firmware C source code is quite easy to understand, and the required modifications are straightforward to implement. In order to make this project successful, it is better to keep it simple.

I would not consider this project as a replacement for any existing instrument on the market (tinySA, nanoVNA etc.). The sole purpose of this project is just to improve the existing hardware so that it will become more usable for common amateur radio measurements (adjustable RBW for improved spectral resolution, shifted IF for NA measurements).
 

Offline KalvinTopic starter

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
If you intend anyway to sample the pre-filtered IF with the ADC, then why limit to "scalar", and not do "vector"?
External directional bridge and RF switches to switch the (single) receiver input between reference signal, refelected singal, DUT output,... are required for both.
As long as TX oscillator, RX oscillator and ADC clock are frequency-locked to the same xtal by rational factors, no frequency drift between them is to be expected either, and phase can be retained.
Eventually I don't see any difference in the required hardware then - it's just software. Do i miss anything?

After some thinking, this is not extraordinary difficult to implement, given that there are external relays and directional coupler/reflection bridge available, and the two ADF4351s can maintain sufficiently stable frequency. Since the tracking generator is sending a sinusoidal signal, the RX can use whatever shifted IF and ADC sample clock that is suitable. For example the RX could use 12kHz IF offset, thus the ADC sample-clock needs to be at least 24kHz, preferably more like 4x, which makes signal detector quite simple. Since the ADC sample-rate can now be reduced from 1Ms/s down to 48Ks/s, ADC can be kept running continuously between different measurements, STM32F103 can process the samples in real-time, and the phase information between tracking generator and the LO can be preserved -> it is quite possible to implement a VNA with this hardware, external directional coupler/reflection bridge and external signal relays.

On the other hand, using this hardware as a spectrum analyzer with digital, adjustable RBW, the ADC sample rate needs to be high enough so that the complete 120kHz band of the analog RBW/low-pass filter can be sampled without aliasing, probably up to 1Ms/s. In this mode the signal can be sampled into a RAM buffer using ADC+DMA, and then processes the RAM buffer without actual real-time requirements.
« Last Edit: May 15, 2021, 01:40:53 pm by Kalvin »
 

Offline KalvinTopic starter

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
About an firmware bootloader: It is most useful if the onboard application firmware could be updated over the USB port without opening the enclosure. The device firmware contains two parts: 1) the firmware bootloader and 2) actual application part of the firmware. The firmware bootloader needs to be installed only once using the JTAG/SWD interface/tools, and after that the application part of the firmware can be updated over the USB port without any tools.

The firmware bootloader resides in the Flash memory page(s) that won't be erased during firmware update process. After each device reset the program starts executing from the firmware bootloader, which will compute and validate the checksum of application part of the Flash memory. If the checksum is valid, the execution continues from the application run-time initialization which eventually calls the application's main() function (this needs some linker magic). If the firmware application checksum is invalid, the bootloader will take over the USB-serial port and open a command interface for updating the application firmware.

The application firmware needs to have a command for erasing/invalidating the application part of the firmware itself. After erasing/invalidating the application firmware, the device will reset itself, and the firmware bootloader will notice that the application part of the firmware needs to be updated, and will open the command interface for updating the application firmware.

Pretty standard stuff.

Does someone know a pointer for a small & simple STM32F103 bootloader which could be used as a basis for this kind of bootloader?
« Last Edit: May 15, 2021, 03:53:18 pm by Kalvin »
 

Offline radiolistener

  • Super Contributor
  • ***
  • Posts: 3279
  • Country: ua
I was thinking would it be possible to remove the DC-blocking capacitor from the mixer output in order to improve low-frequency performance near DC. The output of the 120 kHz low-pass filter will then have a DC offset, but it can be removed after sampling the signal by computing the average of the buffered samples and then subtracting this average from the samples.

The problem of DC bias is that it add offset for the ADC input. ADC max/min limits are fixed and adding offset on the input leads to remove dynamic range. If ADC is overloaded you can't fix it by remove DC bias from ADC output because it don't allows you to restore dynamic range.

Using IIR filter on DSP side to remove DC offset is useful for analog quadrature detector, which may have a slightly phase shift difference between I and Q branches. LTDZ don't use quadrature detector, it use usual mixer which folds negative and positive frequencies.

So there is no advantage of removing DC block capacitor, much better solution is to use some IF (measure magnitude at some frequency shift) to keep DC frequency in outside of your digital filter.


About an firmware bootloader: It is most useful if the onboard application firmware could be updated over the USB port without opening the enclosure.

The bootloader is useful for mass production, it allows to deliver firmware to millions users who don't have knowledge about MCU and don't have JTAG/SWD adapter.

If you're want to develop and debug your own firmware for your own usage, then bootloader is complete useless for you. You're needs JTAG/SWD which allows you to upload firmware directly and use in circuit debugging (you can run your firmware step-by-step in the source code on the live hardware).

« Last Edit: May 15, 2021, 04:43:42 pm by radiolistener »
 

Offline KalvinTopic starter

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
About an firmware bootloader: It is most useful if the onboard application firmware could be updated over the USB port without opening the enclosure.

The bootloader is useful for mass production, it allows to deliver firmware to millions users who don't have knowledge about MCU and don't have JTAG/SWD adapter.

If you're want to develop and debug your own firmware for your own usage, then bootloader is complete useless for you. You're needs JTAG/SWD which allows you to upload firmware directly and use in circuit debugging (you can run your firmware step-by-step in the source code on the live hardware).

Like I wrote above, I do not want to open the enclosure if I just want to update the firmware. It has been quite possible to develop the firmware without JTAG/SWD-tools and the ability to single-step the code. Those tools are handy at the initial phase of the project, but after a while those tools are not that important anymore.
 

Offline radiolistener

  • Super Contributor
  • ***
  • Posts: 3279
  • Country: ua
Like I wrote above, I do not want to open the enclosure if I just want to update the firmware.

LTDZ don't have enclosure. Without JTAG/SWD you will be unable to debug your code, so it is must have if you want to develop your own firmware.

The bootloader is only useful to upload already existing firmware (designed, compiled and tested) for these users who don't have JTAG/SWD adapter. It is not intended for firmware development.

The bootloader is the same as installer engine on windows. It allows to install already existing and prepared software for end-users. But it is complete useless if you develop and debug your own software.
« Last Edit: May 15, 2021, 04:50:05 pm by radiolistener »
 

Offline radiolistener

  • Super Contributor
  • ***
  • Posts: 3279
  • Country: ua
It has been quite possible to develop the firmware without JTAG/SWD-tools and the ability to single-step the code. Those tools are handy at the initial phase of the project, but after a while those tools are not that important anymore.

No. This is impossible. You can compile code and upload it to device through the bootloader, but you won't be able to debug your code on the live hardware.

Another issue is that upload firmware through the bootloader is more complicated than through JTAG/SWD. The bootloader needs to get your hands on device to enable bootloader mode with solder some jumper on PCB or by selecting appropriate menu on the device firmware (if it has such ability). You will need to damage your LTDZ PCB (cut some wires and solder jumpers) in order to enter into bootloader mode. Sometimes the bootloader mode is just unavailable. JTAG/SWD don't needs any action before upload firmware, it just uploaded by your IDE directly just after compilation and with no needs to do something.

STM32F103 has the bootloader in OTP memory, in order to enter into bootloader mode you're needs to solder jumpers for specific pins. As I know these jumpers don't available on LTDZ PCB. And I'm not sure if serial port for the bootloader is the same as used for PC communication on LTDZ device.

Just believe me, you will get a lot of trouble in order to use bootloader for firmware development. It will be much cheaper to buy JTAG/SWD adapter like ST-LINKv2 than trying to use the bootloader.

If you're not experienced with STM32 programming and debugging, I think using the bootloader from OTP memory for STM32F103C8T6 will be almost impossible for you with LTDZ, which PCB don't have switch to enter into the bootloader.
« Last Edit: May 15, 2021, 05:30:42 pm by radiolistener »
 

Offline KalvinTopic starter

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
We are probably talking about two different bootloaders here. Yes, there is an on-chip bootloader in STM32F103, created by the chip manufacturer, which can be activated by a certain GPIO-pin and device reset. There is even an application note for that: AN3155 USART protocol used in the STM32 bootloader. It seems that you are referring to this factory programmed bootloader here.

What I am talking about is a bootloader which is a part of the firmware and resides in the on-chip Flash memory (the user erasable Flash memory). This bootloader is customized for updating the application firmware in the Flash memory using the standard USB-UART-serial port. The on-chip Flash memory is partitioned into two parts: Few Flash pages are reserved for the bootloader itself, and the rest of the Flash memory pages are used for the application part of the firmware. No need to use jumpers any more for updating the firmware.
« Last Edit: May 15, 2021, 06:30:35 pm by Kalvin »
 

Offline KalvinTopic starter

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
It has been quite possible to develop the firmware without JTAG/SWD-tools and the ability to single-step the code. Those tools are handy at the initial phase of the project, but after a while those tools are not that important anymore.

No. This is impossible. You can compile code and upload it to device through the bootloader, but you won't be able to debug your code on the live hardware.

It has been possible to develop firmware successfully with only a serial port available for printf(), and LTDZ is no exception. Only very seldom, as the last resort, there has been a real need to use JTAG/SWD-tools for debugging. Single-stepping a real-time system is quite often useless or even impossible, as the word "real-time" will suggest.
 

Offline radiolistener

  • Super Contributor
  • ***
  • Posts: 3279
  • Country: ua
What I am talking about is a bootloader which is a part of the firmware and resides in the on-chip Flash memory (the user erasable Flash memory).

What is the reason for such bootloader? It still needs JTAG/SWD to program it and for debugging. It consume a lot of memory. It requeres some jumper or communication protocol command to enter into bootloader, it needs to develop firmware uploader tool etc. Only headache and no any advantage.

This bootloader is customized for updating the application firmware in the Flash memory using the standard USB-UART-serial port.

You can't use existing USB-UART for printf, because it already used for communication protocol.

This bootloader is customized for updating  The on-chip Flash memory is partitioned into two parts: Few Flash pages are reserved for the bootloader itself, and the rest of the Flash memory pages are used for the application part of the firmware. No need to use jumpers any more for updating the firmware.

And it still needs JTAG/SWD to upload such bootloader and if something going wrong and bootloader page will be erased, you will need JTAG/SWD to upload it again.

When you try to use JTAG/SWD, you will realize that bootloaders is a nightmare, which consume MCU memory, needs special software and additional actions and  just provide you with a very basic functionality for upload firmware and nothing else.

It's complete useless, I'm implemented user bootloader and know what I'm talking about.  ;)

Using the bootloader for firmware development is like using Update Service in the Windows for software development. It's complete useless for such purposes.
 

Offline radiolistener

  • Super Contributor
  • ***
  • Posts: 3279
  • Country: ua
It has been possible to develop firmware successfully with only a serial port available for printf(), and LTDZ is no exception.

Yes, it is possible to develop firmware without debugging... if you're a guru in STM32 development and have a long experience in using it with ADF4351 and other peripherals.

But if you're a guru, you're already understand why JTAG/SWD is must have why it simplify your life.

The same as a guru can write software with no using IDE. He can, but he don't want do it in such way. If you're novice it will be just crazy to start programming in such way.

Only very seldom, as the last resort, there has been a real need to use JTAG/SWD-tools for debugging. Single-stepping a real-time system is quite often useless or even impossible, as the word "real-time" will suggest.

No. If you're working on firmware for MCU, you will needs JTAG/SWD for incircuit debugging. Otherwise you will just don't know what is going on in the MCU if it don't works as expected. For example, you upload firmware but it don't works and don't printf debug message on serial port for unknown reason. Without JTAG it's just a lot of headache to understand why it don't works as expected and what is happened.

JTAG/SWT is used 100% time for firmware development and debugging. The bootloader is useless for such purpose. For example, I never used bootloader for debugging and don't know who is using it for that.

Anyway, in order to develop and debug your bootloader you will needs JTAG/SWD.

The bootloader for firmware development is just like a fifth leg for a dog.

I'm sure for almost 100% that you will not implement firmware if you start to fight with bootloader problems instead of firmware debugging with JTAG/SWD adapter.
« Last Edit: May 15, 2021, 07:13:58 pm by radiolistener »
 

Offline KalvinTopic starter

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
Let's say I have a friend who also has this LTDZ hardware. So, my friend brings it to me, I open the enclosure, flash the firmware once using the JTAG/SWD-tools, and close the enclosure. Now, when there is a new version of firmware available, I can email the firmware to him (or my he can download it from Github), and he is able to update his LTDZ without opening the enclosure or without any special development tools - just like magic. All he needs is a PC software supporting the new protocol and a USB-cable.

Same thing even if I do it for me: If I just make a simple fix in the firmware, I prefer to do it using this bootloader instead of doing the whole chore of opening the enclosure, flashing the firmware using the tools, and finally closing the enclosure again.
« Last Edit: May 15, 2021, 07:19:06 pm by Kalvin »
 

Offline radiolistener

  • Super Contributor
  • ***
  • Posts: 3279
  • Country: ua
Let's say I have a friend who also has this LTDZ hardware. So, my friend brings it to me, I open the enclosure, flash the firmware once using the JTAG/SWD-tools, and close the enclosure. Now, when there is a new version of firmware available, I can email the firmware to him

I suggest to write firmware first and then thinking on how to delivery it to the end-user.

PS: Self updating firmware is dangerous for user who don't have JTAG/SWD, because it may fail and brick the device. And he will be unable to restore it, because he don't have JTAG/SWD.

The user with bootloader can brick his device and it will be like magic. No needs to open enclosure, no needs for jumpers, just brick device through USB. I had such experience when debug bootloader. :)

Another issue is that you will need to edit memory map in the linker file. It will be complicated for novice.
« Last Edit: May 15, 2021, 07:24:54 pm by radiolistener »
 

Offline KalvinTopic starter

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
PS: Self updating firmware is dangerous for user who don't have JTAG/SWD, because it may fail and brick the device. And he will be unable to restore it, because he don't have JTAG/SWD.

What is so special about this that makes it dangerous? What are the places in the update process where the system can brick itself during the firmware update? Is there a way that can prevent this from happening, I mean is there a way that the firmware could be updated in a safe manner without bricking the device?
 

Offline radiolistener

  • Super Contributor
  • ***
  • Posts: 3279
  • Country: ua
What are the places in the update process where the system can brick itself during the firmware update?

if you write your own bootloader, it is placed in erasable memory page, so it can be easily erased due to some random error or mistake in the firmware. Or just random power off during update. If bootloader is erased, you will not be able to use it anymore. You will needs JTAG/SWD to upload it again.

The bootloader placed in OTP memory cannot be erased, because it is placed in one time programmable memory. But it requires special jumpers configuration at MCU reset in order to enter into bootloader mode. As I mentioned before, you will need to desolder/solder components, cut wires or add jumpers on LTDZ PCB in order to enter into bootloader mode. And after update it will needs to restore original configuration in order to allow to run firmware instead of bootloader.

Is there a way that can prevent this from happening, I mean is there a way that the firmware could be updated in a safe manner without bricking the device?

Yes, do it through JTAG/SWD adapter. If something bad happens you always can upload firmware again.  :)
« Last Edit: May 15, 2021, 07:37:22 pm by radiolistener »
 

Offline KalvinTopic starter

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
Another issue is that you will need to edit memory map in the linker file. It will be complicated for novice.

True. Although I have not worked with STM32F103 before, it should not be too different from other Cortex-M3 processors.
 

Offline radiolistener

  • Super Contributor
  • ***
  • Posts: 3279
  • Country: ua
NanoVNA uses a hack to use OTP bootloader from user code. It jumps into OTP firmware code from user menu on device.

Since bootloader needs the same MCU state as after hardware reset, it use a trick. It write special value into RAM variable with fixed address and then perform MCU reset. After reset it check RAM variable and if there is request for bootloader it jumps into OTP bootloader. It allows to use DfuSeDemo tool to upload firmware with no needs to switch BOOT jumper.

But I'm not sure that this way will works for STM32F103C8T6. As I know some part of memory are write protected from bootloader in STM32F103C8T6, so it needs very special configuration for the code to compile into specific pages.

Using STM32F103C8T6 OTP bootloader may needs to buy external USB-serial and solder wires directly to the chip pins. But I'm not sure, because I never used bootloader on STM32F103C8T6.

For JTAG/SWD no needs to change something, JTAG/SWD connector is already present on LTDZ PCB.

I think it will be more easy for your friend to buy ST-LINKv2 than modify LTDZ PCB by adding hardware switch for bootloader mode and by adding external USB-serial adapter, which may cost more than ST-LINKv2.
« Last Edit: May 15, 2021, 08:49:16 pm by radiolistener »
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf