Warning about NanoVNA clones and comparison with Siglent VNA

[The Bootloader]

Hi!

Before I could afford a proper VNA (thank you Siglent), I naturally got a NanoVNA because they are so cheap.
In fact, I ended up with two NanoVNAs. One from a cheap AliExpress seller (around $35), and one from the original creator (around $80)

I recently got a Siglent SSA3021X Plus which I converted to a Siglent SVA1032X (thanks again Siglent).

I did a small comparison of the three instruments in real-world applications (antenna S11 testing and filter S21 testing)

The biggest takeaway is that the cheap clone produces measurements which are drowned out by its own noise, presumably due to poor shielding of the different sections.

The "original" NanoVNA is much more useable, and I would trust it for basic antenna tuning or filter design.

For higher quality quantitative measurements, nothing beats a proper VNA, but this is significantly more expensive.

Hope you enjoy this test as much as I enjoyed doing this. Comments and suggestions welcome as always.

Video link:

[The Bootloader]

I realize the YouTube compression algorithm messed up the colors of the graphs. I selected those colors because they are supposed to be visible for the majority of people including most of our colorblind peers.

The original image files are attached.

Additionally, a few things I forgot to mention in the video and that may be valuable for interpretation:
- The screenshot of the NanoVNA shows 101 points. However, NanoVNA-App did capture all the 801 points
- I left each instrument running for more than 1 hour before starting filming this video, to ensure thermal equilibrium and prevent too much drift between the calibration and the measurements
- The clone NanoVNA has an offset of -20dB for S11 measurements, as described in the video. But strangely for S21 measurements, this offset changed to -10dB. I compensated accordingly when plotting the data.

[EggertEnjoyer123]

If you look at the specs of the cheap clone, you'll see that it's only designed to go up to 900MHz. I think the more expensive SAA2 is designed for up to 3GHz.

[The Bootloader]

Correct! Usually when testing I go past the design boundaries which allows to see where the limit actually is.

Even way below 900MHz, the measurements on the clone are very noisy, whereas the genuine one still works okay.

[gf]

Note that the SAA2 is not the "original NanoVNA".

Actually it is Hugen’s NanoVNA-H which is derived from the original NanoVNA, with some improvements added. The original NanoVNA and NanoVNA-H support only up to 300 MHz natively, while up to 900 Mhz are only supported via harmonics, with significantly reduced performance. So don't expect too much beyond 300MHz.

OTOH, SAA V2.2 is a completely different design, and it was a priori designed for 3GHz. The V2.2 design is not generally better than the original NanoVNA. Up to 300MHz, the original NanoVNA and NanoVNA-H are still supposed to outperform the SAA V2.2 (depending on the used firmware and settings).

Btw, was your SAA V2.2 really an original V2.2 from the Tindie store, or a V2.2 clone? AFAIK, the original SAA V2.2 is no longer produced and sold for quite some time, but they sell only V2 Plus4 and newer SAA2 models. Almost all SAA V2.2 units sold today by various sellers are clones (which may or may not perform as well as the original V2.2 did).

[The Bootloader]

Thank you for the history! I always found the naming scheme of all these NanoVNAs super confusing.

My V2.2 is indeed from the Tindie store I linked from the first post. I also linked the exact place where I bought the clone.

If I recall properly, I found out about the NanoVNA on AliExpress and I was like "WOW a VNA for $35, awesome".
Then I read about it and realized the clones were crap and I should have bought an original from the Tindie store, which I did, around 2020 so that's why I got the V2.2

Fast forward recently, I needed to use a VNA. Completely forgot about the V2.2, took the clone instead. Had completely sketchy measurement results which were caused by the calibration load having bad solder joint inside it, and behaving like a capacitor. Fixed it, but still got dubious results, and got the Siglent.

Then, I found the V2.2 in a box and I was like.. oh wait, I have this one too? And that's why I had the idea to make a video to compare all of them and I am glad I did, finally puts some data about the "bad clones".

It would have been nice to compare with a more recent NanoVNA, but I don't feel like buying one now that I have the Siglent

[The Bootloader]

Here is (attached) what the inside of the failed 50 ohms load looked like. This is the load that came with the clone NanoVNA, so even the calibration load was bad, not just the VNA. I attached what it actually measured like, with a VNA calibrated with a proper 50 ohms load.

We can clearly see that this is a "dry" solder joint, the center lead of the resistor has not been weted properly by the solder.

I put a tiny bit of flux, remelted the solder, and this load now reads (Smith chart) very close to a Mini-Circuits ANNE-50L+ on my Siglent up to 3.2GHz (see attached) so I consider it fixed.

The point of this message being, whatever you use to calibrate your VNA is as important as the instrument. Do not blindly trust what you get especially with cheaper clones, always cross-check your calibration kit if you do not have a proper traceable one.

[joeqsmith]

Hi!

Before I could afford a proper VNA (thank you Siglent), I naturally got a NanoVNA because they are so cheap.
..

I'm interested in seeing how you compare these VNAs and plan to watch your video in the next day or so.   I would expect the Siglent would outperform the low cost units above 300MHz but I wonder below 1MHz how it stacks up.   I wonder also how the Siglent does with narrow band measurements.   

I am using a Mini Circuits ANNE as a reference load as well.  I purchased eight of them for about $10/ea and then sorted them using a set of Agilent metrology grade standards.  One of them had really good return loss.   A second one was decent enough that I use it with these low cost VNAs when working above 1GHz.   Looking forward to watching.   

Could you do me a favor:

1) Power on the Siglent and allow it to warm up for at least a half hour.
2) Set the VNA to sweep over the entire frequency range using an IFBW of somewhere around 100Hz (make a note of this).  Use something around 800 to 1600 points. 
2) Using your best cables and standards, perform a SOLT calibration
3) Attach a load to each port and measure S21.   
4) Save the data to a Touchstone file and upload it here.  You may need to rename it or ZIP it up.  Maybe name the file to include the model, IFBW, range.  Something meaningful.   

I have collected data for all of my low cost VNAs as well as for the LibreVNA.  It would be nice to include the Siglent.  Do this and I will post the updated plots here. 

If there is other data you would like to see, just ask.

[807]

Considering the vast difference in price, the SAA2 held up very well against the Siglent.

Recently bought a NanoVNA-F v2. Haven't even taken it out of the box yet. There's such a vast number of different versions to choose from. I finally settled on the F v2, as it has a metal enclosure & seemed to get favourable reviews.

[nctnico]

Considering the vast difference in price, the SAA2 held up very well against the Siglent.

From what I understand from others is that this particular Siglent model isn't very good where it comes to VNA performance.  Especially at low frequencies (which is relevant for doing power distribution network measurements). So it is no surprise a super low cost device shows equal performance. I have considered the SSA3021X Plus to get a low cost VNA but came to the conclusion that it is not worth buying for use as a VNA.

[The Bootloader]

Hi,

1) Power on the Siglent and allow it to warm up for at least a half hour.
2) Set the VNA to sweep over the entire frequency range using an IFBW of somewhere around 100Hz (make a note of this).  Use something around 800 to 1600 points. 
2) Using your best cables and standards, perform a SOLT calibration
3) Attach a load to each port and measure S21.   
4) Save the data to a Touchstone file and upload it here.  You may need to rename it or ZIP it up.  Maybe name the file to include the model, IFBW, range.  Something meaningful.   

I tried, but I cannot set the IFBW to anything other than 10kHz. The setting is greyed out and locked at 10kHz. I do have the capability to change the sweep time, I tried increasing it significantly, but it does not change the IFBW.

In the specs of the SVA1032X from Tequipment, they do write that the IFBW for the VNA feature is 10kHz:
https://www.tequipment.net/Siglent/SVA1032X/Spectrum-Analyzers/

However, on the website of Siglent NA, they write "Minimum RBW 1Hz" but I suspect this applies to the Spectrum Analyzer feature and not the VNA feature.

The SVA1032X I have has all options enabled so I doubt it's an issue about software options.

I looked into the Siglent Programming Guide for this instrument. The corresponding SCPI command seems to be ":BWIDth?", I tried and it returns 10000.
I tried issuing ":BWIDth 100", it returns success, but when querying the bandwidth again it's still stuck at 10000, and it does not change on the GUI either.

So... it looks like the IFBW is locked at 10kHz.

The datasheet confirms:


So, I proceeded to still capture the data for you, with that 10kHz IFBW. See attached.

I believe the most valuable measurement is the one where I averaged 100 readings, it displays the true noise floor of the Siglent SVA1032X:



I attached all measurement raw data along with a file describing all measurement conditions.

[Mechatrommer]

Considering the vast difference in price, the SAA2 held up very well against the Siglent.

not just the price, but BW difference. "real" 4GHz and above VNA's are simply unaffordable. but credit to OP bootloader for making a review comparing them with more expensive unit and revealed how cloned NanoVNA fair up - NanoVNA V2.2, LiteVNA and KC901V owner here.

[The Bootloader]

Alright, alright, I like this, we are finding interesting things together!

I took the averaged data I posted above from the Siglent (100 readings 1600 points 100k-3.2G), and plotted it against averaged data from the SAA2 V2.2 (200 readings 801 points 10M-1.5G)
[EDIT] Adding measurements from the clone NanoVNA (50 readings 801 points 10M-1.5G)

Results:

[joeqsmith]

Thanks for posting that data.  As promised the attached shows how your VNA compares with some others.  The large step in the LiteVNA's data is where it switches to use harmonics.

As you can see, my original NanoVNA performs very well at the lower frequencies (<1MHz).  I never used it above 300MHz.    I never found stable enough firmware for the H4 to use it.   The LiteVNA64 cost about $120 and if you don't care about narrow band measurements and want to experiment beyond 300MHz, it's my  choice.

***
Looks like I forgot to include the V2Plus4 and V2Plus.  It's limited to 4GHz and has similar noise as the LiteVNA.  The V2Plus was a little worse than the V2Plus4.  I understand they jacked up the prices for it and I doubt many people would consider it now.   

[joeqsmith]

I mentioned that my friend had sent me their first LiteVNA which exhibited very poor noise.  This was due to the designer's not considering the specs of the USB standards.   I had added a series diode to the power lines which improved it.   I then set about attempting to further improve its noise floor by changing the bypass, increasing the IF gain and removing the TVS diodes.  I also changes some other values.  First attachment shows my original stock LiteVNA compared with my modified one.

The second attachment shows how the modified one compares with your Siglent.  Easier to see without all the clutter.   You can see the twp lay on top of one another all the way up to 3GHz. 

The Lite does allow setting the IFBW and I can also average and smooth if you like to make it look better but I do not typically use it this way. 

***
pictures are out of order....

[Mechatrommer]

I thought you stated you mod your siglent to 3ghz version. Why dont plot to 3ghz?

[The Bootloader]

I thought you stated you mod your siglent to 3ghz version. Why dont plot to 3ghz?

I was hoping no one would notice
Because I mistakenly assumed the NanoVNA was only good up to 1GHz. Oops
I realize the SAA2 is actually rated until 3GHz, so you are right, I could have plotted it until 3GHz.

The main objective of this video was to show the issues with the NanoVNA clones, not do a metrology-grade comparison. I think we can clearly see that at least until 1.5GHz, the genuine NanoVNA compares favorably with the Siglent VNA, and the clone NanoVNA is terrible.

So, mission accomplished I guess?

[The Bootloader]

Alright, more goodies for you!

Is the bad performance of the clone NanoVNA caused by the lack of shielding?

Well we are going to find out, and you are going to learn how I make custom RF shields too!



Spoiler alert, data/plot attached.

[Geoff-AU]

The main objective of this video was to show the issues with the NanoVNA clones, not do a metrology-grade comparison. I think we can clearly see that at least until 1.5GHz, the genuine NanoVNA compares favorably with the Siglent VNA, and the clone NanoVNA is terrible.

So, mission accomplished I guess?

Yeah.  This is news to a lot of people so I'm glad you have the energy to demonstrate it.  Clones are hot garbage, buying most electronics via ebay/amazon is hot garbage.  RTL-SDRs should only be purchased from the rtl-sdr.com blog official sources and so on. 

The official NanoVNA v2 (whichever one goes up to 4GHz) is bloody AMAZING for the price.  The isolation degrades at high frequencies >2GHz but gee.. it's so cheap and so good you can use it for a lot of stuff.  Even though your dynamic range at the top end isn't great, it's still giving you GOOD data rather than lies so you can rely on it for what it can tell you.

Is the bad performance of the clone NanoVNA caused by the lack of shielding?

It's caused by karma.  If you buy a clone, you get what you deserve 

[Mechatrommer]

I thought you stated you mod your siglent to 3ghz version. Why dont plot to 3ghz?

I was hoping no one would notice
Because I mistakenly assumed the NanoVNA was only good up to 1GHz. Oops
I realize the SAA2 is actually rated until 3GHz, so you are right, I could have plotted it until 3GHz.

and then i got it backward, i thought you plotted up to 1.5GHz for siglent and 3GHz for SAA2, its actually the other way around when i look at it back here at home. btw what surprised me from your report, is how come cloned Nano becomes a joke past 400MHz albeit using same components/BOM? and probably same OSHW gerber file.

[The Bootloader]

btw what surprised me from your report, is how come cloned Nano becomes a joke past 400MHz albeit using same components/BOM? and probably same OSHW gerber file.

Right! I have no idea. This is why I started doing a video adding shields, I was expecting an impressive improvement... surely it's just a lack of shielding? oh boy I was wrong. Although adding shielding did seem to fix a few artifacts particularly that weird dip around 1100MHz.

The next hypothesis I have is inferior components (capacitors, inductors, fake ICs, rejected ICs, or simply a BOM mistake causing them to use the wrong ICs), and/or materials (PCB substrate?). It would definitely be interesting to identify what is causing this.

The fact that my particular clone is missing shields is not encouraging. It's not something that they could have missed. They left the shields out on purpose. So who knows what other shortcut they may have taken to produce these

[Mechatrommer]

shield only to shield EMI, if your room is clear of EMI, i think there should be no difference shield or no shield... the dip caused by shield i imagine is due to some resonance or impedance modifier when geometry/space changed by adding shield. btw... looking back at your video in OP, the cloned one is different pcb design (sma connectors at the side) compared to ori SAA2 sma at the bottom. that may explain the very different performance.

[joeqsmith]

Your original NAnoVNA is similar to what I show.  I'm not sure why you feel its a problem.    You have the Siglent.  Use it to look at the output signal.  At 300MHz, the NanoVNA will change to using harmonic mode.  As you go up frequency, the VNA will continue to change bands and which harmonic is used.   I'm guessing it was designed for 300MHz.   Of course, the S/N ratio is going to get much worse the higher up you go, shielding or not.   

That was a benefit to the LiteVNA.  Now that the firmware supports harmonics you can see where it changes.  They have limited it to 9.3GHz which like the original NanoVNA at 1.5G, is poor.  You can see that in the plots I provided. 

Of course, you need to be aware that when you are using harmonics, that fundamental frequency with all it's power is still present.  It can lead to problems with the measurement and damage of your device and VNA.   Think of testing a wideband amplifier.   

And of course, with these low cost VNAs you have a squarewave drive.  Another can of worms....

I watched your 1st video.  While easy enough to follow along, I was really hoping for you showing a lot more than just a couple of antennas and filters.  Maybe consider a detailed review at some point once you have learned more about it.

Video demonstrating an amplifier being tested on my low cost VNAs with squarewave drive vs my vintage tech.

[Mechatrommer]

And of course, with these low cost VNAs you have a squarewave drive.  Another can of worms....

i think designer already figured it out and dealt with its shortcoming in IF...

I watched your 1st video.  While easy enough to follow along, I was really hoping for you showing a lot more than just a couple of antennas and filters.  Maybe consider a detailed review at some point once you have learned more about it.

imho its enough to show one example that its broke to prove that its broke...

[thinkfat]

Just a remark: The "nanoVNA-H clone" is not a clone of the SAA2 V2.2. They have a very different hardware architecture. A fair comparison would be between a genuine nanoVNA-H and the Aliexpress clone. There's just no way adding a couple of shields would get them to the same level.