NanoVNA Custom Software

[DiSlord]

Send scan command
scan start freq_start
scan stop freq_stop
After device only change strart and stop points, not made sweep (just start it if no paused)
if after you send
data 0
Device return current data set (and if at this moment not complete scan - data not fully correct)
You must wait sweep complete (need add delay)

So if used segment scan and call
scan start 1000000
scan stop 2000000
data 0  - at this moment device in most cases not complete current sweep measure (last part of data contain prev scan measure, yes all good if you not change sweep range and so on)
scan start 2000000
scan stop 3000000
data 0 - in this case first part data contain new measured result, last part - contain any data from prev scans)
You get not correct result if not add delays (yes you can calculate it, but .. not good)

If use sweep command
sweep start stop points
points - optional, if not set - used max possible for device points
Device set start stop frequency (and points count if set), after made sweep and pause
So next received command
data 0
or
data 1
Always return correct result
This old format for receive data from device, work on any firmware (not related to my)

After v0.7 i extend this command by add mask, if set possible get result after scan, not need call 'frequencies' or 'data 0' and so on, this allow save some time
In my last fw to mask also added uncalibrated data ask (this allow in all cases get uncalibrated data, not depend from options selected on device)
Also added binary command scan_bin (it return data in binary format as v2, not need additional data convert), but this command supported only by my firmware.

[joeqsmith]

I use a handshake with the data collection thread.  The old firmware may have cleared the buffer and resampled but I always toss out the first data set.   While I display the data, I don't use it for any measurements.   This caused the person trying to use your firmware to ask about the odd waveshapes.  Again, cosmetic.    It sounds like making these changes still requires the first data to be tossed out, so I still don't understand what more faster means.

If the sweep command is causing the problem, are you suggesting that making two separate sweep calls rather than one is causing the poor performance?   If so, I can try that combination as well but it seems if you are testing your firmware with:

scan start stop CRLF
data 0 CRLF
data 1 CRLF

This is how I should test it.  Again, tossing out the first sweep after the scan is sent. 

[DiSlord]

scan start stop CRLF
data 0 CRLF
data 1 CRLF
In this case not need drop first scan, you should always get correct result after this commands.

Faster - not need wait additional thread sync in device for receive and exec shell commands (save some ms)

scan_bin more faster - not need made lot of float calculations for convert data in text format, less data send.

In most cases need get only s11 or s21 data, if use 'scan start stop' device always made 2 channel measure, if ask get only one channel data - sweep made 1.5x faster
example how receive only s11 data
scan start stop points 0b011
result list of
freq s11_re s11_im

[joeqsmith]

Again maybe a language problem but if I am only looking at S11, I send data 0, data 0, data 0 ....   I've always done this to speed it up.  When I first sniffed their software, I believe they sent data 0, data 1, frequency (repeat) which seemed stupid.  I know the frequencies, so that was dropped.  I then stopped asking for both data sets.  Again, it's been like that over a year.  With the new software, it's even further simplified.  Not having to drop that first data would help somewhat.  I suspect what you are suggesting is that the time to calculate the floating point values in firmware is taking a lot of time and ditching it would somehow speed up the over all data rates to the PC. 

It sounds like you are now suggesting that scan start stop will somehow cause it to collect both S11 and S21 but not send them.   This would be a detriment unless it actually improves on the poor performance from using the sweep command. 

I think now that I understand what is meant by more faster and the gains are small over what I have today, the real question is back to the poor performance.  Above all else, the data needs to be good.  If we can't get past that, the rest is useless.   It will be very interesting to see if how the ranges are set has anything to do with the actual data being sent back.  I wouldn't think it would have any effect on it but it's an easy test so we will see. 

[joeqsmith]

It's going to be a real pisser if I find out the hardware has a problem...   Still the box is pretty fancy and nice to look at.

[DiSlord]

>Again maybe a language problem but if I am only looking at S11, I send data 0, data 0, data 0 ....   I've always done this to speed it up.
when a scan command is received, the device does not know what data will be requested later (you send in future 'data 0' or 'data 1'), therefore it always measures two channels.

If you explicitly indicate in the mask that only one channel will be needed, then only it will be measured (which is faster).

But since you are not using a mask, it doesn't matter

[joeqsmith]

H4 with the stock firmware (0.5.0) looking at the 100uF//1uF.   Very similar to what we see with the old hardware and year old firmware. 

Running the regression test with this combination will not pass but outside of what appears to be the same old problem from a year ago, it seems alright.  It does not appear that the poor performance is related to the H4's hardware.

[joeqsmith]

Just using a dumb terminal, if I send "scan 100000 1000000" it returns without an error but the screen does not reflect the changes.  Sending "sweep start 100000" does change the start frequency on the screen. 

I then tried "scan 100000 1000000 101", no change. 

So the command that is wrong to use works but the one that fixes everything doesn't work, at least with 0.5.0.  Nice.   There's not much else I can do with that. 

****
Next I reloaded your firmware 1.0.38 and again using the terminal program I send the same commands.  Again, the H4s LCD does not reflect the changes.  So, I sent the frequencies command and it seems to report correct range.  I sent the data 0 command but the screen still shows the wrong range.

I then made the change to my software to use the scan rather than the sweep.   I left your 1.0.38 firmware installed in the H4.  It appears to have no effect (which I would expect) and the performance is again very poor. 

[joeqsmith]

Next I reinstalled the original 0.5.0.   The firmware behaves a bit different but it does appear to update the range.  Once again, we can see it had no effect on the data and the performance is decent. 

[joeqsmith]

Next I attached the original NanoVNA with my year old firmware to see what effects the scan command has with it.

help
Commands: help exit info threads version reset freq offset dac saveconfig clearconfig data frequencies port stat gain power sample sweep touchcal touchtest pause resume cal save recall trace marker edelay capture
ch>

Notice, there is no scan command.  So while the sweep command may be wrong, it certainly works and it much more compatible.  I think we have our answer to what the cause of the poor performance is.  Sadly, the standard firmware for the H4 isn't good enough to use.  I have packed away the H4 and won't bother with it.  On the plus side, it does appear that talking with it the wrong way works and it does put out good enough data to measure crystals.   I think that was all they were after.   

Back to writing manuals.... 

[DiSlord]

Hmm i check again Hugen v0.5.0 (from 21/02/2020)
https://github.com/hugen79/NanoVNA-H/releases/tag/0.5.0

scan command present in code (it added in 24/09/2019)

scan command does not change the device settings, and is intended only for receiving data from the computer

PS About perfomance - used 12k IF (possible see it on measure in 12k and 24k?) better in work on higher frequency > 50k. Old fw use 5k IF
Need made research better IF for < 50k work

Thanks

[joeqsmith]

Hmm i check again Hugen v0.5.0 (from 21/02/2020)
...
scan command present in code (it added in 24/09/2019)

You would need to search this thread to find when the members here had started to fix the bugs in the firmware.   Looks like I made that first demo video in August 2019.   This gives you some idea of the timeline.   Once the major problems had been corrected, there was no need for me to try other firmware.   

scan command does not change the device settings, and is intended only for receiving data from the computer
Thanks

The language makes it difficult for me to understand what you are trying to convey.   Of course the scan command changes the device settings.     
It's odd how they decided to change how the H4's screen functions once you control it remotely.  I had not really paid attention to it until I ran these tests for you.     

PS About perfomance - used 12k IF (possible see it on measure in 12k and 24k?) better in work on higher frequency > 50k. Old fw use 5k IF
Need made research better IF for < 50k work

Thanks

You claim one of the features of your firmware is supporting 800Hz but the old firmware will far outperform what you have at lower frequencies.   At higher frequencies, your firmware can really mess up the data.   I suspect your filters have done more harm than good.   BTW,  while I did not show it, using the scan versus the sweep had no effect when changing the bandwidth filters.  The data still exhibits the same poor behavior. 

As you said:

I tried to explain what you are doing wrong.  Even on the old firmware, this is incorrect.  ... On this I will no longer recommend anything, do as you want

I now tend to agree with you.  We have done about all we can.  You're happy with what you wrote and I am happy to continue to incorrectly use the old firmware and get good results.  We both come out winners. 

[joeqsmith]

A draft manual for 2.0 is now available.  Of course the TOC was updated along with the what's new.  The main changes are at the end where it describes the difference in the new version and the new PDN measurement feature.   

[joeqsmith]

If the "native protocol" of all these VNA variants is getting to be a tower of babylon, what may be desirable is a rosetta stone.

Program client software based on an abstraction of WHAT the VNA hardware is supposed to be able to do (e.g. measure parameters discretely vs. frequency points), not HOW the VNA hardware may be asked to get that data and how it does the data acquisition.

Then all that is needed for clients is a fairly generic stable interface for "data requests" that pretty much any VNA can somehow or other satisfy (given range limits, et. al.).

For the wrapper that supports that generic interface for a particular model of VNA then that would be reduced to something simple / stable and easily tested / regression tested since the API of the VNA exercised by its interface wrapper implementation would be simple, consistent, and possible to optimize (or work-around) completely arbitrarily without modifying either the SW client code or the VNA protocol implementation / VNA firmware.

One could transparently talk to GPIB, Ethernet, USB, serial port, whatever VNAs with the same client side API.

Obviously if there are certain "access patterns" which are important to be able to represent to optimize the VNA's throughput / latency in responding to them then defining the API interfaces so they can express asking for a batch / vector of data or whatever could be interesting to consider.  The client could ask for anything as fancy as it wants as a "batch transaction" / request vector within the limits of the defined API but the API implementation could freely simplify or optimize that as much as it can for a particular back-end device protocol.

At first I thought you were referring to these low cost VNAs that this thread is about, until I saw your mentioning of Ethernet and GPIB.   It seems like you are interested in writing a generic VNA program.  This sounds like an interesting project but I am not sure how you would pull it off.   Normally, when it comes to test equipment, if I see generic software, it will have very limited capabilities. 

I have a few VNAs and each is very unique.  It's not just the physical layer, or the protocols used.  The systems are totally different.   One has a mechanical interface that can't be controlled remotely.  In this case, I prompt the user what they need to do.   The features they support can also be very different.   For example, I have one that can do spectrum analysis.   If you want to measure S11, you have to manually insert a directional coupler.   My less crappy VNAs support log or linear sweeps.  They also have basic features,  like attenuators.  The attenuators may require manual setting or they can be remotely set.   Then there are countless options you are not going to find in every VNA.  To top it off, you would somehow need to rent or procure all these VNAs, learn how to use them, maintain them.

Some flavor of my software runs on all of my VNAs.   There is a common library which for the most part contains all the number crunching stuff.  For each VNA, there is a top level program that supports their unique feature sets.  Finally, there is an interface layer.     

For the low cost VNAs, the problems I have ran into are poor documentation and unstable firmware. 

It will be interesting to see what you come up with and what instruments you support.   Maybe start up a new thread for your generic VNA software. 

[joeqsmith]

Thanks for the post.  It was a good read.

I own a modest amount of test equipment and most of it is controlled remotely.  Most is from major brands.  The documentation is well written and the systems are fairly bug free.  For me, putting together software for them has never been easier.   Its rare I need software that would say, record data from a meter.  It's more common that I want to read the meter as part of a much larger setup involving other equipment all tied together.   

The VNA is no different.  If you look at my software, you will find it has the ability to control a linear stage.  This isn't something you will normally find in any VNA software.  Having the ability quickly and effortlessly change the software to run more complex experiments is really powerful.       

It is certainly possible to make external calls from LabView.  I would frequently use MSVC to write code that needed to execute faster or more efficiently.  However, I have not had to resort to that in several years.  The modern PCs are so fast I now just use LabView for anything PC related.   

From my experience with that first Nano, it became apparent that with it being opened source, a lot of people were interested in making their own changes to try and improve the firmware.  It seemed like it would be a never ending battle to find something that worked well enough to use.   It was bad enough that I put together a simple regression test to automatically check the firmware of the week.  Much of the changes were focused on features I really had no use for rather than addressing the problems. 

Not surprising, none of my vintage VNAs are open source.  I'm not against it but when you have people making changes who really don't understand the problems they can cause, it's doing more harm than good.  In the case of the original NanoVNA, having the source allowed a few people to address the basic problems and get it working.  So there can be an advantage depending on the skills of the people working on it. 

At least for these low cost VNAs, my demands for the firmware and hardware are simple.   Document your code, allow the software to control what ever peripherals are attached and send down good data.   Getting good data off the Nano was always a problem.  Not because the protocol was complex or the PC lacks performance.  It was just bad firmware.   Oddly, more commands were added rather than correcting the root problems which had nothing to do with the protocol.  Once the locals sorted out the basic problems, there was no need to do any more with it.  I am surprised that much of what I saw with the early firmware is still a problem with the latest releases. 

If people want to try and run my software with the original NanoVNA, the manual is pretty clear about different firmware/hardware combos being a problem.  All the software in the world isn't going to make bad firmware/hardware good.  We need a solid foundation to build on. 

Looking at the LibreVNA, it seems like they are having problems with their software and firmware.  While the moderators of some of the groups (for example https://groups.io/g/NanoVNAV2) have chosen to make them private (which I see it as a dick move),  the LibreVNA group seems to want to sensor the posts.  That was enough to turn me off of it.   Hopefully OWOs team will roll something out that can measure lower frequencies,  with a higher resolution (at least at lower frequencies) and an increased upper frequency range beyond the capabilities of the V2Plus4.   Even if they kept it closed source, as long at the interface is documented,  I would get one based on my experience with the V2Plus4.   Even if they just improved the V2Plus4 to allow it to fully replace the original NanoVNA, I would get one.   

I was mostly referring to the low cost ones here, I was just extending the example how the same basic "I want to measure X at frequency Y" approach is vendor / model / VNA / interface neutral and could apply to high end units or low cost ones.

I gather that the tower of babylon of both interface hardware as well as interface apis & protocols is a fairly major annoyance with respect to cross platform applications that control and access test equipment in general.  It seems like it doesn't need to be so bad these days one could have a simple HTTP/REST or JSON or XML or database or something based neutral exchange layer and then the instrument driver can go off and get the data however it wants, and the client applications can fetch/use the data however they want.

So far I've just got one nanovna model and no "boat anchor" devices.  But even just with that it's getting hard to keep track of what nanovna variants there
are and what the differences in their hardware capabilities are not to mention what differences exist in their protocols and firmwares with respect to how to talk to them from a client application and what works / doesn't.

Eventually I'd like to upgrade to measure to 6GHz, low frequency / narrow bandwidth crystal parameter measurement, and PDNs but given what I see in this thread and the mailing lists being full of "but X doesn't work on model Y, and Z doesn't work on firmware Q" it's seeming out of hand.

I get that they're trying to build it down to a cost level but it seems so penny wise pound foolish to have all the variants of these tiny little MCUs running mutually incompatible firmware as the interface.  They could just have made it so the processor board was pluggable e.g. arduino interface or something and then one could at least upgrade the CPU and the host interface (e.g. ethernet) so one wouldn't be quite so limited in terms of MCU and interface quirks / limitations by spending a few more optional dollars on a better controller.

Anyway I'd be willing to write a wrapper and API layer for these things in spare time but I don't have nearly enough hardware variants to make it very useful in the sense I couldn't port / test things to different devices without those protocols / devices available.

And in the client end since in your code's case you run labview I take it that'd mean it'd be more oriented to a C-API + DLL or better yet something over IP/UDP/TCP/HTTP or something that'd be much more portable cross client sw / platform.

I suppose if there was a simple suite of unit / system tests and a half decent client API / protocol in a few variants that might be enough to get some of these VNA firmware developers / manufacturers to be interested enough to support it instead of making everything firmware / device unique.

One thing I don't understand is the whole firmware / protocol business.  I gather that the nanovna v2 being open hardware (and open source, maybe, I think?) is something that's not representative of what other devices are / will be.  So I don't even know if there's open firmware / protocol documentation for any / many of the other low cost VNAs at this point or what.  I saw mention to sniffing the protocol of something or other so I guess that's one way to reverse engineer at least what the serial command / response sequences are for a given firmware/unit, though if that's really all there is to rely on for most of these devices then it's a bit sad state of affairs.

I am wondering if there is NOTHING relevant / compelling about the "boat anchor" APIs, SCPI, whatever for "lowest common denominator" basic VNAs that nobody thought practically of just reusing those protocols exactly or if needed extending them slightly vs. reinventing the wheel incompatibly....

[joeqsmith]

Robert Feranec, PCB Layout & Decoupling - Explained why it's so complicated

I've watched a few of his interactive interviews.  He does a good job with them. 
**
If it's not obvious, the reason including this series here is it explains the basics of PDN design, simulation and measurement.  In part 3, they use the Bode 100.  Could we use the $50 NanoVNA and a couple hundred bucks of hardware to make this same measurement?   

[Knuddel]

A draft manual for 2.0 is now available.  Of course the TOC was updated along with the what's new.  The main changes are at the end where it describes the difference in the new version and the new PDN measurement feature.

Hi Mr. Smith,

I would like to try out your new PDN measurement feature and may have the chance to borrow some DC-blockers and maybe a professional common mode transformer for short time.
Did you upload the two announced versions 2.0 for both VNAs already?
The last version that is visible for me is p1.08 and for the V2Plus4 it's p1.04 in your GitHUB repository.

However, as for my privat use I would like to build up the blockers and the CMT like yours, would it be possible to post the partlist and the 'how to' for this in more detail some time?

Thank you very much for all this investigations and education up to now!

[joeqsmith]

A draft manual for 2.0 is now available.  Of course the TOC was updated along with the what's new.  The main changes are at the end where it describes the difference in the new version and the new PDN measurement feature.

Hi Mr. Smith,

I would like to try out your new PDN measurement feature and may have the chance to borrow some DC-blockers and maybe a professional common mode transformer for short time.
Did you upload the two announced versions 2.0 for both VNAs already?
The last version that is visible for me is p1.08 and for the V2Plus4 it's p1.04 in your GitHUB repository.

However, as for my privat use I would like to build up the blockers and the CMT like yours, would it be possible to post the partlist and the 'how to' for this in more detail some time?

Thank you very much for all this investigations and education up to now!

Hello.   I haven't released the new versions to the public domain yet.  I am not sure when you will see them.   The document is being reviewed.   I am still finding minor problems in the software.  For the first time today, I tried it on the the V2Plus4.  Once things seem fairly stable, I plan to put together a video demonstrating the new features, most likely using the old hardware.    It all takes time.

Someone wrote me how my software would calibrate their V2Plus4 but would throw checksum errors when running sweeps.  I've never ran into this and it really makes no sense.  So I would like to see if they can supply enough information to replicate what they are seeing.  That would be a major bug and makes the software worthless if that is really what is going on.  I would like to sort that out before releasing it as well. 
 
The DC blocks shouldn't pose much of a problem but the transformer took some effort to sort out.   I am holding out for JohnG's transformer as it sounds like they have a lot of expertise in this area.   They don't seem to agree with how Brian collected the data or some of the measurements that have been made.  So give them a little time and let's see what they come up with. 

If you have a access to a VNA, it would be interesting to measure the performance of the transformer you plan to borrow. 

[joeqsmith]

Running the new software on the V2 Plus 4.   It's really too bad this thing can't out perform that original hardware.  Every time I use it, I am reminded just how slow the old hardware/firmware is. 

While I have not seen any sort of checksum error I wonder if there is a problem with the newer firmware.  I have never has a reason to change mine and it was shipped with git-20201010-86c7055.   If you check their store, they don't list it.

What's really odd is when they claim they can cal it but not sweep.  Both require data to be read.  It's the same section of code.  And if there is a checksum, I flag the error and throw out the data.   So I don't see how it would calibrate.   I use the same commands to control the V2Plus no matter if I am running a calibration or sweeping.  It's a really odd problem assuming it actually exists. 

[Knuddel]

There are no checksum errors from my unit.
I have one I believe from the first lot of 2021 delivered in April.

The firmware version is git-20201013-32077fd

I am using the Microsoft port driver 10.0.18362.1 that was installed automatically by Windows 10.

I added the through measurement of a Nooelec Inc. 50kHz - 8GHz SMA DC block after normalization with a simple through with 201 point segmented sweep both.

It is important to rewrite the sweep points setting to 201 after each segmented sweep.
Otherwise if this is above 1024 the software hangs sometimes...

[joeqsmith]

With the V2Plus4 supporting over 1000 points, I'm not even sure I tested the segmented option with the old software and would not be too surprised if there were problems with it.   With 2.0, the code for the segmented sweep has been changed a fair amount.  I just ran eight consecutive sweeps with 1024 points and saw no problems.   

Starting 40KHz lower than the unit supports may be part of the problem.  Then again, your firmware/hardware combo may provide acceptable results down to 10KHz.   It would be very easy to coerce values to the limits of what the Nano supports but I see no need for it.  The software wasn't written for the beginner and I assume the users knows what they are doing.  Taking this approach makes the software more generic and if in the future the limits are changed, there is no need to release new software to support them. 

It's also very possible the older software did not restore the original values and you would have to reset everything between runs (which would include the number of data points).   The firmware may not like the software setting values out of range.   Now that I have a need for segmented sweeps, this feature was rewritten to make it easier to use. 

Good to know that you are not seeing the checksum errors with the newer firmware.   That person has yet to respond with any additional information.   

[joeqsmith]

I think we have our checksum problem sorted out.  They were not using the stable release and also some of DisLords firmware.  The checksum faults appear to be when they are using DisLord's firmware.   I have no plans to try and support the firmware of the week so no problem.   

[joeqsmith]

Been busy with other projects but Flipper and I spent some time going over the entire document.   They had some good input that makes it a little easier to read.   It's now available on-line.

They were asking about the firmware for the original hardware.   I rolled back after running into problems with the latest releases. 
https://github.com/qrp73/NanoVNA-Q 

I haven't had a lot of time for testing the software.  It appears stable enough.  Hopefully I will have some time over the next week to put together a short demo and get it uploaded.  If there's something you would like to see, now's the time to ask. 

[joeqsmith]

...
I would like to try out your new PDN measurement feature and may have the chance to borrow some DC-blockers and maybe a professional common mode transformer for short time.
...

Making some progress and have started working on the video to demo the new software.   Are you planning to try to use your V2Plus4 or one of the other VNAs?   The software for it still needs some work and may not be released at the same time. 

I have added the image of the firmware I am using in the original NanoVNA to Github along with a link to their repository.   

Hope to have the video and software for the original uploaded this weekend.   

[Alextsu]

Looking at the LibreVNA, it seems like they are having problems with their software and firmware.  While the moderators of some of the groups (for example https://groups.io/g/NanoVNAV2) have chosen to make them private (which I see it as a dick move),  the LibreVNA group seems to want to sensor the posts.  That was enough to turn me off of it.   Hopefully OWOs team will roll something out that can measure lower frequencies,  with a higher resolution (at least at lower frequencies) and an increased upper frequency range beyond the capabilities of the V2Plus4.   Even if they kept it closed source, as long at the interface is documented,  I would get one based on my experience with the V2Plus4.   Even if they just improved the V2Plus4 to allow it to fully replace the original NanoVNA, I would get one.   

[/quote]

Hi Joe,
Just a short comment to make the things clear. What's concerning the LibreVNA group at groups.io.
I have asked Jan and he told me that this is unofficial group. It had been created by unknown person before the official LibreVNA group started, so nothing could be done about it.
In any case, You can reach Jan directly.