Products > Test Equipment
Pocket-Sized 6 GHz 1 TS/s ET Scope
joeqsmith:
--- Quote ---Is this in your custom software, or our software? The check should not be required for shorted inputs - it's there only for low-sample situations where the data is very noisy. May be worth tracing down where the NaNs are coming from.
--- End quote ---
My software. It's very possible I am still not understanding your fitter math. Attached data is with short applied to channel 1. No CDF data falls within the 0.9 to 0.1 range. For now it just sets the NaN as you don't define what to do in this condition and I didn't ask.
Sweeping DC (single ended only), appears. linear.
SJL-Instruments:
--- Quote from: joeqsmith on February 03, 2024, 01:46:48 am ---My software. It's very possible I am still not understanding your fitter math. Attached data is with short applied to channel 1. No CDF data falls within the 0.9 to 0.1 range. For now it just sets the NaN as you don't define what to do in this condition and I didn't ask.
Sweeping DC (single ended only), appears. linear.
--- End quote ---
Thanks for the clarification. In this situation, the official software reverts to the 0.5 CDF interpolation method.
***
In fact, we revert to the interpolation method unless there are at least 2 points in the 0.5-0.9 range and 2 points in the 0.1-0.5 range. This is to prevent single outliers from blowing up the fit.
joeqsmith:
Thanks for that info. I'll add that check to my software as well. Goal of this software is really just to give me an easy way to test your hardware/firmware while I wait for your software to catch up.
Tonight's test, collecting some long high res data sets. Well... compared to my typical 1-4ns tests, its long...
joeqsmith:
--- Quote from: SJL-Instruments on February 03, 2024, 02:04:44 am ---In fact, we revert to the interpolation method unless there are at least 2 points in the 0.5-0.9 range and 2 points in the 0.1-0.5 range.
--- End quote ---
I want to make sure I understand this simple statement. So 2 data points of 0.5 gives us 2 points in the 0.5-0.9 range and 2 points in the 0.1-0.5 range. Details details...
SJL-Instruments:
--- Quote from: joeqsmith on February 03, 2024, 02:13:33 pm ---
--- Quote from: SJL-Instruments on February 03, 2024, 02:04:44 am ---In fact, we revert to the interpolation method unless there are at least 2 points in the 0.5-0.9 range and 2 points in the 0.1-0.5 range.
--- End quote ---
I want to make sure I understand this simple statement. So 2 data points of 0.5 gives us 2 points in the 0.5-0.9 range and 2 points in the 0.1-0.5 range. Details details...
--- End quote ---
Luckily, this cannot happen, since the returned values are expressed as an integer fraction over 255 (odd). :)
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version