General > General Technical Chat
How much accuracy do YOU really need?
David Hess:
The old solution for more accuracy and precision than your instrument can return is to make a null or slideback measurement using a precision accurate DC reference source.
For instance the Tektronix 7A13 differential comparator allows an analog oscilloscope to make voltage measurements to 4 digits of resolution and 0.1% accuracy.
ebastler:
--- Quote from: madires on February 09, 2023, 02:57:27 pm ---accuracy <-> precision <-> resolution
--- End quote ---
Umm... what do you mean by the "<->" symbol?
Gregg:
Slide rules are another example of how much accuracy the old timers really needed.
tggzzz:
--- Quote from: Gregg on February 10, 2023, 10:19:49 pm ---Slide rules are another example of how much accuracy the old timers really needed.
--- End quote ---
12/24m long scales in some cases, giving 4 to 5 digits, e.g. https://en.wikipedia.org/wiki/Fuller_calculator
David Hess:
It really depends on the circuit and application. With precision linear circuits, it is possible for an error of greater than 1 count out of 100,000, or even 1 million, to indicate an error which requires fixing. These are the same kinds of circuits where it may be mandatory to have a voltmeter with gigaohm+ input resistance for useful measurements.
More than once I have diagnosed the existence of a problem in a precision analog-to-digital converter because the least significant digit flickered more than 1 count when it should not have. I have also done the same a couple times now with digital counters.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version