General > General Technical Chat
How much accuracy do YOU really need?
<< < (5/7) > >>
David Hess:
The old solution for more accuracy and precision than your instrument can return is to make a null or slideback measurement using a precision accurate DC reference source.

For instance the Tektronix 7A13 differential comparator allows an analog oscilloscope to make voltage measurements to 4 digits of resolution and 0.1% accuracy.
ebastler:

--- Quote from: madires on February 09, 2023, 02:57:27 pm ---accuracy <-> precision <-> resolution

--- End quote ---

Umm... what do you mean by the "<->" symbol?
Gregg:
Slide rules are another example of how much accuracy the old timers really needed.
tggzzz:

--- Quote from: Gregg on February 10, 2023, 10:19:49 pm ---Slide rules are another example of how much accuracy the old timers really needed.

--- End quote ---

12/24m long scales in some cases, giving 4 to 5 digits, e.g. https://en.wikipedia.org/wiki/Fuller_calculator
David Hess:
It really depends on the circuit and application.  With precision linear circuits, it is possible for an error of greater than 1 count out of 100,000, or even 1 million, to indicate an error which requires fixing.  These are the same kinds of circuits where it may be mandatory to have a voltmeter with gigaohm+ input resistance for useful measurements.

More than once I have diagnosed the existence of a problem in a precision analog-to-digital converter because the least significant digit flickered more than 1 count when it should not have.  I have also done the same a couple times now with digital counters.
Navigation
Message Index
Next page
Previous page
There was an error while thanking
Thanking...

Go to full version
Powered by SMFPacks Advanced Attachments Uploader Mod