EEVblog Electronics Community Forum
Products => Test Equipment => Topic started by: petemate on November 13, 2020, 12:58:40 am
-
Hi guys,
Why is it that the Keithley 2000, which is a 6½ digit DMM, shows 9½ digits when you read data through GPIB or RS232? For instance, the display shows 0.999563V, but the remote read value is 0.999562592V (output as 9.99562502E-01). Whats the cause or purpose of all these extra and probably redundant digits?
-
It's filtering it just gives you the actual result over remote rather than display @ valid 6.5D on display. Maybe you want the full result to process later.
-
Why is it that the Keithley 2000, which is a 6½ digit DMM, shows 9½ digits when you read data through GPIB or RS232? For instance, the display shows 0.999563V, but the remote read value is 0.999562592V (output as 9.99562502E-01). Whats the cause or purpose of all these extra and probably redundant digits?
It sent you a floating point number, binary floating point numbers are imprecise when converted to decimal representation. :-//
-
The number is just a floating point number. If one carefully looks at the data, not all values are possible, but there are discrete steps. These steps are not 10 µV to get exact 6.5 digit resolution for the 10 V range. However they are also not a simple 1 µV or 0.1 µV but more a odd number (and slightly different between individual meters), as the calibration is with a numerical scale factor. The floating point number avoids extra errors from rounding, but the extra digits have limited value.
-
the extra digits have limited value.
Yep. Any thoughts that it might secretly be accurate to 9 digits will be wrong.
-
Thanks for your replies! But why not just round the "remote value" it to the value presented on the display? Why give the full, yet unreliable number?
-
Thanks for your replies! But why not just round the "remote value" it to the value presented on the display? Why give the full, yet unreliable number?
That's like asking why 1/3 has a lot of decimal places when you write it in base 10.
Conversion between number bases isn't always neat and tidy.
Edit: I'm assuming this transmission is using binary floating point numbers.
-
More digits are also usually available in ASCII mode results. One still has a similar rounding problem: The result from the ADC is usually with a somewhat better resolution, but the steps are not so simple to represent. The additional digits avoid/reduce a rounding error. The principle is that the quantization steps from the ADC may be at some 2.34567 µV or a similar odd number and it is thus difficult to represent that in a short decimal form: either you get rounding errors or additional digits with very limited value.
So the longer result send to the computer can be a little better than the 6.5 digit display, though usually not much. On the other side it does not cost much sending 1 or 2 additional bytes and the computer usually does not mind. The highest speed modes may / will show less digits or use binary data.
-
They might have wanted meters with different resolutions to report in exactly the same numeric format for interoperability.