Math channel got me confused...

[eTobey]

Info:
I post this here, so that others could benefit from. It got me confused, because i expected the math channel to display the exact same data, not beeing influenced by the vertical scale. I have the SDS800X HD, but i guess this would apply to other scopes too.

When saving data from a math channel, from a certain scale onwards, signal values are cut off:
Since pure data has nothing to do with an opamp, that could only go to a certain value, i would have NEVER expected this. It makes no sense at the memory level, since saving the value 0, or the value 6537478245 would not make a difference. See picture "same data...": The same waveform (at a different scale), but the signal is cut off at ~0.5V.

Same cause, but different result:
When the math channel is scaled, it looses precision (see picture "precision loss").

Seems to be a 16 bit datatype. So the precision of the saved data would be roughly:
Precision = vertical_scale* 8.53 / 65520

vertical_scale= needed_precision * 65520 / 8.53

(8.53 = max vertical extend in divisions / 65520 values for datatype)

(Yes, the labels of the traces are bad, but thats due to the stupid keyboard in this scope, where i dont want to do 3 clicks to have a spacebar or a "-" added. I copy filenames over, to not introduce errors.)