What's the bandwidth?
Looks like normal noise of the internal preamp / driver / ADC. Knock it down to a lower bandwidth setting (or turn on averaging) and it should decrease accordingly.
The scale of the "signal" doesn't matter, because it's a few counts at the ADC, not actually a few volts at the input.
For further reading, check out the user manual in great detail (it should specify what the noise floor and bandwidth is on various ranges), and read up on theory like the Shannon "noisy channel" theorem, Johnson-Nyquist noise, and etc.
Tim