Yep, its picking up RF/EMF noise in the air.
Totally normal.
Should go to solid zero when you short the probes together.
Some meters have added hackery/code to try and show 0.0000 if it looks like very small noise, but this is bad because if you actually have a very low signal your meter may lie to you and says its zero.