The expected range for the input bias is in the 10 pA range. So most normal ampmeters will not show anything. With 10 Mohm input resistance this may just get 100 µV, but possibly also less.
The input offst is often less than 1 LSB, maybe in the 10 µV range. Additional offset can come from thermal effects at the protection and the connectors.
For the test with the batteries, there is a chance that the battery volage may not be stable. With only 6.2 voltage left, chances are it is pretty dead and can change quite a bit.
The asymmetry shown looks more like a gross error, like hum, a loose connectio or dead battery.