What happens if you keep pressing the one-shot button?
Create a table:
Trial ChA ChB
1 5.5mV 6mV
2 ? ?
etc.
You're doing noise, so you have to do statistics! If there is a clear trend between these data series, then one may well be noisier. So far, your two data points ("looks like 5.5, 6" and single shot "1.8, 1.56") aren't suggesting much of a trend.

The scope tells you it's reading some amount of mV, but that's a convenience only; ultimately, it's only measuring what appears at the sampling circuit. Which is the sum of internal noise (the analog to digital converter isn't perfect, and makes mistakes), buffering and amplification (random thermal noise in the circuits, amplified by a lot because the converter itself probably senses ~1V signals), and what's actually at the input connector itself (which isn't much compared to the rest, even if terminated in 50 ohms and nothing else). And when it performs a sample operation, it simply pulls that number out of the random cloud: the expected value of a given sample is simply the amplitude of the noise, period. So from any given series of samples, it's simply going to be... random, no matter how fast or slow you sample at.
The noise amplitude depends on bandwidth, however. If you have the scope configured for 100MHz (or more?) bandwidth, and the noise spectrum is flat, then reducing it to 20MHz BW will yield sqrt(100/20) ~= 2.2 times less noise. The reduction is even more significant if you set an even lower bandwidth: there are probably settings for digital filtering (as a percentage of displayed rate), or "high res mode", or averaging mode (with averaging, the expected reduction for un-triggered components is sqrt(1/N), so for N=100, the noise goes down by 10x).
Tim