Probes, probes, probes...!
You mention that you're using the scope to measue the ripple and noise on a PSU under load, and this is a very difficult thing to do correctly and accurately without some experience with scope probes and their limitations. (It's something I've used as a practical test as part of an EE job interview, very few people got it right).
The main problems are common mode noise and RF pick-up, and these issues can be reduced (not entirely eliminated) by choosing a probe which allows you to make a very short ground connection between the probe and the UUT. Typically this involves removing the ground wire with the croc clip, and making a connection instead using a grounded sleeve which the probe should have right next to the tip.
Hint: always try probing the exact point where you make your GND connection to the UUT. In theory, of course, this shoud always measure 0V exactly, but it won't. Remember at the very least to subtract whatever ripple and noise you measure at the GND point from whatever you measure on the supply itself.
If you don't probe correctly, it's very, *very* easy to make a noise measurement which is wildly inaccurate and incredibly misleading.