My opinion has been for a some time that scope FFT is usually good only for signals that you know beforehand, regardless of the scope price point. Due to aliasing and other issues, real spectrum analyzer beats it hands down. I wouldn't even try comparing my Agilent MSO6034A FFT to my Rohde&Schwarz FSV7, the difference is so big when things get serious.
For coarse sanity check, you can use Fourier series to calculate approximate amplitudes of harmonics of square wave. For example, 50% duty cycle, and 5 Vpp square (amplitude 2.5V), the fundamental frequency peak should be at 4*2.5/pi = 3.18 Vpeak, or 2.25 V RMS. 3rd harmonic should then be at 1/3 of this value etc. I don't know if Rigol FFT is supposed to show RMS values.
Conventional spectrum analyzer works just like a superheterodyne radio receiver. Spectrum analyzers use nowadays FFT, but since signal is bandlimited, no aliasing occurs. Sensitivity and dynamic range are also usually much better. Typical dynamic range is something like 80 dB and sensitivity goes down to microvolts. Downside of spectrum analyzer is that input is not so abuse-proof than scope. You can destroy the input mixer with too large input signal or DC voltage. Also, the frequency range typically begins around 10 kHz, but with special options, you can get downto something like 20 Hz.
Typical use for a spectrum analyzer for me is to measure the cleanliness of digital circuits (less noise means better layout) and switch mode power supplies and of course, diagnosing EMI problems. EMI problems also usually require logarithmic frequency axis to have sensible display due to wide bandwidth. With a tracking generator, spectrum analyzer can be used to check high frequency response of power supply decoupling etc.
One application is that you can look at your oscillator signal with a very narrow bandwidth, just to check if your power supply noise modulates the oscillator (any signal produces sidebands around the oscillator main signal). See the attached screenshots for some measurements I have made with the FSV7. Perhaps one can judge from them how the scope FFT would work with these.
Regards,
Janne