The USB is not connected, the Arduino Nano is used as a cheap "CPU module" and there is not much else on the main board except of a few 5mA leds which are driven by a PWM (directly from the AVR), no long wires or anything. This could make some noise, but then I would expect to see a lot of harmonics and not a single line at 120MHz. It looks like some kind of oscillation, but there are no DC/DC regulators (just the LDO on the Arduino) and no Opamps or something similar that could oscillate.
I didn't expect any problems with this product and was very surprised.