Moving the power supply outside the box all 50hz noise with its harmonics are virtually 0.
What happens if you leave the power supply in the box, but add some insulation material around it to avoid any 50Hz currents flowing through its metal case into the analyser?
If the electrolytic capacitors are the actual problem, it wouldn't make a difference if you put the power supply outside the box or leave it inside.
My guess would be a bad connection to GND somewhere near the analogue frontend, injecting the 50Hz current into the signal path.
It could also be some bad emc filter capacitor near the mains input.
Since you see 50Hz, it can't be anything after the bridge rectifier, because rectifying a sinewave doubles its frequency. Therefore bad capacitors show up as 100Hz + harmonics.