Compliant mounting ( blob of acetoxy free electronics grade silicone) on the capacitor, or simply using a different manufacturers part ( or even a different batch) will probably lower the noise, along with using a different capacitor type, a small film unit might fit, or a PCB mount electrolytic using the same pad spacing.