Little bit more info is needed here.
What microphone do you use, what cable (must be shielded) what housing does your pre-amp have, where is its power coming from, how do you connect to your computer?
The mic is some Jaycar cheap thing at the moment, it simply has a transducer in it, completely passive device.
The pre-amp is in a full metal enclosure, grounded.
Mic cable is XLR, shielded, etc.
Cable shield is connected to the enclosure.
Cable ground is connected to the 0V rail.
All grounding is wired using a star configuration.
The connection to the PC is RCA -> Stereo Jack, shielded of course.
Also of note... when disconnected from the PC there is no noise at all, clearly the PC is the source.