50dB is very reasonable for most analogue quadrature mixers, there's probably not much point trying to polish that any more, environmental effects such as temperature will start to become significant.
I am pretty familiar with the way the IQ image correction works in SDR Sharp, I did some work with the author of the software as well as doing my own IQ correction algorithms. I can assure you that it isn't "easily fixed in software", it is actually quite complex to get it right. It is not solvable analytically, you have to use numerical methods.
SDR sharp uses a stochastic method, i.e., it makes random attempts to correct for phase and amplitude channel mismatch, takes an FFT, and then correlates the effects of the guesses on upper and lower sidebands, maximizing the aggregated power delta between opposite FFT bins. Over time, the more guesses make for better image rejection.
The method I used is similar, but it's iterative rather than stochastic, narrowing down phase and gain imbalance successively. While this attains the imbalance match more quickly it's a bit harder to implement.
If you're unable to get a balance, then whatever you're using to sample might not be simultaneously sampling. There have been instances of sound cards sampling 1 sample out on the left channel to the right channel which is impossible to correct for with I/Q imbalance correction alone.