I've not done any simulations, but my advice would be to flood the bottom side with solder. A copper shim could work (if soldered onto both the board and connector), and would give a little bit more mechanical strength. Don't use aluminium since you won't be able to solder to it (among other issues) The shim won't magically "fix" the impedance mismatch issues, unless the board layout is already designed for it.
Comparing the mechanical drawings of the connectors for various board thicknesses, the pin diameter stays the same, it's only the height of the bottom ground "pins" that varies. The width of the RF PAD on the PCB does vary between board thicknesses, and is different than the width of a 50 ohm track. Note that the connector is designed for two-layer PCB (not four layer). The suggested footprint assumes some particular dielectric (FR-4 with 1 oz/ft^2 copper), so perhaps they are not optimal for you either. Designing the PCB RF track width to match 50 ohm impedance is a much more important than a perfect coax to PCB transition.
Since you're using SMA, I'm assuming it's lower frequency (less than a few GHz), I doubt that the connector is so critical from an electric standpoint. If you care about precision, you'd be avoiding SMA (and use 3.5mm instead). Assuming your board isn't large (significant fractions of a wavelength), the accuracy of your transmission line impedance wouldn't have a large effect on performance.