To give you some idea, I helped design a VHF telemetry system that's in use on several satellites in LEO. At the horizon they're of the order of 3,000km distant and have a downlink power of about 30mW and can be decoded with a simple dipole or 1/4 wave. The receiver channel bandwitdth is about 3kHz to help deal with doppler, but the occupied bandwidth is about 1.5kHz and the raw BPSK bit rate is 1200baud. After a concatenated FEC scheme the actual data rate is about 500bps.
You can trade off bandwidth/bit rate against power, but keep in mind that the narrower bandwidth will mean it's harder and harder to correlate a signal, i.e., you need a lot of processing power to fish the signal out. Your reference oscillator in the transmitter will likely be not too accurate in relation to carrier and channel bandwidth.
The choice of modulation and coding schemes is key here, but also in relation to the efficiency of your transmitter. You might find that operating FSK is more efficient to transmit than BPSK based on pure power, but BPSK can be demodulated at much lower SNRs.
The FEC I use includes both a convolution and a block encoder with interleaving so that a transmision affected by single bit, burst and fading can all be recovered from.
Also the choice of frequency band is key: there's an awful lot of noise at lower frequencies, and making directional antennas to reject noise from the warm earth is much harder. You may find a sweet spot higher up in UHF for example. Edit: and smaller antennas will be more efficient too.