One source of "pop" will be the output 250uF (or whatever you've actually used) DC decoupling capacitor charging to 1/2 of Vs during switch-on. The LM386, being designed for single-supply operation, biases itself internally to 1/2 of the supply voltage.
Removing this the naive way by simply putting a relay between capacitor and speaker will just delay this charging until the relay switches on - you'll need to find another way to pre-charge the capacitor before connecting the speaker, such as a resistor a good deal higher than your speakers impedance between capacitor output and ground, then relay between that and the speaker, and just make the time constant before the relay switches on longer than the time it'll take to charge the cap.
A "better" way would be to use a dual-rail amplifier of a type that doesn't pop, but this may not be practical for your design.