Interesting as I was just thinking about a project yesterday as a way to A/B audio to compare the two sources - switching them without (noticable) delay, and without artifacts.
Moffy is spot on - wait for the zero cross of the current source (source 1), then disable source 1, wait for the zero cross of the next source (source 2) then enable that. Given its audio there really won't be a long time between the zero crossings of two different sources, so there shouldn't be any noticable delay.
Zero crossing detection would have to be fairly quick, and, once again as Moffy said, you'll need two, or four if you're switching stereo sources.
A cheaper way to redice clicks/pops is to ramp down/up sources, this is often used in commerical gear, but I dismissed this idea as even though it happens very quickly, it could sitll create a noticable drop, which isn't ideal for my application. If you're just switching sources to an amplifier - manually switching, rather than trying to switch active sourcesduring playback, then that could be easier.