Rock and a hardplace.
The ground loop tonight was obvious yet. However what was also apparent is that it doesn't like to be pulled low at the audio ground point. As soon as I connected the scope to the breadboard the audio got mangled as the scope was pulling the audio GND down to earth. Without the ground connected the scope showed my signals modulated on top of a 50Hz sine wave (and a mess).
There has, come to think, always been a low level hum on the headphones when the speakers are powered on, often I disconnect them or turn them off, I put it down to a ground loop, but it's more likely coupling of that 90VAC leakage being grounded to the PC audio connection.
So if I earth reference the PSU there will be a loop formed between the PC ground and the speakers ground and potentially the leakage and other noise will just "freewheel" around the loop creating more EMF than I would like into the bargain of adding hum to the audio.
Are audio transformers a thing? Would that even have an effect? Would it not couple through the 90VAC as well?
I'm thinking maybe 10 years out of these is long enough. Which brings me to a second problem. I don't want to spend £500 on a small amp and speakers and if I go down the Amazon no-brand route, I run a high probability of encountering another leaky SMPS. I suppose, at least with amps you can look at the back and see if they have a ground lifter / lug.