The ultimate floating supply is a battery. The only leakage to the Real World is what you accidentally allow.
The simple, generic answer is simply a power source which has neither end hard-wired to some other circuit reference (like "ground"). Of course, beyond that it is a matter of what you are doing and how much isolation do you NEED vs. how much isolation does the power supply allow.
And there are two ends of the spectrum for "isolation". At the low-end, how much leakage is there between the floating circuit and the rest of the world. And at the other end, how far above (or below) ground can you take the floating output without breakdown failure? Perhaps most hobby or commercial bench supplies may be capable of several 10s of volts (or maybe even 100V or more?) But, of course there are special experiments that require much higher isolation than that.
I am reminded of a high-voltage experiment that used a Van de Graaff generator to create the thousands of volts required for the experiment. But they needed an isolated small voltage at the top to power some instrumentation circuit. But how to get it? Some creative person thought of simply using a small DC motor/(generator) as the top pulley for the belt. So the motion of the belt created the very high voltage, and also turned the little generator at the top. Brilliant!
