There's a considerable difference between floating a scope to break a ground loop that would otherwise cause a large circulating current through the probe ground, but with only a volt or two of difference between the ground points in question, and floating a scope to attempt to make a differential measurement with a probe ground clip on a non-ground circuit node. The latter is *ALWAYS* unsafe and if mains or high voltages are involved, is potentially lethal.
However using an isolation transformer to power the scope with output Neutral strapped to output Ground, and an adequately sized grounding connector bonding its output ground to the engine block, steel or aluminum hull or sacrificial anode on a boat is just about the only way to avoid the risk of high circulating currents in the ground if the boat doesn't have a permanent shorepower installation with properly bonded ground. Use a RCD on the output of the isolating transformer, connected *AFTER* the Neutral to Ground strap. N.B. the isolating transformer output is *NOT* floating, as it has been re-referenced to ground. Its Line terminal is 'hot' and although its Neutral is grounded, from a safety viewpoint it should also be handled as if it was 'hot'.