Connecting the grounds together isn't the explosive setup. Here's an example where things can go boom.
1. Your circuit/device is plugged into a wall outlet and is ground referenced (i.e., 0V is at the same potential as the ground of the outlet).
2. You want to see the voltage change across a component in the circuit/device. Let's take a resistor somewhere in the middle of the circuit (i.e., neither side of it is at 0V), for example.
3. The lower-potential side of the resistor is at, say, 18V at a particular moment in time, relative to ground.
4. The higher-potential side of the resistor is at, say, 24V at the same moment in time, relative to ground.
5. In the hopes of seeing the 6V across the resistor, you want to connect the ground wire of the scope probe to the low-potential side of the resistor and the probe to the high-potential side.
6. BAM! The 18V at that point of the circuit has been shorted to ground through the scope.
On the other hand, if the circuit/device was floating/isolated from ground, then,
1. The low-potential side of that resistor is not 18V above ground since it has no ground reference.
2. Connecting the ground of the scope to that point of the circuit then makes it 0V, relative to ground. No short circuit occurs because the only ground is still the one provided by the scope's ground wire.
3. Then, connecting the probe of the scope to the high-potential side of the resistor shows 6V, because the low side is at 0V (due to the scope's ground being connected to it) and what used to be the circuit's 0V before connecting the scope is now at -18V relative to the scope's ground.
Thus, the problem is not that an isolated circuit is grounded by the scope's ground wire. The problem occurs when the circuit is already grounded and the scope's ground wire is placed anywhere in the circuit except where it's 0V.
That was a bit long. Hopefully, it made sense and I didn't derp the details. Corrections are, of course, welcome.