Aside from the power rating, power losses are affecting the circuit in two more ways.
The current passed through the potentiometer must be large enough to power the motor. That means: low resistance. And since a potentiometer also passes current to the ground, you are creating low-resistance path across the power supply. A lot of energy does nothing except heting up things.
If a considerable portion of the current coming into the potentiometer is diverted through its taper, each leg sees hugely different current. At low currents that’s not an issue. But if the current is large enough to cause noticeable self-heating, each leg reaches a different temperature, slightly shifting resistances. So the output slowly drifts as the device heats up. You would need a potentiometer that can dissipate heat well for that to not happen.
The transistor may be seen as a kind of separator between the potentiometer and the load, making the output of the potentiometer independent of the motor itself and removing restrictions on potentiometer’s parameters.
If that was not a fan: a motor is not a constant resistance. Due to back-EMF the resistance seen by the source depends on the load. If that would be a simple voltage divider, any change in load would change voltage applied to the motor. The reason a fan is an exceptional case is that it is constant load after spinning up. Unless you stick your finger in it.