Well good design is to have protection as close to the source.
The fuse is directly at the input terminal, so it is as close to the source as possible.
Also, typically you want to have as good of a current return path as possible with the lowest impedance. If you have a fuse, you cannot achieve this because you add the fuse resistance to the 0V resistance, and you would also for example in board have to narrow all paths to connect to this fuse pad.
That is a nonsensical argument, as all paths will unavoidably narrow down to the power supply wires. Whether that narrow point is located a few millimeters further to the right than without the fuse is irrelevant. Also, that argument would apply equally to the non-GND connection. There is nothing special about the ground side as far as effects of high impedance supply paths within the circuit are concerned.
This can increase the voltage on the ground relative to the supply 0V and can cause issues in your electronics.
No, it can't. Also, the fuse in the non-GND connection equally has resistance and will change the potential on the non-GND supply rail. That also doesn't cause problems in your electronics. Also, both power supply wires have resistance and make it so that the potential of both supply rails on your PCB have a potential that is different from the respective potentials inside the power supply. That also doesn't cause problems in your electronics.
The only reason the potential shift due to the fuse's resistance can cause problems is
if there are alternative paths from the power supply's respective pole into the device that bypasses the fuse, because that then changes the I/O voltages on those connections, causes power supply currents through those other connections, and causes the fuse to be ineffective (because it blowing doesn't stop the current through the alternative paths ... and also, potentially, it doesn't blow in the first place because it doesn't see all the current in the first place).
But that in principle applies to either side. It's just that usually there is only one potential that is the reference potential for all connections into a device, which then is what we usually also call ground, which then, if those other connections could ever end up connecting to the power supply's ground, is why one shouldn't place a fuse on that rail. But the point is that if there aren't ever any alternative (low impedance) paths from the power supply ground to device ground, then there is nothing special about whatever supply rail we declare "ground" within the circuit that would cause any problems ... and also, if for whatever reason, there could be alternative paths to the non-GND rail of the power supply, then the same problem arises.
Putting the fuse in the non-GND is the convention because the vast majority of circuits don't have alternative paths from their non-GND supply rail to the power supply non-GND rail, and also, if there are alternative paths from a device supply rail to the respective power supply rail, that's almost always the ground rail. But fundamentally, what matters is that there are no paths that bypass the fuse. If there aren't, you can in principle put the fuse on either side, doesn't matter as far as correct function and safety are concerned. If there are, it doesn't matter which rail it is, you have a problem.
Secondly due to safety reasons. If your circuit is inside a machine and your fuse blows the circuit still has voltage applied to it but no return path so someone could try and repair, it and get an electric shock.
If someone repairs a device that could have dangerous voltages inside without disconnecting the power supply "because the fuse is blown" ... they're an idiot.