Almost all lab DC CV/CC supplies have floating output. Some also provide earth terminal for ease of wiring. Some don't, because it adds weight/cost and most importantly, size on front panel. You don't need to have earth connection on PSU, especially when you do not understand what it is for.
If you are worried about turn on spikes of PSU and still want to keep PSU connected with cables all the time, make adapter cable with switch or relay. Something like this:
DUT - cable - switch - cable - PSU
Then first turn on PSU, then turn on the switch to power your circuit. This way there will be no spikes.
You can add fuse if you like, but why would you need that complexity with constant current PS?