i am not sure if this is a just for you or a commercial product, if just for you, then, it doesnt even need to be isolated on the product if your external supply is isolated,
my point was, if your output "ground" is not at the same ground as your input you have a problem,
if you are aware of this then your not too likely to go and blow the thing up,
or you design a physical switch for the output that disconnects the ground aswell so your safe as long as the output is off,
it just comes down to what range of senarios you want to cover, something i have been beating myself up about on my own higher power varient,