Hm, I think nobody understood my question here.
Perhaps because you seem to have two different questions here, and based on a misunderstanding.
So, when I used the word "ground" I was thinking of the ground connection found on the power cable that goes between the unit and the wall socket that is linked to ones fuse box. I was not referring to the third output on the power supply, that is labeled "ground".
The "third output" on the power supply (typically a green color connector labeled "Ground" is actually CONNECTED to the ground connection on the power cord -> wall socket -> fuse box. That terminal is there for you convenience. Normally no current flows through that connection whether the circuit is powered or not, whether the power supply is turned on or off.
I was wondering if a power supply that ordinarily would be powering a circuit board if the circuit board was connected to the power supply, but if "turning on the power" without having the power supply connected to anything to be powered, I was wondering if electricity is going places inside the power supply (maybe going back to ground/earth), or, if a power supply then is simply understood as being idle where the + and - output connectors aren't touching anything.
It is not "going places" any more than your utility mains power is "going places" when you turn out the light in your room. If you open the switch, the voltage is still there, but with no load, no current will flow.
I have sometimes heard/read that "earth" is a "reference", but what does that mean? I've always thought that ground, being real ground would just suck up and lead away any current that would otherwise be harmful if not shunted off to a ground wire going down into the soil. Presumably, the earth wire would handle a maximum amount of current before disintegrating due to the heat created.
Ground is used for many purposes in electricity, electronics, RF, and other fields. In electricity, its primary use is a common "reference" so that "ground" at your house is essentially the same as at the far end of the road (or way back at the generating plant). It is used as a safety circuit that conducts fault currents away from YOU so that you are not electrocuted. In this sense, the safety ground (green/yellow) wire in the power cord under normal conditions conducts NO CURRENT AT ALL. It is only there as a "safety feature" so that if there is a problem inside an electrical appliance (such as your bench power supply, etc.), any fault current will go to the green-wire safety ground and not through your body.
In electronics, we use the word "ground" most often to refer to the designated reference point (0 volts) in a circuit. Regardless of whether it is connected to the crust of the planet, or whether it is in a quad-copter drone flying 100s of m above the crust of the planet. The green terminal on a bench supply is provided as a convenience. If you wish to connect your circuit to the crust of the earth, it is a simple matter of a jumper wire over to that green terminal. But, if designed properly no significant current will be flowing through that connection.
If RF, especially for low frequencies, the crust of the earth is used as a "counterpoise", or essentially the "return path" of the RF signal or the other-half of a dipole antenna, etc. This is an extremely crude explanation that would be widely criticized over in the RF forum. LF and MW transmitter towers frequently have an array of wires buried in the soil around the antenna which form the counterpoise to the tower part that sticks up into the air.
Ref:
https://en.wikipedia.org/wiki/Counterpoise_(ground_system)