Three terminal voltage regulators like the 78xx have three terminals to be connected. The input terminal is connected to the supply voltage, the common terminal is connected to the common ground or reference rail, and the output terminal with the regulated voltage is connected to the load. Three terminals is the minimum number to achieve voltage regulation and the device will not work unless all terminals are connected as per the data sheet.
On switches: As far as interrupting the current is concerned a switch will work just as well if placed either in the out or the return wire, however it will do a better job of isolating active voltages if it is placed in the live wire. If the switch is placed in the return wire then the source voltage will still reach the load and this could in some circumstances be a source of danger to components or humans. It is safer to place the switch where it both isolates the voltage and interrupt the current.