Besides all the good comments so far, using a switch like that can lead to two problems
1) If you're switching with full load current, you'll arc, either damaging the switch if not properly rated, or creating havoc for the load
2) Cutting the input to a 7805 regulator could run the risk of Vout > Vin due to the load's input capacitance.
I've always been told that having Vout > Vin on a 78-style regulator is bad for the regulator, maybe even destroying it.
Different datasheets say anything from nothing to outright warning against it.
Since the cure is a reverse diode across the Vin and Vout terminals, I always put one.
and
3) Also, there is no adj pin on a 7805, it's gnd.
I don't know what " a seven segment display, controlled through a diode matrix," means. Like the old frequency displays on desktop PC cases? If all you want is a more or less accurate display of voltage, get a nice ebay voltmeter module.
I whacked one into my super-old college-project power supply.
http://www.ebay.ca/itm/0-99-9V-Panel-Meter-Digital-Voltmeter-RED-LED-DC-One-Decimal-Digit-Display-/330813058644?pt=LH_DefaultDomain_0&hash=item4d05feae54