The uln2003 is gonna have a significant voltage drop.
I also wouldn't tie the D+ and D- wires together. Consider analog mux chips or mechanical relays.
Mechanical relays would be easiest, as you'd have the lowest resistance, and they're cheap.. a 5v DPDT relay is 1$ and let's you switch the D+ and D- wires between 2 ports , so 3 relays would be enough for the data wires.
Here's an example of a DPDT relay :
https://www.digikey.com/en/products/detail/cit-relay-and-switch/J104D2C5VDC-20S/12503179See datasheet at page 2 for how the pins of relay are positioned.
The ground wire can be kept always wired, for the voltage wire you'd have to use either relay or have it hardcoded on the selector switch.
You could use a 4 position slide switch with 2 sets of contacts , a set could be used to route the voltage, the other could be used to turn on the specific relays
relay 1 - default off = port A , relay 1 - on = port B
relay 2 - default off = port C , relay 2 - on = port D
Relay 3 connects either relay 1 outputs or relay 2 outputs to computer port
relay 3 - default off = relay 1 , relay 3 - on = relay 2
So :
port A : relay 1 off, relay 2 off (don't care) , relay 3 off
port B : relay 1 on , relay 2 off (don't care) , relay 3 off
port C : relay 1 off (don't care) , relay 2 off , relay 3 on
port D : relay 1 off (don't care) , relay 2 on , relay 3 on
So worst case scenario, you're gonna 2 relays energized, consuming maybe 30-50mA from 5v.
The design isn't safe to switch between ports live, as there's a delay until a relay turns off and until the other relay turns on, so if you slide between positions it could be that data wires of two ports would be simultaneously connected.
But, you can simply add a on/off switch to disconnect the whole box from 5v input, wait a second or so, change the slider position, turn back on ... and you don't have to reboot the computer.
You'll need to add one diode to each relay for protection but that's easy, if you're a beginner you'll find tutorials on the internet about why that's needed..