If you are driving it from an USB connection, so I suppose it's in a PC? (explains also the 500mA)
BUT... If you know a computer, it's all driven by an ATX power supply like an example below:
It has standard connections, Just buy a splitter, so you can split the power out of it (most likely one for DVD driver or something like that) so you can get a power line with 5V power (red color) without a problem...
So then you can easy use 5V without blowing your USB port by accident...
Just a quick... it's the same power as the one comming out of the USB port...
On topic: If you want to power something, you need to know if it's a power supply that can source only or can sink current on it's output... The power supply in a computer can't do that (only source). If one of the 2 power supplies can sink current, then you can have a problem, becose when there is a different in potential, it will sink the current to maintain it's voltage level...
Example: the usb give 5.02V and your other supply give 4.98V... If your other supply can sink current, current will flow out of the USB driver, into your other supply... Not a huge problem, your devices will still be powered, but you are only able to drive 500mA out of your UBS port, so that may cause for problems...
2 Solution:
(...)
If you do not want that then you could add a diode to stop current flowing from A to B.
Or disconnect the 5V lines of A and B and only have them share the ground.