You can connect each power source to the load through a Schottky diode. Then the source with the highest voltage will power the load. The other diode will be reverse-biased, so no current will flow through it. This assumes the battery voltage is less than the USB 5V.
If you want to eliminate the diode voltage drop in the battery line, you can use a traditional load sharing circuit, which is one diode, a P-channel mosfet, and a resistor. The attached circuit shows a battery charger being used, but it should work the same if there's no charger. USB voltage, if present, turns off the mosfet, which isolates the battery. Otherwise, the gate pull-down resistor turns on the mosfet. The orientation of the mosfet is a bit unusual, depending on how you look at it.