The half on the VBUS side is pretty obviously to prevent the 5V supply from leaking into the USB host which is a big no-no, the other would do the opposite but nothing the OP mentioned allows to find out whether that's a requirement or could be ommitted.
So OP... you're asking us if removing the diode is OK, but really the answer to that depends on the circuit that supplies the 5V, of which we know nothing.