I haven't done much Java, but I have used JSSC which worked quite well
https://github.com/scream3r/java-simple-serial-connector.
As for a direct USB interface, it's a great idea but I don't think any OS supports that directly. The idea of USB is that it is device oriented, i.e. mouse, keyboard, modem, disk etc. Having a plain old IO port to make it easy for developers seemed to escape the USB designers. The closest thing to a raw pipe is CDC serial, which is a subset of the whole comms device class, designed to include modems, fax machines etc, which were assumed to be the only things that would require COM port type functions.
If anyone has tried it, can libusb be used to create a cross platform "direct" comms? Or at least a uniform device interface.