I would prefer MicroSD card in SPI mode.
(+ It's smaller than ordinary USB pen drive.)
That could be a disadvantage in some situations - much easier to lose!
SD/MicroSD is certainly the simplest way to get from PC-style media to a MCU, but SD card readers are slightly less ubiquitous than USB drives. Using SD cards in SPI mode can also be rather slow, especially on newer, larger cards, particularly for writes.
Some PIC24 and PIC32s have USB OTG, with examples for talking to USB memory. The smallest option is a 28 pin ( DIP/SO/SSOP or QFN) If you want to keep the USB stuff seperate you could use a USB OTC MCU as USB host, with a simple UART interface to your device.
As USB support is typically via manufacturer-supplied frameworks, there may be good reasons to "contaminate" your code with them, or be forced to use them to get the USB part working.