but if the device is powered directly off the battery this will vary from around 4V fully charged to lower voltages. So would it be reasonable to expect a memory chip that has the built in USB interface to work?
Would it be reasonable to expect any device to work out of spec?
Whereas we can reasonably expect all of these to be internally powered at 3.3V, given their size and cost, it's very likely they use a linear regulator. Given again the cost factor, it's also very likely they include lousy LDOs that are not that low drop-out at all. A very common "LDO" that we can see in many cheap devices is the ubiquitous AMS1117 which is dead cheap, for instance. It has over 1V of drop-out voltage, so given that for most of the useful life of a LiPo battery, the voltage is around 3.7V-3.8V, you'd end up with 2.7V or less. So not only may some ICs misbehave, but one of the most problematic point I see would be a significant mismatch in the logic 'high' voltage between the host and the memory stick, potentially causing problems with the USB data lines.
The power supply stage of those memory sticks is likely to be pretty different from one device to another.
Powering USB devices directly from a LiPo battery looks like a pretty lousy design here IMO.