Of course.
But keep in mind that you can short everything else also, and those others may be much less tolerant.
An output of a single OpAmp can be one, but it can be internally protected also.
So better learn a habit that everything stay put when power is on.
USB is a hot plug type connector, it can be disconnected anytime.
If you think a Windows message telling that it's now safe to disconnect, it's different.
Old Windows versions used delayed write style, so it's a software based message.