An alternative is an IDE tray / hot swap bay of some kind
Hot-swapping IDE devices requires support from the IDE controller; and those are really rare.
Hot-swapping SATA devices also requires some support from the controller/motherboard, but that is fortunately common; check the motherboard manual. Often there is at least one port with an eSATA connector (or a cable from the motherboard SATA connector with an eSATA female at the other end) with hot-swap support. Sometimes most or all SATA motherboard connectors support hot-swap, although there may be some limitations.
USB-IDE adapters can be hot-swapped with the same limitation: you'll want to park the drive before you power it down, and power up the controller/bridge before you power up the drive. Many docks do this internally, so all you need to worry about is turning the drive off in software before powering down the dock.
(I use USB/eSATA docks constantly; currently, most used one is an old USB 2.0 Akasa DuoDock, that supports both 2.5" and 3.5" SATA drives. I use a two-inch thick soft foam pad underneath for acoustic isolation, as my old 3.5" drives are noisy.)