Possibly, what's happening is, without power, the synchronous transistor isn't holding, and the voltage gets unclamped and bad things happen. Probably a schottky diode from GND to VCC, and another from OUT to VCC, will help, but also a pair clamping the switching node, GND to SW, SW to VCC.
Short circuits in general are startlingly fast and violent transients, and getting them anywhere near a semiconducting device is sure to cause problems. It could be something as unexpected as sheer ground bounce, for all we know.
Tim