Resistance wire (very fine) and a firecracker (if they're available in your area, anyway..).
Electrolytics aren't rated for operation at less than -3V, or much more positive than their rating. But that's hardly a guarantee for failure; 6.3V caps can operate at -5V (5V supply, installed backwards) for months before failure.
Tantalums are likely to fail at -3V, but I wouldn't really recommend one for a prank, they're a bit lively for that. Well, so is the firecracker, but that's a limited pop and little or no fire, which a mouse should contain quite well.
Don't suppose you can find a very small smoke / fog generator that's USB powered... actually... I'm sure such a useless novelty already exists... You wouldn't get the suspicious crackle-pop sound and burnt electronics smell though, at least not without doping the fog juice with phenol and depolymerized bisphenol derivatives.
Tim