I have done some research.
The NiMH is cheap £4.80 for the 4.8V/2550mAh pack. But it needs to be charged at C/10 (or less) to be allowed to over charge without damage. Still the charging has to stop at some point and that would require some components. Then, if power is lost, the battery has to be disconnected after say 2-3 mins of alarms buzzing, else it would go on for ever even when you do not want it, and would drain the battery when the device is off. More components required then,
The .47F supercap would provide 2T cycles of discharge to go from 99.33% charge to 95%, into a 30R load, in almost 3 minutes. According to my calculations that is. 3 minutes is long enough to sound the alarm buzzer. And we do not care if the supercap gets drained completely, it is not a NiMH battery, and it cannot overcharge!
However this super cap would be like a super-short and will probably cause all sorts of harm. So it would need to be charged through some limiting resistor/LM317 in CC, but for discharge it would need a diode, which would then drop 0.4V, so the MCU would get 4.5V or thereabouts, not 5V.
This is what I have come up with.