Exactly, the time constant isn't there -- for nanoseconds to microseconds, you need ceramic caps; for microseconds to milliseconds, you need electrolytics (roughly including solid tantalum and aluminum polymer). Supercaps just barely begin to be useful in the milliseconds; they're best in the seconds range. Still further out, various types of cells serve seconds to minutes (the new lithium titanate nanoparticle stuff?), to minutes to hours (LiPo, ion, NiMH, etc.), to hours or days (lead acid, etc.).
Interesting to note, you're largely not storing energy here. I mean, you are, a
lot of it, but that's not
why you're doing it. It's not energy for the sake of energy: energy is only incidental to keeping the supply impedance low as hell.
If we had capacitors that have C ~ 0 for most of their voltage range, then C ~ nominal for voltages within say 10% of operating voltage -- we'd be glad as hell! Supply voltage can ramp up quickly from zero, because there's almost nothing on it, then in the last few percent, a lot of charging work is needed, storing only the energy over that range. Then under normal load fluctuations, only a percent of whatever of that is actually used -- as bypass gets better, voltage change gets smaller and energy exchange drops towards zero.
So, the fact that supply ripple is small, is proof that it's not about energy storage -- if it were, voltage would be allowed to change more, to make more efficient use of those capacitors.
As it turns out, such devices do in fact exist -- poled (electret) ceramics. Whereas an ordinary ceramic has maximum capacitance around zero bias, and less and less at elevated voltage, these have an electric field frozen in, so their maximum capacitance "zero bias" point falls at some offset instead!
I don't think this works at low voltages though; the example I'm aware of is designed for industrial application, 400V or thereabouts. (For sure, 0.7V is just about "zero bias" for most parts I've seen, no worries there.
)
Tim