Is there any reason to not use bypass caps?

[Dago]

Doesn't this has to do more with your voltage regulation?

If you can hit the core voltage then do you need bypass caps?

Not really no. With fast transients (such as caused by lets say digital edge transitions which can have a bandwidth of up to several GHz) the decoupling cap supplies all the energy because the inductance to the regulator is humongous. Switching regulators usually have a bandwidth of a few MHz.