Decoupling capacitors are usually ceramic ones, in the 0.01 uF to 0.1uF ... With ceramic capacitors it's sometimes important to be aware of voltage bias , see :
https://www.maximintegrated.com/en/app-notes/index.mvp/id/5527 Probably wouldn't matter for a microcontroller, but for other devices, maybe you'd want to use 25v or higher rated ceramic capacitors.
If you mean some bulk electrolytic capacitors like let's say 100-470uF or something like that, then yeah, 6.3v should be the minimum. Note though that often 10v rated capacitors of such low values may have the same diameter and sometimes can be cheaper than 6.3v rated capacitors.
And for lower capacitances like 100uF or lower, voltage ratings like 25v and 35v are common, because you get around the same diameter and height regardless of voltage so makes no sense restricting yourself to 6.3v or 10v rating.