Thanks for the nods-of-approval for the 1uF MLCC, I'll switch to those. I'll also throw some across the LED's 100uF tantulum caps.
Across the LEDs? I hope that's a typo
.
Tantalum is a bit iffy, but USB is kinda current limited with some source resistance usually there limiting inrush so it's not a huge issue. Still, remember to derate voltage, 10V or 16V part is needed for 5V bus, and look up the inrush current specification and max ripple current from the datasheet.
USB spec limits allowable bus capacitance (to limit inrush duration), if I recall correctly that's some 47µF, so you need to add a precharge circuit to be able to use more. It's getting some sort of PITA.
If you don't have precharge, you can't have much capacitance, and if you don't have much capacitance, the USB sees all those 1A peaks, in which case you would need to enumerate and ask for more current, or hope everything's OK.
Its disappointing that it's not possible to calculate this analytically.
Of course you can, but the equations get really complex when you include "everything", including cable ESR, ESL, capacitor C, ESR, ESL, and possibly C, ESR and ESL of another set of (different type of) capacitors. Spice handles coming up with all the equations and solving them for you. Tools are there to help.