Check your units!!!
Amperes are a unit of current flow. Ah are charge, namely 1 amp for 1 hour (3600 s).
A device that uses 500mA will use 500mAh of capacity after one hour, and so on.
A coulomb is an amp-second.
So what you need is 500mA * 2s = 1 C, and 1C/(5V-4V) = 1F.
If it's getting 5V from a device that won't backflow, you can just tack on the capacitor, and it should be alright. Well, the converter / charger might not be the happiest starting up into 1F... but that's a separate issue.
If it doesn't prevent backflow, then you'll need a diode, otherwise other loads on the accessory circuit will discharge it through the converter. Which sucks, because if the load needs its 5V, you don't have many options there; you may need an "ideal diode" circuit.
On the upside, 10F+ caps aren't terrifically bulky or expensive, so you can spare plenty of extra. And for those cold cranking mornings, if applicable.
Tim