A good rule of thumb is to keep ripple under 20% or so for electrolytics at mains frequency or 2nd harmonic. A small cap like that (say 100 or even 47uF) might not even get very warm, but it won't have a long life regardless.
220 or 470uF sounds fine for that, yes.
And yeah, the switcher doesn't care, and except for hold-up near zero crossing, you could just about get away with nothing. If you wanted a higher power factor and longer life, maybe use ceramic or film capacitors -- the ripple will probably make audible buzz from ceramics, but won't otherwise do anything beyond normal aging, and film are fine. The big downside is the high cost of ceramics that size (10s of uF at 50V will be a half dozen or so 1210 chips), or the bulk -- and cost -- of films that size.
Aluminum polymers are, in many ways, the dense, polarized equivalent of film caps at low voltages; but they're also not great at high ripple, I think. Better than electrolytic, certainly. A 50V 47uF part probably would last quite a while here.
Better question: why not rectify mains instead, i.e. use an offline power supply in the first place? 3.3 and 5V outputs are cheap and plentiful. Maybe not with great noise performance, but also not hard to filter.
Tim