The buzzing is probably due to electrostriction of the dielectric, i.e. the capacitor is physically expanding (straining) due to applied voltage. I'm not sure why it's impulsive (hence the maybe perceptibly rapid-fire-pulses sound), maybe something to do with the dielectric popping into position due to voids in the construction, or something about the material.
There's also some noise due to the magnetics (the analogous magnetostriction effect), usually moreso than the capacitors, but apparently not so much in this case. Give or take if the PFC even behaves with those caps removed (eh, it probably still does, there's still the gray caps further upstream of the rectifier).
The startup sound is less frequent, sounds like it's having a couple of startup attempts before kicking on fully. This is normal behavior, or if not intended then at least not bad behavior. Supplies are usually made with a "hiccup" characteristic, implemented by powering it by itself -- if the output is overloaded (which is the case, momentarily, during startup as the big capacitors all charge up), the controller doesn't get enough power to keep running so it goes into shutdown, and waits for a new cycle to begin (timed by the large value bleeder resistor from the mains DC bus).
The sound may also change with a load applied versus idling. If it's just idling, I'd expect the sound to be more of a variable whine, due to the output being switched on only every once in a while to keep its voltage up. The sound being from magnetics (or a few likely ceramic caps in the path). But eh, apparently you have noisy film caps this time!
Tim