Capacitors are weird. I played with a few of different, mostly film types (in 1uF and up range) when learning to use my SR560. If I remember correctly, quietest were PP (didn't have a large polystyrene at the time). PPS were next. Acrylics, regular electrolytics and tantalums were fairly noisy and, let's say, they "behaved". (talking .1-10Hz here). Also, noise appeared to be proportional to voltage. And all that with me being careful about temperature - caps have fairly bad tempco: change in capacity, by definition, with constant charge, causes proportional (to change in capacity) change in voltage on terminals.
It seems that my experience is in agreement with Jim Williams's (AN 124). He had to resort to using crazy expensive (to this day) wet tantalum...