Ceramics and Tants are fast and best used to suppress spikes and therefore commonly used for Decoupling needs.
When you say fast, the "speed" of a capacitor is a function of its capacitance right? Or is there something else going on?
The "speed" is essentially the time constant C * ESR (or the cutoff frequency proportional to 1/x of this). It's largely a matter of technology and construction, not capacitance (you can make a bigger capacitor that should be slower, but it's going to have lower ESR, so it stays the same speed).
The time constant of batteries is in the seconds to hours range; supercapacitors, seconds to fractions of a second; electrolytics, milliseconds to microseconds; and most others (ceramic, film, etc.) microseconds and below.
This determines the range of frequencies these components are best suited to bypassing. Lower frequencies (longer times) are most efficient for a given type, while moderate frequencies can be filtered effectively, if not necessarily with low losses. Example: you might use an electrolytic to bypass some digital logic, where the frequencies are high but the average current is low; but not for a switching regulator, where both frequency and current are high, and power dissipation would be excessive. Sometimes, the loss is a helpful addition; an excellent use of electrolytic or tantalum capacitors is in parallel with smaller ceramic or film capacitors, which have quite low ESR which can resonate with circuit inductances.
It's noteworthy that the time constant of tantalum capacitors is not much less than most electrolytic capacitors. But, the response is far more stable, and much simpler. Electrolytics have all sorts of transmission line and ionic diffusion effects (a result of their wound construction and electrolytic nature), and ESR is vastly higher at low temperatures; the impedance measured in circuit (C, ESR and ESL) vary in all sorts of subtle ways. Tantalums are basically ESR (with a small tempco), with ESL limited to that due to lead length and basically nothing else. (The ESR is due to the "electrolyte", a conductive solid conformed around the tantalum pellet. It's more conductive than electrolyte, and the conductive path is direct.)
Tim