The choice of energy storage component is driven by how long the energy needs to be stored for, how fast (peak power), and whatever's cheapest.
Viewed from this angle, it is clear that solar is best served by batteries. Batteries are the slowest electrical component, with time constants from seconds to hours or days. Solar needs to store energy for less than a day, with peak power equal to the installation size, which turns out to be ideal for most battery chemistries (including lead acid and lithium ion).
Supercaps are great at storing small amounts of energy for long periods of time ("CMOS backup"), or delivering high power over short periods of time (seconds). An example might be a boost circuit in an electric car, where energy is drawn from a slow battery, into a fast capacitor, then dumped into the motor and vehicle in a few seconds -- a race car might need to deliver a megawatt over a few seconds to accelerate competitively. But it might only need a fraction of that for cruise, which can be drawn from the battery while the supercaps are recharged for another boost.
But supercaps are also quite expensive and bulky. It very often happens that, it's cheaper or more compact to use enough batteries to handle the peak power, which gets extra charge capacity as a byproduct. Or conversely, if you need the extra charge, you get the power for free -- which is how Tesla's cars are so powerful. They don't need to be, but they have the electrical power available, and it's not much cost (considering it's already a luxury vehicle) to connect that power to the wheels, so... they do!
This back-and-forth continues onto smaller time scales as well, where we go from supercaps to electrolytic caps, to aluminum-polymer and film capacitors, to ceramics and transmission lines. Switching power supplies, for example, have large power demands on the microsecond time scale, but it's more economical to use relatively oversized electrolytic capacitors to satisfy that demand, rather than film caps which are more bulky and costly. Though the balance changes a bit at high voltages, where film caps are actually less bulky for the same power or current rating (or electrolytics aren't available at all!).
On the smallest time scales, we have transmission lines -- storing energy, nanoseconds at a time, as wave energy propagating near the speed of light. The most notable case is the "Z Machine", which uses an extremely pure water dielectric to deliver petawatts in about a hundred nanoseconds!
There can also be economies of scale. Thermal plants (solar thermal, usually) store heat energy in enormous tanks of molten salt: the high melting point, and relatively high specific heat, keeps the process temperature up, even after the sun sets, allowing a steam turbine (usually) to operate reasonably efficiently around the clock. Pumped fluid systems include water going up and down from a mountain lake, or air pumped into an underground chamber. These have their losses, but apparently not so bad they can't be used for day-to-day demands, as results from solar and wind sources.
Electrical storage is actually one of the least efficient possible, in terms of space -- you can only make a capacitor as small as a few atoms, and you can only put as much energy into those atoms as they'll store before ionizing. Ionization is chemistry, and then you have a battery, see?

Tim