Have you actually measured the solar cell output in actual open circuit and load conditions in your real-world usage setup?
The voltage and current out of the panel will vary a lot depending on whether it has a load, how much light it receives, the angle of the panel to the sun, and the temperature of the cells.
If you got an inexpensive system from a place like Harbor Freight, consider that they may have used the absolute best case spec and used some fuzzy math to rate the system; par for the course when marketing gets involved.
What you will actually get out of the system may be considerably less than you think. (Seriously, think 50%-80% less than the box says)
It appears that your pots are not connected correctly to the LEDs, unless you are just using them for their nominal value, in which case you'd be better off with standard resistors; much cheaper too.
Even if you are wiring them correctly and just drew them wrong in your schematic, it'd still be cheaper to use standard resistors, unless you have some special reason you need to vary the brightness of the LEDs. Don't know what that would be, perhaps you could fill us in?
You'll also need a resistor for the LED with the fans.
You must be using a seriously massive heat sink with big fans to dissipate that 140+W of power the 5V regulator will be generating at full load. A switching regulator would be much more efficient and generate almost no heat. Don't know how much power the 12V regulator is dissipating, but easy for you to calculate as you should have the specs on the fans.
A better way to go would be to use a 12V switching regulator to run the fans, and as a pre-reg for the 5V reg. You'll be dissipating about 50W that way, but that's still a lot of wasted power.
Might want to watch Dave's video on thermal design and figure out how hot that junction will be and if it's within the spec of the regulator. I'd bet not, at least in the current setup, but you haven't given enough detail to even get close to a WAG.