I would in the ideal case power the fans with th Peltier Element but I don't know if I will get enough energy out.
If you plan on using the peltier on the heatsink to power it's own fans to cool down the heatsink while having surplus power for a USB charger, you are barking up the wrong tree as you are trying to make a system that is more than 100% efficient with a petlier device that is typically as poor as 10% efficient or less if you have a cheap ebay one.
A peltier must have a lot of delta T to produce any meaningful power, which means an extremely hot surface on one side and a humongous heatsink on the other, note that if it is being used for generation you are effectively insulating the hot side by using this setup.
If you wanted to produce a proper USB charger out of a peltier you would probably start by deciding what you are attaching it to and having a reasonable datasheet for the peltier (if possible). When you know what kinds of hot temperatures you expect, you can work out your heatsink properties (which should not be using fans as you lose your small margin of efficiency straight away). With that known you can work out your expected delta T and use that information along with your datasheet or other typical datasheets to determine what kinds of voltages and current you will expect.
Once you have that I would recommend doing something like a buck-boost regulator that has 90% or better efficiency so that you can squeeze every last bit of the little amount of power you are generating. You will probably need an undervoltage lockout as well. As previously stated you will need to pay extra attention to your input ripple on your buck-boost otherwise you will destroy your peltier after a few hours\days.
This is not the kind of project you can just throw together if you wanted to have a useful charger. If you are going for just a learning exercise then do it anyway but don't expect to have something thats very useful and don't expect your peltier to last very long if you go for a buck-boost approach without lots of design work for your input ripple filtering