I have 4000mah 3.7V battery and 4000mah means that it draws 4A for the whole time in one hour (https://www.ifihomes.com/blog/how-much-time-it-takes-to-charge-a-solar-power-bank/).
Is this means that it's output current is 4A? What is the meaning of draw here? Is this mean that 4A is the maximum current it holds?
How much voltage and wattage of solar cell need to charge this battery?
The 4000 mA-hr rating means it can deliver 4 amps for one hour or .4 amps for 10 hours. Note thare will be a maximum current limit on the battery.
You need a charger to control charge to the battery. This will avoid damage to the battery and potential fire.
The energy in the battery is roughly the amp-hr rating times the voltage. The voltage changes during discharge and goes down. So you need to integrate to get an accurate energy store value.
The rough numbers would be 4 amp-hr * 3.7 volts= 14.8 watt-hr.
The power depends on the current. So if you are discharging for 10 hours with a current of .4 amps the power is 0.4 amps * 3.7 volts=1.48 watts.
watt-hr or kW-hr is energy.
watt or kW is power.