The battery charging circuit most probably measures the voltage of the battery to determine a fully charged state. A fully charged Li-ion battery will never reach 5V, so the circuitry might not be able to withstand that.
To drop the 5V down to 4. something Volts, you could connect one or two diodes in series or hook up an LDO regulator set at 4.4V or thereabouts.