Yes, definitely 18650 Li-ion. But I guess the replacement batt will end up the same as it appears the charging "circuit" is relying on the over charge cutoff of the protection PCB (cuts off at 4.5V or higher) rather than terminate at 4.2V as proper chargers do.
I would add a shunt regulator IC / circuit to burn off the excess voltage. A TL431 and a transistor could work.