My battery also was a bloated mess, finally had to fix it. I actually managed to get away with replacing the battery with a capacitor, since I were too lazy to solder wires to the the PCB again (already got the 5V USB OTG to USB-Power wire mod installed). Only downside is, that you have to wait a few seconds till the camera "failed" to turn on automatically, because it sees a dead battery initially. Then you can actually turn on the camera normally, and it will see a 100% charged battery. So I guess the step down is technically still the superior solution, since apparently it turns on automatically right away? For someone feeling lazy this might be the easier solution, like if you already connected the power from the OTG port, or just want to power the camera externally and are too scared to solder on the main PCB.
I tested a 10V 220uF, which worked fine, but in the end used a 6.3V 1000uF cap I had laying around from salvaging.
PS: Ofc insulated the battery protection PCB and everything in (kapton) tape, before closing everything up.