If you find that the charger is powering itself from the load then there's probably a big reverse protection diode across the regulator which is a good thing.
The sparks on inserting the banana clip should have raised the red flag to me that there was current flowing, but I thought, maybe I'm just charging the output capacitor. Then the thing powered up and I pulled the + banana quickly.
The Tenma throws no sparks, it just registers the battery voltage.
So my take away is.
1. Only charge batteries with the PSU if I really need the extra kick (or accuracy) from them.
2. Power the power supply first.
3. Turn the load on at a sensibly high voltage and sensibly low current.
Then connect the battery and adjust the current up. Hopefully preventing any back feed. Except of course if there is a power failure. Then the PSUs are on their own. The DPS5020 a new board is about £30, the Tenma would be a completely replacement at £100 ish. Of course worst case is fire and death LOL