I can see what you mean about the voltage dropping when the charger sees a full battery and cuts off.
No, it is probably never supposed to cut out. Just run at 14.5V. I think it could be overheating and cutting in/out due to internal protection.
The voltage there should not drop to below 12v with a good battery and very little power draw, should it?
Float voltage of a lead acid battery != load voltage. A 12V lead acid battery has a charge voltage of 13.8V, a float voltage of 12V, and a load voltage of 10.5V.
Charge a 12V battery to 13.8V, then disconnect it. The float votage will drop to 12.5V in a matter of minutes, doing nothing. And it will eventually settle at around 12V for a good while, until self discharge brings it lower.
Charge a 12V battery to 13.8V, then draw a significant load. The voltage will drop to AT LEAST something under 12V, immediately. Under a great enough load, voltage will drop to ~10.5V on a fully charged battery.
Unless your lights are all LED or flourescent, it is seeing a significant load. Jumping between 11.X and 14.X V is more or less what you would see if the regulator were cutting in/out on a fairly charged battery under load. Might jump down to 10V or so, if the battery is low or the load is high. This is what you'd expect, else I wouldn't have suggested it.
Voltage jumping between 11.9V and 14.5V, specifically, is more or less exactly what you would see with a good, fully charged battery, under a partularly light load, when the regulator cuts in/out. Which, oddly, enough, is exactly what you described as your situation.
I had similar symptoms with a vehicle. The battery was randomly dying. And so I measured the voltage like you did, and I found essentially the same results that you did. And then I hooked up a panel meter to the power rail and monitored it while operating said vehicle. And I found out the voltage would randomly switch between 14.0-14.7V and 10.0-11.5V, or so, the range of higher voltages dependent on current rpm. The range of lower voltages dependent on state of charge on the battery. Actually, this was not quite random. Usually, the regulator would cut out after the engine was hot and the vehicle was stopped and restarted. So drive to work and back, and you'd be fine. Stop for gas, and the vehicle would run off the battery the rest of the trip. If you made it home, the vehicle wouldn't start the next day. I googled and found out that said vehicle was known to have regulator failures, particularly in warm weather. I ordered a replacement regulator. In the interim, I placed a fan over the regulator that would run whenever the ignition was on. The panel meter showed 14.xV all the time, thereafter. I kept said vehicle for another 6 years, and it started every time. I never did change out the regulator. Still have it in my parts box.