I live not quite in the middle of nowhere (you can see there from here), and I too deal with unreliable power.
Not only does it fail several times a year (some years "many"), but it often "flails around" as it fails or comes back, resulting in brownouts and short on/off cycles.
At one time I had many small UPS, but I found that keeping them tested and in working batteries was a lot of bother and I was often less reliable about this than the power.
What I finally settled on:
- Everything I care about is on a good quality surge suppressor.
- For things that are OK with a sudden/unexpected but clean power-off, I built little "power controllers" which function like a self-hold relay.
When the power goes OFF, it stays OFF until I deal with it.
- For things that don't want to be suddenly powered OFF, but are normally off when I'm not present, I use standard "desktop" UPSs, so I can just shut down when power-outage.
- In my equipment room, where my server and several other things that are on all the time and/or must shut down properly reside:
I have a big APC UPS running off a lawn tractor battery. Lasts longer and cheaper to replace than the gell-cells, but you need to tweak the charging circuit and have external venting.
I also added forced air cooling when running on battery and put a 20A circuit breaker in the battery line (this will depend on how much power your equipment draws).
To manage it I built a custom STM32 based control system:
- Monitors UPS to determine when on-battery.
- Controls the UPS shutdown signal.
- Has large solid-state relay to defer returning power to the equipment.
- Monitors the 5V rail of the server to determine when it is ON.
- Modified server adding opto-isolator and connector to allow STM32 to "push" the power-button. This is set up to initiate an orderly shutdown of the server.
When power goes out:
- If outage is <5mins, STM32 does nothing USP keeps everything running.
- If outage is >5mins, STM32 shuts down server, and once it has powered OFF, shuts down the UPS (killing it's own power)
This also "unplugs" the other network switches, bridges, interfaces etc. in the equipment room.
When power returns:
- USP comes back online automatically.
- solid-state relay remains OFF, blocking power to all but the STM32
- Once power has been stable for 30 mins, STM32 enables power to all equipment, then power-on the server.
- If power fails before 30 mins, shutdown UPS and wait for it to come back on again.
- If power fails after 30 mins, go back to "When power goes out".
This has worked really well, and I no longer have to worry about the server & equipment when I am away from the premises. I've not had a single problem with the full-time equipment since.
Dave