We are moving to a very standardized setup so that Dave can have someone else on standby should the worst happen to me. The configuration will consist of three servers.
- Stock Standard cPanel Server - For Dave's Convenience, General Hosting, etc.
- Two servers configured with a HTTP stack.
The two servers will very likely be configured using the following (note, this is a quick list off the top of my head at this time):
- Nginx HTTP server + PHP
- MariaDB using Galera to keep the two servers in sync, cPanel server will likely be used as the Arbiter
- Since very little outside of the database changes on disk, `unison` will likely be used to sync the HTTP directories, this simplifies the sync and allows each host to operate independently should the other die, unlike configurations with NFS deployments.
- The two servers will be active/active, keepalived will likely be used to handle automatic failover between the two hosts.
- Server configuration will likely be deployed via Puppet or SaltStack, I have not yet decided. This if done properly will ensure configuration is consistent across both nodes, as well as future proofing for possible growth
This is a pretty common way to configure things, making it possible for any competent admin with fail over hosting experience capable of managing the servers without too much trouble.
This all sounds very good. I was originally worried with the idea that Dave would end up with custom tweaked servers that are only understood by one person, but if you are going to use a tools like Puppet or SaltStack for the configuration, it should be possible to have a documented procedure for quickly rebuilding servers from scratch.
You could make it even easier by using containers such as Docker, but it is one more thing you have to keep going. Containers would probably separate the http/nginx stack, Simple Machines and the database into separate containers with the actual data perhaps with its own location. It does make it extremely easy to backup the actual running containers and to make it trivial to get it running on new PCs/VMs - even if it is just to have a debugging on a home/office PC that is running exactly the same containers as the Xeon servers.
It could be that Puppet effectively provides a similar speed of recovery and flexability. The actual size of the EEVblog customisation could easily be pretty small, and so the cost of rebuilding may not be significant. I am pretty ignorant of Puppet and SaltStack so I am not sure how over time, you ensure that they can rebuild the server with 100% of the current mods. If Gnif is not there, will rebuilding a server be a task that for the new admin person is a hope-it-works-or-else leap in the dark?
Is MariaDB a bottlekneck at all? If it is handling the load without a problem, then there is probably no need to touch it.
One of the fastest and most stable relational databases for large data is, believe it or not, SQlite3 - provided you have lots of RAM. There is a project
http://bedrockdb.com that makes SQlite3 a replicated and geo-redundant dB. It provides MySQL compatibility if required. Bedrock does provide features like automatic fallover to a slave, and automatic recovery back to the master. If DB issues are a bit of a headache, Bedrock is worth a look.
https://twit.tv/shows/floss-weekly/episodes/456As a matter of interest, how big is the DB and the associated attached files?
Overall though, this is a great move. I think it is time to move to dedicated servers. The redundant server plan sounds great. If there are a few hiccups along the way, so be it.