Oh dear god, did i read that right and the site uses hostgator? /puke
Really though this site seems big enough to get hosting donations and use a modern day dual data center high availability setup either active/active or active/passive... or at least a T1 DC Host
We are very likely moving to a active/active setup, waiting on final decisions from Dave
How about the happy medium? Monitoring on the server. (Why doesn't your current hosting provider do this, tho?) (Sorry, gnif) Being able to get in and do your thing is good. Having a single point of failure is Very Bad - what if gnif gets hit by a bus after you move?
Sent from my Nexus 6P using Tapatalk
Can I make a request for the big move? Can we please at long last at least test transitioning the DB to UTF-8 so posts can stop being mangled?
The new configuration will give us room to wriggle and test stuff like this,
but it wont happen until after this work is completed.
How about the happy medium? Monitoring on the server.
Aren't there decent hosting server companies here in Australia with decent kit?
Why be at the mercy of a server overseas that can go suss or belly up any time,
or do 'changes' without notice or apology,
and or no one picks up the phone or replies to emails,
and when they do reply it's more or less a "have a nice day" placebo support...
The U.S. is a politically unstable place at the best of times, why have your bread and butter 'over there' instead of here just in case shtf?
No offence meant to Americans, but would they have their blog server parked in Australia? I don't think so...
Keep it local
We are moving to a very standardized setup so that Dave can have someone else on standby should the worst happen to me. The configuration will consist of three servers.
- Stock Standard cPanel Server - For Dave's Convenience, General Hosting, etc.
- Two servers configured with a HTTP stack.
The two servers will very likely be configured using the following (note, this is a quick list off the top of my head at this time):
- Nginx HTTP server + PHP
- MariaDB using Galera to keep the two servers in sync, cPanel server will likely be used as the Arbiter
- Since very little outside of the database changes on disk, `unison` will likely be used to sync the HTTP directories, this simplifies the sync and allows each host to operate independently should the other die, unlike configurations with NFS deployments.
- The two servers will be active/active, keepalived will likely be used to handle automatic failover between the two hosts.
- Server configuration will likely be deployed via Puppet or SaltStack, I have not yet decided. This if done properly will ensure configuration is consistent across both nodes, as well as future proofing for possible growth
This is a pretty common way to configure things, making it possible for any competent admin with fail over hosting experience capable of managing the servers without too much trouble.
This all sounds very good. I was originally worried with the idea that Dave would end up with custom tweaked servers that are only understood by one person, but if you are going to use a tools like Puppet or SaltStack for the configuration, it should be possible to have a documented procedure for quickly rebuilding servers from scratch.
You could make it even easier by using containers such as Docker, but it is one more thing you have to keep going. Containers would probably separate the http/nginx stack, Simple Machines and the database into separate containers with the actual data perhaps with its own location. It does make it extremely easy to backup the actual running containers and to make it trivial to get it running on new PCs/VMs - even if it is just to have a debugging on a home/office PC that is running exactly the same containers as the Xeon servers.
It could be that Puppet effectively provides a similar speed of recovery and flexability. The actual size of the EEVblog customisation could easily be pretty small, and so the cost of rebuilding may not be significant. I am pretty ignorant of Puppet and SaltStack so I am not sure how over time, you ensure that they can rebuild the server with 100% of the current mods. If Gnif is not there, will rebuilding a server be a task that for the new admin person is a hope-it-works-or-else leap in the dark?
Is MariaDB a bottlekneck at all? If it is handling the load without a problem, then there is probably no need to touch it.
One of the fastest and most stable relational databases for large data is, believe it or not, SQlite3 - provided you have lots of RAM. There is a project http://bedrockdb.com that makes SQlite3 a replicated and geo-redundant dB. It provides MySQL compatibility if required. Bedrock does provide features like automatic fallover to a slave, and automatic recovery back to the master. If DB issues are a bit of a headache, Bedrock is worth a look.
As a matter of interest, how big is the DB and the associated attached files?
If Gnif is not there, will rebuilding a server be a task that for the new admin person is a hope-it-works-or-else leap in the dark?
A server once bootstrapped can be deployed with puppet in a matter of minutes in a completely automated manner, which is the entire point .
Is MariaDB a bottlekneck at all? If it is handling the load without a problem, then there is probably no need to touch it.
The only changes that will be made here is the replication configuration so we can load balance reads between the servers. SMF was not designed with multiple masters in mind, ie insert ID problems, so writes will have to be directed to a single server, but this will be done with keepalived, not the database.One of the fastest and most stable relational databases for large data is, believe it or not, SQlite3 - provided you have lots of RAM. There is a project http://bedrockdb.com that makes SQlite3 a replicated and geo-redundant dB. It provides MySQL compatibility if required. Bedrock does provide features like automatic fallover to a slave, and automatic recovery back to the master. If DB issues are a bit of a headache, Bedrock is worth a look.
Changing databases is not an option, we require the feature set of MySQL.As a matter of interest, how big is the DB and the associated attached files?On disk the database is currently consuming 10GB, the fileset well over 100GB, but very little data on disk outside of the database changes each day.
If in the course of keeping the server running, you have to update anything, change the settings, add modules or whatever, can you do it first in Puppet and then let Puppet make the changes, or do you have to keep the server and Puppet modified in parallel?
Bedrock is MySQL compatible, but I do understand - you have it running on MySQL/MariaDB and so life is easier if you do not change it. It is hard to be 100% compatible.
10GB is not too bad. I was only mentioning Bedrock in case you needed a big speed boost. Bedrock/SQLite3 can hold the whole 10G database in RAM while also writing it to the drives, so it can be exceptionally fast.
I am sure the daily DB changes probably can be synced in seconds, so as long as running Simple Machines off a synced database does not cause stability problems, I do like the plan.
May be upgrade and allow more MB and freedom in posting/attaching pictures here?
Everytime I need to attach a picture here my eyes are rolling up in the sky.
Last night I had a dream:QuoteMay be upgrade and allow more MB and freedom in posting/attaching pictures here?
Everytime I need to attach a picture here my eyes are rolling up in the sky.
Just on the use of Unison, does that mean if one server gets the website hacked, then the hacks are duplicated to the other server?
how much more complicated / expensive / etc would it be if dave buys his own hardware to put inside his house to function as a server, assuming there is a low cost way?
and would that possibly be the best way to remove all server issues?
..., even something as simple as a rodent/marsupial chewing into some cabling, you quickly realize how frail your connection actually is.
Last night I had a dream:QuoteMay be upgrade and allow more MB and freedom in posting/attaching pictures here?
Everytime I need to attach a picture here my eyes are rolling up in the sky.
Last night I had a dream:QuoteMay be upgrade and allow more MB and freedom in posting/attaching pictures here?
Everytime I need to attach a picture here my eyes are rolling up in the sky.
Really bad idea for lots of reasons. Not all the planet has uber broadband. Loading threads with off site images is bad enough now on DSL2.
There are several stable SMF mods that can auto strip images down to size but you can find yourself in a position with a mod (generally built by one person) that doesn't keep compatibility with a patch or major upgrade of the core software.