eevblog back up again

[miguelvp]

Wow, that was a long FUBAR from HostGator!
Even my HostMonster sites went down but not for that long but I don't have dedicated servers!

[tautech]

Glad I didn't miss much as a storm put our power down for 10 hours in NZ

[EEVblog]

Wow, that was a long FUBAR from HostGator!

That's a rare Cluster FUBAR I think 

[BravoV]

They should rename the company's name to HungLowGator or something sounds similar.

[rdl]

I have to say, I absolutely hate that cutesy Cloudflare error page. As if I didn't know. Just gtfoomf. seriously, "cloud"?

[SeanB]

I just went to bed, then in the morning went to work. ah well, it was a busy day today, nibbled by rats and mice things.

[Monkeh]

That error page is the only thing CloudFlare contributes, too. Useful.

[amyk]

I have to say, I absolutely hate that cutesy Cloudflare error page. As if I didn't know. Just gtfoomf. seriously, "cloud"?

It's far more informative than most other "cutesy" error pages I've seen.

[rdl]

Maybe. My web browser is pretty capable on its own. I know it can tell when a connection has timed out. That cloud page is intrusive and annoying, it's more like they are bragging how their service is working fine and pointing fingers to place blame elsewhere. It adds nothing.

[Richard Crowley]

Browsers are more and more "dumbing down" their messages to make them attractive to low-experience users. I agree with amyk that it is a better error report than most we see.

I think most of you are completely missing the point of what CloudFlare does.  When it is doing its job (making an "ordinary" website be able to handle massive hits by various methods including cacheing, etc.) you never see it and you think you are talking directly to the target website.  Criticizing things we don't understand isn't a very positive indicator of intelligence and maturity.

[zapta]

Maybe. My web browser is pretty capable on its own. I know it can tell when a connection has timed out. That cloud page is intrusive and annoying, it's more like they are bragging how their service is working fine and pointing fingers to place blame elsewhere. It adds nothing.

The connection from the browser to the San Jose site was just fine. The problem was in serving that request, hence the internal error page. More useful than just throwing an Internal Server Error / 500.

[Richard Crowley]

So let's hear reports of where CloudFlare was reporting as a location. This will demonstrate how geographically diverse the service is.

Zapata says that CloudFlare was reporting from San Jose.
Since I am in the Pacific Northwest of the US, I was seeing the CloudFlare server in Seattle.

[ve7xen]

So let's hear reports of where CloudFlare was reporting as a location. This will demonstrate how geographically diverse the service is.

Zapata says that CloudFlare was reporting from San Jose.
Since I am in the Pacific Northwest of the US, I was seeing the CloudFlare server in Seattle.

There's a list of CloudFlare POPs at their status page: http://www.cloudflare.com/system-status

Maybe. My web browser is pretty capable on its own. I know it can tell when a connection has timed out. That cloud page is intrusive and annoying, it's more like they are bragging how their service is working fine and pointing fingers to place blame elsewhere. It adds nothing.

Your web browser is not capable of generating a useful error message when it is not privy to the connection problem. The connection to CloudFlare that your browser is actually making works fine. The problem is between CloudFlare and the backend server. For this reason, only CloudFlare could (and should) be the one generating the error. The alternative would be, I don't know, sending a blank page back or something, which is just dumb.

Also a cloud image has been standard for 'the internet' in networking diagrams for at least a decade. Extending that to a pictogram like this one makes perfect sense, nevermind that they've called the service CloudFlare and have a cloud for an icon...

[echen1024]

I am in Houston. I was seeing CloudFlare in Dallas, which was down.

[Richard Crowley]

Well, if it had been CloudFlare that was down you wouldn't know where it was or whether it was down. The fact that you saw the error page indicates that CloudFlare was UP, but the eevblog server itself was down. Which is exactly what the error page said.  Your browser does not have that information to display even if it wanted to.

[Richard Crowley]

Can the forum server be split across two (or more) servers? With MySQL replication between them.

CloudFlare effecctively caches the website contents across 24 servers around the planet.  No fooling around with SQL or anything else. Dave can maintain his server as it if were just a single box (which it is), and CloudFlare automatically caches the content out to a local server near you.

According to the article Dave referenced, it was a server-farm wide outage of the network infrastructure.  Pretty odd that the downtime was so very long.  But then Dave called it a "cluster FUBAR".  I strongly suspect that the inner workings of the internet is far more complex than most of us realize.

[miguelvp]

Can the forum server be split across two (or more) servers? With MySQL replication between them.

At a cost, Dave could use Cloud computing from Google Cloud, Amazon Could, Windows Azure, Verizon's Terremark, RackSpace Could services, etc.
Pretty much just use the over 1 billion dollar companies. Everything virtualized and load balanced.

But even if they are not too expensive, they will be more expensive than HostGator. Also setting up the servers to be scalable is not really that simple he'll need to hire someone to do it right.

[Richard Crowley]

I'm specifically targeting the single point of failure

But when it comes down to it, there is always a single point of origin.  Your SQL replication scheme only works if the master copy is available and participating, for example.

and slow time to recover. Servers in a single datacenter are a single point of failure if there is a site wide problem, however multiple servers in geographically dispersed locations are some insurance against that.

But the SQL replicated scheme has exactly the same vulnerability.  I agree that it seems like an extraordinarily long downtime for such a major outage of both HostGator and BlueHost (which may be the same company for all we know).

Cloudfare may (not that I have noticed) improve responsiveness for static pages but a forum DB is not static pages.

Yes, that does seem odd. CloudFlare appears to have simply rolled over and played dead instead of serving up old content. Perhaps that is a user-configurable behavior?

I strongly suspect these problems are the result of some technician walking around with two fingers up his nose. Or a screwdriver.

Its hard to find good help.  But then it seemed to take an extraordinarily long time to recover. I was very surprised that a "Multi-Server Outage" took the better part of a day to fix a network hardware problem.

[jancumps]

...

I'm specifically targeting the single point of failure and slow time to recover. Servers in a single datacenter are a single point of failure if there is a site wide problem, however multiple servers in geographically dispersed locations are some insurance against that...

Because high availability and disaster recovery come at a cost, it's a risk and financial discussion.
For a website that doesn't have a critical role, it is often better economics to take the hit of an outage than to take measures for high availability or failover.

[jancumps]

Hehe, just noticed that my posting time is off now that the site is back. says it's 5:52 while it's 7:52 over here.
I'll have to visit my profile options