General > General Technical Chat
The BIG EEVblog Server Fire
<< < (34/36) > >>
Monkeh:

--- Quote from: floobydust on April 13, 2021, 04:10:30 am ---
--- Quote from: Monkeh on April 13, 2021, 01:37:39 am ---What on earth makes you think that's a wooden roof?
--- End quote ---

I see a (modified) bitumen roof with wood fascia. A "fire suppression system" does nothing if that lights up.
It's common problem in building fires here, the roof and trusses (which are above all the sprinklers) can start burning and the fire crawls along the roof. Firefighters have to dump water despite the interior not being on fire at all.
I would think buildings in the UK have similar issues, with flat tar roofs?

--- End quote ---

I see a structure which will have steel trusses. The deck could be anything up to and including pre-cast concrete panels.
ejeffrey:
Most cloud compute platforms offer some form of preemptable / flexible instance class that the provider can shut down at any time.  This allows them to reduce over-provisioning while serving peak loads to high paying customers but I'm sure it can also be used to throttle for power consumption purposes due to distribution capacity, backup generator size, or cooling.  You then pay less -- possibly much less -- per CPU hour.  It's not usually useful for a webserver since you can't control when users come to your site.  You can't get an HTTPS request and just say "I'll respond to that when the spot price drops."  There might be situations where a moderate amount of downtime isn't a big deal but it isn't the normal situation for commercial web hosting.

Cancelling low priority or low urgency jobs is much more effective than force throttling CPU frequency.  Even cancelling high priority jobs is sometimes the best approach if it gets you through a crunch without outright failure.  Throttling CPUs not a great way to apply backpressure.  At first it does nothing but reduce the sleep/idle time fraction.  Then it generates congestion.  Only when the congestion degrades the service enough people stop using it does it really reduce the workload.  Essentially you put human behavior in your feedback loop, and require poor quality of service to be effective.
floobydust:

--- Quote from: Monkeh on April 13, 2021, 04:27:54 am ---
--- Quote from: floobydust on April 13, 2021, 04:10:30 am ---
--- Quote from: Monkeh on April 13, 2021, 01:37:39 am ---What on earth makes you think that's a wooden roof?
--- End quote ---

I see a (modified) bitumen roof with wood fascia. A "fire suppression system" does nothing if that lights up.
It's common problem in building fires here, the roof and trusses (which are above all the sprinklers) can start burning and the fire crawls along the roof. Firefighters have to dump water despite the interior not being on fire at all.
I would think buildings in the UK have similar issues, with flat tar roofs?

--- End quote ---

I see a structure which will have steel trusses. The deck could be anything up to and including pre-cast concrete panels.

--- End quote ---

The roof overhang on the loading docks is all wood, for many of the buildings. They might be cheaply built, whatever era they are from. There's no rooftop A/C on any of the buildings so they might not support any weight beyond snow load.
Point is, a datacenter should be constructed entirely of non-combustible building materials IMHO.
james_s:

--- Quote from: floobydust on April 13, 2021, 04:10:30 am ---
--- Quote from: Monkeh on April 13, 2021, 01:37:39 am ---What on earth makes you think that's a wooden roof?
--- End quote ---

I see a (modified) bitumen roof with wood fascia. A "fire suppression system" does nothing if that lights up.
It's common problem in building fires here, the roof and trusses (which are above all the sprinklers) can start burning and the fire crawls along the roof. Firefighters have to dump water despite the interior not being on fire at all.
I would think buildings in the UK have similar issues, with flat tar roofs?

--- End quote ---

Wood roofs are typical on that sort of building, I don't recall the name they call buildings of that style but the walls are cast in place reinforced concrete with wooden beams running across, supporting a plywood roof that is covered in a weatherproof outer layer. All three of the light industrial complexes my friends' shop has been been built in that way.
bd139:

--- Quote from: NiHaoMike on April 13, 2021, 02:45:44 am ---
--- Quote from: duckduck on April 12, 2021, 06:41:00 pm ---That's a great idea. One issue I can see for a hosting company is that they allow their customers to manage/reinstall to OS and apps. It would be difficult to enforce the installation of power-management software. It would be great if servers had a (let's say) 5 volt input, and when it dropped to below 1 volt, the BIOS would throttle the CPU down. Then you "just" run a 5 volt line run off of non-UPS, non-generator mains to each server and you're golden.

--- End quote ---
I have hacked some PCs to do just that by using a small MOSFET to pull the PROCHOT line to ground.

--- Quote from: bd139 on April 12, 2021, 07:10:03 pm ---That probably wouldn't work. If you're running near your CPU provision, you can enter a thing called "load hysteresis" which may be irrecoverable. This is where your load average goes above the total capacity and the CPUs can never catch up with the workload.  It requires adding much much more capacity than you had to start with before you can being the demand you had originally. Either that or breaking a huge chunk of your incoming load to recover.

--- End quote ---
That doesn't sound like something that should happen with a robustly designed service, wouldn't that mean a hacker who wants to take it down can just DDoS it for a short time and let queue overflow continue it for much longer than the initial attack?

--- End quote ---

DDoS is not something you handle at the service level. There’s no way to sink one. I mean how do you scale up something (taking our shit as an example) from aggregate average 700Mbit out to say 40Gbit out which is your saturation boundary? Even if you can scale up enough nodes horizontally it’s unlikely to be sensible or even reasonable to add that capacity. I mean we’re not going to add 57x the database capacity on the fly when our customer base is fairly static.

In this case we pay the provider to handle it. They do traffic analysis and if they detect an incoming DDoS then they drop the traffic from it at their network edge.

Edit: there is a mid ground though which is perfectly valid where you are unexpectedly successful. This can make or break a product. I’ve seen both outcomes.
Navigation
Message Index
Next page
Previous page
There was an error while thanking
Thanking...

Go to full version
Powered by SMFPacks Advanced Attachments Uploader Mod