Author Topic: Proof that software as service/cloud based, will never work for long term ...  (Read 151533 times)

0 Members and 1 Guest are viewing this topic.

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 15244
  • Country: fr
Translation:
"As a service" : "Cash Cow"
 
The following users thanked this post: SeanB

Offline paulca

  • Super Contributor
  • ***
  • Posts: 4245
  • Country: gb
"What could possibly go wrong?"
Current Open Projects:  STM32F411RE+ESP32+TFT for home IoT (NoT) projects.  Child's advent xmas countdown toy.  Digital audio routing board.
 

Online tautech

  • Super Contributor
  • ***
  • Posts: 29346
  • Country: nz
  • Taupaki Technologies Ltd. Siglent Distributor NZ.
    • Taupaki Technologies Ltd.
https://www.theregister.com/2024/07/19/microsoft_365_azure_outage_central_us/
LOL, it's worldwide !

But smart dudes previously active members here are fixing it bit by bit.  :-X
Avid Rabid Hobbyist.
Some stuff seen @ Siglent HQ cannot be shared.
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 4245
  • Country: gb
https://www.theregister.com/2024/07/19/microsoft_365_azure_outage_central_us/
LOL, it's worldwide !

But smart dudes previously active members here are fixing it bit by bit.  :-X

I am basking with my coffee on a misserable wet summer morning working for the irish civil service and thus not effected either.  Yet.  I am SO glad I got out of the US banks right now :)
"What could possibly go wrong?"
Current Open Projects:  STM32F411RE+ESP32+TFT for home IoT (NoT) projects.  Child's advent xmas countdown toy.  Digital audio routing board.
 

Online tautech

  • Super Contributor
  • ***
  • Posts: 29346
  • Country: nz
  • Taupaki Technologies Ltd. Siglent Distributor NZ.
    • Taupaki Technologies Ltd.
Avid Rabid Hobbyist.
Some stuff seen @ Siglent HQ cannot be shared.
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 4245
  • Country: gb
In telecoms industry most large companies are legally required to not be vendor locked and maintain contracts with multiple vendor for critical infra..  Or at least I believe it WAS that way.

Later I discovered that the entirety of large digital government wings in the UK are 100% vendor locked in Azure.... and single vendor.  I was shocked and even asked if this was the case of if they had a hidden replica in AWS.... nope.  No idea what I was talking about either.

That article's question to government is real and absolutely needs addressed.

We can watch companies go bust and laugh, but when government departments start falling over and public services get impacted it's a different ball game IMHO.

...  and or pensions.
« Last Edit: July 19, 2024, 10:40:10 am by paulca »
"What could possibly go wrong?"
Current Open Projects:  STM32F411RE+ESP32+TFT for home IoT (NoT) projects.  Child's advent xmas countdown toy.  Digital audio routing board.
 

Online tom66

  • Super Contributor
  • ***
  • Posts: 6965
  • Country: gb
  • Electronics Hobbyist & FPGA/Embedded Systems EE
The Crowdstrike issue has nothing to do with the cloud, it's a botched antivirus update that is BSODing systems.  It is a massive cock up on their part, but it would happen regardless of the 'cloud'.

At this point I bet lawyers are rubbing their hands with glee as Crowdstrike may be sued for substantial damages.
 

Offline berke

  • Frequent Contributor
  • **
  • Posts: 259
  • Country: fr
  • F4WCO
https://www.theregister.com/2024/07/19/microsoft_365_azure_outage_central_us/
Does anyone know when exactly this started?  Earliest Tweet I found (from Downdetector) is timestamped 2024-07-19 00:23 UTC.
 

Online PlainName

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: va
The Crowdstrike issue has nothing to do with the cloud, it's a botched antivirus update that is BSODing systems.

Exactly this. And, so far as I can make it, it's not a Microsoft or Windows issue, despite every headline saying it's Windows. It just happens that the software runs on Windows - if it were a Mac, say, it would be different software and ditto for Linux.
 

Offline Andy Chee

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: au
The Crowdstrike issue has nothing to do with the cloud, it's a botched antivirus update that is BSODing systems.

Exactly this. And, so far as I can make it, it's not a Microsoft or Windows issue, despite every headline saying it's Windows. It just happens that the software runs on Windows - if it were a Mac, say, it would be different software and ditto for Linux.
https://x.com/george_kurtz/status/1814235001745027317

Quote
CrowdStrike is actively working with customers impacted by a defect found in a single content update for Windows hosts. Mac and Linux hosts are not impacted.

 

Online PlainName

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: va
Yes, it's not a Windows thing per se. Like if your car tyre blows out and your car crashes, it's not a Volkswagon issue, just that those types of tyres are used on those cars. You off-roader won't be affected because it uses different tyres, although they do the same job.
 

Offline coromonadalix

  • Super Contributor
  • ***
  • Posts: 6559
  • Country: ca
i hope governments and institutions will sue ....  while i understand a few things,  what is not ok   is the dependency / no redundancy on theses things

this year we saw  many similar things happened
 

Offline madires

  • Super Contributor
  • ***
  • Posts: 8129
  • Country: de
  • A qualified hobbyist ;)
Seems to be the new normal at CrowdStrike. Just a few weeks back:
-  CrowdStrike bug maxes out 100% of CPU, requires Windows reboots (https://www.thestack.technology/crowdstrike-bug-maxes-out-100-of-cpu-requires-windows-reboots/)

And they did it also with linux in April:
- Bug#1069642: linux-image-6.1.0-20-amd64: kernel panic after 2024-04-20 (https://www.mail-archive.com/debian-kernel@lists.debian.org/msg136190.html)
- linux-image-6.1.0-20 killed all my debian VMs (https://www.reddit.com/r/debian/comments/1c8db7l/linuximage61020_killed_all_my_debian_vms/)

BTW, Microsoft 365 was down too:
- https://www.timesnownews.com/technology-science/microsoft-365-down-full-list-of-apps-and-services-facing-outage-globally-article-111855390
Caused by a configuration change within the Azure backend infrastructure.
 

Online tautech

  • Super Contributor
  • ***
  • Posts: 29346
  • Country: nz
  • Taupaki Technologies Ltd. Siglent Distributor NZ.
    • Taupaki Technologies Ltd.
When you let monkeys loose.....




Avid Rabid Hobbyist.
Some stuff seen @ Siglent HQ cannot be shared.
 
The following users thanked this post: nctnico

Offline JMK

  • Newbie
  • Posts: 9
  • Country: au
i hope governments and institutions will sue ....  while i understand a few things,  what is not ok   is the dependency / no redundancy on theses things

this year we saw  many similar things happened

Do you mean "throwing it against the wall and hoping it sticks isn't good enough"?

Redundancy costs time, effort and money. How can that be possible when professional bosses only know how to reduce costs and increase charges for the corporate bottom line. And, of course, that bonus. ;D
 
The following users thanked this post: SeanB

Offline madires

  • Super Contributor
  • ***
  • Posts: 8129
  • Country: de
  • A qualified hobbyist ;)
Why the fuss? It's business as usual, i.e. the antivirus kills the OS. Happens all the time. It's the side effect of security snake oil.
 

Offline coppercone2

  • Super Contributor
  • ***
  • Posts: 10517
  • Country: us
  • $
I bet they wish they had a dedicated isolated mainframe for logistics right now run on 74 series cards :-DD

It seems like global infrastructure is on a hair trigger....
 

Offline Karel

  • Super Contributor
  • ***
  • Posts: 2259
  • Country: 00
« Last Edit: July 19, 2024, 05:16:28 pm by Karel »
 

Online tautech

  • Super Contributor
  • ***
  • Posts: 29346
  • Country: nz
  • Taupaki Technologies Ltd. Siglent Distributor NZ.
    • Taupaki Technologies Ltd.
I bet they wish they had a dedicated isolated mainframe for logistics right now run on 74 series cards :-DD

It seems like global infrastructure is on a hair trigger....
Monkeys can wreak anything.

From a guy in the middle of fixing things:
They found out the root cause earlier
They loaded a driver (.sys file) which was all zeroes......

Avid Rabid Hobbyist.
Some stuff seen @ Siglent HQ cannot be shared.
 

Offline mikeb1279

  • Newbie
  • Posts: 6
  • Country: au
The Crowdstrike issue has nothing to do with the cloud, it's a botched antivirus update that is BSODing systems.  It is a massive cock up on their part, but it would happen regardless of the 'cloud'.

At this point I bet lawyers are rubbing their hands with glee as Crowdstrike may be sued for substantial damages.

I'm going off the top of my head here, but isn't Crowdstrike Falcon an entirely cloud based SaaS product? In which case this would be exactly one of my big problems with SaaS. A third party company complete baulking your environment by pushing in changes you have no control over and no ability to roll back.

We had a Sev2 outage at work for two weeks recently for something similar. SaaS should be relegated to nice to have non-business critical functions and the sooner the better.....
« Last Edit: July 19, 2024, 09:58:37 pm by mikeb1279 »
 
The following users thanked this post: SiliconWizard

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 15244
  • Country: fr
This Crowdstrike debacle is hilarious!

And yes, claiming it's not cloud-based is quite funny. This is exactly how they define it themselves:
Quote
Powered by the CrowdStrike Security Cloud, the CrowdStrike FalconĀ® platform leverages real-time indicators of attack, threat intelligence(...)
« Last Edit: July 19, 2024, 10:21:20 pm by SiliconWizard »
 

Online tom66

  • Super Contributor
  • ***
  • Posts: 6965
  • Country: gb
  • Electronics Hobbyist & FPGA/Embedded Systems EE
The Crowdstrike issue has nothing to do with the cloud, it's a botched antivirus update that is BSODing systems.  It is a massive cock up on their part, but it would happen regardless of the 'cloud'.

At this point I bet lawyers are rubbing their hands with glee as Crowdstrike may be sued for substantial damages.

I'm going off the top of my head here, but isn't Crowdstrike Falcon an entirely cloud based SaaS product? In which case this would be exactly one of my big problems with SaaS. A third party company complete baulking your environment by pushing in changes you have no control over and no ability to roll back.

We had a Sev2 outage at work for two weeks recently for something similar. SaaS should be relegated to nice to have non-business critical functions and the sooner the better.....

The point is the fault hasn't occurred because the cloud went down or even because the software is cloud based (that really is a buzzword), but because some numpty somewhere (who is presumably going to find a P45 equivalent on their desk soon enough) pushed a bad update.

This could well have occurred in the days before 'cloud' was even a term that had been dreamed up. 

 

Offline mikeb1279

  • Newbie
  • Posts: 6
  • Country: au
The Crowdstrike issue has nothing to do with the cloud, it's a botched antivirus update that is BSODing systems.  It is a massive cock up on their part, but it would happen regardless of the 'cloud'.

At this point I bet lawyers are rubbing their hands with glee as Crowdstrike may be sued for substantial damages.

I'm going off the top of my head here, but isn't Crowdstrike Falcon an entirely cloud based SaaS product? In which case this would be exactly one of my big problems with SaaS. A third party company complete baulking your environment by pushing in changes you have no control over and no ability to roll back.

We had a Sev2 outage at work for two weeks recently for something similar. SaaS should be relegated to nice to have non-business critical functions and the sooner the better.....

The point is the fault hasn't occurred because the cloud went down or even because the software is cloud based (that really is a buzzword), but because some numpty somewhere (who is presumably going to find a P45 equivalent on their desk soon enough) pushed a bad update.

This could well have occurred in the days before 'cloud' was even a term that had been dreamed up.

But one of the reasons it was so serious IS because it's part of the SaaS model (notice SaaS here, rather than general cloud). As someone pointed out above, anti-viruses has been screwing OS's since the dawn of time. On-prem IT can manage this by patching non-critical boxes first to test etc (and they did). With the SaaS model you just get numpties pushing code straight to prod and if it breaks everything and you can't do anything about it then 'too bad, so sad'.
 

Online PlainName

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: va
Quote
As someone pointed out above, anti-viruses has been screwing OS's since the dawn of time.

Which is actually preferable to being hit by ransomware or worse. At least problems with AV (and they are actually few) aren't malicious.

Quote
On-prem IT can manage this by patching non-critical boxes first to test etc

With other stuff, sure. But AV is often pushing out critical patches for 0-day exploits, and if you hang around a week for IT to try it on some spare kit you might be too late to apply it. While your IT bods are having a good play the bad guys are deconstructing it to find the hole it patches, and then hope they get to your setup before your IT people finally give the OK and think about rolling it out.

It's a matter of risk, and effectively you're outsourcing the testing and stuff to a third party who should know their onions - your local IT bods generally don't have a clue because they don't have the mindset of do-badders. Just think of how many security holes there are all over the place (requiring AV to stop them being exploited) - the developers don't have the mindset to see them, and IT support are not really any different (and if they were, you wouldn't want to be employing them).

 

Offline Halcyon

  • Global Moderator
  • *****
  • Posts: 5902
  • Country: au
https://www.theregister.com/2024/07/19/microsoft_365_azure_outage_central_us/
LOL, it's worldwide !

But smart dudes previously active members here are fixing it bit by bit.  :-X

Well yesterday was fun. Thankfully we are 50/50 Microsoft/Apple so our business operations weren't impacted too much. Each time could still do their job, albeit at reduced capacity. This is why I always say not to put your eggs in one basket. Those guys who rely on Windows endpoints and servers and Hyper-V will still be picking up the pieces this weekend.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf