Yes it’s about “attack surface”. The attack surface needs to be minimised and that is extremely difficult on cloud platforms even with proper architectural design. All it takes is one fuck up on an S3 bucket policy and you’re screwed. There is no isolation late r in front of that, no physical separation, even if you use the subnet endpoints only because the S3 API is exposed everywhere. You are instantly up shit creek.
AWS provided late last year an additional isolation layer to help customers from making that particular mistake (because it was a common one, as you observed). Sort of a set of suspenders to go with the technically OK, but sometimes misused, belt that was always there.
https://aws.amazon.com/blogs/aws/amazon-s3-block-public-access-another-layer-of-protection-for-your-accounts-and-buckets/Eventually as customers learn to fear this they pay for people, processes and software to manage this and then the cost savings shrivel up.
But most of the time, mid size enterprises actually cost more up front in “the cloud” on operational expenditure. It’s easier writing off a monthly credit card bill than a capital expenditure though. And this isn’t helped by the cloud proponents and sales folk constantly buzzing around like flies around shit selling the overall cost savings lie.
For my day job, we're in the cloud for development speed and agility, not cost savings. It costs slightly more in total, but I also know damn well that our dev teams can launch services to the public in days not months, no one has to negotiate queueing up to get their project delivered, and we've more or less eliminated the annual "scale up for next holiday" project that we used to start in Feb and run through September each year. Our monthly AWS bill has two commas and it's totally worth it.
For my own personal work, I also mostly host in AWS on my own nickel. Not having to think about a lot of the randoms ops tasks is freeing.
One comedy thing here I experienced recently is a £165k SQL server box that lasts 3 years costs £890k a year to run in AWS without any other infrastructure considered. There’s enough cash left over by not using AWS to fix the rest of the company’s problems but you know, death march...
I tried to find the comparison server you're talking about. I think you've chosen an example which is apples to watermelons by choosing a high-availability multi-AZ server (which your single box obviously is not) and by not contemplating/comparing the purchase of the AWS box as a reserved instance (which is financially analogous to buying your own 3 year hardware), and not counting any of the ping, power, cooling, security, and maintenance costs to run the on-prem server.
I have no AWS financial interests (other than owning mutual funds, so I own some Amazon shares indirectly). They are simply the best game in town in cloud computing and likely to remain that way for the next half-decade. If you're moving an existing operation into the cloud solely to save costs, you're probably going to be disappointed. How much could you possibly be saving over whatever you're doing that's already working? Why spend the effort, dollars, and risk to move something that works?
If you're going to the cloud for speed and agility, you're much more likely to achieve your goal.