37 Signals says cloud repatriation plan has already saved it $1 million David Heinemeier Hansson, CTO of SaaS project management outfit 37Signals, has posted an update on the cloud repatriation project he’s led, writing that it’s already saved the company $1 million. Hansson has previously revealed that his company spent $3.2 million a year on cloud computing, most of it at Amazon Web Services. …
Does he have geographically diverse redundant servers? Didn't read anything about the servers he is setting up in a second location. Didn't read anything in there about backup, surely he's going to have to spend money to back it up, ship backup tapes to secure off site storage, and have a plan (that's regularly rehearsed) for disaster recovery? How about a beefy UPS able to handle those beefy servers, and a generator for longer outages, or is he going to rely on his non existent redundant servers?
People didn't switch to cloud to save money, at least no one with any sense did. If a company thinking about switching to cloud 10 years ago (when it started becoming a big trend) and 5 years ago (when it became "you're still running in your own datacenter?") was willing to invest in beefy servers, VMware licenses, and people who knew how to manage that stuff they could always do it cheaper than cloud. That's all cloud is, after all - that and a lot of automated processes that take a lot of (but not all, of course!) human error out of the equation. It was avoided capital investment that caused them to choose the cloud in the first place, paving the way to collect a big bonus for "saving the company money".
Smart people who switched to cloud did it knowing it wasn't being done to save money, but because it gave them peace of mind that someone else with automated processes that thanks to massive scale are able to catch many of the corner cases that bite you in the ass the first time you see them. They take care of the redundancy and backups, and much of the difficulty of disaster recovery along with it. They didn't need to worry that their UPS was aged and underpowered and needed replacement, or have to deal with noise citations from the city due to their monthly generator tests. They didn't have to worry if their top IT engineer left along with the two best lieutenants leaving you paying for consultants to run things for months who may or may not do a decent job but at least give you someone to point the finger of blame at while you try to hire someone able to replace them without incurring too many "learning on the job" mistakes.
So sure, he'll save money. Even after he spends millions on all the stuff he's left out (or pays later if he thinks he can do without and he's hit with a ransomware attack or weather event that takes power out in the region for days) And he'll collect a bonus for saving the company money. But he won't sleep better, because he'll have a lot more stuff he's on the hook for since this whole thing was his idea, and if the shit hits the fan even if they don't blame him to his face a lot of the higher ups will blame him behind his back. And one way or another he'll have moved on in five years when all this shiny new equipment is long in the tooth, and faced with a hefty bill to replace it his replacement will suggest to the CEO "we could avoid a big capital expense by switching to the cloud" and collect a bonus for the money it saves...
37signals has received a great deal of publicity on the back of this decision but I've not yet been able to locate any real detail about what they've been doing, which makes me a little cautious.
However, their data centre provider has multiple geographic locations available. 37signals also seem to have been using very little vendor-specific cloud technology: apart from S3 storage, it's pretty much all standard virtual machines. And they have a relatively stable base workload. That would make it pretty easy to use the cloud or other data centres as standby.
DHH is the guy behind Ruby on Rails which was largely developed for Basecamp, 37signals' main SaaS offering, so he's not just some C-suite placeholder.
It may be that smart people who switched to cloud did it knowing it wasn't being done to save money, but because it gave them peace of mind, but your peace of mind was very likely bought with other people's money. It's also probably illusory: cloud services do go down, backups get corrupted, vendors cut off small customers on a whim. You still need the backup on other infrastructure, you still need the DR plan.
As I say, I don't know in detail what they're doing, but they seem to have the kind of workload for which self-hosting might work and could be made reliable. If the savings are as they predict, they could afford a few more operations staff and still pocket a decent return. I just wonder whether we'll ever get an independent analysis.
As a SWE who had to learn WAY too much about Rails internals working around various *($#& problems, let me tell you that Rails code sucked rocks in the 1.x - 2.x days. And the early 3.x code was worse. I referred to one of his innovations (which is STILL polluting the ruby ecosystem) as a "typical DHH three-quarter baked solution". It solved his problem pretty well--and made the life of a LOT of other people miserable in the process.
So when it comes to DHH claiming that he's achieving some great success, history shows that he's particularly good at ignoring the larger consequences of his decisions. I think this is a classic case where someone is dramatically dropping uptime while not even noticing.
"People didn't switch to cloud to save money"
In many/most large enterprises the finance decisions are much more subtle than that as which bucket it comes from often rates higher than how big the bill is and moving to cloud simplifies the accounting since it is now just a rental charge rather than capital equipment with support costs and depreciation etc.
The people that actually declare that the company will move to cloud have a limited understanding of the technical side but have listened to the salesmen/accountants and have been assured that it will have no effect on the end-user but will look much nicer on the balance sheet (we have global distributed resources; great uptime blah blah blah). What nobody will mention is the change in stress points since now it is not the ability of the datacentre team to keep all the servers going but all the disparate companies between you and your data who can roll out changes without warning that cause your link to go dark. If it is your datacentre/staff you can send the boys round 'pour encourage les autres' but when it could be any one of six companies who already have your money and consider you a class B customer you may not even be able to get a recorded message when you call them.
People didn't switch to cloud to save money, at least no one with any sense did.
How many did it with sense, or on the promises of some slippery salesman to the CFO?
The points you raise about backup/redundancy/etc are all very valid, but the UPS & redundant A/C side would be the data centres responsibility. While cloud providers promise a totally stress free and safe system, we still see them going down with outages or strange data losses from time to time. When it happens, take a ticket, you are 10,451th in line for support. And then we get to the locked-in aspect...
What this article raises should be considered - your costs might be much less keeping stuff on-premises, but of course that only works out well above a certain size of technical staff levels when you have your own folks to take care of it. There are other cases where cloud makes a lot of sense, for example rapid scalability.
"There are other cases where cloud makes a lot of sense"
Absolutely true. Use the right tool for the right job. I've seen cases where Cloud would have been a phenomenal solution to the problem at hand. I've also seen cases where pushing everything to the Cloud regardless has resulted in hundreds of thousands in lost revenue, because no-one looked at the cost of replicating data across the Cloud to provide DR.
Unfortunately the C-Suite at the top of the larger companies are the ones drinking the Cloud cool-aid and making the decisions. It's almost always "How can I secure my next bonus whilst appearing to save the company money", whilst ignoring the technical concerns of those actually keeping everything running.
"People didn't switch to cloud to save money, at least no one with any sense did."
Kind of true - it's something C-Suite accountants peddle to move costs from Capex to Opex. The problems start because the accountants aren't IT people, and they don't keep a track of the often spiralling costs of Cloud.
"...it's something C-Suite accountants peddle to move costs from Capex to Opex"
Not sure that's correct.... AFAIK beancounters prefer costs to be in Capex rather than Opex, because the expense can be capitalised and added to the balance sheet right now, while the depreciation on the capital assets goes into future balance accounts. One of the great ills of modern capitalism is the short-termism required in this view, because if the only thing that counts is the current stock price, the effects on the company future are ignored.
Also, with opex not only is the money gone straight away, it also probably has to be listed as a recurring expense rather than a one-off, which C-suits also try to minimise in the accounts. (Recurring vs one-off is the same reason why some companies prefer to hire consultants at double the cost of internal employees - even if the consultants are at the company for years, the expense doesn't have to be classified as 'recurrent')
With 'C level' execs expected to provide ginormous profit growth every 90 days or get the chop thanks to Wall St, no one wants to invest even a dime on something that will take even 100 days to start paying pack.
This is the sad situation that most of US and increasingly Worldwide business has to operate under.
All they are measured on is the bottom line profits and not much more it was than 90 days before.
There are some exceptions to this crazyness.
If you are a private company (ie does not have a listing anywhere) then you don't have to suffer the fools on Wall St (and elsewhere). That means sanity.
If you are CEO of a company like Apple then your investment plans are for 3 to 10 years. Yes, even Tim Cook has to take lots of flack if their profits don't exceed what the goons on Wall St have predicted but in the main, he can ignore a dip for 3 or even 6 months because their product line investments are working away to update existing products and introduce new ones on a regular basis.
Otherwise, you are largely at the mercy of Wall St and the scum called Activist Investors who want to do nothing but asset strip a business before selling out and moving on to the next sucker.
Perhaps there needs to be legislation to limit financial statements to annually or whatever interval dividends are paid apart, possibly, from regular statements as to whether the company is still solvent. The fact that that might throw many financial analysts from Wall St & elsewhere out of work is an added bonus.
Someone once told me the opposite with the reasoning that payroll is deductible. I would think the same is true whether your costs are in payroll or a service company.
Of course if you believe you will only ever need to make capital expenditures once and will thereafter just charge rents, that would sound great until you reached the cliff.
Years ago, the people from StackOverflow posted why and how they moved from AWS. You always pay for what you provision, not what you use. The cloud providers attempt to ellide this but the costs are concealed in the charges that give you so much flexibility.
AWS, et al. have no interest in making your operations run more efficiently because the more you use, the more they earn. So, you'll get ElasticSearch on lots of nodes where a single well-designed Postgres DB server might suffice. Backup, replication and scaling are all good arguments but how to do this is now well documented and can be learned and most data centres can often help to set them up and run them, including the right kind of connections to relevant colocation servers. They also provide the UPS and other infrastructure.
Cloud promises flexibility and scale and designs products that make entry quick and simple because they know that once your data is on their infrastructure they have you: exporting data is dfficult, slow and expensive.
The cloud is flexible, and certainly a whole lot more flexibility than on-premise.
Look at services like Azure Functions and AWS Lambda. Lambda is priced at $0.20 per 1M requests. No requests, = no costs. Seems a whole lot more flexible than having a server sitting in a data centre powered up 24/7.
Look at Blob Storage/S3. You pay $0.023 per GB for S3 / $0.00099 per GB for Deep Archive. When I used or architect on-prem solutions for customers, it would be - how much storage do you need? How much growth do you anticipate in the first 12 months? How much backup do you need? How much disk redundancy do you need? All of this added up to a shitload of hardware that's only utilised at 70%, and you are stuck with over-provisioned hardware that is depreciating year on year until it needs replacing with new kit, where we do the same dance all over again.
Now, if you take 500 VMs on Prem, move them to the cloud and try to do the same thing as you did on Prem, well, yep, it's going to cost more than on-prem, but then you shouldn't be working in IT if you signed that madness off.
You're not making a like-for-like and lifecycle comparison. Sure, for any kind of flexible load these services are great. Most businesses have fairly predictable loads with equally predictable spikes where extra capacity is bought in: think of fruit-pickers in the summer or extra deliveries in the run up to Christmas.
But some of these deals are essentially loss-leaders designed to trick you into moving your data onto their platform. But how much is going to cost you to get it back or switch providers? This is stated in virtually every quarterly report to investors.
The cloud is flexible, and certainly a whole lot more flexibility than on-premise
If you have really peaky loads that's valuable. A chain of flower shops will have huge peaks for Valentine's Day, Mother's Day and so forth, providing your servers to handle that load would cost a lot than if you could provision for your average load year round. That's the kind of flexibility cloud is good for.
Most businesses don't have enough peaks and valleys that this helps though.
I think that that's how AWS started: Amazon's datacentres were designed for the peak of pre-Xmas season, and thus were sitting idling around 11/12 months. Then someone thought about renting out that spare computing capacity when not used, and that became a business of its own
People didn't switch to cloud to save money
On the contrary, lots of CEOs did.
, at least no one with any sense did.
Indeed so.
They take care of the redundancy and backups
They will take care of redundancy, if you think to ask them and pay for it. They won't handle backup of anything but configuration details, the rest is your data and it's up to you to apply whatever backup rules you decide are necessary.
>> Does he have geographically diverse redundant servers?
Maybe. But just because cloud providers have geographically diverse redundant servers doesn't necessarily mean that, if one falls over, your workload will smoothly transition to another server in another location. First of all because that feature is usually charged extra, and secondly not all data is even allowed to be moved to other locations.
>> Didn't read anything in there about backup
You mean the same backup required for cloud workloads because providers usually put the onus for data backup on the customer?
>> Smart people who switched to cloud did it knowing it wasn't being done to save money, but because it gave them peace of mind that someone else with automated processes that thanks to massive scale are able to catch many of the corner cases that bite you in the ass the first time you see them.
And yet there have been several massive outages with certain big cloud providers, caused by both equipment failures and human error, and in some cases customers lost data. With no repercussions for the cloud provider because their SLAs are written in a way that they are pretty much not responsible for anything.
Microsoft Azure/Entra certainly takes the cake for the number of outages due to stupid mistakes made by its staff, and that's already on top of a number of massive security fuckups. But AWS and even GCP (which so far has been the most reliable of the big cloud vendors) had outages affecting their customers.
So, really, it looks those "smart people" might not have been so smart after all, and the decision to move to the cloud for better redundancy and security was the exact opposite of smart.
"But just because cloud providers have geographically diverse redundant servers doesn't necessarily mean that, if one falls over, your workload will smoothly transition to another server in another location"
All the extra networking involves adds an extra layer of failure points.
>> Microsoft Azure/Entra certainly takes the cake for the number of outages due to stupid mistakes made by its staff, and that's already on top of a number of massive security fuckups.
Look like Microsoft did it again:
https://www.bleepingcomputer.com/news/microsoft/microsoft-leaks-38tb-of-private-data-via-unsecured-azure-storage/
I'm gobsmacked how any business can consider using this clown show of a cloud provider for anything important.
The initial USP was to save money since you wouldn't need you own administrators and techies to keep things up and running. I guess know that has fallen through they're making up other excuses, such as geographical redundancy.
Cloud is darn expensive and we're already seeing companies (not just 37 Signals) pull back on deployments. Hopefully providers will slash prices to keep growth going forward.
"People didn't switch to cloud to save..." (and/or SaaS) ?
Yeah, if a client asks, I find myself recommending cloud solutions (even though I've never really liked the cloud).
A business "rolling there own" may start out OK but....I assume that eventually they'll encounter issues in acquiring/retaining staff.
Sure, we don't know if it is comparable.
But I'm guessing from the article it's in a DC so things like a UPS are part of the deal.
As to redundancy, how do you know if they had that level of redundancy to begin with - it isn't just globally redundant by being 'in the cloud' anyhow.
You seem to think that the existing setup covered all the things you think he might need to spend money on, or have to spend money on later. I doubt it.
We're very much a small outfit by comparison but there is *no* doubt in my mind that we're better off financially and operationally by not being 'in the cloud'. It isn't even a close competition. It's the difference between 'we exist' and 'we would have run out of cash'
We deploy application, database, network and backup solution as a site template for our internal customer - we've deployed 20 sites in the last 3 or 4 (5?) years. A site runs around 500 application VMs and the solution takes 4 racks. Cloud comes in much more expensive.
I work for a hardware vendor and everything you list in your first paragraph is instinctive for all my customers. If DHH hasn't considered this then he's not the bright chap pretty much *all* of his public comments suggest, and a massive outlier amongst his peers.
Secondly, most of these objections are handled by using a co-location data centre supplier rather than running your own. I see very few business who still own their own data centres apart from some (not all) of the largest multi-nationals. (See my final paragraph below as well). So objections like a UPS and generator are moot, as it's part of the service you are paying for. Added to which Cloud services are often run out of these third party DCs anyway.
I would also contest your argument that moving to the cloud removes a lot of corner cases that might bite you - yes, it removes some, but just introduces others. Otherwise why has SRE suddenly become a thing as the cloud has grown ?
"People didn't switch to cloud to save money, at least no one with any sense did" - seriously? More than half of my customers have, if not an "Everything Cloud" policy, at least a "Cloud First" mantra. The more aggressive/impatient/foolish ones are getting their fingers royally burnt and seeing costs go up by 50%+. IMHO it is a combination of short-term opex avoidance, the desire to be more "agile" and simply an attempt by board-level people to appear more "sexy" that is driving this rush away from on-prem.
Ultimately though this is about scale. It rarely makes sense for consumers of specialist resources to produce it themselves. Who generates their own electricity or owns their own dark fibre for telecoms? But there comes a point where it is cheaper to take over yourself, and this often brings accompanying benefits like greater control - you are no longer forced into your vendors service levels of small, medium or large, can choose when maintenance occurs, are able to cope with unusual edge-cases and so on. As you say, there is some additional responsibility to be inherited if you do this, but you pay your money and make your choice. A final caveat though - I work with large organisations who have the luxury of size to allow some redundancy in people and resources and economies of scale. I would agree that cloud makes a lot more sense for smaller businesses.
If the demand is well understood and stable, then it's perfectly possible that owned kit is lower cost. But, when I looked at this in large orgs around the time that cloud was emerging, it was clear that not only were capex costs very large - usually because the user demand and how the system behaved were not well enough understood before the investments were made - they were usually hidden, as were the opex costs of competing technical solutions and sustaining the necessary infrastructure skills, DR costs (for non-validated disaster recovery techniques), etc.
"37 Signals ops team remains the same size"
If your Ops team can handle your on-prem load, then they were either over-spec'd for cloud or you were doing cloud not quite in the way it was intended. We don't have an Ops team. No one is racking and stacking, installing OSs, running updates, or doing anything related to server IT. Then again, that's the distinction between PaaS and IaaS.
The same size doesn’t mean the same skills set.
Interesting to know the size of the team, I note it was only 6 people in 2012 and in 2022 the entire company was 80 people [ https://world.hey.com/jason/on-company-size-8095488d ].
Would not be surprised if the team has lost “racking and stacking” staff (outsourced) and gained more business systems management and operations staff. Ie. What is considered “ops” has changed.
You don't need "racking and stacking" staff for 8 servers ffs.
People these days seem to think moving out of the cloud can consume dozens of racks and hundreds of servers, servers are very powerful now, have been for some time. Consolidation has been a thing for almost 2 decades with VMware and similar tech.
When I moved my last company out of cloud in early 2012 it was to 8 x HP DL385G7 with 2x12 core CPUs and 192GB of memory (later upgraded to 16 core cpus and 384GB memory). A small HP F200 3PAR array, a couple of small fibrechannel switches, a pair of Citrix Netscaler load balancers, I think just 4 ethernet switches (2x10G and 2x1G), later added a pair of Sonicwalls for IPSec site to site VPN. All of this was in 2 partially filled racks(47U and extra wide/deep) in a 64 sq foot cage. Our last cloud bill was in the ~$70k/mo range when we got out. I still have the email the CTO sent to the company at the time, in part saying it would pay for itself within the year.
Several of my team left eventually to another company(5-7 years later) and the leaders there wanted to move them to cloud, and in their case they tried. I found it hysterical that these pro cloud folks spent more than 3 years trying to get it done, and never got out of their old (VERY poorly managed) data center infrastructure(my data center stuff runs super smooth beyond my expectations). Also first hand accounts of their google cloud bills. Several of those folks are working with me now again at new company, operating the same infrastructure I started back in late 2011. I still have one of the original DL385 G7s in the rack still(turned off, officially retired in 2019), in the rare chance I need a spare test system. Also still have the same original 4 ethernet switches in place, in service since 2011(with no failures). I have their replacements on site and will be replacing them with new gear before end of the year. I'll be honest, replacing running ethernet switches filled with cables is the thing I dread the most about data center gear. I have pretty good cable management(and labeling) but it's not like these run a single VLAN, tons of ports with different configs.
As for those advocating for multiple data centers, geographic separation all that shit. You can go get that if you want, most(~90%+) do not need it. I've been with colo providers for 20 and 1/2 years, of the professional providers I have had 2 full data center outages(both by the same provider in a short time frame) in 2006(moved out of that provider shortly after - that facility went on years later to have a ~50 hour power outage due to a fire in their power room, and spent months running on generator trucks during repairs). Choose a good provider, there are tons out there, I'd guesstimate that a good 75% of them out there would not meet my standards for mission critical operation. The facility I use for my personal colo by contrast does not meet my standards for mission critical operation, but it is cheap, and does a good enough job for my own stuff (they have had maybe 4-5 power outages over the past decade, no redundant power). You get what you pay for.
Also get a good internet provider(or more than one if you want). Personally I like Internap (whom sold their Internet division to Unitas Global about a year ago), and their MIRO architecture with 100% uptime SLA, as I have told them in the past, in my opinion, Internap is the #1 ISP and there is no 2nd place. Have used Internap(now Unitas) networking off and on(mostly on) since 2006. Have had several unique experiences with their support that have made my loyalty strong. Though they have had some stupid issues as well, nowhere NEAR the volume of issues as other providers I have used.
I don't get why so many people are so absolutely sure this guy is wrong - he's reporting what his team have done/are doing, but the cloud groupies are convinced he can't count / it can't be done even for his use case.
His blog that is linked in The Register article does highlight (in my words) "your mileage may vary based on your workloads", but it seems to be a no-brainer for them and from my perspective, it's good to have a little less group think / herd mentality on this.
If I had a $ for every time I'd heard "cloud first strategy" as an edict from a board, irrespective of their ability to execute it well/derive any commercial benefits from it, I wouldn't be reading The Register this afternoon, I'd be on a yacht.....
He always seems to have some kind of agenda. Nothing he writes is ever particularly balanced or sufficiently nuanced. And he doesn't give a toss what anyone thinks - see the latest mess with removal of Typescript from Turbo. He's a narcissistic asshole, it's his way or the highway.
That's why I start from the default position of assuming he's talking crap, and see if he can persuade me otherwise (and he rarely does). The problem is too many people seem to lap up his preachings as some kind of gospel and don't question any of it.
Is his company spending less money than they were? Yes, apparently so.
But is their solution less resilient and less capable of scaling? Probably. Does that actually matter to them or their customers? Probably not.
Cloud isn't inherently bad or expensive - but there's far too little Systems Analysis and Systems Engineering that goes on these days, and far too much "agile" lobbing stuff at a wall and seeing what sticks, so it's not surprising that people end up building stuff that's over- or under-engineered, with cost impacts.
How much time per day do you spend calling out narcissistic morons? There's plenty of them out there for sure, so it could easily be a fully time job......
Cough "Musk". Cough "Trump". Cough "Johnson". Cough "Ramaswamy". Cough "Putin"* (and they're just the more famous ones)
*Might be more sociopath, but certainly some narcissist
Think of the savings he could of had if he did this a decade ago like I did.
http://www.techopsguys.com/2010/10/06/amazon-ec2-not-your-fathers-enterprise-cloud/
Wrote that almost 13 years ago now. Andy jassy tried to get me fired from my job(brother of the CEO of the company I was at) within hours of posting it here. Still applies today. (I don't blog anymore).
I had to fight executives a half dozen times at my last company over a decade. New ones kept coming in thinking they can save money with cloud. They were ALWAYS very confused when the cloud cost numbers were so high.
Companies are constantly bombarded with cloud will save you money BS. I'm sure there are cases where that is true. But they are a minority of cases.
side note in my blog post I mentioned using the Terremark vCloud express cloud for my blog, which I did for a few months as a bridge between colo providers(moved from WA to CA), until I bought a new server and found a new local provider to host with, been with them ever since. One thing I didn't mention on my blog was on one engagement with Terremark(back in early 2010, long before Verizon bought them), they proposed a DR cloud solution for us for a mere $272,000/mo (with $0 setup fee!). What a deal. I explained to them, (I'm always honest, sometimes too much so), that I can buy all of the gear for my DR plan for about $800k (+ some costs for hosting), so $272k/mo is a bit steep. My VP was confused why it was so much(he had budgeted only $250k total for a DR plan, a number he pulled out of his ass without consulting anyone).
They later came back with an updated deal, 10G networking was still "new" for them, at that point in time they only had one data center(in south america) that had it. Though they were in the midst of building a new DC in the Washington DC area I think that had it too. They said we could host there, cost would be about $150k/mo. Only catch is there was a $3 million install fee. (and yes I still have all my emails and their pdf proposal at that time).
I reminded them, I told you already I can buy this stuff for $800k, why would I pay $3M. Their only response was kind of confused "who's going to manage it? do you have the ability to run that yourself?". I explained to them yes, literally it is two cabinets of equipment(keeping in mind we were 100% on prem at the time, so we had about 40 racks of gear already). But they were as confused how we could run this stuff as my management was confused why their costs were so high.
Company I was at later agreed to my $800k plan, only to rip that budget away a month later to give to another massively underfunded project that the VP budgeted numbers from his ass again without consulting anyone, I left very quickly during that, actually interviewed the person who ended up replacing me(solid mid level tech, no mission critical experience), would of paid good money to see the look on his face on his first day when he was told he had the keys to the kingdom as I had left already.
thanks, I have had others tell me the same, just don't have the passion for that stuff that I once did(though it's still my job), most of the "exciting" new things in tech don't excite me at all so ran out of things to write about. Had a good run though, when I started it was with 3 other people and they all quit within 6 months, I went on for about 6-7 years with a few hundred posts.
It takes someone with the security and authority of working in the C suite to challenge anything regarding The Cloud (hallowed be it's name).
At any other level, individuals re potentially committing Career Suicide if/when they point problems with spend, pace, and not-so-recent 'discovery' of not just repatriation costs, but storage 'per-click' access costs for data a company once believed it owned.
I once read about this guy who made a programming mistake running his app on AWS over the weekend and ended up getting a $60.000 bill. There are no spending limits on AWS it seems and you have to make sure manually you don't exceed your self-imposed spending limit.
This is why I completely deleted my AWS account after I was done using it. I needed to create the account because Amazon required one for me to use one of their programming tools. Since it was registered to me (and not the company I worked for as a contractor) I was scared shitless someone would steal my credentials and rake up a huge bill.
Very specious argument against cloud usage, you could say the same about a car, a chainsaw, a kitchen knife and fork! Anything is dangerous and easy for it to get out of control if you're not careful with it.
Don't get me wrong I'm totally a "horses for courses" guy, I think cloud and on-prem have pros and cons, you need to find what works for you but to run from AWS and hide under the bed like a frightened 8 year old seems a little daft. Use some common sense.
We use AWS where I am and
- There are no IAM accounts, it's all controlled by our AD group roles.
- Only 10 admins out of 150 IT people have AWS access and they're granted access by infosec/CTO only
- All AWS admin accounts use 2FA on company phones only
- We have processes that check every 30 mins for anything that spends more than X amount on a resource and it reports to all admins, infosec and all IT managers if overspend happens. So it can be instantly shut down if something is fired up by accident.
- We only ever deploy using scripted Terraform builds. the builds are scanned every 24 hours to ensure they've not been manually adjusted. if so, again reports
- Every week we have a meeting to go over all AWS an Azure spend in all areas, 1 hour a week by 6 people to check and look for savings where we can.
- We shut down all dev/test resources overnight automatically to avoid any extra unwanted spend. We even shutdown some prod kit at weekends to avoid over spend.
AWS is a like a chainsaw, you use it and use ti carefully with the right tools and safety kit.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
