Most AWS Analytics Customers Will Go Serverless, VP Says

(Timofeev-Vladimir./Shutterstock)

Serverless computing has become the latest battleground as cloud service providers compete to make their offerings as simple to use as possible. AWS is right in the game too, launching a slew of serverless offerings at the recent re:Invent conference. Eventually, most AWS customers will adopt serverless analytics services, in part because they’ll be less expensive to use, the company’s VP analytics said.

“We think a majority of them will absolutely” adopt serverless offerings, Rahul Pathak, the vice president of analytics for AWS, told Datanami at the conference two weeks ago. “People make changes at different rates, but we do think it will be compelling for most.”

Prior to re:Invent, AWS offered one serverless analytics service with Athena, its hosted Presto service. At the show, the cloud giant debuted several more, including serverless versions of its hosted Apache Kafka, Kinesis, Elastic MapReduce (EMR), and Redshift offerings. Among the products Pathak is responsible for, only the AWS service for Open Search (the Apache 2 licensed version of Elastic) is not available in a serverless offering. “We do have the most serverless options of any other provider,” he said.

With a serverless offering, customers no longer have to worry about choosing the right EC2 instance to run software in. They don’t even have to think about scaling the instances up or down. It’s all handled automatically with containers and the underlying container and resource management software that AWS has running under the covers.

Anything that takes decision and complexity away from customers is a good thing, Pathak said.

“As you can imagine, it requires customer to do less work and that’s always a win from a customer point of view,” he said. “Anything we can offload from them when it comes to the muck and the undifferentiated pieces of running infrastructure, the better. And I think the other piece is because were able to better dynamically match resources to workloads, most customer end up saving money. So that’s also welcome.”

There’s currently “a ton” of demand for serverless offerings at AWS, Pathak said. While there are a handful of workloads that don’t benefit much from running in a serverless manner, most customers are looking to cut down on the management work and drive up their cloud efficiency

“It’s as close to no work as we can make it,” Pathak said of AWS’s serverless offerings. “So a customer provisions an end point, say for Redshift Serverless. They optionally get to specifically min and max values that they want to scale between. That’s the way of sort of managing costs and budgets. The min can be zero. And then the system will automaticaly scale up and down. It will power itself down to zero if that’s what the customer has required, and it will spin back up in seconds.”

AWS is going serverless with its analytics offerings (Michael Vi/Shutterstock)

AWS and other public cloud providers have been criticized for not doing enough to fight the increased spending on cloud computing. The COVID-19 pandemic has been particularly beneficial to the cloud providers, as companies have accelerated the migration of workloads to the cloud–sometimes at the expense of data management (but that’s a different story).

That begs the question: Considering that AWS stands to lose revenue when it automatically scales down customer workloads with serverless services, will AWS charge a premium for its serverless offerings compared to non-serverless offerings? That answer is no, Pathak said. While the company may lose money on a per-workload basis, AWS stands to do better in the long run, he said.

“Actually, it’s good for us as well, because I think in the long term, the more aligned we are with customers, the better,” he said. “I think we found that, as you drive down the cost of doing work, customers find more work to do. So it’s a win for us in the long term.”

With the serverless offerings, AWS now offers three ways for customers to consume analytics services: running on EC2 instances, running atop Kubernetes, and now serverless. That gives customers more flexibility in how they want to run their analytics services. “Kubernetes is one way to run infrastructure. Different teams make different decisions,” Pathak said.

Pathak declined to provide technical specifics of how AWS’s serverless offering runs, or whether it’s using Kubernetes, which is quickly becoming a popular way to run big data applications in containers.

“We don’t really share the specifics of the underlying architecture. From a customer perspective, all that’s completely abstracted,” he said. “Nothing customers need to think about. It’s really just an end point where you can submit work and connect to using standard tools that you’re already familiar with, and then you get to decide how much of it you want to consume.”

Kubernetes will likely continue as an option for customers running customized versions of EMR, Pathak said.

“EMR is an interesting one because there’s a lot of customizability possible,” he said. “Customer may choose to completely modify what’s running and in those scenarios, I suspect there will be some customers who will stick with instances or Kubernetes. But the majority I suspect will move.”

Related Items:

Why AWS Keeps It Simple

Cloud Getting Expensive? That’s By Design, But Don’t Blame the Clouds

AWS Rocks with New Analytics, AI Services at re:Invent