Amazon S3 Storage Lens

S3 Storage Lens getting started tutorial

S3 Storage Lens delivers organization-wide visibility into object storage usage, activity trends, and makes actionable recommendations to optimize costs and apply data protection best practices. S3 Storage Lens is the first cloud storage analytics solution to provide a single view of object storage usage and activity across hundreds, or even thousands, of accounts in an organization, with drill-downs to generate insights at multiple aggregation levels. Drawing from more than 16 years of experience helping customers optimize their storage, S3 Storage Lens analyzes organization-wide metrics to deliver contextual recommendations that make it easy to take immediate action. \n

Amazon S3 is used to store large shared datasets across tens to hundreds of accounts and buckets, multiple Regions, and thousands of prefixes. With S3 Storage Lens, you can easily understand your storage and analyze it to detect outliers. S3 Storage Lens delivers more than 60 metrics (free metrics and advanced metrics) on S3 storage usage and activity to an interactive dashboard in the S3 Console. Free metrics are provided to all customers at no charge, while advanced metrics can be enabled for a monthly per object monitoring charge to receive additional metrics, prefix-level insights, custom metadata filtering, extended data retention, and the option to publish metrics to Amazon CloudWatch. For advanced metrics pricing details, visit the Amazon S3 pricing page.","id":"collection-text-media#text-media-s3-storage-lens-overview","mediaPosition":"right","mediaAltText":"Video: Maximizing storage cost visibility with Amazon S3 Storage Lens"},"metadata":{"tags":[]}}]},"metadata":{"auth":{},"testAttributes":{}},"context":{"page":{"locale":null,"site":null,"pageUrl":"https://aws.amazon.com/s3/storage-lens/","targetName":null,"pageSlotId":null,"organizationId":null,"availableLocales":null},"environment":{"stage":"prod","region":"us-east-1"},"sdkVersion":"1.0.115"},"refMap":{"manifest.js":"47864e0370","rt-text-media-collection.js":"07ca5af217","rt-text-media-collection.rtl.css":"6e99be202e","rt-text-media-collection.css":"a2f6e3274d","rt-text-media-collection.css.js":"364e78221d","rt-text-media-collection.rtl.css.js":"6e41cc5eeb"},"settings":{"templateMappings":{"hyperlinkText":"hyperlinkText","hyperlinkUrl":"hyperlinkUrl","heading":"heading","mediaAltText":"mediaAltText","mediaPosition":"mediaPosition","mediaUrl":"mediaUrl","subheader":"subheader","bodyContent":"bodyContent","videoOverlayTitle":"videoOverlayTitle","videoThumbnailUrl":"videoThumbnailUrl","videoPlayButtonText":"videoPlayButtonText","dark":"dark"}}}

Overview

S3 Storage Lens delivers organization-wide visibility into object storage usage, activity trends, and makes actionable recommendations to optimize costs and apply data protection best practices. S3 Storage Lens is the first cloud storage analytics solution to provide a single view of object storage usage and activity across hundreds, or even thousands, of accounts in an organization, with drill-downs to generate insights at multiple aggregation levels. Drawing from more than 16 years of experience helping customers optimize their storage, S3 Storage Lens analyzes organization-wide metrics to deliver contextual recommendations that make it easy to take immediate action.

Amazon S3 is used to store large shared datasets across tens to hundreds of accounts and buckets, multiple Regions, and thousands of prefixes. With S3 Storage Lens, you can easily understand your storage and analyze it to detect outliers. S3 Storage Lens delivers more than 60 metrics (free metrics and advanced metrics) on S3 storage usage and activity to an interactive dashboard in the S3 Console. Free metrics are provided to all customers at no charge, while advanced metrics can be enabled for a monthly per object monitoring charge to receive additional metrics, prefix-level insights, custom metadata filtering, extended data retention, and the option to publish metrics to Amazon CloudWatch. For advanced metrics pricing details, visit the Amazon S3 pricing page.

How S3 Storage Lens works

How S3 Storage Lens works

All Amazon S3 customers can access an interactive S3 Storage Lens dashboard in the S3 Console containing pre-configured views to visualize storage usage and activity trends. With S3 Storage Lens free metrics, you receive 28 metrics across various categories at the bucket level, and 14 days of historical data in the dashboard. You can filter your dashboard by summary, cost optimization, data protection, access management, performance, or events to analyze specific metrics for your intended use cases. In addition to the dashboard in the S3 console, you can export metrics in CSV or Parquet format to an S3 bucket of your choice for further use. To learn more about S3 Storage Lens, read the documentation.

Benefits for upgrading to Storage Lens Advanced metrics

Advanced metrics provide insights related to activity (such as request counts), deeper cost optimization (such as S3 Lifecycle rule counts), additional data protection (such as S3 Replication rule counts), and detailed status codes (such as 403 authorization errors).

Prefix Aggregation enables you to drill down to the prefix level to identify your largest, fastest growing, and most frequently accessed prefixes within a bucket.

With Storage Lens Advanced metrics, you get insights based on 15 months of historical data. For the optional metrics export, you can configure any retention period you wish.
 

With Storage Lens Advanced, you can access S3 Storage Lens metrics through Amazon CloudWatch to create alarms, or the CloudWatch API to send metrics to integrated observability partners.

With Storage Lens groups, you can view metrics filtered by object tag, prefix, suffix, age, or size. For example, you can filter metrics by object tag to identify your fastest-growing datasets, or visualize your storage based on object size and age to inform your storage archive strategy.  

Create custom filters with Storage Lens groups

Storage Lens groups aggregate metrics using custom filters based on object metadata. Storage Lens groups help you drill-down into characteristics of your data, such as distribution of objects by age, your most common file types, and more. This information helps you to better understand and optimize your S3 storage. With Storage Lens groups, you can view metrics filtered by object tag, prefix, suffix, age, or size. For example, you can filter metrics by object tag to identify your fastest-growing datasets, or visualize your storage based on object size and age to inform your storage archive strategy. To get started, you can create Storage Lens groups through the S3 console, CLI, or SDK, and attach them to your Storage Lens dashboards. View the documentation to learn more.

Customers

  • Upstox

    Upstox is a leading Indian discount broker who provides financial education and a digital platform for making investments to more than 11 million customers. 

    The path forward is to continue using Amazon S3 Storage Lens to maintain a closer pulse on our storage usage and further improve on cost efficiencies. With the S3 Storage Lens interactive dashboard, we can easily locate the S3 prefix hotspots where increases in cost happen and optimize them with the right retention policies and S3 storage class to further improve cost efficiencies.

    Indranil Chandra, Principal ML & Data Engineer - Upstox
    Read the case study »

  • Pinterest

    Pinterest is a visual discovery engine more than 400 million people use each month to find inspiration for their lives.

    S3 Storage Lens will enhance our visibility into our storage usage, and help us continuously optimize our storage costs. We use the organization-wide view to see aggregated storage usage for our teams across multiple regions, and then drill-down to understand storage growth on a bucket or even prefix level.
     
    Using S3 Storage Lens, we can then create different dashboards for different internal teams, so they self-serve to find ways to optimize storage costs by using a Lifecycle policy or transitioning data to a more cost-effective storage class.

    Yi Yin, Software Engineer - Pinterest

  • Snowflake

    Snowflake delivers the Data Cloud - where organizations mobilize data with near-unlimited scale, concurrency, and performance.

    The ability to use see metrics based on prefixes will be invaluable to us. We will have better insights into usage for particular prefixes including total storage, object counts, average object sizes and break down by current and non-current versions, which can help us debug and see the trends of operations in an economical way. This wasn't previously achievable before S3 Storage Lens.

    Ju-Yi Kuo, Senior Software Engineer - Snowflake

  • Veeva

    Veeva Systems Inc. is a leader in cloud solutions—including data, software, and services—for the global life sciences industry.

    S3 Storage Lens is very simple to setup and configure. We received good insights into our S3 usage, which allowed us to put into action plans to optimize costs in S3. We are now saving thousands of dollars a month by optimizing how we store objects in S3.

    Dan Soble, Vice President of Technical Operations - Veeva

  • Dropsuite

    Dropsuite helps organizations in 139 countries simply back up, recover, and protect their important business information. Its email backup solution currently backs up company data for more than one million individual users.

    Using S3 Storage Lens, we get a macro view with filters at the bucket and Region levels, leading to transformative architecture design conversations. We use insights from S3 Storage Lens to analyze compression algorithms, look at methods of storage, and make decisions about removing and managing data for customers. Without S3 Storage Lens, we wouldn’t have this granular usage data to examine, visualize, and act on. It also gives us visibility into customer data usage patterns. With such valuable information now available to us, we’ve been able to use Amazon S3 storage classes more effectively.

    Manjor Kalyanaraman, Chief Technology Officer - Dropsuite
    Read the case study »

  • DoorDash

    DoorDash connects consumers with their favorite local businesses in more than 25 countries across the globe. A born-in-the-cloud company, DoorDash hosts the majority of its infrastructure, including its commerce platform, on AWS.

    Once we decided to dig deeper into our Amazon S3 usage to identify efficiency opportunities to support future growth, we started our investigation using S3 Storage Lens with the advanced metrics and recommendations. We found the ability to analyze storage usage and activity patterns at an S3 bucket, prefix, and storage class level invaluable in identifying opportunities to optimize costs. Without S3 Storage Lens, it would have taken much more time to examine and visualize Amazon S3 usage, and to improve our storage unit economics.

    Levon Stepanian, Cloud FinOps Lead - DoorDash
    Read the blog »

Videos

Gain insights into storage usage at scale with Amazon S3 Storage Lens (51:07)
Amazon S3 Storage Lens - Demo (4:32)