Real-time data streaming involves collecting and ingesting a sequence of data from various data sources and processing that data in real time to extract meaning and insight. Real-time data streaming enables you to analyze and process data in real time instead of waiting hours, days, or weeks to get answers.","sortDate":"2023-11-09","headlineUrl":"https://aws.amazon.com/what-is/real-time-data-streaming/?trk=faq_card","id":"faq-hub#what-is-real-time-data-streaming","category":"Analytics","primaryCTA":"https://portal.aws.amazon.com/gp/aws/developer/registration/index.html?pg=what_is_header","headline":"What is Real Time Data Streaming? "},"metadata":{"tags":[{"id":"GLOBAL#tech-category#analytics","name":"Analytics","namespaceId":"GLOBAL#tech-category","description":"Analytics","metadata":{}},{"id":"faq-hub#faq-type#what-is","name":"what-is","namespaceId":"faq-hub#faq-type","description":"

what-is","metadata":{}}]}}]},"metadata":{"auth":{},"testAttributes":{}},"context":{"page":{"pageUrl":"https://aws.amazon.com/what-is/real-time-data-streaming/"},"environment":{"stage":"prod","region":"us-east-1"},"sdkVersion":"1.0.129"},"refMap":{"manifest.js":"289765ed09","what-is-header.js":"2e0d22c000","what-is-header.rtl.css":"ccf4035484","what-is-header.css":"ce47058367","what-is-header.css.js":"004a4704e8","what-is-header.rtl.css.js":"f687973e4f"},"settings":{"templateMappings":{"category":"category","headline":"headline","primaryCTA":"primaryCTA","primaryCTAText":"primaryCTAText","primaryBreadcrumbText":"primaryBreadcrumbText","primaryBreadcrumbURL":"primaryBreadcrumbURL"}}}

What is Real Time Data Streaming?


Create an AWS Account
Explore Free Analytics Offers
View free offers for Analytics services in the cloud
Check out Analytics Services
Innovate faster with the most comprehensive set of Analytics services
Browse Analytics Trainings
Get started on Analytics training with content built by AWS experts
Read Analytics Blogs
Read about the latest AWS Analytics product news and best practices

Real-time data streaming involves collecting and ingesting a sequence of data from various data sources and processing that data in real time to extract meaning and insight. \n

Examples of streaming data are log files generated by customers using your mobile or web applications, ecommerce purchases, in-game player activity, information from social networks, financial trading floors, or geospatial services, and telemetry from connected devices or instrumentation in data centers. \n

Real-time data streaming enables you to analyze and process data in real time instead of waiting hours, days, or weeks to get answers. \n

Read about streaming data »","id":"seo-faq-pairs#what-is-real-time-data-streaming","customSort":"1"},"metadata":{"tags":[{"id":"seo-faq-pairs#faq-collections#real-time-data-streaming","name":"real-time-data-streaming","namespaceId":"seo-faq-pairs#faq-collections","description":"

real-time-data-streaming","metadata":{}}]}},{"fields":{"faqQuestion":"What are the components of real-time data streaming?","faqAnswer":"

\"\" \n

Source:  Up to hundreds and thousands of devices or applications that are producing high volumes of continuous data at a high velocity. Examples are mobile devices, web applications (clickstream), application logs, IoT sensors, smart devices and gaming applications.  \n

Stream Ingestion: Simple integration with over 15 AWS services (Amazon API Gateway, AWS IoT Core, Amazon Cloudwatch, and more) that enables you to capture continuous data being produced from thousands of devices in a durable and secure manner. \n

Stream Storage: Choose a solution that meets your storage needs based on scaling, latency, and processing requirements like Amazon Kinesis Data Streams, Amazon Data Firehose, and Amazon Managed Streaming for Apache Kafka (Amazon MSK). \n

Stream Processing: Choose from a selection of services ranging from solutions that require just a couple of clicks to transform and deliver data continuously to a destination like Amazon Data Firehose, to powerful, custom-built, real-time applications and machine learning integration using services like Amazon Managed Service for Apache Flink and AWS Lambda. \n

Destination: Deliver streaming data to a selection of fully integrated data lakes, data warehouses, and analytics services for further analysis or long term storage, like Amazon S3, Amazon Redshift, Amazon OpenSearch Service, and Amazon EMR.","id":"seo-faq-pairs#components-data-streaming","customSort":"2"},"metadata":{"tags":[{"id":"seo-faq-pairs#faq-collections#real-time-data-streaming","name":"real-time-data-streaming","namespaceId":"seo-faq-pairs#faq-collections","description":"

real-time-data-streaming","metadata":{}}]}},{"fields":{"faqQuestion":"What are real-time data streaming use cases?","faqAnswer":"

Real-Time Data Movement \n

Streaming data from hundreds of thousands of devices and performing ETL transformations on high volumes of continuous, high velocity data in real-time allows users to analyze data as soon as it is produced, and then durably store the data in a data lake, data warehouse, or database for further analysis. \n

Learn how WalkMe tackle sequencing challenges as they stream data into a data lake \n

Learn how John Deere perform streaming ETL into their data lake in real-time \n

Real-Time Analytics \n

Analyze data as soon as it is produced and enable real-time decisions across an organization to capitalize on opportunities, enhance customer experiences, prevent networking failures, or update critical business metrics in real-time. \n

Logs: Capture, process and analyze logs from your applications in real-time.  \n

Read about how to analyze user behavior in real-time \n

Learn how to analyze and visualize your VPC network traffcing \n

See how to manage centralized Microsoft Exchange server logs \n

Read about log analytics » \n

Real-time updates: Engage with consumers, gamers, financial traders, and more by poviding real-time updates to critical decisoning metrics, offer reccomendations, and customer experiences.  \n

See how to use real-time gaming analytics to delight your gamers \n

Learn how to deliver trending, personalized news in real-time \n

Clickstream: Get a real-time view of the performance of your web content and user interaction with your applications and websites including user behavior, amount of time spent, popular content, and more.  \n

See how Hearst built a clickstream analytics solution to transmit and process 30 terabytes of data a day from 300+ websites worldwide \n

IoT: Connect to hundreds of thousands of IoT devices and collect, process, and analyze the streaming data in real-time.  \n

Learn how John Deere streams data from tens of thousands of farming vehicles in real-time to optimize food production \n

Event Stream Processing \n

Capture and repsond to events as they happen in real-time across multiple applications. The most common use cases are communication between hundreds of decoupled microservices and maintaining a system of record via Change Data Capture.  \n

Communication between decoupled microservices: When any micro-service is triggered, an event can be sent to a data stream in real-time, and other micro-services can ‘watch’ the stream to see if any event has occurred to trigger the required action.  \n

Learn how Lyft communicates between hundreds of micro-services in real-time \n

Change Data Capture: All changes to data across several applications and databases can be streamed to a central system of record in real-time.  \n

Learn how to load Change Data Capture data from relational databases \n

Learn how to stream Change Data Capture in real-time","id":"seo-faq-pairs#what-are-real-time-data-streaming-use-cases","customSort":"3"},"metadata":{"tags":[{"id":"seo-faq-pairs#faq-collections#real-time-data-streaming","name":"real-time-data-streaming","namespaceId":"seo-faq-pairs#faq-collections","description":"

real-time-data-streaming","metadata":{}}]}},{"fields":{"faqQuestion":"What streaming services are on AWS?","faqAnswer":"

AWS provides several options to work with real-time data streaming.  \n

Next Steps on AWS

Check out additional product-related resources
Innovate faster with the most comprehensive set of analytics services 
Sign up for a free account

Instant get access to the AWS Free Tier.

Sign up 
Start building in the console

Get started building in the AWS management console.

Sign in