Skip to content

Latest commit

 

History

History
109 lines (48 loc) · 4.02 KB

kinesis-data-stream.md

File metadata and controls

109 lines (48 loc) · 4.02 KB
Raw

Kinesis Data Stream 이란?

How it works

Amazon Kinesis Data Streams is a serverless streaming data service that makes it easy to capture, process, and store data streams at any scale.

image

사용 케이스

  1. Steam log and event data: Ingest and collect terabytes of data per day from application and service logs, clickstream data, sensor data, and in-app user events to power live dashboards, generate metrics, and deliver data into data lakes.

  2. Run real-time analytics: Build applications for high-frequency event data such as clickstream data, and gain access to insights in seconds, not days, using AWS Lambda or Amazon Kinesis Data Analytics.

  3. Power event-driven applications: Quickly pair with AWS Lambda to respond to or adjust immediate occurrences within the event-driven applications in your environment, at any scale.

Base structure

image

image

특징 : Easily stream data at any scale

  1. With Amazon Kinesis Data Streams, there are no servers to manage. The on-demand mode eliminates the need to provision or manage capacity required for running applications.

  2. Adjust your capacity to stream gigabytes per second of data with Kinesis Data Streams. Get automatic provisioning and scaling with the on-demand mode.

  3. Pay only for what you use with Kinesis Data Streams, starting as low as $0.015 per hour. With the on-demand mode, you don't need to worry about over-provisioning.

  4. Use built-in integrations with other AWS services to create analytics, serverless, and application integration solutions on AWS quickly.

Benefits of Stream storage

  • Decouple producers & consumers

  • Persistent buffer

  • Collect multiple streams

  • Preserve client ordering

  • Parallel consumption

Kafka 비교

image

image

PK: Partition Key 예) b8e0745459d9a885f1b30541e28e0e6c

[Kinesis Data Stream에 입력한 결과의 예

kinesisResponse: { 
"ShardId": "shardId-000000000000", "SequenceNumber": "49628968760419381815775185092326431149216009853023027202" 
}

Cases

당근마켓: DynamoDB 데이터 변경 이벤트 파이프라인 구축하기

image

AWS Kinesis Data Stream 으로 대용량 데이터 수집을 편하게

image

AWS - Kinesis Data Stream - Hands On! image

Capacity

Write capacity (Maximum)

200 MiB/second, 200,000 records/second

Read capacity (Maximum)

400 MiB/second

Reference

AWS re:Invent 2020: Top 5 best practices for data streaming with Amazon Kinesis

Amazon Kinesis Data Streams

Amazon Kinesis Data Streams와 MSK를 비교해 보기