[B! kinesis] kakku22ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

kakku22 id:kakku22

kinesisã«é–¢ã™ã‚‹kakku22ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (13)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

AWS Lambda ã¯ Kinesis ãŠã‚ˆã³ DynamoDB ã‚¤ãƒ™ãƒ³ãƒˆã‚½ãƒ¼ã‚¹ã®ä¸¦åˆ—åŒ–ä¿‚æ•°ã‚’ã‚µãƒãƒ¼ãƒˆã—ã¾ã™
AWS Lambda ã¯ã€ä¸¦åˆ—åŒ–ä¿‚æ•°ã‚’ã‚µãƒãƒ¼ãƒˆã™ã‚‹ã‚ˆã†ã«ãªã‚Šã¾ã—ãŸã€‚ã“ã®æ©Ÿèƒ½ã«ã‚ˆã‚Šã€Kinesis ã¾ãŸã¯ DynamoDB ãƒ‡ãƒ¼ã‚¿ã‚¹ãƒˆãƒªãƒ¼ãƒ ã® 1 ã¤ã®ã‚·ãƒ£ãƒ¼ãƒ‰ã‚’è¤‡æ•°ã® Lambda å‘¼ã³å‡ºã—ã§åŒæ™‚ã«å‡¦ç†ã§ãã¾ã™ã€‚ã“ã®æ–°æ©Ÿèƒ½ã«ã‚ˆã‚Šã€æ®ç™ºæ€§ã®ãƒ‡ãƒ¼ã‚¿ãƒˆãƒ©ãƒ•ã‚£ãƒƒã‚¯ã§ã‚ˆã‚Šæ©Ÿæ•ãªã‚¹ãƒˆãƒªãƒ¼ãƒ å‡¦ç†ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã‚’æ§‹ç¯‰ã§ãã¾ã™ã€‚ ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆã§ã¯ã€Lambda ã¯ä¸€åº¦ã« 1 ã¤ã®ã‚·ãƒ£ãƒ¼ãƒ‰ã‹ã‚‰ãƒ‡ãƒ¼ã‚¿ãƒ¬ã‚³ãƒ¼ãƒ‰ã® 1 ã¤ã®ãƒãƒƒãƒã§é–¢æ•°ã‚’å‘¼ã³å‡ºã—ã¾ã™ã€‚å˜ä¸€ã®ã‚¤ãƒ™ãƒ³ãƒˆã‚½ãƒ¼ã‚¹ãƒžãƒƒãƒ”ãƒ³ã‚°ã®å ´åˆã€åŒæ™‚ Lambda å‘¼ã³å‡ºã—ã®æœ€å¤§æ•°ã¯ã€Kinesis ã¾ãŸã¯ DynamoDB ã‚·ãƒ£ãƒ¼ãƒ‰ã®æ•°ã«ç‰ã—ããªã‚Šã¾ã™ã€‚ ã“ã‚Œã§ã€Lambda ãŒ 1 (ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆ) ã‹ã‚‰ 10 ã¾ã§ã®ä¸¦åˆ—åŒ–ä¿‚æ•°ã‚’ä»‹ã—ã¦ã‚·ãƒ£ãƒ¼ãƒ‰ã‹ã‚‰ãƒãƒ¼ãƒªãƒ³ã‚°ã™ã‚‹åŒæ™‚ãƒãƒƒãƒã®æ•°ã‚’æŒ‡å®šã§ãã¾ã™ã€‚ãŸã¨ãˆã°ã€ä¸¦åˆ—åŒ–ä¿‚æ•°ã‚’ 2 ã«è¨å®šã™ã‚‹ã¨ã€100 ä»¶ã® Kinesis ãƒ‡ãƒ¼
kakku22 2020/08/14
aws

kinesis

lambda
ãƒªãƒ³ã‚¯
[ã‚¢ãƒƒãƒ—ãƒ‡ãƒ¼ãƒˆ] 1è¡ŒãŸã‚Šã¨ã‚‚ã‚³ãƒ¼ãƒ‰ã¯æ›¸ã‹ãªã„ï¼AWS Glueã§ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ETLãŒå¯èƒ½ã«ãªã‚Šã¾ã—ãŸ | DevelopersIO
å…ˆæ—¥ã®ã‚¢ãƒƒãƒ—ãƒ‡ãƒ¼ãƒˆã§ AWS Glue ãŒã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚° ETL ã‚’ã‚µãƒãƒ¼ãƒˆã™ã‚‹ã‚ˆã†ã«ãªã‚Šã¾ã—ãŸï¼ AWS Glue now supports serverless streaming ETL New â€“ Serverless Streaming ETL with AWS Glue ãªã«ãŒå¬‰ã—ã„ã®ã‹ ã¾ãšã€AWS Glueï¼ŸãƒŠãƒ‹ã‚½ãƒ¬ï¼Ÿã¨ã„ã†æ–¹ã¯ã€æ˜¯éžã€ä»¥ä¸‹ã®å†å…¥é–€ãƒ–ãƒã‚°ã‚’ãŠèªã¿ãã ã•ã„ã€‚ AWS Glue ã¯ãƒ•ãƒ«ãƒžãƒãƒ¼ã‚¸ãƒ‰ã§ã‚µãƒ¼ãƒãƒ¼ãƒ¬ã‚¹ãª ETLï¼ˆæŠ½å‡º/å¤‰æ›/ãƒãƒ¼ãƒ‰ï¼‰ã‚µãƒ¼ãƒ“ã‚¹ã§ã™ãŒã€ã“ã‚Œã¾ã§ãƒ‡ãƒ¼ã‚¿ã‚½ãƒ¼ã‚¹ã¨ã—ã¦ã¯ S3ã€RDSã€Redshiftãƒ»ãƒ»ãƒ»etc ã¨ã„ã£ãŸãƒ‡ãƒ¼ã‚¿ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã®ã¿ãŒå¯¾è±¡ã§ã—ãŸã€‚ãã®ãŸã‚ã€Kinesis Data Streams ã‚„ Apache Kafka ã¨ã„ã£ãŸã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ‡ãƒ¼ã‚¿ã«å¯¾ã—ã¦ Glue ã‚’çµ„ã¿è¾¼ã‚“ã ETL å‡¦ç†ã¯å‡ºæ¥ã¾ã›ã‚“ã§ã—ãŸã€‚ å¾“æ¥ã§ã‚ã‚Œã°ã€ä¾‹ãˆ
kakku22 2020/04/30
aws

glue

spark

kinesis
ãƒªãƒ³ã‚¯
Choosing the Right Messaging Service for Your Distributed App (API305) - AWS re:Invent 2018
kakku22 2020/04/03
äº‹ä¾‹ã”ã¨ã«ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ã‚µãƒ¼ãƒ“ã‚¹ã‚’ã©ã†ä½¿ã„åˆ†ã‘ã‚‹ã‹ï¼Ÿä»•çµ„ã¿ã‚‚ Dive Deep ã—ã¦ã¦ã‹ãªã‚Šè‰¯ã„ã‚»ãƒƒã‚·ãƒ§ãƒ³ï¼

aws

reinvent

sqs

sns

kinesis
ãƒªãƒ³ã‚¯
Amazon Kinesis Data Generatorã‚’ä½¿ç”¨ã—ã¦ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ‡ãƒ¼ã‚¿ã‚½ãƒªãƒ¥ãƒ¼ã‚·ãƒ§ãƒ³ã‚’ãƒ†ã‚¹ãƒˆã™ã‚‹ | Amazon Web Services
Amazon Web Services ãƒ–ãƒã‚° Amazon Kinesis Data Generatorã‚’ä½¿ç”¨ã—ã¦ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ‡ãƒ¼ã‚¿ã‚½ãƒªãƒ¥ãƒ¼ã‚·ãƒ§ãƒ³ã‚’ãƒ†ã‚¹ãƒˆã™ã‚‹ ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ‡ãƒ¼ã‚¿ã‚½ãƒªãƒ¥ãƒ¼ã‚·ãƒ§ãƒ³ã‚’æ§‹ç¯‰ã™ã‚‹å ´åˆã€ã»ã¨ã‚“ã©ã®ãŠå®¢æ§˜ã¯ã€æœ¬ç•ªãƒ‡ãƒ¼ã‚¿ã¨åŒæ§˜ã®ãƒ‡ãƒ¼ã‚¿ã‚’ä½¿ç”¨ã—ã¦ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ‡ãƒ¼ã‚¿ã‚½ãƒªãƒ¥ãƒ¼ã‚·ãƒ§ãƒ³ã‚’ãƒ†ã‚¹ãƒˆã—ãŸã„ã¨è€ƒãˆã¦ã„ã¾ã™ã€‚ã“ã®ã€ãƒ‡ãƒ¼ã‚¿ã‚’ä½œæˆã—ã¦ã‚½ãƒªãƒ¥ãƒ¼ã‚·ãƒ§ãƒ³ã«ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ã™ã‚‹ã“ã¨ã¯ã€ã‚½ãƒªãƒ¥ãƒ¼ã‚·ãƒ§ãƒ³ã‚’ãƒ†ã‚¹ãƒˆã™ã‚‹éš›ã®æœ€ã‚‚é€€å±ˆãªä½œæ¥ã‹ã‚‚ã—ã‚Œã¾ã›ã‚“ã€‚ Amazon Kinesis Streamsã¨Amazon Kinesis Firehoseã‚’ä½¿ç”¨ã™ã‚‹ã¨ã€æ•°åä¸‡ã®ã‚½ãƒ¼ã‚¹ã‹ã‚‰1æ™‚é–“ã«ãƒ†ãƒ©ãƒã‚¤ãƒˆç´šã®ãƒ‡ãƒ¼ã‚¿ã‚’é€£ç¶šçš„ã«æ‰ãˆã¦ä¿å˜ã§ãã¾ã™ã€‚ Amazon Kinesis Analyticsã§ã¯ã€æ¨™æº–SQLã‚’ä½¿ç”¨ã—ã¦ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ã§ã“ã®ãƒ‡ãƒ¼ã‚¿ã‚’åˆ†æžãŠã‚ˆã³é›†è¨ˆã™ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ AWS Managem
kakku22 2020/01/21
KDG ä¾¿åˆ©

aws

kinesis
ãƒªãƒ³ã‚¯
Amazon Kinesis Data Analytics ãŒæ±äº¬ãƒªãƒ¼ã‚¸ãƒ§ãƒ³ã§åˆ©ç”¨å¯èƒ½ã«ãªã‚Šã¾ã—ãŸ | Amazon Web Services
Amazon Web Services ãƒ–ãƒã‚° Amazon Kinesis Data Analytics ãŒæ±äº¬ãƒªãƒ¼ã‚¸ãƒ§ãƒ³ã§åˆ©ç”¨å¯èƒ½ã«ãªã‚Šã¾ã—ãŸ 2019å¹´5æœˆã€Amazon Kinesis Data Analytics ãŒæ±äº¬ãƒªãƒ¼ã‚¸ãƒ§ãƒ³ã§åˆ©ç”¨ã§ãã‚‹ã‚ˆã†ã«ãªã‚Šã¾ã—ãŸã€‚ ã“ã‚Œã¾ã§ã€æ±äº¬ãƒªãƒ¼ã‚¸ãƒ§ãƒ³ã«ã‚ã‚‹ãƒ‡ãƒ¼ã‚¿ã‚½ãƒ¼ã‚¹ï¼ˆAmazon Kinesis Data Streams ã¾ãŸã¯ Amazon Kinesis Data Firehoseï¼‰ã‚’ Kinesis Data Analytics ã«æŽ¥ç¶šã™ã‚‹ãŸã‚ã«ã¯ã€ãƒªãƒ¼ã‚¸ãƒ§ãƒ³ã‚’è·¨ã„ã§æŽ¥ç¶šã™ã‚‹å¿…è¦ãŒã‚ã‚Šã¾ã—ãŸã€‚Kinesis Data Analytics ãŒæ±äº¬ãƒªãƒ¼ã‚¸ãƒ§ãƒ³ã§åˆ©ç”¨å¯èƒ½ã¨ãªã£ãŸã“ã¨ã«ã‚ˆã‚Šã€æ—¥æœ¬å›½å†…ã«ã‚ã‚‹ãƒ‡ãƒ¼ã‚¿ã‚½ãƒ¼ã‚¹ã¨æŽ¥ç¶šã—ã‚„ã™ããªã‚Šã€ã‚¹ãƒˆãƒªãƒ¼ãƒ ãƒ‡ãƒ¼ã‚¿åˆ†æžã®ç’°å¢ƒã‚’ã‚ˆã‚Šã‚·ãƒ³ãƒ—ãƒ«ã«æ§‹ç¯‰ã§ãã‚‹ã‚ˆã†ã«ãªã‚Šã¾ã—ãŸã€‚ Kinesis Data Analyt
kakku22 2019/07/10
aws

kinesis
ãƒªãƒ³ã‚¯
20171019_aws_startup_tech
ãƒ¦ãƒ¼ ã‚¶ãƒ¼ ã®è¡Œå‹•ãƒã‚°ã‚’SQL ã§åˆ†æž ex: Action A ã‹ã‚‰ Action B ã¾ã§ã®æ™‚é–“ã‚’ç®—å‡ºã™ã‚‹SQL SELECT "operator_name", "log_date", "operatability", "total_elapsed_time" FROM (SELECT operator_name, date(created_at) AS log_date, operatability, sum(elapsed_time) AS total_elapsed_time FROM (SELECT operator_name, created_at, lag(operatability) over(partition by operator_name, date(created_at) COALESCE(date_diff('minute', lag(created_at)
kakku22 2017/10/23
Kinesis Firehose äº‹ä¾‹ã ãƒ¼

aws

kinesis
ãƒªãƒ³ã‚¯
ã‚¹ã‚±ãƒ¼ãƒ©ãƒ–ãƒ« tail -f | grep on AWS - KAYAC engineers' blog
ã“ã‚“ã«ã¡ã¯ã€‚çµ„é•·ã“ã¨ @fujiwara (id:sfujiwara) ã§ã™ã€‚ æ—©é€Ÿã§ã™ãŒçš†ã•ã‚“ã€ãƒã‚°ã¯è¦‹ã¦ã„ã¾ã™ã‹? å½“ç„¶è¦‹ã¦ã„ã¾ã™ã‚ˆãã€‚tail -f ã§è¿½ã„ã‹ã‘ã‚‹ã¨ã‹å¤§å¥½ãã§ã™ã‚ˆãã€‚ã¨ã„ã†å‰æã§è©±ã‚’é€²ã‚ã¾ã—ã‚‡ã†ã€‚ ã“ã®è¨˜äº‹ã¯ Tech KAYAC Advent Calendar 2016 ã®24æ—¥ç›®ã®è¨˜äº‹ã§ã™ã€‚ TL;DR å¤šç¨®å¤šæ§˜ãªãƒ›ã‚¹ãƒˆã‹ã‚‰ç™ºç”Ÿã™ã‚‹ãƒã‚°ã®æœ«å°¾ã‚’ tail -f ã§è¿½ã„ã‹ã‘ãŸã„ fluentd + ãƒãƒ¼ã‚«ãƒ«ãƒ•ã‚¡ã‚¤ãƒ«ã§è¡Œã†ã®ã¯ã¡ã‚‡ã£ã¨é¢å€’ Amazon Kinesis + Lambda ã‚’ä½¿ã£ã¦ã‚¹ã‚±ãƒ¼ãƒ©ãƒ–ãƒ«ãªä»•çµ„ã¿ã‚’è€ƒãˆã¦ã¿ãŸ kinesis-tailf ã¨ã„ã†ãƒ„ãƒ¼ãƒ«ã‚‚ä½œã£ãŸ æ—¢å˜ã®ä»•çµ„ã¿ã¨å•é¡Œç‚¹ tech blog.kayac.com å¼Šç¤¾ã® Lobi ã¨ã‚½ãƒ¼ã‚·ãƒ£ãƒ«ã‚²ãƒ¼ãƒ ã§ã¯ã€ã“ã®è¨˜äº‹ã§ç´¹ä»‹ã•ã‚ŒãŸã‚ˆã†ãª Fluentd ã‚’æ´»ç”¨ã—ãŸãƒã‚°é›†ç´„ã®ä»•çµ„ã¿ã‚’æ•´å‚™ã—ã¦ã„ã¾ã™ã€‚ã“ã®
kakku22 2016/12/24
ã“ã‚Œã¯ã™ã”ã„

aws

kinesis

lambda

golang
ãƒªãƒ³ã‚¯
ç§’é–“æ•°ä¸‡ã®ãƒã‚°ã‚’ã„ã„æ„Ÿã˜ã«ã™ã‚‹ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£
AWS Summit Tokyo 2016 Developer Conference (2016/06/03)
kakku22 2016/11/29
ã‚¹ã‚±ãƒ¼ãƒ©ãƒ–ãƒ«ã§è‰¯ã„ã‘ã©ï¼Œçµæ§‹ã‚³ã‚¹ãƒˆã‚‚æŽ›ã‹ã‚Šãã†ãªæ§‹æˆã

cookpad

kinesis

lambda

log
ãƒªãƒ³ã‚¯
Amazon Kinesis Analyticsã¨ES/Kibana4ã§ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ãƒ€ãƒƒã‚·ãƒ¥ãƒœãƒ¼ãƒ‰æ§‹ç¯‰ - Gunosyãƒ‡ãƒ¼ã‚¿åˆ†æžãƒ–ãƒã‚°
ã“ã‚“ã«ã¡ã¯ã€‚é–‹ç™ºãƒ»é‹ç”¨æŽ¨é€²éƒ¨ã®å°å‡ºã§ã™ã€‚ çŒ«ã«ã‚¤ãƒ¤ãƒ›ãƒ³ã‚’å™›ã¿åˆ‡ã‚‰ã‚Œã‚‹ã“ã¨æ•°å›žã€æœ€è¿‘ã®BGMã¯ã‚‚ã£ã±ã‚‰ç’°å¢ƒéŸ³ã§ã™ã€‚ ã€Œè€³ã‹ã‚‰ã†ã©ã‚“ãŒå‡ºã¦ã„ã‚‹ã‚ˆã†ã«ã—ã‹è¦‹ãˆãªã„ã€ã¨ã„ã†å™‚ã®Bluetoothã‚¤ãƒ¤ãƒ›ãƒ³ãŒæ°—ã«ãªã£ã¦ã„ã¾ã™ã€‚ ä»Šå›žã¯ã€Amazon Kinesis Analyticsã¨Elasticsearch/Kibana4ã‚’åˆ©ç”¨ã—ãŸãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ãƒ€ãƒƒã‚·ãƒ¥ãƒœãƒ¼ãƒ‰æ§‹ç¯‰ã«ã¤ã„ã¦ã§ã™ã€‚ Amazon Kinesis Analytics ã¨ã¯ ãƒ€ãƒƒã‚·ãƒ¥ãƒœãƒ¼ãƒ‰ã‚’æ§‹ç¯‰ã—ã¦ã¿ã‚‹ Source Streamã¨Mapping Query Destination ãƒã‚°ãƒ‡ãƒ¼ã‚¿ã‚’æ‹¡å……ã™ã‚‹ Reference Dataã¨Mapping Query Destination ã¾ã¨ã‚ ãŠã¾ã‘ï¼šAmazonES&Kibana4ã®ãƒ€ãƒƒã‚·ãƒ¥ãƒœãƒ¼ãƒ‰å…±æœ‰ Amazon Kinesis Analytics ã¨ã¯ Amazon Kinesis A
kakku22 2016/11/29
ã»ã»ã†

aws

kinesis
ãƒªãƒ³ã‚¯
Fluentd&Kinesis&Lambdaã«ã‚ˆã‚‹æŸ”è»Ÿã§é«˜å¯ç”¨æ€§ãªãƒã‚°åŽé›†åŸºç›¤ã®æ§‹ç¯‰ - Qiita
æ¦‚è¦ Fluentdã«ã‚ˆã‚‹ãƒã‚°åŽé›†åŸºç›¤ã‚’ã€Lambda&Kinesisã‚’åˆ©ç”¨ã—ãŸå½¢ã§æ§‹æˆã‚’å¤‰æ›´ã—ãŸã®ã§ã€ãã®æ™‚ã®ãƒ¡ãƒ¢ã€‚ å…ˆã«ä¸€è¨€ã§æ„Ÿæƒ³ã‚’è¿°ã¹ã‚‹ã®ã§ã‚ã‚Œã°ã€å··ã®å™‚é€šã‚Šã€Kinesis&Lambdaã¯ç´ æ•µã§ã—ãŸã€‚ Fluentd AggregatorãŒæŠ±ãˆã‚‹å•é¡Œç‚¹ ãƒã‚°ãƒ‡ãƒ¼ã‚¿ã‚’Fluentdã®Forwarder & Aggregatorã‚’çµŒç”±ã—ã€S3ã¨Redshiftã«ä¿å˜ã™ã‚‹ã‚ˆã†ãªæ§‹æˆã§ã—ãŸãŒã€Aggregatorã¯å†—é•·åŒ–ã—ã¥ã‚‰ããƒœãƒˆãƒ«ãƒãƒƒã‚¯ã¨ãªã£ã¦ã„ã¾ã—ãŸã€‚ ãƒã‚°åŽé›†ã«æ±‚ã‚ã‚‹è¦ä»¶ ãƒã‚°åŽé›†ã®æ§‹æˆã‚’æ”¹å–„ã™ã‚‹ã«ã‚ãŸã‚Šã€å…ˆã«æŒ™ã’ãŸç›®ã®å‰ã®å•é¡Œç‚¹ã‚’è§£æ±ºã™ã‚‹ã®ã¯å‹¿è«–ã§ã™ãŒã€ä»–ã«ã‚‚æ§˜ã€…ãªè¦ä»¶ãŒã‚ã‚Šã€å¤§åˆ¥ã™ã‚‹ã¨ä»¥ä¸‹4ã¤ã€‚ ä¿¡é ¼æ€§: ãƒ‡ãƒ¼ã‚¿ã‚’ãƒã‚¹ãªãåŽé›†ã™ã‚‹ã“ã¨ãŒã§ãã‚‹ã€‚ å®‰å®šæ€§: ãƒã‚°é‡ã«é–¢ä¿‚ãªãå®‰å®šçš„ãªãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹ã‚’ç¶æŒã§ãã‚‹ã€‚ å†—é•·æ€§: å®‰å…¨ã‹ã¤ç°¡å˜ã«ã‚¹ã‚±ãƒ¼ãƒªãƒ³ã‚°ãŒå¯èƒ½ æ±Žç”¨æ€§: è¤‡æ•°ãƒã‚°ã‚’è¤‡æ•°ã‚µ
kakku22 2016/10/13
è‰¯ã„

aws

kinesis

lambda

fluentd
ãƒªãƒ³ã‚¯
Rubyã‹ã‚‰Amazon Kinesisã‚’æ“ä½œã™ã‚‹ - Tech-Sketch
æ˜¨å¹´11æœˆã€ç±³ãƒ©ã‚¹ãƒ™ã‚¬ã‚¹ã§é–‹å‚¬ã•ã‚ŒãŸã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ã€ŒAWS re:Invent 2013ã€ã§ã€ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ‡ãƒ¼ã‚¿ã‚’ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ã«å‡¦ç†ã™ã‚‹ã‚µãƒ¼ãƒ“ã‚¹ã§ã‚ã‚‹ã€ŒAmazon Kinesisã€ãŒç™ºè¡¨ã•ã‚Œã¾ã—ãŸã€‚ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ‡ãƒ¼ã‚¿ã‚’ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ã«å‡¦ç†...ã¨ã€ãªã«ã‚„ã‚‰é›£ã—ãã†ãªæ„Ÿã˜ãŒã—ã¾ã™ãŒã€APIã€SDKãŒå…¬é–‹ã•ã‚Œã¦ã„ã‚‹ã®ã§ç°¡å˜ã«è©¦ã™ã“ã¨ãŒã§ãã¾ã™ã€‚ä»Šå›žã¯Rubyå‘ã‘AWS SDKã‚’åˆ©ç”¨ã—ã€Amazon Kinesisã‚’æ“ä½œã—ã¦ã¿ã¾ã™ã€‚ Amazon Kinesisã¨ã¯ Amazon Kinesisã¯ã€Amazon Web ServicesãŒæä¾›ã™ã‚‹ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ‡ãƒ¼ã‚¿ã‚’ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ã«å‡¦ç†ã™ã‚‹ç‚ºã®ã‚µãƒ¼ãƒ“ã‚¹ã§ã™ã€‚è¿‘å¹´ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã®å‡¦ç†ã‚„åˆ†æžãŒæ³¨ç›®ã•ã‚Œã¦ãã¾ã—ãŸãŒã€ãƒãƒƒãƒå‡¦ç†ãŒä¸»æµã§ã—ãŸã€‚Amazon Kinesisã‚’åˆ©ç”¨ã™ã‚‹ã¨ã€ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ã«ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ‡ãƒ¼ã‚¿ã®å‡¦ç†ã‚„åˆ†æžãŒå¯èƒ½ã§ã™ã€‚ã¾ãŸã€Ama
kakku22 2015/06/16
aws

kinesis
ãƒªãƒ³ã‚¯
Kinesisã«ãƒªã‚¯ã‚¨ã‚¹ãƒˆã‚’æµã—ã¦Lambdaã‹ã‚‰S3ä¸Šã®JSã‚’å‹•çš„ã«å®Ÿè¡Œã™ã‚‹ - Qiita
æ¦‚è¦ S3 ã« JavaScript ã®ãƒ¢ã‚¸ãƒ¥ãƒ¼ãƒ«ã‚’ç½®ãã€ãã®ãƒã‚±ãƒƒãƒˆåã¨ã‚ªãƒ–ã‚¸ã‚§ã‚¯ãƒˆã‚ãƒ¼ã‚’ Kinesis ã‚¹ãƒˆãƒªãƒ¼ãƒ ã«æµã™ã“ã¨ã§ Lambda ã‹ã‚‰å‹•çš„ã« JS ã‚’å‘¼ã³å‡ºã—å®Ÿè¡Œã—ã¦ã¿ã‚‹ã€‚ ä½•ã«ä½¿ã†ã‚“ã§ã—ã‚‡ã†ãã€‚ ã‚µãƒ³ãƒ—ãƒ«ã‚³ãƒ¼ãƒ‰ ãƒãƒ¼ã‚«ãƒ«ã«æ›¸ãå‡ºã—ãŸãƒ•ã‚¡ã‚¤ãƒ«åã« context.invokeid ã‚’ä½¿ã†ã“ã¨ã§ require ã«ã‚ãƒ£ãƒƒã‚·ãƒ¥ã•ã‚Œã‚‹ã“ã¨ã¯ãªããªã£ãŸã€‚ ã¤ã¾ã‚Šé…ã„ã€‚ ã¤ã¾ã‚Šã‚¹ãƒ†ãƒ¼ãƒˆãƒ¬ã‚¹ã¨è¨€ã£ã¦ã„ã‚‹ã‘ã© require ã®çµæžœã¯ã‚ãƒ£ãƒƒã‚·ãƒ¥ã•ã‚Œã¦ã„ã‚‹ã¿ãŸã„ã€‚ Lambda function var fs = require('fs'); var aws = require('aws-sdk'); var s3 = new aws.S3({apiVersion: '2006-03-01'}); exports.handler = function(event, context) {
kakku22 2015/06/16
aws

kinesis
ãƒªãƒ³ã‚¯
AWS ç™½å¸¯ã‚·ãƒªãƒ¼ã‚ºï¼ˆ19ï¼‰Amazon Kinesis ãƒ¡ãƒ¢ | iret.media
loggly => Apache Kafka => Amazon Kinesis ã«æµã‚Œç€ã„ãŸã€cloudpack ã® ã‹ã£ã±ï¼ˆ@inokaraï¼‰ã§ã™ã€‚ã©ã†ã‚‚ã€‚ ã¯ã˜ã‚ã« Amazon Kinesis ã«ã¤ã„ã¦ä»¥ä¸‹ã®æ—¥æœ¬èªžè³‡æ–™ã‚’èªã¾ã›ã¦é ‚ãã¾ã—ãŸã®ã§ãã‚Œã‚’ã¾ã¨ã‚ã¾ã—ãŸã€‚ Amazon kinesisã§åºƒãŒã‚‹ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ãƒ‡ãƒ¼ã‚¿ãƒ—ãƒã‚»ãƒƒã‚·ãƒ³ã‚°ã¨ãã®æœªæ¥ AWS Black Belt Techã‚·ãƒªãƒ¼ã‚º Amazon Kinesis [åˆéŸ³ãƒŸã‚¯] Kinesis ã§ãƒ•ãƒªãƒ¼ã‚¶ã‚’æ’ƒã¦ï¼ Pythonã§kinesisã€œã±ã„ããï¼ in ç¬¬20å›žAWS User Group Japan æ±äº¬å‹‰å¼·ä¼š2014.04.11 Amazon Kinesis ã¨ã¯ ä¸€è¨€ã§è¨€ã†ã¨ ãƒ•ãƒ«ãƒžãƒãƒ¼ã‚¸ãƒ‰ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚µãƒ¼ãƒ“ã‚¹ AWS ã§ã®ä½ç½®ã¥ã‘
kakku22 2015/06/16
aws

kinesis
ãƒªãƒ³ã‚¯
1