[B! Kinesis] hatano99ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

hatano99 id:hatano99

Kinesisã«é–¢ã™ã‚‹hatano99ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (4)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Amazon Kinesis / S3 / SQS ã®å°Žå…¥ã‚’æ¯”è¼ƒæ¤œè¨Žã—ã¦åˆ†ã‹ã£ãŸãã‚Œãžã‚Œã®ç‰¹å¾´ ï½œ SiTest (ã‚µã‚¤ãƒ†ã‚¹ãƒˆ) ãƒ–ãƒã‚°
2-1. çµŒç·¯ ãƒˆãƒ©ãƒ•ã‚£ãƒƒã‚¯ã®ã‚³ã‚¹ãƒˆãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹ã‚’è‰¯ãã™ã‚‹ç‚ºã®ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆãŒç«‹ã¡ä¸ŠãŒã‚Šã¾ã—ãŸã€‚ ãƒˆãƒ©ãƒ•ã‚£ãƒƒã‚¯è‡ªä½“ã‚’æŒãäº‹è‡ªä½“ã¯ä»¥ä¸‹ã®ã‚ˆã†ãªæ‰‹æ®µã‚’é§†ä½¿ã™ã‚‹ã“ã¨ã§å¯èƒ½ã§ã™ã€‚ ãƒ»ã‚µãƒ¼ãƒãƒ¼ã®å°æ•°ã‚’å¢—ã‚„ã™ ãƒ»ã‚µãƒ¼ãƒãƒ¼ã‚’ã‚ˆã‚Šä¸Šä½ã®ãƒ—ãƒ©ãƒ³ã«å¤‰æ›´ã™ã‚‹ Amazon EC2 ã¯ã©ã¡ã‚‰ã‚‚ç°¡å˜ã«å®Ÿç¾ã§ãã¾ã™ã®ã§ç‰¹ã«å•é¡Œè¦–ã¯ã—ã¦ã¾ã›ã‚“ãŒã€ ç¾åœ¨é€²è¡Œå½¢ã§å¢—ãˆç¶šã‘ã¦ã„ã‚‹ã‚³ã‚¹ãƒˆã‚’æ¸›ã‚‰ã™ãŸã‚ã«ã¯åˆ¥ã®æ‰‹æ®µã‚’è€ƒãˆã‚‹å¿…è¦ãŒã‚ã‚Šã¾ã™ã€‚ 2-2. ç¾çŠ¶åˆ†æž SiTest ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆå§‹å‹•æ™‚ã¯é–‹ç™ºé€Ÿåº¦ã‚’é‡è¦–ã™ã‚‹ãŸã‚ã«ã€ è¨å®šã‚ˆã‚Šè¦ç´„ã‚’é‡è¦–ã™ã‚‹CakePHPã¨ã„ã†ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã‚’é¸æŠžã—ã¦ã„ã¾ã—ãŸã€‚ CakePHPã®æŒ™å‹•ã¯ç´ ç›´ã§é–‹ç™ºã—ã‚„ã™ãä¸æº€ã¯ãªã„ã®ã§ã™ãŒã€é«˜ãƒˆãƒ©ãƒ•ã‚£ãƒƒã‚¯ã‚’æŒãã“ã¨ãŒè‹¦æ‰‹ã§ã™ã€‚ ãã‚‚ãã‚‚ã‚µãƒ¼ãƒãƒ¼ã‚µã‚¤ãƒ‰ã¨ã—ã¦é¸æŠžã—ã¦ã„ã‚‹PHPè‡ªä½“ã®å•é¡Œã§ã‚ã‚‹ç¨‹åº¦ã¾ã§ã—ã‹å‡¦ç†èƒ½åŠ›ãŒå‡ºã¾ã›ã‚“ã€‚ Web Framework Benchmar
hatano99 2017/03/20
CakePHP

ãƒˆãƒ©ãƒ•ã‚£ãƒƒã‚¯

Amazon S3

amazon

Kinesis

ãƒ‡ãƒ¼ã‚¿

s3
ãƒªãƒ³ã‚¯
ã€æ–°æ©Ÿèƒ½ã€‘Amazon Kinesis Firehoseã«Amazon Lambdaã‚’ä½¿ã£ãŸãƒ‡ãƒ¼ã‚¿å¤‰æ›æ©Ÿèƒ½ãŒè¿½åŠ ã€‚Apacheãƒã‚°ãŒç°¡å˜ã«JSONå½¢å¼ã«å¤‰æ›å¯èƒ½ã«ï¼ | DevelopersIO
ã“ã‚“ã«ã¡ã¯ã€ã›ãƒ¼ã®ã§ã™ã€‚ä»Šæ—¥ã¯Kinesis Firehoseã«ãªã‹ãªã‹å¼·åŠ›ãªæ©Ÿèƒ½ãŒè¿½åŠ ã•ã‚ŒãŸã®ã§ã”ç´¹ä»‹ã—ã¾ã™ã€‚ ã“ã‚Œã‚’å¾…ã£ã¦ã„ãŸ ä»Šå›žã”ç´¹ä»‹ã™ã‚‹ã®ã¯ã€Œãƒ‡ãƒ¼ã‚¿å¤‰æ›æ©Ÿèƒ½ã€ã§ã™ã€‚æ‰€è¬‚ETL(Extractã€Transf ormã€Loadï¼‰å‡¦ç†ã¨è¨€ã‚ã‚Œã‚‹ã‚‚ã®ã§ã€æµã‚Œã¦ãã‚‹ãƒ‡ãƒ¼ã‚¿ã‚’ä½¿ã„ã‚„ã™ã„å½¢ã«åŠ å·¥ã—ãŸä¸Šã§å¯¾è±¡ã¨ãªã‚‹ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã‚„ãƒ‡ãƒ¼ã‚¿ãƒ¬ã‚¤ã‚¯ã«æ›¸ãå‡ºã™ã‚‚ã®ã§ã™ã€‚ ä¾‹ãˆã°Apacheã®ãƒã‚°ãƒ‡ãƒ¼ã‚¿ã‚’S3ã«è“„ç©ã—ã¦ãŠã„ã¦ã€å¾Œã»ã©åˆ†æžã«ä½¿ã„ãŸã„ã¨ã—ã¾ã™ã€‚Apacheã®ãƒã‚°ã£ã¦ã“ã‚“ãªæ„Ÿã˜ã§ã™ã‚ˆãã€‚ 127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 ã“ã®çŠ¶æ…‹ã§S3ã«å…¥ã£ã¦ã„ãŸå ´åˆã€åˆ†æžã«ä½¿ã†ã«ã¯ä¸€æ—¦ã“ã®ãƒ‡ãƒ¼ã‚¿ã‚’ã©ã“ã‹ã®ã‚µãƒ¼ãƒã«å–ã‚Šå‡ºã—ã€åˆ†æžç”¨ã«åŠ å·¥ã—ã¦ã€ã‚‚ã†ä¸€åº¦ãƒ‡ãƒ¼ã‚¿ã‚’è²¯ã‚ç›´ã™å¿…è¦ãŒã‚ã‚Šã¾ã™
hatano99 2016/12/22
apache

s3

Kinesis

log

JSON

lambda
ãƒªãƒ³ã‚¯
DynamoDB Streamã®ãƒ‡ãƒ¼ã‚¿ã‚’socket.ioã‚’ä½¿ã£ã¦ãƒ–ãƒ©ã‚¦ã‚¶ã«æµã™ - Qiita
ã‚„ã‚‹ã“ã¨ Twitterã®ã‚¹ãƒˆãƒªãƒ¼ãƒ ãƒ‡ãƒ¼ã‚¿ã‚’Kinesisã‚’çµŒç”±ã—ã¦DynamoDBã«ä¿å˜ã—ã¾ã™ã€‚ DynamoDB Streamã«ãƒ‡ãƒ¼ã‚¿ã‚’æµã—ã€ãƒ–ãƒ©ã‚¦ã‚¶ã§ã‚¹ãƒˆãƒªãƒ¼ãƒ ãƒ‡ãƒ¼ã‚¿ã‚’å¯è¦–åŒ–ã™ã‚‹ æ§‹æˆå›³ Twitter Stream â†’ Kinesis â†’ DynamoDB AWS Lambda Reference Architecture: Real-time Stream Processing awslabsã®ã‚¢ã‚«ã‚¦ãƒ³ãƒˆã«ãã®ã¾ã¾ã‚„ã‚ŠãŸã„ã“ã¨ãŒã‚ã‚Šã¾ã—ãŸã€‚å‹•ã‹ã™ã¨Kinesisã«ãƒ‡ãƒ¼ã‚¿ã‚’æµã—ã¦ã€ã‚ãƒ£ãƒ—ãƒãƒ£ã®é€šã‚ŠDynamoDBã«ãƒ‡ãƒ¼ã‚¿ã‚’ä¿å˜ã—ã¦ãã‚Œã¾ã™ã€‚ DynamoDB â†’ DynamoDB Stream â†’ Lambda â†’ SNS http Notification DynamoDB Streamã®æœ‰åŠ¹åŒ– ã‚ãƒ£ãƒ—ãƒãƒ£ã®ç®‡æ‰€ã‹ã‚‰DynamoDB Streamã‚’æœ‰åŠ¹åŒ–ã—ã¾ã™ ãƒˆãƒªã‚¬ãƒ¼ã‚’æœ‰åŠ¹åŒ–ã—
hatano99 2016/10/25
Kinesis

lambda
ãƒªãƒ³ã‚¯
Kinesis Advantage2ã€ã‚ãƒã‚·ã‚¹ ã‚¢ãƒ‰ãƒãƒ³ãƒ†ãƒ¼ã‚¸2ãƒ»èŒ¶è»¸/èµ¤è»¸é™éŸ³ã€‘ [KB600 / KB600LFQ]-ã‚¨ãƒ«ã‚´ãƒŽãƒŸã‚¯ã‚¹ã‚·ãƒ§ãƒƒãƒ—
å½“ç¤¾ã¯ç±³å›½ã‚ãƒã‚·ã‚¹ç¤¾ã®å›½å†…å”¯ä¸€ã®å…¬å¼è²©å£²ä»£ç†åº—ã§ã™ã€‚ æœ¬ãƒšãƒ¼ã‚¸ã‹ã‚‰è³¼å…¥ã—ãŸè£½å“ã«ã¯ï¼‘å¹´é–“ã®å›½å†…ä¿è¨¼ãŒã¤ãã¾ã™ã€‚ â€»æœ¬è£½å“ã¯ç±³å›½é…åˆ—ã‚ãƒ¼ãƒœãƒ¼ãƒ‰ã«ãªã‚Šã¾ã™ã€‚
hatano99 2013/04/12
****å‚è€ƒ

Kinesis

ã‚ãƒ¼ãƒœãƒ¼ãƒ‰
ãƒªãƒ³ã‚¯
1