[B! kinesis] serihiroã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

serihiro id:serihiro

kinesisã«é–¢ã™ã‚‹serihiroã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (4)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

KinesisClientLiblary(KCL)ã«ãŠã‘ã‚‹LATESTã¨TRIM_HORIZONã®æŒ™å‹•ã«ã¤ã„ã¦ - Qiita
AWS Kinesis Stream APIã§è¨˜è¼‰ã•ã‚Œã¦ã„ã‚‹Shard iterator typeã®ã†ã¡ã€KCL(Kinesis Client Liblary)ã«æ¨™æº–ã§ç”¨æ„ã•ã‚Œã¦ã„ã‚‹ã®ã¯ LATEST ã¨ TRIM_HORIZON ã§ã™ãŒã€ã“ã‚Œã®æŒ™å‹•ã«ã¤ã„ã¦æ°—ã«ãªã‚‹ç‚¹ãŒã‚ã£ãŸã®ã§èª¿æŸ»ã—ã¦ã¿ã¾ã—ãŸã€‚é€”ä¸ã®è¨³æ–‡ã¯é©å½“ã§ã™ã€‚ åŽŸç¾©ã§ã¯ ï¼ˆåŽŸæ–‡ï¼‰LATEST - Start reading just after the most recent record in the shard, so that you always read the most recent data in the shard. ï¼ˆè¨³ï¼‰LATEST - ã‚·ãƒ£ãƒ¼ãƒ‰å†…ã®æœ€æ–°ãƒ¬ã‚³ãƒ¼ãƒ‰ã®ç›´å¾Œã‹ã‚‰èªã¿å–ã‚Šã‚’é–‹å§‹ã—ã¾ã™ã€‚ãªã®ã§ã‚·ãƒ£ãƒ¼ãƒ‰å†…ã®æœ€æ–°ã®ãƒ‡ãƒ¼ã‚¿ã‚’å¸¸ã«èªã¿ã¾ã™ã€‚ ã¨ãªã£ã¦ã„ã¾ã™ã€‚ å®Ÿéš›ã®æŒ™å‹•ã¨ã—ã¦ã¯KCLã¯ã¾ãšDynamoDBã«ã‚¢ãƒ—ãƒª
serihiro 2017/07/24
kinesis
ãƒªãƒ³ã‚¯
fluentdã§Kinesis Streamsã«é€ã£ã¦Lambdaã§èªã‚“ã§S3ã«ä¿å˜ã™ã‚‹ - sambaiz-net
aws-fluent-plugin-kinesisã§Kinesis Streamsã«é€ã‚Šã€Lambdaã§èªã‚“ã§S3ã«ä¿å˜ã™ã‚‹ã€‚ è¦ã™ã‚‹ã«Firehoseã®ã‚ˆã†ãªã“ã¨ã‚’ã‚„ã‚ŠãŸã„ã®ã ã‘ã‚Œã©Tokyoãƒªãƒ¼ã‚¸ãƒ§ãƒ³ã«ã¾ã æ¥ãªã„ã®ã§è‡ªåˆ†ã§ã‚„ã‚‹ã€‚ fluentdã§é€ã‚‹ try_flush_intervalã¨queued_chunk_flush_intervalã¯ãƒ‰ã‚ãƒ¥ãƒ¡ãƒ³ãƒˆã«ã¯è¼‰ã£ã¦ã„ãªã„ãŒã€ ä»¥ä¸‹ã®ãƒšãƒ¼ã‚¸ã«ã‚ˆã‚‹ã¨ãã‚Œãžã‚Œqueueã«æ¬¡ã®chunkãŒãªã„ã¨ãã¨ã‚ã‚‹ã¨ãã®flushã™ã‚‹é–“éš”ã€‚ ã„ãšã‚Œã‚‚ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆã¯1ã ãŒã€ã“ã‚Œã‚’æ¸›ã‚‰ã™ã“ã¨ã§ã‚‚ã£ã¨é »ç¹ã«åãå‡ºã•ã‚Œã‚‹ã‚ˆã†ã«ãªã‚‹ã‚‰ã—ã„ã€‚ Fluentd ã® out_forward ã¨ BufferedOutput ã‚ã¨ã‚·ãƒ£ãƒ¼ãƒ‰ã«æŒ¯ã‚Šåˆ†ã‘ã‚‹ãŸã‚ã®partition_key ã‚’æŒ‡å®šã§ãã‚‹ã€‚ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆã¯ãƒ©ãƒ³ãƒ€ãƒ ã€‚ <source> @type tail path /v
serihiro 2017/07/24
kinesis

lambda
ãƒªãƒ³ã‚¯
ç§’é–“æ•°ä¸‡ã®ãƒã‚°ã‚’ã„ã„æ„Ÿã˜ã«ã™ã‚‹ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£
AWS Summit Tokyo 2016 Developer Conference (2016/06/03)
serihiro 2017/07/20
aws

kinesis
ãƒªãƒ³ã‚¯
[åˆéŸ³ãƒŸã‚¯] Kinesis ã§ãƒ•ãƒªãƒ¼ã‚¶ã‚’æ’ƒã¦ï¼
27. Tweet ã‚’ Stream ã«å…¥ã‚Œã‚‹ Kinesis Stream Twitter4J Kinesis MeCab Dynamo Unity TwitterStream twitterStream = new TwitterStreamFactory().getInstance(); StatusListener listener = new StatusListener() { @Override public void onStatus(Status status) { String key = RandomStringUtils.randomAlphanumeric(10); String json = DataObjectFactory.getRawJSON(status); PutRecordRequest putRecordRequest = new PutRecor
serihiro 2015/03/23
ã“ã‚Œç”Ÿã§è¦‹ãŸã‹ã£ãŸãªï½—

kinesis

aws
ãƒªãƒ³ã‚¯
1