[B! mapreduce][Amazon] beth321ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

beth321 id:beth321

mapreduceã¨Amazonã«é–¢ã™ã‚‹beth321ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (6)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ â€” Amazon EMR â€” AWS
Amazon EMR Apache Sparkã€Trinoã€ãŠã‚ˆã³ä»–ã®ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ãƒ¯ãƒ¼ã‚¯ãƒãƒ¼ãƒ‰ã‚’ç°¡å˜ã«å®Ÿè¡Œã—ã¦ã‚¹ã‚±ãƒ¼ãƒ« Amazon EMR ã¯ã€æ¯”é¡žã®ãªã„æŸ”è»Ÿæ€§ã¨ã‚¹ã‚±ãƒ¼ãƒ«ã«ã‚ˆã‚Šã€åˆ†æžãƒ¯ãƒ¼ã‚¯ãƒãƒ¼ãƒ‰ã‚’åŠ é€Ÿã™ã‚‹ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚µãƒ¼ãƒ“ã‚¹ã§ã™ã€‚EMR ã¯ã€Apache Sparkã€Trinoã€Apache Flinkã€Apache Hive å‘ã‘ã«ãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹æœ€é©åŒ–ãƒ©ãƒ³ã‚¿ã‚¤ãƒ ã‚’å‚™ãˆã¦ãŠã‚Šã€ã‚³ã‚¹ãƒˆã¨å‡¦ç†æ™‚é–“ã‚’å¤§å¹…ã«å‰Šæ¸›ã—ã¾ã™ã€‚ã“ã®ã‚µãƒ¼ãƒ“ã‚¹ã¯ AWS ã¨ã‚·ãƒ¼ãƒ ãƒ¬ã‚¹ã«çµ±åˆã—ã€ãƒ‡ãƒ¼ã‚¿ãƒ¬ã‚¤ã‚¯ãƒ¯ãƒ¼ã‚¯ãƒ•ãƒãƒ¼ã¨ã‚¨ãƒ³ã‚¿ãƒ¼ãƒ—ãƒ©ã‚¤ã‚ºè¦æ¨¡ã®ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ã‚’ç°¡ç´ åŒ–ã—ã¾ã™ã€‚çµ„ã¿è¾¼ã¿ã®è‡ªå‹•ã‚¹ã‚±ãƒ¼ãƒªãƒ³ã‚°ã€ã‚¤ãƒ³ãƒ†ãƒªã‚¸ã‚§ãƒ³ãƒˆãƒ¢ãƒ‹ã‚¿ãƒªãƒ³ã‚°ã€ãƒžãƒãƒ¼ã‚¸ãƒ‰ã‚¤ãƒ³ãƒ•ãƒ©ã‚¹ãƒˆãƒ©ã‚¯ãƒãƒ£ã‚’å‚™ãˆãŸ EMR ã‚’åˆ©ç”¨ã™ã‚‹ã“ã¨ã§ã€ã‚¯ãƒ©ã‚¹ã‚¿ãƒ¼ã®ç®¡ç†ã§ã¯ãªãã‚¤ãƒ³ã‚µã‚¤ãƒˆã®æŠ½å‡ºã«æ³¨åŠ›ã§ãã€å¾“æ¥ã®ã‚½ãƒªãƒ¥ãƒ¼ã‚·ãƒ§ãƒ³ã®é‹ç”¨ä¸Šã®ã‚ªãƒ¼ãƒãƒ¼ãƒ˜ãƒƒãƒ‰ãªã—ã«ã€ãƒšã‚¿ãƒã‚¤ãƒˆè¦æ¨¡ã®åˆ†æžãŒåŠ¹çŽ‡
beth321 2015/03/20
hadoop

amazon_emr

amazon

ã‚¯ãƒ©ã‚¦ãƒ‰

aws

MapReduce

Web Service

webã‚µãƒ¼ãƒ“ã‚¹

amazons3

development
ãƒªãƒ³ã‚¯
Hive on AWS @ COOKPAD - ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰é–‹ç™ºè€…ãƒ–ãƒã‚°
ã“ã‚“ã«ã¡ã‚ã€‚ã‚µãƒ¼ãƒ“ã‚¹é–‹ç™ºæ‹…å½“ã®å‹é–“ã§ã™ã€‚ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã®ï¼‘å¹´ã®æœ€å¤§ã®ãƒ”ãƒ¼ã‚¯ã§ã‚ã‚‹ãƒãƒ¬ãƒ³ã‚¿ã‚¤ãƒ³ãŒçµ‚ã‚ã‚Šã€å°‘ã—è½ã¡ç€ãã‚’ã¨ã‚Šã‚‚ã©ã—ãŸæŠ€è¡“éƒ¨ã‹ã‚‰ãŠå±Šã‘ã—ã¾ã™ã€‚ ã•ã¦ã€å…ˆæ—¥ç§‹è‘‰åŽŸã§ã€Œç¬¬0å›ž AWS User Group - Japanå‹‰å¼·ä¼šã€ãŒé–‹å‚¬ã•ã‚Œã¾ã—ãŸã€‚100äººã‚’è¶…ã™å‚åŠ è€…ã®ä¸ã€AWSã®ã‚¨ãƒãƒ³ã‚¸ã‚§ãƒªã‚¹ãƒˆJeff Barrã•ã‚“ã®è¬›æ¼”ãŒã‚ã£ãŸã‚Šã€å†…å®¹ã®æ¿ƒã„LTãŒç¶šã„ãŸã‚Šã¨ã€éžå¸¸ã«å¤§ç››æ³ã§ã—ãŸã€‚ãã‚“ãªLTã«åƒ•ã‚‚å‚åŠ ã—ã¦ã€ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã®ãƒãƒƒãƒã‚·ã‚¹ãƒ†ãƒ ã¨AWSã¨ã®é€£æºã«ã¤ã„ã¦è©±ã—ã¦ãã¾ã—ãŸã€‚ ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã§ã¯AWSã¨ã—ã¦EC2, S3ã‚’ã¤ã‹ã£ã¦åˆ†æ•£è§£æžç’°å¢ƒã‚’æ§‹ç¯‰ã—ã¦ã€Hiveã‚’ä½¿ã£ãŸãƒ‡ã‚¤ãƒªãƒ¼ã®ãƒã‚°è§£æžã‚’è¡Œã£ã¦ã„ã¾ã™ã€‚LTã§ã¯ãã‚Œã‚‰ã®è©±ã‚’ã—ãŸã®ã§ã™ãŒã€ï¼•åˆ†ã¨é™ã‚‰ã‚ŒãŸæ™‚é–“ã§ã¯é§†ã‘è¶³ã®ç™ºè¡¨ã«ãªã£ã¦ã—ã¾ã£ãŸã®ã§ã€å½“æ—¥ã˜ã£ãã‚Šè©±ã›ãªã‹ã£ãŸç®‡æ‰€ãªã©ã‚’ç¢ºèªã„ãŸã ã‘ã‚Œã°ã¨æ€ã„ã¾ã™ã€‚ [slideshare id=328
beth321 2014/12/03
cloud

ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰

japan

AmazonEC2

hadoop

Hive

cookpad

amazon

MapReduce

aws
ãƒªãƒ³ã‚¯
ç¬¬1å›žã€€Amazon Elastic MapReduceã‚’ä½¿ã†æº–å‚™ã‚’ã™ã‚‹ | gihyo.jp
Amazon Elastic MapReduceã¨ã¯ æ˜¨ä»Šã€ãƒãƒƒãƒå‡¦ç†ã«Hadoopã‚’æ´»ç”¨ã™ã‚‹äº‹ä¾‹ãŒè©±é¡Œã«ãªã£ã¦ã„ã¾ã™ã€‚ãã®ä¸ã§ã‚‚ç‰¹ç†ã™ã¹ããªã®ã¯ã€åˆ†æ•£å‡¦ç†æŠ€è¡“ã§ã‚ã‚‹MapReduceã§ã™ã€‚ã—ã‹ã—ã€MapReduceã‚’å®Ÿéš›ã«ä½¿ã†ã«ã¯ã€æ•°å°ã‹ã‚‰æ•°100å°ã®ã‚µãƒ¼ãƒã‚’ç”¨æ„ã—ã€Hadoopã®ã‚»ãƒƒãƒˆã‚¢ãƒƒãƒ—ã‚‚ã—ãªã‘ã‚Œã°ãªã‚Šã¾ã›ã‚“ã€‚ãã®ã‚ˆã†ãªã‚µãƒ¼ãƒæ§‹ç¯‰ãƒ»ç®¡ç†ã€ã‚»ãƒƒãƒˆã‚¢ãƒƒãƒ—ã®æ‰‹é–“ã‚’ç„¡ãã—ã€ã™ãä½¿ãˆã‚‹ã®ãŒAmazon Web Serviceï¼ˆAWSï¼‰ã®Elastic MapReduceï¼ˆEMRï¼‰ã§ã™ã€‚ å›³1ã€€Amazon Elastic MapReduce AWSã€MapReduceã®èª¬æ˜Žã¯å¤šãã®æ›¸ç±ã€ãƒ–ãƒã‚°ãªã©ã‚’å‚ç…§ã—ã¦ãã ã•ã„ã€‚ã¾ãŸã€EMRã‚’ä½¿ç”¨ã™ã‚‹ã«ã¯ã€ã¾ãšAWSã®ã‚¢ã‚«ã‚¦ãƒ³ãƒˆã‚’ä½œã£ã¦ãŠãå¿…è¦ãŒã‚ã‚Šã¾ã™ãŒã€ãã®æ‰‹é †ã‚‚å‰²æ„›ã•ã›ã¦ã„ãŸã ãã¾ã™ã€‚æœ¬é€£è¼‰ã§ã¯EMRã®ä½¿ã„æ–¹ã€æ°—ã‚’ã¤ã‘ãªã‘ã‚Œã°ã„ã‘ãªã„ç‚¹ãªã©ã‚’é‡ç‚¹çš„
beth321 2014/11/21
Ruby

æƒ…

Hadoop

AWS

MapReduce

EMR

amazon

amazon ec2

ã‚ã¨ã§èªã‚€

ec2
ãƒªãƒ³ã‚¯
Hadoopã‚’æ¥å‹™ã§ä½¿ã£ã¦ã¿ãŸè©± - ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰é–‹ç™ºè€…ãƒ–ãƒã‚°
8æœˆã«å…¥ç¤¾ã—ãŸä½ã€…æœ¨ã§ã™ã€‚ã“ã‚“ã«ã¡ã‚ï¼ å…¥ç¤¾ã—ã¦ã‹ã‚‰ã¯Hadoopã‚’ä½¿ã†ã“ã¨ãŒå¤šãã€æ—¥ã€…ã€å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã¨æ ¼é—˜ã—ã¦ã„ã¾ã™ã€‚å¤§å¤‰ã§ã¯ã‚ã‚Šã¾ã™ãŒã€å€‹äººã§ã¯ãªã‹ãªã‹è§¦ã‚‹ã“ã¨ãŒå‡ºæ¥ãªã„ã‚ˆã†ãªå¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã‚’è§¦ã‚Œã‚‹ã®ã¯æ¥½ã—ã„ã§ã™ã€‚ ã•ã¦ã€Hadoopã¯æœ€è¿‘è‰²ã€…ãªã¨ã“ã‚ã§ä½¿ã‚ã‚Œå§‹ã‚ã¦ãã¦ã„ã‚‹ã¨æ€ã†ã‚“ã§ã™ãŒã€å®Ÿéš›ã«åˆ©ç”¨ã—ã¦ã¿ã¦å›°ã£ãŸäº‹ã‚„tipsãªã©ã€å®Ÿè·µçš„ãªæƒ…å ±ã¯ã¾ã ã‚ã¾ã‚Šå…¬é–‹ã•ã‚Œã¦ã„ã¾ã›ã‚“ã€‚ãã®è¾ºã®æƒ…å ±ã‚’ã¿ã‚“ãªæ±‚ã‚ã¦ã„ã‚‹ã¯ãšâ€¦ï¼ï¼ ãã“ã§ã€åƒ•ãŒå®Ÿéš›ã«è§¦ã£ã¦ã¿ã¦å›°ã£ãŸäº‹ã‚„Hadoopã‚’ä½¿ã†ä¸Šã§ãƒã‚¤ãƒ³ãƒˆã ã¨æ€ã£ãŸã“ã¨ãªã©ã‚’ç¤¾å†…å‹‰å¼·ä¼šã§ç™ºè¡¨ã—ãŸã®ã§å…¬é–‹ã—ã¦ã¿ã¾ã™ã€‚Hadoopã‚’ä½¿ã£ã¦ã„ã‚‹ï¼ˆä½¿ã„ãŸã„ã¨æ€ã£ã¦ã„ã‚‹ï¼‰æ–¹ã®å‚è€ƒã«ãªã‚Œã°å¹¸ã„ã§ã™ã€‚ [slideshare id=2711363&doc=20091214tech blog-091213183529-phpapp02] Hadoopã®åˆ©ç”¨ã¯ã¾ã ã¾ã è©¦è¡ŒéŒ¯èª¤ã®é€£ç¶š
beth321 2014/11/19
ã‚ã¨ã§èªã‚€

ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰

hadoop

Amazon

presentation

cookpad

mapreduce

ruby

development

è³‡æ–™
ãƒªãƒ³ã‚¯
Amazon Elastic MapReduceã®ä½¿ã„æ–¹â”€Hadoopã‚ˆã‚Šæ‰‹è»½ã«ã¯ã˜ã‚ã‚‹å¤§è¦æ¨¡è¨ˆç®— è¨˜äº‹ä¸€è¦§ | gihyo.jp
ç¬¬8å›žAmazon Elastic MapReduceã®ãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹ã‚’ãƒ¢ãƒ‹ã‚¿ãƒªãƒ³ã‚°ã™ã‚‹ å°æž—éš† 2011-05-12
beth321 2014/11/08
elastic map reduce

Amazon

hadoop

ã‚ã¨ã§

aws

mapreduce

*ã‚ã¨ã§

emr

ã‚ã¨ã§èªã‚€

cloud
ãƒªãƒ³ã‚¯
Amazon.co.jp: ä¸¦è¡Œã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒ†ã‚£ãƒ³ã‚°æŠ€æ³• â€•å®Ÿè·µãƒžãƒ«ãƒã‚³ã‚¢/ãƒžãƒ«ãƒã‚¹ãƒ¬ãƒƒãƒ‰ãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°: ã‚¯ãƒ¬ã‚¤ãƒ»ãƒ–ãƒ¬ãƒƒã‚·ãƒ£-ã‚º, åƒä½æ²»éƒŽ: æœ¬
beth321 2014/11/08
æœ¬

mapreduce

programming

èªã¿ãŸã„æœ¬

book

multithread

æœ¬

ãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°

amazon
ãƒªãƒ³ã‚¯
1