[B! MapReduce] conceal-rsã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

Rust/OGAWA id:conceal-rs

MapReduceã«é–¢ã™ã‚‹conceal-rsã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (6)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

é–‹ç™ºãƒ¡ãƒ¢: ãƒãƒ¼ã‚«ãƒ«MapReduceã®æ€§èƒ½
Kyoto Cabinetã«MapReduceã‚’å®Ÿè£…ã—ãŸã¨ã„ã†è©±ã¯å‰å›žæ›¸ã„ãŸãŒã€ãã®Luaãƒã‚¤ãƒ³ãƒ‡ã‚£ãƒ³ã‚°ã§ã‚‚MapReduceã‚’ã‚µãƒãƒ¼ãƒˆã—ãŸã€‚ã¾ãŸã€Kyoto Tycoonã¨ãã®ã‚¹ã‚¯ãƒªãƒ—ãƒˆè¨€èªžæ‹¡å¼µã§ã‚‚MapReduceã‚’ã‚µãƒãƒ¼ãƒˆã—ãŸã€‚ä»Šå›žã¯ãã®æ€§èƒ½ã«ã¤ã„ã¦è§£èª¬ã™ã‚‹ã€‚ ãƒãƒ¼ã‚«ãƒ«MapReduceã®ãƒ„ãƒœ ä¸–ã«è¨€ã†MapReduceã¯åˆ†æ•£å‡¦ç†ã®ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã ã‘ã‚Œã©ã‚‚ã€KC/KTã®ã€Œãƒãƒ¼ã‚«ãƒ«MapReduceã€ã¯åˆ†æ•£å‡¦ç†ã‚’è¡Œã‚ãªã„ã€‚åˆ†æ•£å‡¦ç†ã‚’ã—ãªã‹ã£ãŸã‚‰ãƒ‡ãƒ¼ã‚¿å‡¦ç†èƒ½åŠ›ãŒä¸ŠãŒã‚‰ãªã„ã˜ã‚ƒãªã„ã‹ã¨æ€ã†ã‹ã‚‚ã—ã‚Œãªã„ã‘ã‚Œã©ã‚‚ã€ãã†ã¨ã‚‚é™ã‚‰ãªã„ã®ã ã€‚å‰å›žã‚‚æ›¸ã„ãŸã‘ã©ã‚‚ã€MapReduceãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯éƒ¨åˆ†ã‚’ã†ã¾ãå®Ÿè£…ã™ã‚‹ã¨ã€æ™‚é–“åŠ¹çŽ‡ã¨ç©ºé–“åŠ¹çŽ‡ã®åŒæ–¹ã‚’å‘ä¸Šã•ã›ã‚‹ã“ã¨ãŒã§ãã‚‹ã€‚ç‰¹ã«ã‚ãƒ£ãƒƒã‚·ãƒ¥ã¨ã‚½ãƒ¼ãƒˆã®éƒ¨åˆ†ã«å·¥å¤«ãŒã‚ã‚‹ã€‚ MapReduceã¯ã€ãƒªãƒã‚¸ãƒˆãƒªå†…ï¼ˆKCã§ã¯ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ãƒ•ã‚¡ã‚¤ãƒ«å†…ï¼‰ã®å„ãƒ¬ã‚³ãƒ¼ãƒ‰ã‹ã‚‰ã‚ãƒ¼ã¨å€¤
conceal-rs 2010/11/10
*ã‚ã¨ã§

kvs

MapReduce

Performance
ãƒªãƒ³ã‚¯
KVSã‚’Webã‚¢ãƒ—ãƒªã®ãƒ¡ã‚¤ãƒ³ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã«ã—ãŸã‚‰ã€é›†è¨ˆãƒãƒƒãƒã‚’MapReduceã§ã‚„ã‚‹ã¯ã‚ã«ãªã£ãŸã§ã”ã–ã‚‹ã®å·» - Tous Les Jours æ”»é˜²è¨˜
ã¨ã„ã†é¡Œã§ã€RubyKansaiå‹‰å¼·ä¼š#39ã§ç™ºè¡¨ã•ã›ã¦ã„ãŸã ãã¾ã—ãŸã€‚ å†…å®¹ã¯MapReduceã¨KVSã®é–¢ä¿‚ã€ãŠã‚ˆã³è‡ªä½œã®MapReduceå‡¦ç†ç³»ã§ã‚ã‚‹TinyMapReduceã®ç´¹ä»‹ã«ã¤ã„ã¦ã€‚å®Ÿè£…ã«ã¯DRubyã‚’ä½¿ã£ã¦ã„ã¾ã™ã€‚ã¨ã„ã£ã¦ã‚‚TinyMapReduceã¯ã‚³ãƒ¼ãƒ‰ã‚µã‚¤ã‚º200è¡Œä»¥ä¸‹ã§ã€å¯¾éšœå®³æ€§ã‚‚ãªãã‚µãƒ³ãƒ—ãƒ«ã®åŸŸã‚’å‡ºãªã„ã‚‚ã®ã§ã™ãŒã€‚ã€‚ TinyMapReduceæ¨™æº–æ·»ä»˜ã®Sampleã¯ã€ï¼‘ã€œ100ä¸‡ã¾ã§ã®è‡ªç„¶æ•°ã®ä¸ã«2ã®å€æ•°ã¨3ã®å€æ•°ãŒãã‚Œãžã‚Œã„ãã¤å«ã¾ã‚Œã‚‹ã‹ã‚«ã‚¦ãƒ³ãƒˆã™ã‚‹ã‚¿ã‚¹ã‚¯ã§ã™ï¼ˆçµæžœä¿å˜ã«SimpleResourceã‚’ä½¿ã£ã¦ãŠã‚Šã€åˆ¥é€”å°Žå…¥ãŒå¿…è¦ï¼‰ã€‚Workerã®æ•°ã‚’å¢—ã‚„ã—ã¦ã„ãã“ã¨ã§ã€ã‚¿ã‚¹ã‚¯ã®å®Ÿè¡Œæ™‚é–“ãŒã‚·ãƒ¼ãƒ ãƒ¬ã‚¹ã«çŸç¸®ã™ã‚‹ã“ã¨ãŒã‚ã‹ã‚‹ã¨æ€ã„ã¾ã™ã€‚ ã¨ã„ã†ã‚ã‘ã§ã€ä½¿ç”¨ã—ãŸã‚¹ãƒ©ã‚¤ãƒ‰ã‚’ç½®ã„ã¦ãŠãã¾ã™ã€‚ TinyMapReduce on rubyView more presentat
conceal-rs 2010/03/30
MapReduce

ruby

kvs

druby

cloud

NoSQL
ãƒªãƒ³ã‚¯
Hive on AWS @ COOKPAD - ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰é–‹ç™ºè€…ãƒ–ãƒã‚°
ã“ã‚“ã«ã¡ã‚ã€‚ã‚µãƒ¼ãƒ“ã‚¹é–‹ç™ºæ‹…å½“ã®å‹é–“ã§ã™ã€‚ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã®ï¼‘å¹´ã®æœ€å¤§ã®ãƒ”ãƒ¼ã‚¯ã§ã‚ã‚‹ãƒãƒ¬ãƒ³ã‚¿ã‚¤ãƒ³ãŒçµ‚ã‚ã‚Šã€å°‘ã—è½ã¡ç€ãã‚’ã¨ã‚Šã‚‚ã©ã—ãŸæŠ€è¡“éƒ¨ã‹ã‚‰ãŠå±Šã‘ã—ã¾ã™ã€‚ ã•ã¦ã€å…ˆæ—¥ç§‹è‘‰åŽŸã§ã€Œç¬¬0å›ž AWS User Group - Japanå‹‰å¼·ä¼šã€ãŒé–‹å‚¬ã•ã‚Œã¾ã—ãŸã€‚100äººã‚’è¶…ã™å‚åŠ è€…ã®ä¸ã€AWSã®ã‚¨ãƒãƒ³ã‚¸ã‚§ãƒªã‚¹ãƒˆJeff Barrã•ã‚“ã®è¬›æ¼”ãŒã‚ã£ãŸã‚Šã€å†…å®¹ã®æ¿ƒã„LTãŒç¶šã„ãŸã‚Šã¨ã€éžå¸¸ã«å¤§ç››æ³ã§ã—ãŸã€‚ãã‚“ãªLTã«åƒ•ã‚‚å‚åŠ ã—ã¦ã€ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã®ãƒãƒƒãƒã‚·ã‚¹ãƒ†ãƒ ã¨AWSã¨ã®é€£æºã«ã¤ã„ã¦è©±ã—ã¦ãã¾ã—ãŸã€‚ ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã§ã¯AWSã¨ã—ã¦EC2, S3ã‚’ã¤ã‹ã£ã¦åˆ†æ•£è§£æžç’°å¢ƒã‚’æ§‹ç¯‰ã—ã¦ã€Hiveã‚’ä½¿ã£ãŸãƒ‡ã‚¤ãƒªãƒ¼ã®ãƒã‚°è§£æžã‚’è¡Œã£ã¦ã„ã¾ã™ã€‚LTã§ã¯ãã‚Œã‚‰ã®è©±ã‚’ã—ãŸã®ã§ã™ãŒã€ï¼•åˆ†ã¨é™ã‚‰ã‚ŒãŸæ™‚é–“ã§ã¯é§†ã‘è¶³ã®ç™ºè¡¨ã«ãªã£ã¦ã—ã¾ã£ãŸã®ã§ã€å½“æ—¥ã˜ã£ãã‚Šè©±ã›ãªã‹ã£ãŸç®‡æ‰€ãªã©ã‚’ç¢ºèªã„ãŸã ã‘ã‚Œã°ã¨æ€ã„ã¾ã™ã€‚ [slideshare id=328
conceal-rs 2010/03/09
hive

hadoop

AmazonEC2

MapReduce

cookpad
ãƒªãƒ³ã‚¯
ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã¨Hadoop - ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰é–‹ç™ºè€…ãƒ–ãƒã‚°
ã¯ã˜ã‚ã¾ã—ã¦ã€‚ä»Šå¹´ã®5æœˆã«å…¥ç¤¾ã—ãŸå‹é–“@ã•ãŒã™ãƒãƒ¼ãƒ ã§ã™ã€‚ å…¥ç¤¾ã—ã¦ã‹ã‚‰ã¯ã€ãªã‹ãªã‹å¤§å¤‰ãªã“ã¨ã‚‚å¤šã„ã§ã™ãŒã€æœ€è¿‘ã¯ãŠé…’å¥½ããŒé›†ã¾ã£ã¦æœˆæ›œã‹ã‚‰é£²ã¿åˆã† ã€Œå‹é–“ä¼šã€ãªã‚‹ã‚‚ã®ã‚‚ç™ºè¶³ã—ã¦ã€ä»•äº‹é¢ã§ã‚‚ä»•äº‹ä»¥å¤–ã®é¢ã§ã‚‚å¯†åº¦ã®é«˜ã„æ¯Žæ—¥ã‚’éŽã”ã—ã¦ã„ã¾ã™ï¼ ã•ã¦ã€åƒ•ã¯ã€Œã•ãŒã™ã€ãƒãƒ¼ãƒ æ‰€å±žã¨ã„ã†ã“ã¨ã§ã€æ™®æ®µã¯ãƒ¬ã‚·ãƒ”ã‚’ã€Œã•ãŒã™ã€ãƒ¦ãƒ¼ã‚¶ã®æº€è¶³åº¦ã‚’ä¸Šã’ã‚‹ãŸã‚ã«ã€ ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã®æ¤œç´¢ã¾ã‚ã‚Šã«ã¤ã„ã¦ã€ã„ã‚ã„ã‚ãªé–‹ç™ºã‚’è¡Œã£ã¦ã„ã¾ã™ã€‚ ä¸€æ–¹ã§ã€ãƒ¦ãƒ¼ã‚¶ã®ã€Œã•ãŒã™æ¬²æ±‚ã€ã«ã¤ã„ã¦æ·±ãçŸ¥ã‚‹ãŸã‚ã«ã€å¤§è¦æ¨¡ãªãƒ‡ãƒ¼ã‚¿è§£æžã‚’è¡Œã„ã€æ¬²æ±‚ã®åˆ†æžã‚’è¡Œã†æ©Ÿä¼šã‚‚å¢—ãˆã¦ãã¾ã—ãŸã€‚ ã¨ã“ã‚ãŒã€ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã®ãƒã‚°ã¯è†¨å¤§ãªæ•°ãŒã‚ã‚‹ã®ã§ã€ä¸€å£ã®ãƒ‡ãƒ¼ã‚¿è§£æžã¨è¨€ã£ã¦ã‚‚é€šå¸¸ã®ãƒãƒƒãƒå‡¦ç†ã ã¨é–“ã«åˆã‚ãªã„ãŸã‚ã€ åˆ†æ•£å‡¦ç†ç’°å¢ƒã®å¿…è¦æ€§ãŒé«˜ã¾ã£ã¦ãã¾ã—ãŸã€‚ ãã“ã§ã€ã¾ãšã¯æ‰‹è»½ã«è©¦ã›ã‚‹åˆ†æ•£å‡¦ç†ã®çŽ‹é“ã¨ã„ã†ã“ã¨ã§ã€æœ€è¿‘ã§ã¯Hadoopã‚’ä½¿ã£ãŸãƒ‡ãƒ¼ã‚¿è§£æžç’°å¢ƒã‚’æ•´å‚™ã—ã¦ã„ã¾ã™ã€‚
conceal-rs 2009/09/16
MapReduce

Hadoop

cookpad

ã‚ã¨ã§èªã‚€
ãƒªãƒ³ã‚¯
asemanfar - Pandemic -- because information needs to be contagious
Pandemic is a map-reduce framework. You give it the map, process, and reduce methods and it handles the rest. It's designed to serve requests in real-time, but can also be used for offline tasks. It's different from the typical map-reduce framework in that it doesn't have a master-worker structure. Every node can map, process, and reduce. It also doesn't have the concept of jobs, everything is a r
conceal-rs 2009/04/14
gems

ruby

MapReduce
ãƒªãƒ³ã‚¯
Amazon EC2ï¼S3ï¼Œåˆ†æ•£å‡¦ç†åŸºç›¤Hadoopã®è¨å®šæ”¯æ´æ©Ÿèƒ½ã€ŒElastic MapReduceã€ã‚’æä¾›
ç±³Amazon.comå‚˜ä¸‹ã®ç±³Amazon Web Servicesï¼ˆAWSï¼‰ã¯ç±³å›½æ™‚é–“2009å¹´4æœˆ2æ—¥ï¼Œå¤§é‡ãƒ‡ãƒ¼ã‚¿ã®åˆ†æ•£å‡¦ç†ã‚’æ”¯æ´ã™ã‚‹ã‚µãƒ¼ãƒ“ã‚¹ã€ŒAmazon Elastic MapReduceã€ã®ç±³å›½å‘ã‘ãƒ™ãƒ¼ã‚¿æä¾›ã‚’é–‹å§‹ã—ãŸã¨ç™ºè¡¨ã—ãŸã€‚AWSã®ã‚¯ãƒ©ã‚¦ãƒ‰ãƒ»ã‚µãƒ¼ãƒ“ã‚¹ã¨ï¼Œã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã®åˆ†æ•£å‡¦ç†ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ ã€ŒHadoopã€ã‚’çµ„ã¿åˆã‚ã›ï¼Œå¤§é‡ãƒ‡ãƒ¼ã‚¿ã‚’æ‰±ã†éš›ã«å¿…è¦ã¨ãªã‚‹è¨å®šä½œæ¥ã®è² æ‹…ã‚’è»½æ¸›ã™ã‚‹ã€‚ Amazon Elastic MapReduceã‚’åˆ©ç”¨ã™ã‚‹ã¨ï¼Œãƒ›ã‚¹ãƒ†ã‚£ãƒ³ã‚°ãƒ»ã‚µãƒ¼ãƒ“ã‚¹ã€ŒAmazon Elastic Compute Cloudï¼ˆAmazon EC2ï¼‰ã€ã¨ã‚ªãƒ³ãƒ©ã‚¤ãƒ³ãƒ»ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ãƒ»ã‚µãƒ¼ãƒ“ã‚¹ã€ŒAmazon Simple Storage Serviceï¼ˆAmazon S3ï¼‰ã€ã§å‹•ã‹ã™åˆ†æ•£ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã«å‰²ã‚Šå½“ã¦ã‚‹æ¼”ç®—ãƒªã‚½ãƒ¼ã‚¹ã®é‡ã‚’ç°¡å˜ã«æŒ‡å®šã§ãã‚‹ã¨ã„ã†ã€‚Amazon EC2ã®ã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹
conceal-rs 2009/04/03
amazon s3

MapReduce

åˆ†æ•£

Hadoop
ãƒªãƒ³ã‚¯
1