[B! MapReduce] kimimasaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

kimimasa id:kimimasa

MapReduceã«é–¢ã™ã‚‹kimimasaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (7)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

ã„ã¾å†æ³¨ç›®ã®åˆ†æ•£å‡¦ç†æŠ€è¡“
å®Ÿé‹ç”¨ã®éšœå®³å¯¾å¿œæ™‚é–“æ¯”è¼ƒã«è¦‹ã‚‹ã€ãƒã‚°ç®¡ç†åŸºç›¤ã®åŠ¹æžœ ï¼ˆ2017/5/9ï¼‰ ãƒã‚°åŸºç›¤ã®æ§‹ç¯‰æ–¹æ³•ã‚„åˆ©ç”¨æ–¹æ³•ã€å®Ÿéš›ã®æ¡ˆä»¶ã§ä½¿ã£ãŸã¨ãã®äº‹ä¾‹ãªã©ã‚’ç´¹ä»‹ã™ã‚‹é€£è¼‰ã€‚ä»Šå›žã¯ã€å®Ÿæ¡ˆä»¶ã‚’äº‹ä¾‹ã¨ã—ã€ãƒã‚°ç®¡ç†åŸºç›¤ã®æœ‰ç”¨æ€§ã‚’ã€éšœå®³å¯¾å¿œæ™‚é–“æ¯”è¼ƒã‚‚äº¤ãˆã¦ç´¹ä»‹ Chatworkã€LINEã€NetflixãŒé€²ã‚ã‚‹ãƒªã‚¢ã‚¯ãƒ†ã‚£ãƒ–ã‚·ã‚¹ãƒ†ãƒ ã¨ã¯ä½•ã‹ ï¼ˆ2017/4/27ï¼‰ ã€Œãƒªã‚¢ã‚¯ãƒ†ã‚£ãƒ–ã€ã«é–¢é€£ã™ã‚‹å¹¾ã¤ã‹ã®ç”¨èªžã«ã¤ã„ã¦è§£èª¬ã—ã€ãƒªã‚¢ã‚¯ãƒ†ã‚£ãƒ–ã‚·ã‚¹ãƒ†ãƒ ã‚’å®Ÿç¾ã™ã‚‹ãŸã‚ã®ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã‚’ç´¹ä»‹ã—ã¾ã™ Fluentdï¼‹Elasticsearchï¼‹Kibanaã§ä½œã‚‹ãƒã‚°åŸºç›¤ã®æ¦‚è¦ã¨æ§‹ç¯‰æ–¹æ³• ï¼ˆ2017/4/6ï¼‰ ãƒã‚°åŸºç›¤ã‚’å®Ÿç¾ã™ã‚‹Fluentdï¼‹Elasticsearchï¼‹Kibanaã«ã¤ã„ã¦ã€æ§‹ç¯‰æ–¹æ³•ã‚„åˆ©ç”¨æ–¹æ³•ã€å®Ÿéš›ã®æ¡ˆä»¶ã§ä½¿ã£ãŸã¨ãã®äº‹ä¾‹ãªã©ã‚’ç´¹ä»‹ã™ã‚‹é€£è¼‰ã€‚åˆå›žã¯ã€ãƒã‚°åŸºç›¤ã®æ§‹ç¯‰ã€åˆ©ç”¨æ–¹æ³•ã«ã¤ã„ã¦ ãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°ã¨ãƒ“ãƒ«ãƒ‰ã€Androidã‚¢ãƒ—ãƒªé–‹
kimimasa 2009/07/06
hadoop

MapReduce

åˆ†æ•£å‡¦ç†

ã‚¯ãƒ©ã‚¦ãƒ‰
ãƒªãƒ³ã‚¯
ãã®ã‚½ãƒ•ãƒˆï¼Œå£²ã‚‹ï¼Ÿå£²ã‚‰ãªã„ï¼Ÿ
ã€ŒGoogleã®å‡„ã•ã¯ï¼ŒFailure-oblivious computingï¼ˆã‚¨ãƒ©ãƒ¼å¿˜å´åž‹ã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒ†ã‚£ãƒ³ã‚°ï¼‰ã®ã‚ˆã†ãªæ–°ã—ã„æ¦‚å¿µã‚’ï¼Œä¸–ç•Œã§åˆã‚ã¦å¤§è¦æ¨¡ã«å–ã‚Šå…¥ã‚Œã¦ã„ã‚‹ã“ã¨ã ã¨æ€ã„ã¾ã™ã€ï¼ï¼ãã†èªžã‚‹ã®ã¯ï¼Œæ¥½å¤©æŠ€è¡“ç ”ç©¶æ‰€ã®æ£®æ£å¼¥æ‰€é•·ã€‚4æœˆã«å§‹ã‚ãŸã‚¯ãƒ©ã‚¦ãƒ‰ãƒ»ã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒ†ã‚£ãƒ³ã‚°ã®å–æã‚‚ï¼Œéšåˆ†ã¨é ã„ã¨ã“ã‚ã¾ã§æ¥ãŸãªï¼ï¼ã¨ç†è€…ã¯æ€ã‚ãšã«ã¯ã„ã‚‰ã‚Œãªã‹ã£ãŸã€‚ ã‚¨ãƒ©ãƒ¼å¿˜å´åž‹ã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒ†ã‚£ãƒ³ã‚°ï¼ˆfailure-oblivious computingï¼‰ã¨ã¯ï¼Œç±³Massachusetts Institute of Techno logyï¼ˆMITï¼‰ã®Martin Rinardæ°ã‚‰ãŒæå”±ã—ãŸæ¦‚å¿µï¼ˆRinardæ°ã‚‰ã®è«–æ–‡ï¼‰ã€‚ãƒ¡ãƒ¢ãƒªãƒ¼ã«ä½•ã‚‰ã‹ã®ã‚¨ãƒ©ãƒ¼ãŒç™ºç”Ÿã—ãŸã¨ã—ã¦ã‚‚ï¼Œãƒ—ãƒã‚°ãƒ©ãƒ ã®å‡¦ç†ã‚’åœæ¢ã™ã‚‹ã®ã§ã¯ãªãï¼Œã‚¨ãƒ©ãƒ¼ã‚’ç„¡è¦–ã—ã¦å‡¦ç†ã‚’ç¶™ç¶šã™ã‚‹ã¨ã„ã†è€ƒãˆæ–¹ã ã€‚æ¥½å¤©æŠ€è¡“ç ”ç©¶æ‰€ã®æ£®æ°ã¯ï¼Œã“ã†ä¾‹ãˆãªãŒã‚‰èª¬æ˜Žã™ã‚‹ã€‚ ãƒ‡ãƒ¼ã‚¿ã‚’ã‚ãƒ¼ã¨å€¤ï¼ˆV
kimimasa 2009/04/15
ã‚¯ãƒ©ã‚¦ãƒ‰

cloud

hadoop

MapReduce

SaaS

KeyValue

ãƒã‚¿
ãƒªãƒ³ã‚¯
Hadoopã®è§£æžè³‡æ–™
ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹åˆ†æ•£ã‚·ã‚¹ãƒ†ãƒ ã€ŒHadoopã€ã«é–¢ã™ã‚‹è§£æžè³‡æ–™ã‚’å…¬é–‹ã•ã›ã¦é ‚ã„ã¦ãŠã‚Šã¾ã™ã€‚ã“ã®èª¿æŸ»ã¯NTTãƒ¬ã‚¾ãƒŠãƒ³ãƒˆæ ªå¼ä¼šç¤¾æ§˜ã¨å…±åŒã§è¡Œã„ã¾ã—ãŸ(ãƒ—ãƒ¬ã‚¹ãƒªãƒªãƒ¼ã‚¹)ã€‚ Hadoopè§£æžè³‡æ–™(PDF), æœ€çµ‚æ›´æ–°: 2008/08/25, å…¬é–‹: 2008/08/25 Hadoopã®å®Ÿéš›ã®ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«æ–¹æ³•ãªã©ã«ã¤ãã¾ã—ã¦ã¯ã€å¼Šç¤¾å¤ªç”°ã«ã‚ˆã‚‹ä»¥ä¸‹ã®è¨˜äº‹ã‚‚ã”å‚è€ƒä¸‹ã•ã„ã€‚ Hadoopã€hBaseã§æ§‹ç¯‰ã™ã‚‹å¤§è¦æ¨¡åˆ†æ•£ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚·ã‚¹ãƒ†ãƒ Hadoopã®ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã¨ã‚µãƒ³ãƒ—ãƒ«ãƒ—ãƒã‚°ãƒ©ãƒ ã®å®Ÿè¡Œ è¤‡æ•°ãƒžã‚·ãƒ³ã¸Hadoopã‚’ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã™ã‚‹
kimimasa 2009/04/15
hadoop

MapReduce

KeyValue

opensource

ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹

Apache

oss
ãƒªãƒ³ã‚¯
key-value stores: Anti-RDBMS: A list of distributed key-value stores | Richard Jones, Esq.
Please Note: this was written January 2009 - see the comments for updates and additional information. A lot has changed since I wrote this. Perhaps youâ€™re considering using a dedicated key-value or document store instead of a traditional relational database. Reasons for this might include: You're suffering from Cloud-computing Mania. You need an excuse to 'get your Erlang on' You heard CouchDB was
kimimasa 2009/04/15
ã‚¯ãƒ©ã‚¦ãƒ‰

ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹

cloud

database

DB

Storage

memcached

KeyValue

MapReduce
ãƒªãƒ³ã‚¯
ã€Œã‚ãƒ¼ãƒ»ãƒãƒªãƒ¥ãƒ¼åž‹ãƒ‡ãƒ¼ã‚¿ã‚¹ãƒˆã‚¢ã€é–‹ç™ºè€…ãŒå¤§é›†åˆã—ãŸå¤œ
ã€Œç™ºè¡¨è€…ãŒè‡ªåˆ†ã‚ˆã‚Šã‚‚è‹¥ã„äººã°ã‹ã‚Šã ã€ã€‚å¤–è¦‹ãŒ20ä»£ã«ã—ã‹è¦‹ãˆãªã„æ±äº¬å·¥æ¥å¤§å¦ã®é¦–è—¤ä¸€å¹¸å‡†æ•™æŽˆï¼ˆ1973å¹´ç”Ÿï¼‰ã®é©šãã•ã¾ãŒã€å°‘ã—é¢ç™½ã‹ã£ãŸã€‚2009å¹´2æœˆ20æ—¥ã®å¤œã€å¤šãã®Webä¼æ¥ãŒæ³¨ç›®ã™ã‚‹ã€Œã‚ãƒ¼ãƒ»ãƒãƒªãƒ¥ãƒ¼åž‹ãƒ‡ãƒ¼ã‚¿ã‚¹ãƒˆã‚¢ã€ã‚’é–‹ç™ºã™ã‚‹è‹¥æ‰‹æŠ€è¡“è€…ãŒã€æ±äº¬ãƒ»å…æœ¬æœ¨ã®ã‚°ãƒªãƒ¼æœ¬ç¤¾ã«ä¸€å ‚ã«ä¼šã—ãŸã€‚ ã‚ãƒ¼ãƒ»ãƒãƒªãƒ¥ãƒ¼åž‹ãƒ‡ãƒ¼ã‚¿ã‚¹ãƒˆã‚¢ï¼ˆã¾ãŸã¯ã‚ãƒ¼ãƒ»ãƒãƒªãƒ¥ãƒ¼åž‹ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ï¼‰ã¯ã€å¤§é‡ã®ãƒ¦ãƒ¼ã‚¶ãƒ¼ã¨ãƒ‡ãƒ¼ã‚¿ã‚’æŠ±ãˆã€ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã®ãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹å•é¡Œã¨ã‚³ã‚¹ãƒˆé«˜ã«é ã‚’æ‚©ã¾ã›ã‚‹Webä¼æ¥ãŒæ³¨ç›®ã™ã‚‹æŠ€è¡“ã§ã‚ã‚‹ã€‚è¨˜è€…ã¯åŒæ—¥ã«é–‹å‚¬ã•ã‚ŒãŸã€ŒKey-Value Store å‹‰å¼·ä¼šã€ã«å‚åŠ ã•ã›ã¦ã‚‚ã‚‰ã£ãŸã€‚åˆå¾Œ7æ™‚ã‹ã‚‰11æ™‚ã¾ã§ã€ã‚ãƒ¼ãƒ»ãƒãƒªãƒ¥ãƒ¼åž‹ãƒ‡ãƒ¼ã‚¿ã‚¹ãƒˆã‚¢ã‚’é–‹ç™ºãƒ»ç ”ç©¶ã™ã‚‹è‹¥æ‰‹æŠ€è¡“è€…ãŒç«‹ã¦ç¶šã‘ã«ç™»å ´ã—ã€1äºº15åˆ†ã®æŒã¡æ™‚é–“ã§æˆæžœã‚’ç™ºè¡¨ã—ã€è°è«–ã‚’é‡ãã‚‹ã¨ã„ã†é›†ã¾ã‚Šã ã€‚ å‘¼ã³ã‹ã‘äººã§ã‚ã‚‹ãƒ—ãƒªãƒ•ã‚¡ãƒ¼ãƒ‰ã‚¤ãƒ³ãƒ•ãƒ©ã‚¹ãƒˆãƒ©ã‚¯ãƒãƒ£ãƒ¼ï¼ˆPFIï¼‰æœ€é«˜æŠ€è¡“è²¬ä»»è€…
kimimasa 2009/04/15
memcached

Storage

database

DB

cloud

ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹

MapReduce
ãƒªãƒ³ã‚¯
ã‚¢ãƒžã‚¾ãƒ³ã€ã€ŒHadoopã€ã‚’åˆ©ç”¨ã—ãŸã€ŒAmazon Elastic MapReduceã€ã®ãƒ™ãƒ¼ã‚¿ã‚µãƒ¼ãƒ“ã‚¹ã‚’é–‹å§‹
Amazonã®åä¼šç¤¾Amazon Web Servicesï¼ˆAWSï¼‰ã¯ç±³å›½æ™‚é–“4æœˆ2æ—¥ã€ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã®ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã€ŒHadoopã€ã‚’åˆ©ç”¨ã—ã¦å¤§é‡ã®ãƒ‡ãƒ¼ã‚¿è§£æžã‚’è¡Œã†ã€æ–°ãŸãªã‚¯ãƒ©ã‚¦ãƒ‰ã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒ†ã‚£ãƒ³ã‚°ã®ãƒ™ãƒ¼ã‚¿ã‚µãƒ¼ãƒ“ã‚¹ã‚’é–‹å§‹ã—ãŸã€‚ ã€ŒAmazon Elastic MapReduceã€ã¨å‘¼ã°ã‚Œã‚‹ã“ã®ã‚µãƒ¼ãƒ“ã‚¹ã¯ã€é›†ä¸çš„ãªãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚’å¿…è¦ã¨ã™ã‚‹ä¼æ¥ã€ç ”ç©¶è€…ã€ã‚¢ãƒŠãƒªã‚¹ãƒˆã‚’å¯¾è±¡ã«é–‹ç™ºã•ã‚ŒãŸï¼ˆç™ºè¡¨è³‡æ–™ï¼‰ã€‚Hadoopã¯Yahooãªã©ã®ä¼æ¥ã‚‚æŽ¡ç”¨ã—ã¦ã„ã‚‹ãŒã€Clouderaãªã©ã®æ–°èˆˆä¼æ¥ã¯Hadoopã‚’ä¼æ¥ãƒ‡ãƒ¼ã‚¿ã‚»ãƒ³ã‚¿ãƒ¼ã¨ã—ã¦åˆ©ç”¨ã™ã‚‹ãŸã‚ã®ã‚µãƒ¼ãƒ“ã‚¹ã‚’æä¾›ã—ã¦ã„ã‚‹ã€‚ Amazon Elastic MapReduceã¯ã€åŒç¤¾ã®ã€ŒElastic Compute Cloudã€ï¼ˆEC2ï¼‰ãŠã‚ˆã³ã€ŒSimple Storage Serviceã€ï¼ˆS3ï¼‰ä¸Šã§Hadoopã‚’ç¨¼åƒã•ã›ã‚‹ã€‚é¡§å®¢ã¯Amazon Ela
kimimasa 2009/04/10
MapReduce

amazon

hadoop
ãƒªãƒ³ã‚¯
Groovin' High:map reduceã§æ„Ÿã˜ã‚‹æœªæ¥ - livedoor Blogï¼ˆãƒ–ãƒã‚°ï¼‰
ã€Œã—ã‚‰ãªãƒ¼ã„ã€ã£ã¦äººã«ã€ä½•äººã‹ä¼šã£ãŸã®ã§ã€ã¡ã‚‡ã£ã¨è§£èª¬ã—ãªãŒã‚‰è©±ã‚’ã™ã™ã‚ã¾ã™ã€‚ ã„ã¤ã‚‚å¤§é‡ã®ãƒ‡ãƒ¼ã‚¿ã‚’å‡¦ç†ã—ã¦ã„ã‚‹GoogleãŒã€ã€Œãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰ã§ã“ã‚“ãªã“ã¨ã‚„ã£ã¦åŠ¹æžœã‚’ä¸Šã’ã¦ã¾ã™ã€‚ã€ã¨ã„ã†è«–æ–‡ã‚’2004å¹´ã«ç™ºè¡¨ã—ãŸã€‚ ãã“ã«æ›¸ã‹ã‚Œã¦ã„ã‚‹ã®ã¯ã€åˆ†æ•£å‡¦ç†ã®ãŸã‚ã®ãƒ‡ã‚¶ã‚¤ãƒ³ãƒ‘ã‚¿ãƒ¼ãƒ³ã€ŒMapReduceã€ã€‚ å¤§é‡ãƒ‡ãƒ¼ã‚¿ã‚’ã„ãã¤ã‹ã®å˜ä½ã«åˆ†å‰²ã—ã¦ã€ã‚ãƒ¼ã¨å€¤ã«åˆ†å‰²ã™ã‚‹å‡¦ç†ï¼ˆMapï¼‰ã¨ã€ãã‚Œã‚’é›†è¨ˆã™ã‚‹å‡¦ç†ï¼ˆReduceï¼‰ã«åˆ†ã‘ã‚‹äº‹ã§ã€å‡¦ç†å˜ä½åŒå£«ã®é–¢é€£æ€§ã‚’å¸Œè–„åŒ–ã—ã€ãã®çµæžœåˆ†æ•£ä¸¦åˆ—å‡¦ç†ã‚’å¯èƒ½ã«ã™ã‚‹ã€‚ã¨ã„ã†ã‚‚ã®ã€‚ ã“ã®ãƒ‡ã‚¶ã‚¤ãƒ³ãƒ‘ã‚¿ãƒ¼ãƒ³ã‚’å®Ÿè£…ã—ãŸã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹è£½å“ãŒã€ä¸–ç•Œä¸ã«ã†ã˜ã‚ƒã†ã˜ã‚ƒã‚ã‚‹ã‚“ã ã‘ã©ã€ä¸€ç•ªæœ‰åãªã®ãŒã€Javaã§å®Ÿè£…ã•ã‚ŒãŸã€ŒHadoopã€ã€‚ Javaã§å®Ÿè£…ã•ã‚Œã¦ã‚‹ã‹ã‚‰ã€Javaã§åˆ©ç”¨ã™ã‚Œã°ã„ã„ã‚“ã ã‘ã©ã€ã“ã„ã¤ã«ã¯streamingã¨ã„ã†æ©Ÿèƒ½ã‚‚æè¼‰ã•ã‚Œã¦ã„ã¦ã€ä»–ã®è¨€èªžã§ã‚‚MapReduceã‚’å®Ÿè£…ã§ã
kimimasa 2009/03/20
MapReduce

hadoop

cloud

ã‚¯ãƒ©ã‚¦ãƒ‰
ãƒªãƒ³ã‚¯
1