[B! map reduce] kanbayashiã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

kanbayashi id:kanbayashi

map reduceã«é–¢ã™ã‚‹kanbayashiã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (23)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Kafkaã«ã‚ˆã‚‹ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ å‡¦ç†
This document discusses Kafka, a distributed messaging system originally created by LinkedIn and now an Apache project. It provides an overview of how Kafka works and how various companies use it, including for log processing (LinkedIn), analytics (Facebook, Twitter, Google), and integrating with other techno logies like Hadoop, Zookeeper, HBase and Storm. It also covers Kafka's scalability, perfor
kanbayashi 2013/06/30
hadoop

map reduce

mapreduce

linkedin

slide share

slideshare

storm

stream

realtime
ãƒªãƒ³ã‚¯
S4
S4 - Presentation Transcript S4: the open-source distributed stream computing platformNachi Ueno è‡ªå·±ç´¹ä»‹ æ—¥æœ¬ã®ã©ã“ã‹ã®ç ”ç©¶æ‰€ã¨å‘¼ã°ã‚Œã¦ã„ã‚‹çµ„ç¹”ã§æ—¥ã€…åƒã„ã¦ã„ã¾ã™ æœ€è¿‘ã¯ã€Hadoop,OpenStack,ã‚¹ã‚±ãƒ¼ãƒ«ã™ã‚‹RDBMSã¨ã‹ã«èˆˆå‘³ãŒã‚ã‚Šã¾ã™ SimCloudã¨ã„ã†ã‚¯ãƒ©ã‚¦ãƒ‰ç®¡ç†ãƒ„ãƒ¼ãƒ«ã‚’ä½œã£ã¦ã¾ã™ã€€->ã€€OSSå…¬é–‹äºˆå®šã€€ http://wiki.openstack.org/simcloud Twitter @nati OpenStackeré”ã¨@ç¯‰åœ° S4 (Simple Scala ble Streaming System) ã€€http://s4.io/ S4 is a general-purpose, distributed, scala ble, partially fault-tole
kanbayashi 2013/06/30
hadoop

distributed

s4

stream

map reduce

mapreduce

slide share

slideshare
ãƒªãƒ³ã‚¯
ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ åˆ†æ•£å‡¦ç†ã®å¸¸è˜ã‚’Apache S4ã§èº«ã«ã¤ã‘ã‚‹
ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ åˆ†æ•£å‡¦ç†ã®å¸¸è˜ã‚’Apache S4ã§èº«ã«ã¤ã‘ã‚‹ï¼šãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã®å¸¸è˜ã‚’Javaã§èº«ã«ã¤ã‘ã‚‹ï¼ˆ6ï¼‰ï¼ˆ1/2 ãƒšãƒ¼ã‚¸ï¼‰ Hadoopã‚’ã¯ã˜ã‚ã€Javaè¨€èªžã‚’ä½¿ã£ã¦æ§‹ç¯‰ã•ã‚Œã‚‹ã“ã¨ãŒå¤šã„ã€Œãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã€å‡¦ç†ã®ãŸã‚ã®ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ï¼ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã‚’ç´¹ä»‹ã—ãªãŒã‚‰ã€å¤§é‡ãƒ‡ãƒ¼ã‚¿ã‚’æ´»ç”¨ã™ã‚‹ãŸã‚ã®æŠ€è¡“ã®å¸¸è˜ã‚’èº«ã«ä»˜ã‘ã¦ã„ãé€£è¼‰ Hadoopã®å¼±ç‚¹ã€Œãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ åˆ†æ•£å‡¦ç†ã€ã¨ã¯ ã€Œãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã€å‡¦ç†ã®ãŸã‚ã«Hadoopã‚’ç”¨ã„ã‚‹ã¨ã€ã€Œè¤‡æ•°ã®ãƒžã‚·ãƒ³ã«å¤§é‡ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚’åˆ†æ•£ã—ã¦é£›èºçš„ã«æ€§èƒ½ã‚’å‘ä¸Šã™ã‚‹ã€ã“ã¨ãŒå®¹æ˜“ã«ã§ãã¾ã™ã€‚ ã¨ã“ã‚ãŒHadoopã®å¼±ç‚¹ã¨ã—ã¦ã€ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã‚’ã„ã£ãŸã‚“è“„ç©ã—ã€ãƒãƒƒãƒã§ä¸€æ‹¬å‡¦ç†ã™ã‚‹å½¢æ…‹ã§å‡¦ç†ã™ã‚‹ã®ã§ã€å‡¦ç†ãƒ‡ãƒ¼ã‚¿ãŒç™ºç”Ÿã—ã¦ã‹ã‚‰ã€ãã‚Œã«å¯¾ã™ã‚‹å‡¦ç†çµæžœãŒå¾—ã‚‰ã‚Œã‚‹ã¾ã§ã€å¿…ãšã‚¿ã‚¤ãƒ ãƒ©ã‚°ãŒç™ºç”Ÿã—ã¾ã™ã€‚ã“ã®ãŸã‚ã€ã‚¯ãƒ¬ã‚¸ãƒƒãƒˆã‚«ãƒ¼ãƒ‰ã®ä¸æ£ã‚¢ã‚¯ã‚»ã‚¹æ¤œçŸ¥ã€ã‚»ãƒ³ã‚µãƒ‡ãƒ¼ã‚¿ãªã©ã§ã®ç•°å¸¸å€¤æ¤œå‡ºã®ã‚ˆã†ãªãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ãª
kanbayashi 2013/06/30
stream

ã‚¹ãƒˆãƒªãƒ¼ãƒ

s4

apache s4

distributed

hadoop

map reduce

mapreduce
ãƒªãƒ³ã‚¯
http://storm-project.net/
kanbayashi 2013/06/12
storm

distributed

hadoop

map reduce

stream

stream data

online
ãƒªãƒ³ã‚¯
GPUã¸ã®MapReduceã®é©ç”¨ã«é–¢ã™ã‚‹èª¿æŸ» - nishio-dens's diary
æ¦‚è¦ è¿‘å¹´ã€GPUã®æ€§èƒ½ã¯é£›èºçš„ã«å‘ä¸Šã—ã¦ãŠã‚Šã€ã‚°ãƒ©ãƒ•ã‚£ãƒƒã‚¯å°‚ç”¨ã®å‡¦ç†è£…ç½®ã¨ã—ã¦ã§ã¯ãªãã€æ•°å€¤è¨ˆç®—ç‰ã®æ±Žç”¨å‘ã‘ã®å‡¦ç†ã«åˆ©ç”¨ã™ã‚‹ã€GPGPUã«é–¢ã™ã‚‹ç ”ç©¶ãŒç››ã‚“ã«è¡Œã‚ã‚Œã¦ã„ã‚‹ã€‚GPUã¯å†…éƒ¨ã«å¤šãã®ã‚³ã‚¢ã‚’å‚™ãˆã¦ãŠã‚Šã€NVIDIAç¤¾ã®GeForce GTX 580ã§ã¯ã€512å€‹ã‚‚ã®ã‚³ã‚¢ã‚’ä¿æŒã—ã¦ã„ã‚‹ã€‚ã“ã‚Œã‚‰ã®ã‚³ã‚¢ã™ã¹ã¦ã‚’åŠ¹çŽ‡ã‚ˆãåˆ©ç”¨ã™ã‚‹ã“ã¨ã§ã€GPUã®æŒã£ã¦ã„ã‚‹é«˜ã„ãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹ã‚’å¼•ãå‡ºã™ã“ã¨ãŒå‡ºæ¥ã‚‹ãŒã€GPUã®é«˜ã„ä¸¦åˆ—æ€§ã‚’åˆ©ç”¨ã™ã‚‹ãŸã‚ã«ã¯GPUç‰¹æœ‰ã®å‡¦ç†ã‚’å®Ÿè£…ã™ã‚‹å¿…è¦ãŒã‚ã‚Šã€GPUãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°ã«ãªã˜ã¿ã®ç„¡ã„åˆ©ç”¨è€…ã«ã¨ã£ã¦ã¯å‡¦ç†ã®è¨˜è¿°ãŒå›°é›£ã§ã‚ã‚‹ã€‚ ãã“ã§ã€GPUã‚’å¤§é‡ã®ã‚³ã‚¢ã‚’æŒã£ãŸãƒ—ãƒã‚»ãƒƒã‚µã ã¨è€ƒãˆã€ã“ã‚Œã‚‰ã«å¯¾ã—ã¦MapReduceã‚’é©ç”¨ã™ã‚‹ã“ã¨ã‚’æ¤œè¨Žã™ã‚‹ã€‚è¦ªã—ã¿ã‚„ã™ã„MapReduceã‚¤ãƒ³ã‚¿ãƒ¼ãƒ•ã‚§ãƒ¼ã‚¹ã«ã‚ã¦ã¯ã‚ã¦å‡¦ç†ã‚’è¨˜è¿°ã™ã‚‹ã ã‘ã§ã€GPUã®é«˜ã„ä¸¦åˆ—æ€§ã‚’ç”Ÿã‹ã—ãŸãƒ—ãƒã‚°ãƒ©ãƒ ãŒå®Ÿè£…å¯èƒ½ã¨ãªã‚‹ã€‚åŠ
kanbayashi 2011/11/06
gpgpu

map reduce

mapreduce

data mining

datamining

gpu

research

survey
ãƒªãƒ³ã‚¯
ã“ã£ã¡ã¯æœ¬ç‰©ã®MapReduceã ï¼ ã‚°ãƒ¼ã‚°ãƒ«ãŒAppEngine-MapReduceã‚’ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã§é–‹ç™ºä¸
ã“ã£ã¡ã¯æœ¬ç‰©ã®MapReduceã ï¼ ã‚°ãƒ¼ã‚°ãƒ«ãŒAppEngine-MapReduceã‚’ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã§é–‹ç™ºä¸ ã‚°ãƒ¼ã‚°ãƒ«ã¯Google App Engineä¸Šã§MapReduceå‡¦ç†ã‚’å®Ÿç¾ã™ã‚‹ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã‚’é–‹ç™ºä¸ã ã¨ã€å…ˆæ—¥è¡Œã‚ã‚ŒãŸã‚¤ãƒ™ãƒ³ãƒˆGoogle I/Oã§æ˜Žã‚‰ã‹ã«ã—ã¦ã„ã¾ã™ã€‚ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã®ãƒ›ãƒ¼ãƒ ãƒšãƒ¼ã‚¸ã‚‚Google Codeä¸Šã«ã€Œappengine-mapreduce - Project Hosting on Google Codeã€ã¨ã—ã¦å…¬é–‹ã•ã‚Œã¦ã„ã¾ã™ã€‚ Reduceå‡¦ç†ã‚„Javaç‰ˆã¯ã“ã‚Œã‹ã‚‰ 1ã¤å‰ã®è¨˜äº‹ã€Œã‚°ãƒ¼ã‚°ãƒ«ã«ã‚ˆã‚‹MapReduceã‚µãƒ¼ãƒ“ã‚¹ã€ŒBigQueryã€ãŒç™»å ´ã€‚SQLãƒ©ã‚¤ã‚¯ãªå‘½ä»¤ã§å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿æ“ä½œã€ã§ã¯ã€ã‚°ãƒ¼ã‚°ãƒ«ãŒSQLãƒ©ã‚¤ã‚¯ãªå‘½ä»¤ã‚’ç”¨ã„ã¦å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã®ã‚µãƒ¼ãƒ“ã‚¹ã‚’æä¾›ã™ã‚‹ã“ã¨ã‚’ãŠä¼ãˆã—ã¾ã—ãŸã€‚ è¨˜äº‹ã§ã‚‚æ›¸ã„ãŸã¨ãŠã‚Šã€ã“ã‚Œã¯å†…éƒ¨ã§MapReduceã‚’ä½¿ã£ã¦ã„ã‚‹ã‹ã©
kanbayashi 2010/10/03
gae

google app engine

map reduce

google

*ã“ã‚Œã¯ã™ã”ã„

oss
ãƒªãƒ³ã‚¯
Hadoopã‚’è¡¨è¨ˆç®—ã®ã‚ˆã†ã«ä½¿ãˆã‚‹ã€ŒInfoSphere BigInsightsã€ã€IBMãŒç™ºè¡¨
IBMãŒHadoopã‚’ãƒ™ãƒ¼ã‚¹ã«ã—ãŸãƒ‡ãƒ¼ã‚¿åˆ†æžã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã€ŒInfoSphere BigInsightsã€ã‚’ã€å…ˆé€±ãƒãƒ¼ãƒžã§é–‹å‚¬ã•ã‚ŒãŸåŒç¤¾ã®ã‚¤ãƒ™ãƒ³ãƒˆã€ŒIBM Information On Demand Europe 2010ã€ã§ç™ºè¡¨ã—ã¾ã—ãŸã€‚ InfoSphere BigInsightsã¯ã€Apache Hadoopã‚’ãƒ™ãƒ¼ã‚¹ã«ã—ãŸåˆ†æžã‚¨ãƒ³ã‚¸ãƒ³ã®ã€ŒBigInsights Coreã€ã¨ã€ãã®ä¸Šã§å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã‚’Webãƒ–ãƒ©ã‚¦ã‚¶ã‚’åˆ©ç”¨ã—ã¦ã‚¹ãƒ—ãƒ¬ãƒƒãƒ‰ã‚·ãƒ¼ãƒˆã®ã‚ˆã†ãªãƒ¦ãƒ¼ã‚¶ãƒ¼ã‚¤ãƒ³ã‚¿ãƒ¼ãƒ•ã‚§ã‚¤ã‚¹ã§åˆ†æžã‚’è¡Œãˆã‚‹ã€ŒBigSheetsã€ã®2ã¤ã‹ã‚‰æ§‹æˆã•ã‚Œã¾ã™ã€‚ BigSheetsã§è¡¨è¨ˆç®—ã®ã‚ˆã†ã«ãƒ‡ãƒ¼ã‚¿åˆ†æž BigInsights Coreã®ãƒ™ãƒ¼ã‚¹ã¨ãªã£ã¦ã„ã‚‹Hadoopã¯ã€æ•°ç™¾ä¸‡ä»¶ãªã©å¤§è¦æ¨¡ãªãƒ‡ãƒ¼ã‚¿ã‚’å¤§é‡ã®ã‚µãƒ¼ãƒã§åˆ†æ•£å‡¦ç†ã™ã‚‹ãŸã‚ã®ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã®ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã§ã™ã€‚Yahoo!ã‚„Facebookã€æ—¥æœ¬ã§ã¯Cookpad
kanbayashi 2010/05/26
hadoop

ibm

splead sheet

map reduce
ãƒªãƒ³ã‚¯
ã‚°ãƒ¼ã‚°ãƒ«ã«ã‚ˆã‚‹MapReduceã‚µãƒ¼ãƒ“ã‚¹ã€ŒBigQueryã€ãŒç™»å ´ã€‚SQLãƒ©ã‚¤ã‚¯ãªå‘½ä»¤ã§å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿æ“ä½œ
ã€Œæ•°å…†ä»¶ã®ãƒ‡ãƒ¼ã‚¿ã‚‚å¯¾è©±çš„ã«ã€é«˜é€Ÿã«åˆ†æžã§ãã‚‹ã€ã€‚ã‚°ãƒ¼ã‚°ãƒ«ã¯5æœˆ19æ—¥ã«ã“ã®ã‚ˆã†ãªè¡¨ç¾ã§æ–°ã—ã„ã‚µãƒ¼ãƒ“ã‚¹ã€ŒBigQueryã€ã®ç™»å ´ã‚’ç´¹ä»‹ã™ã‚‹ã‚¨ãƒ³ãƒˆãƒªã‚’ã€ãƒ–ãƒã‚°ã«ãƒã‚¹ãƒˆã—ã¦ã„ã¾ã™ã€‚ ã‚°ãƒ¼ã‚°ãƒ«ãŒå…¬é–‹ã—ãŸBigQueryã¯ã€Hadoopã‚„ãƒ‡ãƒ¼ã‚¿ã‚¦ã‚§ã‚¢ãƒã‚¦ã‚¹ãªã©ã‚’ç”¨ã„ã¦å¤šãã®ä¼æ¥ãŒè¡ŒãŠã†ã¨ã—ã¦ã„ã‚‹å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ï¼ˆã„ã‚ã‚†ã‚‹ã€ŒBig Dataã€ï¼‰ã®åˆ†æžã‚’ã€ã‚°ãƒ¼ã‚°ãƒ«ã®ã‚¯ãƒ©ã‚¦ãƒ‰ã§å¯èƒ½ã«ã—ã¾ã™ã€‚åˆ©ç”¨è€…ã¯Google StorageçµŒç”±ã§å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã‚’è»¢é€ã—ã€SQLãƒ©ã‚¤ã‚¯ãªå‘½ä»¤ã«ã‚ˆã£ã¦æŠ½å‡ºã‚„åˆ†æžã‚’è¡Œã„ã¾ã™ã€‚ ã¾ã‚‹ã§ã‚°ãƒ¼ã‚°ãƒ«ãŒå¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã®MapReduceã‚’ãƒ›ã‚¹ãƒ†ã‚£ãƒ³ã‚°ã—ã€ãã®æ©Ÿèƒ½ã‚’ã‚µãƒ¼ãƒ“ã‚¹ã¨ã—ã¦æä¾›ã™ã‚‹ã‚ˆã†ãªã‚‚ã®ãŒBigQueryã¨ã„ãˆã¾ã™ï¼ˆãŸã ã—å…¬é–‹ã•ã‚ŒãŸã€ŒBigQueryã€ã®èª¬æ˜Žã«ã¯ã€å†…éƒ¨ã§MapReduceã‚’åˆ©ç”¨ã—ã¦ã„ã‚‹ã®ã‹ã©ã†ã‹ã®è¨˜è¿°ã¯ãªã„ã®ãŸã‚ã€MapReduceã€Œçš„ã€ãªã‚µãƒ¼ãƒ“ã‚¹ã¨è¡¨ç¾ã™ã¹ãã‹ã‚‚ã—ã‚Œ
kanbayashi 2010/05/26
google

map reduce

google storage

sql
ãƒªãƒ³ã‚¯
http://agilecatcloud.com/2009/11/09/hadoop-%EF%BC%9A-twitter-%E3%82%88%E3%80%81%E3%81%8A%E5%89%8D%E3%82%82%E3%81%8B%EF%BC%81/
kanbayashi 2010/03/10
Twitterã‚‚Hadoop(pig?)ã‚’ä½¿ã£ã¦ã„ã‚‹ / pigã¯SQLãƒ©ã‚¤ã‚¯ã®DSLã«ã‚ˆã‚‹Hadoopä¸Šã§ã®åˆ†æ•£å‡¦ç†è¨˜è¿°ã‚’æä¾›ã™ã‚‹ / Hiveã‚‚åŒã˜ã‚ˆã†ãªäº‹ãŒã§ããŸã‚ˆã†ãªã€‚

twitter

hadoop

map reduce

pig
ãƒªãƒ³ã‚¯
Google Code Archive - Long-term storage for Google Code Project Hosting.
Code Archive Skip to content Google About Google Privacy Terms
kanbayashi 2010/01/21
Cloud MapReduce was initially developed at Accenture Technology Labs. It is a MapReduce implementation on top of the Amazon Cloud OS.

cloud

map reduce

oss

*ã‚ã¨ã§è©¦ã™

ec2

amazon ec2

accenture
ãƒªãƒ³ã‚¯
IBM Developer
IBM Developer is your one-stop location for getting hands-on training and learning in-demand skills on relevant techno logies such as generative AI, data science, AI, and open source.
kanbayashi 2009/12/07
"RESTful API ã«ã‚ˆã‚‹ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã‚¢ã‚¯ã‚»ã‚¹ã€MapReduce ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã«ã‚ˆã‚‹ã‚¯ã‚¨ãƒªã€é–¢æ•°åž‹è¨€èªž Erlang ã«ã‚ˆã‚‹å®Ÿè£…ã€ã¨ã„ã£ãŸä»Šè©±é¡Œã®æœ€æ–°æŠ€è¡“ã‚’å–ã‚Šå…¥ã‚ŒãŸ Apache CouchDBã¨ã„ã†éžãƒªãƒ¬ãƒ¼ã‚·ãƒ§ãƒŠãƒ«ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ãŒæ³¨ç›®ã•ã‚Œã¦ã„ã¾ã™"

couchdb

db

apache

map reduce

MapReduce
ãƒªãƒ³ã‚¯
blog.katsuma.tv
å‰å›žã€JavaScriptã§Map Reduceã®ã‚³ãƒ¼ãƒ‰ãŒæ›¸ã‘ã‚‹Hadoop Streamingã«ã¤ã„ã¦ç´¹ä»‹ã—ã¾ã—ãŸã€‚ æ¨™æº–å…¥å‡ºåŠ›ã•ãˆã‚µãƒãƒ¼ãƒˆã•ã‚Œã¦ã‚ã‚Œã°ã€ä»»æ„ã®ã‚³ãƒ¼ãƒ‰ã§Map Reduuceã®å‡¦ç†ãŒæ›¸ã‘ã‚‹ã€ã¨ã„ã†ã‚‚ã®ã§ã—ãŸãŒã€ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã¯ãã‚‚ãã‚‚é¢å€’ãã•ãŒã‚Šã€‚ã‚³ãƒ¼ãƒ‰ã‚‚æ›¸ãã®ã‚‚é¢å€’ã§ã™ã€‚ ã¨ã€ã„ã†ã‚ã‘ã§ã€ä»Šå›žã¯ã‚‚ã†ã‚³ãƒ¼ãƒ‰ã™ã‚‰æ›¸ã‹ãšã«SQLãƒ©ã‚¤ã‚¯ã§Map Reduceã§ãã‚‹Hiveã¨ã„ã†ãƒ—ãƒãƒ€ã‚¯ãƒˆã«ã¤ã„ã¦ã€ã¾ã¨ã‚ãŸã„ã¨æ€ã„ã¾ã™ã€‚ Hive Hiveã¨ã¯ã€ç°¡å˜ã«è¨€ã†ã¨Hadoopä¸Šã§å‹•ä½œã™ã‚‹RDBã®ã‚ˆã†ãªã‚‚ã®ã§ã™ã€‚ HDFSãªã©ã®åˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ä¸Šã«å˜åœ¨ã™ã‚‹ãƒ‡ãƒ¼ã‚¿ã«å¯¾ã—ã¦ã€HiveQLã¨ã„ã†SQLãƒ©ã‚¤ã‚¯ãªè¨€èªžã§æ“ä½œã§ãã¾ã™ã€‚ ã§ã€é¢ç™½ã„ã®ãŒHiveQLã®æ“ä½œã¯åŸºæœ¬çš„ã«Map Reduceã®ãƒ©ãƒƒãƒ‘ãƒ¼ã«ãªã£ã¦ã„ã‚‹ã“ã¨ã€‚ è¦ã™ã‚‹ã«ã€SELECTæ–‡å®Ÿè¡Œã™ã‚‹ã¨è£ã§Map&Reduceã®ã‚¿ã‚¹ã‚¯ãŒèµ°ã‚Šå‡º
kanbayashi 2009/12/04
[*ã“ã‚Œã¯ã™ã”ã„[facebook]

hadoop

distributed

map reduce

mapreduce

katsuma
ãƒªãƒ³ã‚¯
Sector/Sphere: High Performance Distributed Data Storage and Processing
High Performance Distributed File System and Parallel Data Processing Engine Sector/Sphere supports distributed data storage, distribution, and processing over large clusters of commodity computers, either within a data center or across multiple data centers. Sector is a high performance, scala ble, and secure distributed file system. Sphere is a high performance parallel data processing engine tha
kanbayashi 2009/12/02
research

storage

distributed

map reduce

MapReduce

gfarm

file system

filesystem
ãƒªãƒ³ã‚¯
Amazon Elastic MapReduceã‚’ä½¿ã£ã¦ã¿ãŸ - moratorium
Amazon Elastic MapReduceã‚’ä½¿ã£ã¦ã¿ãŸ 2009-04-03 (Fri) 3:06 Amazon EC2 é€£æ—¥ã®EC2ãƒã‚¿ã§ã™ã€‚æœ¬æ—¥ã€Amazonã‹ã‚‰Elastic MapReduceã¨ã„ã†ã‚µãƒ¼ãƒ“ã‚¹ãŒãƒªãƒªãƒ¼ã‚¹ã•ã‚Œã¾ã—ãŸã€‚å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿å‡¦ç†æŠ€è¡“ãŒä¸€æ°—ã«æ°‘é–“ã®æ‰‹ã«ä¸‹ã‚Šã¦ãã‚‹ã€ã¾ã•ã«é©å‘½çš„ãªã‚µãƒ¼ãƒ“ã‚¹ã ã¨æ€ã„ã¾ã™ã€‚ Amazon Elastic MapReduce Amazon ElasticMapReduce ç´¹ä»‹ãƒ“ãƒ‡ã‚ª With Hadoop, Amazon Adds A Web-Scale Data Processing Engine To Its Cloud Computer by techcrunch.com Elastic MapReduceã¯ã€Googleã®åŸºç›¤æŠ€è¡“ã®ä¸€ã¤ã§ã‚ã‚‹MapReduceã‚’æ™‚é–“å˜ä½èª²é‡‘ã§å®Ÿè¡Œã§ãã‚‹ã‚µãƒ¼ãƒ“ã‚¹ã§ã™ã€‚MapReduceã«ã¤ã„ã¦ã¯ä»¥
kanbayashi 2009/08/24
ec2

map reduce

amazon

kzk_mover

elastic map reduce
ãƒªãƒ³ã‚¯
Big Data Platform - Amazon EMR - AWS
Amazon EMR Serverless is a new option in Amazon EMR that makes it easy and cost-effective for data engineers and analysts to run applications built using open source big data frameworks such as Apache Spark, Hive or Presto, without having to tune, operate, optimize, secure or manage clusters.
kanbayashi 2009/04/02
amazon

aws

map reduce

MapReduce

*ã“ã‚Œã¯ã™ã”ã„

vm

ec2

distributed

*ã‚ã¨ã§èªã‚€
ãƒªãƒ³ã‚¯
POPè¾žæ›¸.com
Please report how to reproduce the circumstances of this error to: Thanks for your help making this a better site.
kanbayashi 2008/09/09
amazon

ec2

cloud

hadoop

map reduce

distributed filesystem
ãƒªãƒ³ã‚¯
Googleç¤¾ã®åŸºç›¤æŠ€è¡“ã€ŒMapReduceã€ã«ã¤ã„ã¦ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã«èžã
Googleç¤¾ã®åŸºç›¤æŠ€è¡“ã€ŒMapReduceã€ã«ã¤ã„ã¦ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã«èžã ç±³Google Inc. Christophe Biscigliaæ° ç±³Google Inc.ã§å¤§è¦æ¨¡ä¸¦åˆ—ãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°ã®æ•™è‚²ã‚³ãƒ¼ã‚¹ã€ŒAcademic Cluster Computing Initiativeï¼ˆACCIï¼‰ã€ã®ç«‹æ¡ˆè€…ã§ã‚ã‚‹Senior Software Engineerã®Christophe Biscigliaæ°ã«ï¼Œã‚¤ãƒ³ã‚¿ãƒ“ãƒ¥ãƒ¼ã™ã‚‹æ©Ÿä¼šã‚’å¾—ãŸã€‚Biscigliaæ°ã¯ç¾åœ¨ï¼Œä¸å›½ãƒ»ä¸Šæµ·ã§ã‚µãƒ¼ãƒ“ã‚¹ã®é–‹ç™ºã«æºã‚ã£ã¦ã„ã‚‹ã€‚ â€•â€•Googleã®ã‚µãƒ¼ãƒ“ã‚¹ã‚’æ”¯ãˆã‚‹æŠ€è¡“ã¨ã—ã¦ï¼ŒACCIã§æ•™ãˆã¦ã„ã‚‹ã®ã¯ä½•ã‹ã€‚ Christophe Biscigliaæ°ã€€ä¸¦åˆ—å‡¦ç†ã®ãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°/ãƒ¢ãƒ‡ãƒ«ã§ã‚ã‚‹ã€ŒMapReduceã€ã¨åˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ãƒ»ã‚·ã‚¹ãƒ†ãƒ ã§ã‚ã‚‹ã€ŒGFSï¼ˆGoogle File Systemï¼‰ã€ã ã€‚ã“ã‚Œã‚‰ã‚’å®Ÿè£…ã—ãŸã€ŒHadoop
kanbayashi 2008/07/23
åŸºæœ¬çš„ã«ã¯Google App Engineã¯ãƒ‡ãƒ¼ã‚¿APIã ã€‚å¤§ããªãƒ†ãƒ¼ãƒ–ãƒ«ã‚’å¤§é‡ã®ãƒ¦ãƒ¼ã‚¶ãƒ¼ãŒã‚¢ã‚¯ã‚»ã‚¹ã§ãã‚‹ã‚¹ã‚±ãƒ¼ãƒ©ãƒ“ãƒªãƒ†ã‚£ã‚’ç¢ºä¿ã™ã‚‹ã€‚ãƒãƒ¼ãƒ¬ãƒ™ãƒ«ã®ãƒ•ã‚¡ã‚¤ãƒ«ãƒ»ã‚·ã‚¹ãƒ†ãƒ ã«ã¯GFSã‚’ä½¿ã£ã¦ã„ã‚‹ã€‚

google

map reduce

google app engine
ãƒªãƒ³ã‚¯
HTTPãƒ™ãƒ¼ã‚¹ã«ã‚ˆã‚‹MapReduceãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯Â·HTTPMR MOONGIFT
å¤§è¦æ¨¡ãªãƒ‡ãƒ¼ã‚¿ã‚’åˆ†æ•£å‡¦ç†ã™ã‚‹ãŸã‚ã®æŠ€è¡“ã¨è¨€ãˆã°MapReduceã ã€‚é€šå¸¸ã®ä¼æ¥ã§ã¯é›£ã—ã„ã€æ•°ä¸‡å°ã®ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒ†ã‚£ãƒ³ã‚°ã‚’é§†ä½¿ã—ãŸãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚’å¯èƒ½ã«ã™ã‚‹Googleã®æ ¹å¹¹ã‚’ã•ã•ã›ã‚‹ä¸€æŠ€è¡“ã«ãªã£ã¦ã„ã‚‹ã€‚ å‡¦ç†ã®ä¸€è¦§ ãã‚“ãªMapReduceã¯ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã§å®Ÿè£…ã•ã‚Œã‚‹ã‚‚ã®ã‚‚ã‚ã‚‹ãŒã€æœ¬æ ¼çš„ã«å®Ÿè£…ã™ã‚‹ã«ã¯ãƒãƒ¼ãƒ‰ã‚¦ã‚§ã‚¢ã‚„ã‚¤ãƒ³ãƒ•ãƒ©ã®å˜åœ¨ãŒå¿…è¦ã«ãªã‚‹ã€‚ã ãŒã€ã“ã‚Œã‚’ä½¿ãˆã°ãƒãƒ¼ãƒ‰ã‚¦ã‚§ã‚¢ã‚‚ç„¡ç”¨ã§MapReduceã‚’ä½“æ„Ÿã§ãã‚‹ã€‚ ä»Šå›žç´¹ä»‹ã™ã‚‹ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ãƒ»ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã¯HTTPMRã€Google App Engineä¸Šã§å‹•ä½œã™ã‚‹MapReduceå®Ÿè£…ã ã€‚ HTTPMRã¯Google App Engineä¸Šã§å‹•ä½œã™ã‚‹ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã§ã€HTTPãƒ™ãƒ¼ã‚¹ã§MapReduceã®ã‚ˆã†ã«åˆ†æ•£å‡¦ç†ã‚’è¡Œãˆã‚‹ã‚ˆã†ã«ãªã‚‹ã€‚ãƒªã‚¯ã‚¨ã‚¹ãƒˆã¯ãƒ©ãƒ³ãƒ€ãƒ ã«é¸ã°ã‚ŒãŸã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ã‚¿ä¸Šã§å®Ÿè¡Œã•ã‚Œã‚‹ã€‚å„ãƒªã‚¯ã‚¨ã‚¹ãƒˆã¯æ•°ç§’ã§ã‚¿ã‚¤ãƒ ã‚¢ã‚¦ãƒˆã™ã‚‹ã‚ˆã†ã«ãªã£
kanbayashi 2008/07/16
google

google app engine

map reduce

grid

parallel
ãƒªãƒ³ã‚¯
YLUG ç¬¬88å›žã‚«ãƒ¼ãƒãƒ«èªæ›¸ä¼š Googleã®åŸºç›¤ã‚¯ãƒãƒ¼ãƒ³Hadoopã«ã¤ã„ã¦
ç™ºè¡¨è€…: æ±äº¬å¤§å¦ å¤ªç”°ä¸€æ¨¹ ã•ã‚“å†…å®¹ï¼šGoogleã§ã¯ï¼‘æ—¥ã«ä½•Tã‚‚ã®ãƒ‡ãƒ¼ã‚¿ãŒå‡¦ç†ã•ã‚Œã€æ¤œç´¢ãƒ»åºƒå‘Šç‰ã®ã‚µãƒ¼ãƒ“ã‚¹ã«æ´»ã‹ã•ã‚Œã¦ã„ã¾ã™ã€‚ã“ã®ã‚ˆã†ãªè†¨å¤§ãªãƒ‡ãƒ¼ã‚¿ã‚’å‡¦ç†ã™ã‚‹ç‚ºã®åŸºç›¤æŠ€è¡“ã¨ã—ã¦GoogleFile Systemã¨MapReduce ãŒä½¿ã‚ã‚Œã¦ã„ã‚‹äº‹ãŒè«–æ–‡ã§ç™ºè¡¨ã•ã‚Œã¦ã„ã¾ã™ã€‚ä»Šå›žã¯ãã®ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã‚¯ãƒãƒ¼ãƒ³ã§ã‚ã‚‹Hadoopã®æ¦‚è¦ã¨å®Ÿè£…ã«ã¤ã„ã¦ç™ºè¡¨ã—ã¾ã™ã€‚
kanbayashi 2008/07/15
*ã‚ã¨ã§è¦‹ã‚‹

hadoop

niconico

map reduce

google

linux

ylug
ãƒªãƒ³ã‚¯
Google App Engine - Google Code
Ã£Â‚Â¹Ã£Â‚Â¿Ã£ÂƒÂ¼Ã£ÂƒÂˆ Ã£Â‚Â¬Ã£Â‚Â¤Ã£ÂƒÂ‰ App Engine Ã£Â‚Â¢Ã£Â‚Â«Ã£Â‚Â¦Ã£ÂƒÂ³Ã£ÂƒÂˆÃ£ÂÂ«Ã£ÂƒÂÃ£Â‚Â°Ã£Â‚Â¤Ã£ÂƒÂ³Ã£ÂÂ—Ã£ÂÂ¾Ã£ÂÂ™Ã£Â€Â‚ App Engine SDK Ã£Â‚Â’Ã£ÂƒÂ€Ã£Â‚Â¦Ã£ÂƒÂ³Ã£ÂƒÂÃ£ÂƒÂ¼Ã£ÂƒÂ‰Ã£ÂÂ—Ã£ÂÂ¾Ã£ÂÂ™Ã£Â€Â‚ Ã£Â‚Â¹Ã£Â‚Â¿Ã£ÂƒÂ¼Ã£ÂƒÂˆ Ã£Â‚Â¬Ã£Â‚Â¤Ã£ÂƒÂ‰Ã£Â‚Â’Ã¥ÂÂ‚Ã§Â…Â§Ã£ÂÂ—Ã£ÂÂ¾Ã£ÂÂ™Ã£Â€Â‚ Ã£Â‚Â¢Ã£ÂƒÂ—Ã£ÂƒÂªÃ£Â‚Â±Ã£ÂƒÂ¼Ã£Â‚Â·Ã£ÂƒÂ§Ã£ÂƒÂ³ Ã£Â‚Â®Ã£ÂƒÂ£Ã£ÂƒÂ©Ã£ÂƒÂªÃ£ÂƒÂ¼Ã£ÂÂ®Ã£Â‚ÂµÃ£ÂƒÂ³Ã£ÂƒÂ—Ã£ÂƒÂ« Ã£Â‚Â¢Ã£ÂƒÂ—Ã£ÂƒÂªÃ£Â‚Â±Ã£ÂƒÂ¼Ã£Â‚Â·Ã£ÂƒÂ§Ã£ÂƒÂ³Ã£Â‚Â’Ã£ÂÂ”Ã¨Â¦Â§Ã£ÂÂÃ£ÂÂ Ã£ÂÂ•Ã£ÂÂ„Ã£Â€Â‚
kanbayashi 2008/07/13
google

google app engine

grid

*ã“ã‚Œã¯ã™ã”ã„

map reduce
ãƒªãƒ³ã‚¯
1 2 æ¬¡ã®ãƒšãƒ¼ã‚¸