[B! Hadoop] limeGã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

limeG id:limeG

Hadoopã«é–¢ã™ã‚‹limeGã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (6)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

MapReduceã«ãƒãƒ£ãƒ¬ãƒ³ã‚¸ã—ã‚ˆã†
ä»Šå›žã¯ã„ã‚ˆã„ã‚ˆã€MapReduceã‚’ä½¿ã£ã¦ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚’å®Ÿè·µã—ã¦ã¿ã¾ã™ã€‚ãã®å‰ã«ã¾ãšã€MapReduceã®ä»•çµ„ã¿ã‚’å¤§ã¾ã‹ã«ç†è§£ã—ã¦ãŠãã¾ã—ã‚‡ã†ã€‚ Hadoopã®ç‰¹å¾´ã®ä¸€ã¤ã¯ã€ã€Œä¸¦åˆ—å‡¦ç†å¯èƒ½ãªå°ã•ãªã‚¿ã‚¹ã‚¯ã«åˆ†å‰²ã—ã€ç©ºã„ãŸCPUã«é †æ¬¡å‰²ã‚Šå½“ã¦ã‚‹ã€ã¨ã„ã†ç‚¹ã§ã™ã€‚ã©ã‚“ãªã«å°æ•°ãŒå¤šãã¦ã‚‚ã€å…¨ä½“ã®å‡¦ç†ã‚’ç‹¬ç«‹ã—ãŸå°ã•ãªã‚¿ã‚¹ã‚¯ã«åˆ†å‰²ã—ã¦ä¸¦è¡Œå‡¦ç†ã™ã‚‹ã“ã¨ã§ã€CPUãªã©ã®è³‡æºã‚’åŠ¹çŽ‡çš„ã«ä½¿ã†ã“ã¨ãŒã§ãã‚‹ã¨ã„ã†ã‚‚ã®ã§ã™ã€‚ã“ã®ä»•çµ„ã¿ã‚’å®Ÿç¾ã—ã¦ã„ã‚‹ã®ãŒã€MapReduceã§ã™ã€‚ MapReduceã¯ã™ã¹ã¦ã®ãƒ‡ãƒ¼ã‚¿ã‚’éžå¸¸ã«ã‚·ãƒ³ãƒ—ãƒ«ãªå½¢ã§è¡¨ç¾ã—ã¾ã™ã€‚å¤§ããåˆ†ã‘ã¦ã€ã€Œkey-valueãƒšã‚¢ã€ã¨ã€Œkeyvaluesãƒšã‚¢ã€ã®2ã¤ã®ãƒ¢ãƒ‡ãƒ«ãŒã‚ã‚Šã¾ã™ï¼ˆå›³1ï¼‰ã€‚å¾Œè€…ã®valueã«ã€Œsã€ãŒä»˜ã„ã¦ã„ã‚‹ã¨ã“ã‚ãŒé•ã„ã¾ã™ã€‚ key-valueãƒšã‚¢ã¯ãã®åã®é€šã‚Šã€ 1å€‹ã®ã€Œkeyã€ã¨1å€‹ã®ã€Œvalueã€ãŒãƒšã‚¢ã«ãªã£ã¦ã„ã‚‹ãƒ‡ãƒ¼ã‚¿ã§ã™ã€‚ä¾‹ãˆã°ã€ k
limeG 2012/03/16
Hadoop
ãƒªãƒ³ã‚¯
åˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ HDFSã‚’ä½“é¨“ã™ã‚‹
GoogleãŒä½¿ã£ã¦ã„ã‚‹åˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã€ŒGFSã€ã®ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã«ã‚ˆã‚‹å®Ÿè£…ãŒã€Hadoopã®ã€ŒHDFSã€ã§ã™ã€‚ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã¨ã„ã†ã¨ã€Windowsã®ã€ŒFAT32ã€ã‚„ã€ŒNTFSã€ã€Linuxã®ã€Œext3ã€ã‚„ã€Œext4ã€ãªã©ãŒæœ‰åã§ã™ãŒã€HDFSã¯ãã‚Œã‚‰ã¨ã¯ã¡ã‚‡ã£ã¨ä½¿ã„å‹æ‰‹ãŒé•ã„ã¾ã™ã€‚ã“ã“ã§ã¯ã€å®Ÿéš›ã«HDFSã‚’ä½¿ã£ã¦ã¿ã¾ã™ã€‚ ã¨ã¯ã„ãˆå¤šæ•°ã®PCã‚’ç”¨æ„ã™ã‚‹ã®ã¯å¤§å¤‰ã§ã—ã‚‡ã†ã‹ã‚‰ã€ã“ã“ã§ã‚‚ç–‘ä¼¼åˆ†æ•£ãƒ¢ãƒ¼ãƒ‰ã§å‹•ã‹ã—ã¾ã™ã€‚åŒã˜ãƒžã‚·ãƒ³ä¸Šã«è¤‡æ•°ã®HDFSã®ãƒ‡ãƒ¼ãƒ¢ãƒ³ã‚’ç«‹ã¡ä¸Šã’ã‚‹æ–¹æ³•ã§ã™ã€‚HDFSã®ãƒ‡ãƒ¼ãƒ¢ãƒ³ã¨ã¯å‰ã§èª¬æ˜Žã—ãŸã‚ˆã†ã«ã€NameNodeã‚„DataNodeã§ã™ã€‚æ“¬ä¼¼åˆ†æ•£ãƒ¢ãƒ¼ãƒ‰ã§ã‚‚ã€è¤‡æ•°ã®ãƒ‡ãƒ¼ãƒ¢ãƒ³ãŒå”èª¿ã—ã¦å‹•ãã“ã¨ã§ã€HDFSã‚’åˆ©ç”¨ã§ãã‚‹ã‚ˆã†ã«ãªã‚Šã¾ã™ã€‚å®Ÿéš›ã«HDFSã‚’ä½“é¨“ã—ã¦ã¿ã‚‹å‰ã«ã€HDFSã¨ã¯ã©ã†ã„ã†ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ãªã®ã‹ç°¡å˜ã«ç†è§£ã—ã¦ãŠãã¾ã—ã‚‡ã†ã€‚ ã™ã§ã«èª¬æ˜Žã—ãŸã¨ãŠã‚Šã€HDFSã§ã¯Nam
limeG 2012/03/16
Hadoop
ãƒªãƒ³ã‚¯
Hadoopã‚’ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã—ã‚ˆã†
Hadoopã‚’å‹•ã‹ã™ã«ã¯ã€å°‘ãªãã¨ã‚‚1å°ã®Linuxãƒžã‚·ãƒ³ãŒå¿…è¦ã«ãªã‚Šã¾ã™ã€‚å®Ÿéš›ã«åŠ¹æžœã‚’å¾—ã‚‹ãŸã‚ã«ã¯ã€å·¨å¤§ãªãƒ‡ãƒ¼ã‚¿ã¨ã€ç‰©ç†çš„ã«å¤šæ•°ã®PCã‚’ç”¨æ„ã™ã‚‹å¿…è¦ãŒã‚ã‚Šã¾ã™ã€‚ Hadoopã«ã¯3ã¤ã®å‹•ä½œãƒ¢ãƒ¼ãƒ‰ãŒã‚ã‚Šã€ã©ã®ãƒ¢ãƒ¼ãƒ‰ã‚’å‹•ã‹ã™ã‹ã«ã‚ˆã£ã¦å¿…è¦ãªç’°å¢ƒãŒç•°ãªã‚Šã¾ã™ï¼ˆè¡¨1ï¼‰ã€‚æœ¬è¨˜äº‹ã§ã¯ã€1å°ã®ãƒžã‚·ãƒ³ä¸Šã§Hadoopã®å„ãƒ‡ãƒ¼ãƒ¢ãƒ³ã‚’èµ·å‹•ã™ã‚‹ã€Œç–‘ä¼¼åˆ†æ•£ãƒ¢ãƒ¼ãƒ‰ã€ã‚’å‹•ã‹ã—ã€Hadoopã®å‹•ä½œã‚’ç¢ºèªã—ã¾ã™ã€‚ Linuxç’°å¢ƒã®æº–å‚™ ã¾ãšæœ€åˆã«ã€LinuxãŒå‹•ä½œã™ã‚‹ç’°å¢ƒã‚’æº–å‚™ã—ã¦ãã ã•ã„ã€‚ã‚‚ã—å¤ã„PCãªã©ã€ç¾åœ¨ä½¿ã£ã¦ã„ãªã„PCãŒã‚ã‚Œã°ã€ãã“ã«ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã™ã‚Œã°ã„ã„ã§ã—ã‚‡ã†ã€‚ãã®ã‚ˆã†ãªä½™ã£ãŸPCãŒç„¡ã‘ã‚Œã°Windowsãƒžã‚·ãƒ³ä¸Šã®ä»®æƒ³åŒ–ã‚½ãƒ•ãƒˆã§ã€Linuxã‚’ä½¿ã†ã¨ã„ã†æ–¹æ³•ã‚‚ã‚ã‚Šã¾ã™ã€‚ Linuxã®ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã¯æ¥µã‚ã¦ç°¡å˜ã§ã™ã€‚Linuxã‚’ã“ã‚Œã¾ã§ã¾ã£ãŸãè§¦ã‚ŒãŸã“ã¨ãŒãªã„äººã§ã‚‚ã€GUIã®æ“ä½œã§ç°¡å˜ã«ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã§ãã¾ã™ã€‚L
limeG 2012/03/16
Hadoop
ãƒªãƒ³ã‚¯
Hadoopã£ã¦ä½•ãŒã™ã”ã„ã®
Hadoopã¯ã€æ™®åŠå“ã®PCã‚µãƒ¼ãƒãƒ¼ã‚’å¤šæ•°ã¤ãªã’ã¦ã€èŽ«å¤§ãªæƒ…å ±ã‚’å‡¦ç†ã§ãã‚‹ã‚ˆã†ã«ã—ãŸã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ãƒ»ã‚½ãƒ•ãƒˆã‚¦ã‚¨ã‚¢ã§ã™ã€‚Yahoo!ã‚„Facebookãªã©ã®Webã‚µãƒ¼ãƒ“ã‚¹ã®é–‹ç™ºè€…ãŒå¤šæ•°ã€Hadoopã®é–‹ç™ºã«å‚åŠ ã—ã¦ã„ã¦ã€ã“ã‚Œã‚‰ã®ä¼æ¥ã«ã‚ˆã£ã¦å®Ÿéš›ã«æ´»ç”¨ã•ã‚Œã¦ã„ã¾ã™ã€‚ æœ¬é€£è¼‰ã§ã¯ã¾ãšã€ã„ã¾ç†±ã„è¦–ç·šã‚’é›†ã‚ã¦ã„ã‚‹Hadoopã‚’å®Ÿéš›ã«è§¦ã£ã¦ã„ãã“ã¨ã§ã€Webã‚µãƒ¼ãƒ“ã‚¹ã‚’æ”¯ãˆã¦ã„ã‚‹ã€Œå¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã€ã‚’ä½“é¨“ã—ã¦ã„ãã¾ã™ã€‚ ã¾ãšæœ€åˆã«ã€Hadoopã¨ã¯ä½•ã‹ã€ä½•ãŒã€Œã†ã‚Œã—ã„ã€ã®ã‹ã€ã‚’èª¬æ˜Žã—ã¾ã™ã€‚ãã‚Œã«ã¯ã€ã€Œã‚¹ã‚±ãƒ¼ãƒ«ã‚¢ã‚¦ãƒˆã€ã¨ã„ã†è¨€è‘‰ã®æ„å‘³ã‚’çŸ¥ã£ã¦ãŠãå¿…è¦ãŒã‚ã‚Šã¾ã™ã€‚Hadoopã®ãƒ¡ãƒªãƒƒãƒˆã¯ã€ã‚¹ã‚±ãƒ¼ãƒ«ã‚¢ã‚¦ãƒˆã®å•é¡Œç‚¹ã‚’å…‹æœã—ã¦ã„ã‚‹ã¨ã“ã‚ã«ã‚ã‚‹ã‹ã‚‰ã§ã™ã€‚ ã‚¹ã‚±ãƒ¼ãƒ«ã‚¢ãƒƒãƒ—ã¨ã‚¹ã‚±ãƒ¼ãƒ«ã‚¢ã‚¦ãƒˆ ã‚ãªãŸãŒã€ä¼æ¥ã®Webã‚µãƒ¼ãƒãƒ¼ã®ç®¡ç†è€…ã ã£ãŸã¨ã—ã¾ã—ã‚‡ã†ã€‚ ã‚µãƒ¼ãƒãƒ¼ã‚’ç«‹ã¡ä¸Šã’ãŸå½“åˆã¯ã€Webã‚µã‚¤ãƒˆã®è¨ªå•è€…ã‚‚å°‘ãªãã€1å°ã®PCã§
limeG 2012/03/16
Hadoop
ãƒªãƒ³ã‚¯
ï¼»ç¬¬1å›žï¼½ä»Šã•ã‚‰èžã‘ãªã„Hadoopã®åŸºç¤Ž
OSSï¼ˆã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã‚½ãƒ•ãƒˆã‚¦ã‚¨ã‚¢ï¼‰ã®ãƒŸãƒ‰ãƒ«ã‚¦ã‚¨ã‚¢ã€ã€ŒHadoopã€ãŒæ³¨ç›®ã•ã‚Œã¦ã„ã‚‹ã€‚å¤§é‡ã®ãƒ‡ãƒ¼ã‚¿ã€ã¨ã‚Šã‚ã‘ãƒã‚°ã€ãƒ†ã‚ã‚¹ãƒˆã€ç”»åƒã¨ã„ã£ãŸéžæ§‹é€ åŒ–ãƒ‡ãƒ¼ã‚¿ã‚’å–ã‚Šæ‰±ã†ãŸã‚ã®é«˜é€Ÿãªæƒ…å ±å‡¦ç†åŸºç›¤ã‚’å®‰ä¾¡ã«æ§‹ç¯‰ã§ãã‚‹ã‹ã‚‰ã ã€‚ãã“ã§æœ¬é€£è¼‰ã§ã¯Hadoopã‚’åŸºç¤Žã‹ã‚‰èª¬æ˜Žã™ã‚‹ã€‚ æœ¬é€£è¼‰ã¯æƒ…å ±ã‚·ã‚¹ãƒ†ãƒ éƒ¨é–€ã®ä¼ç”»æ‹…å½“è€…ã‚„åˆ©ç”¨éƒ¨é–€ã®ITæ´»ç”¨æŽ¨é€²è€…ãªã©ãŒå¯¾è±¡ã€‚åºç›¤ã§ã¯Hadoopã®åŸºæœ¬ã‚„ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ãƒ¼ã‚’è§£èª¬ã€‚å›žã‚’è¿½ã£ã¦åˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã®æ“ä½œã¨MapReduceå‡¦ç†ã‚’ã‚³ãƒ¼ãƒ‰ã§ç¢ºèªã€‚ãƒ¦ãƒ¼ã‚¹ã‚±ãƒ¼ã‚¹ã‚„ã‚¯ãƒ©ã‚¹ã‚¿ç®¡ç†ã«ã‚‚è¨€åŠã™ã‚‹ã»ã‹ã€2012å¹´ã«äºˆå®šã•ã‚Œã¦ã„ã‚‹æ¬¡æœŸãƒ¡ã‚¸ãƒ£ãƒ¼ãƒªãƒªãƒ¼ã‚¹0.23ã®æƒ…å ±ãªã©ã®æœ€æ–°ãƒˆãƒ”ãƒƒã‚¯ã‚‚å–ã‚Šè¾¼ã‚“ã§ã„ãã€‚ ãƒ‹ãƒ¼ã‚ºé«˜ã¾ã‚‹ã€Œãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã€ã®åŸºç›¤ æ—¥æœ¬ã«ã¨ã£ã¦2011å¹´ã¯ã€Œãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã€å…ƒå¹´ã ã£ãŸã¨è¨€ã£ã¦ã‚ˆã„ã®ã§ã¯ãªã„ã§ã—ã‚‡ã†ã‹ã€‚ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã«é–¢ã‚ã‚‹è£½å“ã‚„ã‚½ãƒªãƒ¥ãƒ¼ã‚·ãƒ§ãƒ³ãŒå›½å†…å¤–ã®ãƒ™ãƒ³ãƒ€ãƒ¼ã‹ã‚‰ç™ºè¡¨ã¨ãªã‚Šã€å¤š
limeG 2012/03/16
Hadoop
ãƒªãƒ³ã‚¯
å¹³æˆï¼’ï¼‘å¹´åº¦ç”£å¦é€£æºã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢å·¥å¦å®Ÿè·µäº‹æ¥ï¼ˆé«˜ä¿¡é ¼ã‚¯ãƒ©ã‚¦ãƒ‰å®Ÿç¾ç”¨ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢é–‹ç™ºï¼ˆåˆ†æ•£åˆ¶å¾¡å‡¦ç†æŠ€è¡“ç‰ã«ä¿‚ã‚‹ãƒ‡ãƒ¼ã‚¿ã‚»ãƒ³ã‚¿ãƒ¼ã®é«˜ä¿¡é ¼åŒ–ã«å‘ã‘ãŸå®Ÿè¨¼äº‹æ¥ï¼‰ï¼‰äº‹æ¥æˆæžœå ±å‘Šæ›¸
limeG 2012/03/16
Hadoop

Cloud
ãƒªãƒ³ã‚¯
1