[B! Hadoop] heppokonekoã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

heppokoneko id:heppokoneko

Hadoopã«é–¢ã™ã‚‹heppokonekoã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (6)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Hadoopã‚’ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã—ã‚ˆã†
Hadoopã‚’å‹•ã‹ã™ã«ã¯ã€å°‘ãªãã¨ã‚‚1å°ã®Linuxãƒžã‚·ãƒ³ãŒå¿…è¦ã«ãªã‚Šã¾ã™ã€‚å®Ÿéš›ã«åŠ¹æžœã‚’å¾—ã‚‹ãŸã‚ã«ã¯ã€å·¨å¤§ãªãƒ‡ãƒ¼ã‚¿ã¨ã€ç‰©ç†çš„ã«å¤šæ•°ã®PCã‚’ç”¨æ„ã™ã‚‹å¿…è¦ãŒã‚ã‚Šã¾ã™ã€‚ Hadoopã«ã¯3ã¤ã®å‹•ä½œãƒ¢ãƒ¼ãƒ‰ãŒã‚ã‚Šã€ã©ã®ãƒ¢ãƒ¼ãƒ‰ã‚’å‹•ã‹ã™ã‹ã«ã‚ˆã£ã¦å¿…è¦ãªç’°å¢ƒãŒç•°ãªã‚Šã¾ã™ï¼ˆè¡¨1ï¼‰ã€‚æœ¬è¨˜äº‹ã§ã¯ã€1å°ã®ãƒžã‚·ãƒ³ä¸Šã§Hadoopã®å„ãƒ‡ãƒ¼ãƒ¢ãƒ³ã‚’èµ·å‹•ã™ã‚‹ã€Œç–‘ä¼¼åˆ†æ•£ãƒ¢ãƒ¼ãƒ‰ã€ã‚’å‹•ã‹ã—ã€Hadoopã®å‹•ä½œã‚’ç¢ºèªã—ã¾ã™ã€‚ Linuxç’°å¢ƒã®æº–å‚™ ã¾ãšæœ€åˆã«ã€LinuxãŒå‹•ä½œã™ã‚‹ç’°å¢ƒã‚’æº–å‚™ã—ã¦ãã ã•ã„ã€‚ã‚‚ã—å¤ã„PCãªã©ã€ç¾åœ¨ä½¿ã£ã¦ã„ãªã„PCãŒã‚ã‚Œã°ã€ãã“ã«ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã™ã‚Œã°ã„ã„ã§ã—ã‚‡ã†ã€‚ãã®ã‚ˆã†ãªä½™ã£ãŸPCãŒç„¡ã‘ã‚Œã°Windowsãƒžã‚·ãƒ³ä¸Šã®ä»®æƒ³åŒ–ã‚½ãƒ•ãƒˆã§ã€Linuxã‚’ä½¿ã†ã¨ã„ã†æ–¹æ³•ã‚‚ã‚ã‚Šã¾ã™ã€‚ Linuxã®ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã¯æ¥µã‚ã¦ç°¡å˜ã§ã™ã€‚Linuxã‚’ã“ã‚Œã¾ã§ã¾ã£ãŸãè§¦ã‚ŒãŸã“ã¨ãŒãªã„äººã§ã‚‚ã€GUIã®æ“ä½œã§ç°¡å˜ã«ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã§ãã¾ã™ã€‚L
heppokoneko 2012/03/13
Hadoop
ãƒªãƒ³ã‚¯
ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã‚’è¿Žãˆæ’ƒã¦â€¦è¶…é«˜é€ŸåŒ–ã™ã‚‹ã€Œãƒãƒƒãƒå‡¦ç†ã€
æœ¬é€£è¼‰ã§ã¯ã€æƒ…å ±ã‚·ã‚¹ãƒ†ãƒ ã®æ¬¡ä»£ã‚’æ‹…ã†ITã®7å¤§ãƒˆãƒ¬ãƒ³ãƒ‰ã‚’é †æ¬¡ç´¹ä»‹ã—ã¦ã„ãã€‚ä»Šå›žã¯ã€åˆ†æ•£å‡¦ç†ã®å¾¹åº•ã«ã‚ˆã£ã¦è¶…é«˜é€ŸåŒ–ã‚’ç‹™ã†æœ€æ–°ã®ã€Œãƒãƒƒãƒå‡¦ç†ã€æŠ€è¡“ã‚’è¦‹ã¦ã„ã“ã†ã€‚ ã€åˆ†æ•£ãƒãƒƒãƒå‡¦ç†ã€‘ã€€Hadoopåž‹ã®åˆ†æ•£å‡¦ç†ã§ã€ãƒãƒƒãƒã‚’è¶…é«˜é€ŸåŒ– éŠ€è¡Œã®å‹˜å®šç³»ã‚·ã‚¹ãƒ†ãƒ ãªã©ã«æŽ¡ç”¨ã•ã‚Œã€å¤ãã‹ã‚‰ã‚ˆãçŸ¥ã‚‰ã‚Œã‚‹ãƒãƒƒãƒå‡¦ç†ã¯ã€ä»Šã‚„ã€æƒ…å ±ã‚·ã‚¹ãƒ†ãƒ ã®æœ€å…ˆç«¯åˆ†é‡Žã ã€‚å¤šãã®ã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ã‚¿ãƒ¡ãƒ¼ã‚«ãƒ¼ãŒæä¾›ã—ã¦ã„ãŸç‹¬è‡ªä»•æ§˜ã®ãƒ¡ã‚¤ãƒ³ãƒ•ãƒ¬ãƒ¼ãƒ ã‹ã‚‰ç¾åœ¨ä¸»æµã®ãƒ‘ã‚½ã‚³ãƒ³ã‚µãƒ¼ãƒãƒ¼ã«è‡³ã‚‹ã¾ã§ç¶¿ã€…ã¨ç¶šããƒãƒƒãƒå‡¦ç†ã‚’ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ãƒ»ã‚½ãƒ•ãƒˆã‚¦ã‚¨ã‚¢ã«å§”ãã€ã—ã‹ã‚‚åˆ†æ•£å‡¦ç†ã‚’å¾¹åº•ã™ã‚‹ã“ã¨ãŒå¤§ããªãƒˆãƒ¬ãƒ³ãƒ‰ã«ãªã‚Šã¤ã¤ã‚ã‚‹ã€‚ è¥¿é‰„ã‚¹ãƒˆã‚¢ã¯ã€ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã®åˆ†æ•£ãƒãƒƒãƒå‡¦ç†ã‚½ãƒ•ãƒˆã€ŒHadoopï¼ˆãƒãƒ‰ã‚¥ãƒ¼ãƒ—ï¼‰ã€ã‚’ä½¿ã£ãŸåŸºå¹¹æ¥å‹™ã‚·ã‚¹ãƒ†ãƒ ã‚’2011å¹´9æœˆæœ«ã«ç¨¼åƒã•ã›ã‚‹ã€‚åº—èˆ—ã‹ã‚‰é›†ã¾ã‚‹POSï¼ˆè²©å£²æ™‚ç‚¹æƒ…å ±ç®¡ç†ï¼‰ãƒ‡ãƒ¼ã‚¿ã®é›†è¨ˆãƒ»åˆ†æžã‹ã‚‰ä¼šè¨ˆã¾ã§ã‚’æ‹…ã†ã‚·ã‚¹ãƒ†ãƒ ã§ã€ã€Œ4ï½ž5æ™‚é–“ã‚’è¦ã—ã¦ã„ãŸ
heppokoneko 2011/12/13
æƒ…å ±ã‚·ã‚¹ãƒ†ãƒ

Hadoop

*ã‚ã¨ã§èªã‚€
ãƒªãƒ³ã‚¯
ãƒãƒƒãƒˆä¸Šã§æ€¥å¢—ã™ã‚‹ã€Œéžæ§‹é€ åŒ–ãƒ‡ãƒ¼ã‚¿ã€ ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã®æ´»ç”¨ãŒãƒ“ã‚¸ãƒã‚¹ã‚’åˆ¶ã™
ãƒãƒƒãƒˆä¸Šã§æ€¥å¢—ã™ã‚‹ã€Œéžæ§‹é€ åŒ–ãƒ‡ãƒ¼ã‚¿ã€ ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã®æ´»ç”¨ãŒãƒ“ã‚¸ãƒã‚¹ã‚’åˆ¶ã™ æƒ…å ±çˆ†ç™ºã¨ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã®æ´»ç”¨ ï½žæ–°ãŸãªã‚µãƒ¼ãƒ“ã‚¹æ©Ÿä¼šã¨å®Ÿç¾ã®ãŸã‚ã®åŸºç›¤æŠ€è¡“ï½ž --- é‡Žæ‘ç·åˆç ”ç©¶æ‰€ å–ç· å½¹ä¼šé•· è—¤æ²¼ å½°ä¹… æ° é‡Žæ‘ç·åˆç ”ç©¶æ‰€ã¯ã€æƒ…å ±çˆ†ç™ºã¨ãã‚Œã‚’å¼•ãèµ·ã“ã™ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã®æ´»ç”¨ã«å–ã‚Šçµ„ã‚“ã§ã„ã‚‹ã€‚ãã‚‚ãã‚‚ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã¨ã¯ä½•ã ã‚ã†ã‹ã€‚ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã¨ã¯ã€æ—¢å˜ã®æŠ€è¡“ã§ã¯ç®¡ç†ã§ããªã„ã»ã©ã«ãƒœãƒªãƒ¥ãƒ¼ãƒ ãŒå¢—ãˆã€è¤‡é›‘åŒ–ã—ãŸãƒ‡ãƒ¼ã‚¿ã®ã“ã¨ã‚’æŒ‡ã™ã€‚ ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã®ãƒ‡ãƒ¼ã‚¿æ§‹é€ ã¯ã€æ¥å‹™ãƒ‡ãƒ¼ã‚¿ãªã©ã‚’æ‰±ã†å¾“æ¥ã®RDBMSï¼ˆãƒªãƒ¬ãƒ¼ã‚·ãƒ§ãƒŠãƒ«ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ç®¡ç†ã‚·ã‚¹ãƒ†ãƒ ï¼‰ã«æ ¼ç´ã§ããªã„éžæ§‹é€ åŒ–ãƒ‡ãƒ¼ã‚¿ãŒå¤§åŠã‚’å ã‚ã‚‹ã€‚éžæ§‹é€ åŒ–ãƒ‡ãƒ¼ã‚¿ã¯ãƒãƒƒãƒˆä¸Šã§æ€¥å¢—ã—ã¦ã„ã‚‹ã€‚ä¸–ç•Œã§æ•°å„„äººãŒç™»éŒ²ã™ã‚‹Facebookã‚„Twitterãªã©ã‚½ãƒ¼ã‚·ãƒ£ãƒ«ãƒ¡ãƒ‡ã‚£ã‚¢ã®åˆ©ç”¨æ‹¡å¤§ã«åŠ ãˆã€å¤§å®¹é‡ã®æ˜ åƒãƒ‡ãƒ¼ã‚¿ã®ãƒ“ãƒ‡ã‚ªã‚µã‚¤ãƒˆã¸ã®æŠ•ç¨¿ãŒå¢—ãˆã¦ã„ã‚‹ã‹ã‚‰ã ã€‚ ã“ã®ã»ã‹ã€ã‚³ãƒ¼ãƒ«ã‚»ãƒ³ã‚¿ãƒ¼ã¸ã®å•ã„åˆã‚
heppokoneko 2011/09/30
"In-Database Analyticsã¯ãƒ‡ãƒ¼ã‚¿åˆ†æžæ©Ÿèƒ½ã‚’DWHã«çµ„ã¿è¾¼ã¿ã€ãƒ‡ãƒ¼ã‚¿ã®æŠ½å‡ºãƒ»å¤‰æ›ã®æ™‚é–“ã‚’çŸç¸®ã™ã‚‹ã¨ã¨ã‚‚ã«ã€è¶…ä¸¦åˆ—ãƒ—ãƒã‚»ãƒƒã‚·ãƒ³ã‚°æŠ€è¡“ãªã©ã‚’çµ„ã¿åˆã‚ã›ã¦é«˜é€Ÿå‡¦ç†ã™ã‚‹""Hadoopã¯RDBMSã®ã‚ˆã†ãªè¤‡é›‘ãªå•ã„åˆã‚ã›å‡¦ç†ã«ä¸å‘ãã ãŒã€åˆ†

Hadoop

IT
ãƒªãƒ³ã‚¯
ã‚°ãƒ¼ã‚°ãƒ«ç™ºã€ŒHadoopã€ã€æ—¥æœ¬ä¼æ¥ã‚‚åˆ©ç”¨ã¸
Hadoopã¯ã€ã‚°ãƒ¼ã‚°ãƒ«ãŒæ¤œç´¢ã‚¨ãƒ³ã‚¸ãƒ³ç”¨ã«é–‹ç™ºã—ãŸãƒãƒƒãƒå‡¦ç†ã‚·ã‚¹ãƒ†ãƒ ã‚’åŸºã«é–‹ç™ºã•ã‚ŒãŸã€ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã‚½ãƒ•ãƒˆã ã€‚ã‚°ãƒ¼ã‚°ãƒ«ãŒé–‹ç™ºã—ãŸåˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã€ŒGoogle File Systemï¼ˆGFSï¼‰ã€ã‚’æ¨¡ã—ãŸã€ŒHadoop Distributed File Systemï¼ˆHD FSï¼‰ã€ã¨ã€ãƒ‡ãƒ¼ã‚¿å‡¦ç†æ©Ÿæ§‹ã€ŒMapReduceã€ã‚’æ¨¡ã—ãŸã€ŒHadoop MapReduceã€ã§æ§‹æˆã™ã‚‹ã€‚ ç±³å›½ã§ã¯ç±³VISAã‚„ç±³JPãƒ¢ãƒ«ã‚¬ãƒ³ãƒ»ãƒã‚§ãƒ¼ã‚¹ã®ã‚ˆã†ãªå¤§æ‰‹é‡‘èžæ©Ÿé–¢ãŒã€ãƒãƒƒãƒå‡¦ç†ã«Hadoopã‚’ä½¿ç”¨ã™ã‚‹ã€‚ ãã®HadoopãŒã„ã‚ˆã„ã‚ˆã€æ—¥æœ¬ä¼æ¥ã§ã‚‚ä½¿ã‚ã‚Œå§‹ã‚ãŸã€‚ä¾‹ãˆã°æ¥½å¤©ã¯ã€ã‚ã‚‹å•†å“ã«å¯¾ã™ã‚‹ãŠè–¦ã‚å•†å“ã‚’ãƒªã‚¹ãƒˆã‚¢ãƒƒãƒ—ã™ã‚‹ã€Œãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰å‡¦ç†ã€ã«Hadoopã‚’ä½¿ç”¨ã™ã‚‹ã€‚NTTãƒ‡ãƒ¼ã‚¿ã¯ã€å…¨å›½ã®æ¸‹æ»žæƒ…å ±ã‚’ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ã«å¯è¦–åŒ–ã™ã‚‹ã‚·ã‚¹ãƒ†ãƒ ã®æ§‹ç¯‰ã«Hadoopã‚’æŽ¡ç”¨ã—ãŸã€‚ä¸‰è±UFJã‚¤ãƒ³ãƒ•ã‚©ãƒ¡ãƒ¼ã‚·ãƒ§ãƒ³ãƒ†ã‚¯ãƒŽãƒã‚¸ãƒ¼ã‚‚Hadoopã‚’ä½¿ã£
heppokoneko 2011/08/31
Hadoop
ãƒªãƒ³ã‚¯
ã‚ãƒ¼åˆ†å¸ƒãŒåã‚‹å ´åˆã¯æ€§èƒ½å‘ä¸Š
æœ€å¾Œã«12ãƒŽãƒ¼ãƒ‰ã®ç’°å¢ƒã§ã€è¤‡æ•°ã®Hadoopã‚¸ãƒ§ãƒ–ã‚’å®Ÿè¡Œã—ãŸå ´åˆã®å‡¦ç†æ€§èƒ½ã«ã¤ã„ã¦èª¬æ˜Žã™ã‚‹ã€‚é€šå¸¸ã¯ã€1å°ã®ã‚µãƒ¼ãƒãƒ¼ä¸Šã§è¤‡æ•°ã®å‡¦ç†ã‚’åŒæ™‚ã«å®Ÿè¡Œã™ã‚‹ã¨ã€ãƒªã‚½ãƒ¼ã‚¹ã®ç«¶åˆãŒç™ºç”Ÿã—ã¦æ€§èƒ½ãŒä½Žä¸‹ã™ã‚‹ã€‚Hadoopã§åŒæ§˜ã®å•é¡ŒãŒç™ºç”Ÿã™ã‚‹ã®ã‹èª¿ã¹ãŸã€‚ æ¤œè¨¼ã§ã¯ã€åŒä¸€ã®Hadoopã‚¯ãƒ©ã‚¹ã‚¿ãƒ¼ä¸Šã§åŒä¸€å‡¦ç†å†…å®¹ã®Hadoopã‚¸ãƒ§ãƒ–ã‚’äºŒã¤å‹•ä½œã•ã›ãŸã€‚ã‚ãƒ£ãƒƒã‚·ãƒ¥ã®å½±éŸ¿ã‚’æŽ’é™¤ã™ã‚‹ãŸã‚ã«ã€å„ã‚¸ãƒ§ãƒ–ã¯åˆ¥ã®ãƒ‡ãƒ¼ã‚¿ã‚’å‡¦ç†ã™ã‚‹ã‚ˆã†ã«ã—ã¦ã„ã‚‹ã€‚ çµæžœã¯ã‚ãƒ¼ã®åˆ†å¸ƒãŒä¸€æ§˜ãªå ´åˆã¨ã¹ãåˆ†å¸ƒã®å ´åˆã§ç•°ãªã£ãŸï¼ˆå›³9ï¼‰ã€‚
heppokoneko 2011/08/28
Hadoop
ãƒªãƒ³ã‚¯
100ä¸‡ä»¶ã§ã¯å°‚ç”¨ãƒ„ãƒ¼ãƒ«ãŒæœ€é€Ÿ
å®Ÿéš›ã«æ§‹ç¯‰ã™ã‚‹Hadoopã®ã‚·ã‚¹ãƒ†ãƒ ã§ã¯ï¼ˆaï¼‰ã‚¤ãƒ³ãƒãƒ¼ãƒˆã‚„ï¼ˆdï¼‰ã‚¨ã‚¯ã‚¹ãƒãƒ¼ãƒˆã®ã‚ˆã†ã«ã€æ‰±ã†ãƒ‡ãƒ¼ã‚¿é‡ã«ä¾å˜ã—ã€Hadoopã®ãƒŽãƒ¼ãƒ‰æ•°ã‚’å¢—ã‚„ã—ã¦ã‚‚æ€§èƒ½ãŒå‘ä¸Šã—ãªã„å‡¦ç†ãŒã‚ã‚Šã€ãã“ãŒãƒœãƒˆãƒ«ãƒãƒƒã‚¯ã«ãªã‚Šå¾—ã‚‹ã€‚ã„ã‹ã«åŠ¹çŽ‡è‰¯ãRDBMSã‹ã‚‰ãƒ‡ãƒ¼ã‚¿ã‚’ã‚¤ãƒ³ãƒãƒ¼ãƒˆ/ã‚¨ã‚¯ã‚¹ãƒãƒ¼ãƒˆã™ã‚‹ã‹ãŒéžå¸¸ã«é‡è¦ã ã€‚ ã“ã“ã§ã¯ã€ï¼ˆaï¼‰ã‚¤ãƒ³ãƒãƒ¼ãƒˆã«ç„¦ç‚¹ã‚’å½“ã¦ã€ã€ŒJDBCãƒ‰ãƒ©ã‚¤ãƒã‚’ä½¿ç”¨ã—ã¦æ¨™æº–SQLã§ã‚¢ã‚¯ã‚»ã‚¹ã€ã€Œç±³ClouderaãŒæä¾›ã™ã‚‹ãƒ‡ãƒ¼ã‚¿è»¢é€ãƒ„ãƒ¼ãƒ«sqoopã‚’ä½¿ç”¨ã€ã€ŒMySQLã®ç‹¬è‡ªæ©Ÿèƒ½ã‚’åˆ©ç”¨ã—ãŸãƒ€ãƒ³ãƒ—ã€ã®3é€šã‚Šã®æ–¹æ³•ã‚’è©¦ã—ãŸï¼ˆå›³4ï¼‰ã€‚
heppokoneko 2011/08/24
ã‚·ã‚¹ãƒ†ãƒ

Hadoop

IT
ãƒªãƒ³ã‚¯
1