[B! hadoop] shinichitomitaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

shinichitomita id:shinichitomita

hadoopã«é–¢ã™ã‚‹shinichitomitaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (6)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Hadoopã€ã‚»ã‚ãƒ¥ãƒªãƒ†ã‚£å¼·åŒ–ã§Kerberoså®Ÿè£…ä¸
Hadoopã«ã¯ç¾åœ¨ã€ä¼æ¥ãªã©ã§åˆ©ç”¨ã™ã‚‹éš›ã«ã¯ã‚»ã‚ãƒ¥ãƒªãƒ†ã‚£é¢ã§ã®ä¸å®‰ãŒã‚ã‚Šã€ãã®ãŸã‚ã®æ”¹è‰¯ã«å–ã‚Šçµ„ã‚“ã§ã„ã¾ã™ã€‚Yahoo!ã®Hadoop Developmentãƒãƒ¼ãƒ ã«æ‰€å±žã™ã‚‹Owen O'Malleyæ°ã«ã‚ˆã‚‹ã€ç¾åœ¨ã©ã®ã‚ˆã†ãªä»•çµ„ã¿ã§Hadoopã®ã‚»ã‚ãƒ¥ãƒªãƒ†ã‚£ã‚’å¼·åŒ–ã—ã‚ˆã†ã¨ã—ã¦ã„ã‚‹ã®ã‹ã‚’èª¬æ˜Žã™ã‚‹ãƒ—ãƒ¬ã‚¼ãƒ³ãƒ†ãƒ¼ã‚·ãƒ§ãƒ³ãŒã€ŒHadoop User Group March Meeting Recapã€ã®ãƒšãƒ¼ã‚¸ã§å…¬é–‹ã•ã‚Œã¦ã„ã¾ã™ã€‚ Kerberosã§ã‚·ãƒ³ã‚°ãƒ«ã‚µã‚¤ãƒ³ã‚ªãƒ³ã‚’å®Ÿç¾ ãƒ—ãƒ¬ã‚¼ãƒ³ãƒ†ãƒ¼ã‚·ãƒ§ãƒ³ã®ä¸»ãªãƒã‚¤ãƒ³ãƒˆã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ ä»Šå›žã®ä¸»ãŸã‚‹ç›®çš„ã¯ã€ã€Œèªè¨¼ã•ã‚Œã¦ã„ãªã„ãƒ¦ãƒ¼ã‚¶ãƒ¼ãŒHDFSã«ã‚¢ã‚¯ã‚»ã‚¹ã§ããªã„ã‚ˆã†ã«ã™ã‚‹ã“ã¨ã€ã¨ãªã£ã¦ã„ã¾ã™ã€‚ ç¾åœ¨ã®Hadoopã«ã¯ã€NameNodeã‚„JobTrackerã€DataNodeãªã©ã§ã®ãƒ¦ãƒ¼ã‚¶ãƒ¼èªè¨¼ãŒãªã„ã“ã¨ãŒã‚»ã‚ãƒ¥ãƒªãƒ†ã‚£ä¸Šã®èª²é¡Œã«ãªã£ã¦ã„ã‚‹ã¨æŒ‡æ‘˜ã€‚
shinichitomita 2010/04/11
hadoop

kerberos
ãƒªãƒ³ã‚¯
blog.katsuma.tv
å‰å›žã€JavaScriptã§Map Reduceã®ã‚³ãƒ¼ãƒ‰ãŒæ›¸ã‘ã‚‹Hadoop Streamingã«ã¤ã„ã¦ç´¹ä»‹ã—ã¾ã—ãŸã€‚ æ¨™æº–å…¥å‡ºåŠ›ã•ãˆã‚µãƒãƒ¼ãƒˆã•ã‚Œã¦ã‚ã‚Œã°ã€ä»»æ„ã®ã‚³ãƒ¼ãƒ‰ã§Map Reduuceã®å‡¦ç†ãŒæ›¸ã‘ã‚‹ã€ã¨ã„ã†ã‚‚ã®ã§ã—ãŸãŒã€ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã¯ãã‚‚ãã‚‚é¢å€’ãã•ãŒã‚Šã€‚ã‚³ãƒ¼ãƒ‰ã‚‚æ›¸ãã®ã‚‚é¢å€’ã§ã™ã€‚ ã¨ã€ã„ã†ã‚ã‘ã§ã€ä»Šå›žã¯ã‚‚ã†ã‚³ãƒ¼ãƒ‰ã™ã‚‰æ›¸ã‹ãšã«SQLãƒ©ã‚¤ã‚¯ã§Map Reduceã§ãã‚‹Hiveã¨ã„ã†ãƒ—ãƒãƒ€ã‚¯ãƒˆã«ã¤ã„ã¦ã€ã¾ã¨ã‚ãŸã„ã¨æ€ã„ã¾ã™ã€‚ Hive Hiveã¨ã¯ã€ç°¡å˜ã«è¨€ã†ã¨Hadoopä¸Šã§å‹•ä½œã™ã‚‹RDBã®ã‚ˆã†ãªã‚‚ã®ã§ã™ã€‚ HDFSãªã©ã®åˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ä¸Šã«å˜åœ¨ã™ã‚‹ãƒ‡ãƒ¼ã‚¿ã«å¯¾ã—ã¦ã€HiveQLã¨ã„ã†SQLãƒ©ã‚¤ã‚¯ãªè¨€èªžã§æ“ä½œã§ãã¾ã™ã€‚ ã§ã€é¢ç™½ã„ã®ãŒHiveQLã®æ“ä½œã¯åŸºæœ¬çš„ã«Map Reduceã®ãƒ©ãƒƒãƒ‘ãƒ¼ã«ãªã£ã¦ã„ã‚‹ã“ã¨ã€‚ è¦ã™ã‚‹ã«ã€SELECTæ–‡å®Ÿè¡Œã™ã‚‹ã¨è£ã§Map&Reduceã®ã‚¿ã‚¹ã‚¯ãŒèµ°ã‚Šå‡º
shinichitomita 2009/09/12
hadoop

mapreduce

sql

facebook
ãƒªãƒ³ã‚¯
blog.katsuma.tv
ä¹…ã€…ã®Blogæ›´æ–°ã€ã¨ã„ã†ã‚ã‘ã§ãƒªãƒãƒ“ãƒªãŒã¦ã‚‰JavaScriptã§è»½ãéŠã‚“ã§ã¿ãŸã„ã¨æ€ã„ã¾ã™ã€‚ ã„ã¾ã€å··ã§æµè¡Œã£ã¦ã‚‹MapReduceã®ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹å®Ÿè£…Hadoopã¯ã€ŒHadoop Streamingã€ã¨ã„ã†æ¨™æº–å…¥å‡ºåŠ›ã§ãƒ‡ãƒ¼ã‚¿ã®ã‚„ã‚Šã¨ã‚ŠãŒã§ãã‚‹ä»•çµ„ã¿ã‚’ä½¿ã£ã¦ã€ Hadoopã®å®Ÿè£…è¨€èªžã§ã‚ã‚‹Javaã«ã¨ã‚‰ã‚ã‚Œãšã€Rubyã‚„Perlãªã©ä»–ã®è¨€èªžã§ã‚‚Mapï¼‹Reduceã®å‡¦ç†ãŒã§ãã‚‹ã“ã¨ãŒï¼‘ã¤ã®ã‚¦ãƒªã«ãªã£ã¦ã„ã¾ã™ã€‚ ã§ã€åƒ•ãŸã¡webã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã¯ã¿ã‚“ãªJavaScriptå¤§å¥½ããªã®ã§ã€ã€ŒJavaScriptã§ã‚‚Map Reduceã‚„ã‚ŠãŸã„ï¼ã€ã¨ã„ã†æµã‚Œã«ãªã‚‹ã®ã¯å¿…ç„¶ã§ã™ã€‚ ãã“ã§ã€è©¦è¡ŒéŒ¯èª¤ã§ã„ã‚ã„ã‚è©¦ã—ã¦ã¿ã‚‹ã¨å‰²ã¨ã•ã£ãã‚Šå‡ºæ¥ãŸã®ã§ãã®ãƒ¡ãƒ¢ã‚’æ®‹ã—ã¦ãŠããŸã„ã¨æ€ã„ã¾ã™ã€‚ ç’°å¢ƒã®æ•´å‚™ Mac OSXä¸Šã®VMWare Fusionã«CentOSã®ä»®æƒ³ãƒžã‚·ãƒ³ã‚’ï¼’å°ç«‹ã¡ä¸Šã’ã¦ã€ç’°å¢ƒã‚»ãƒƒãƒˆã‚¢ãƒƒãƒ—ã—ã¾ã—ãŸã€‚
shinichitomita 2009/08/03
> cat input/file1 | js script/map.js | js script/reduce.js

hadoop

spidermonkey

javascript

mapreduce
ãƒªãƒ³ã‚¯
Littel-hadoop
ç§ãŸã¡ãƒªãƒƒãƒ†ãƒ«ã¯ã€GoogleåŒ–æ™‚ä»£ã®åˆ‡ã‚Šæœã€Hadoop(ãƒãƒ‰ã‚¥ãƒ¼ãƒ—)æŠ€è¡“ã‚’æ´»ã‹ã—ã¦ã€å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã®åŽé›†ã‚„å‡¦ç†ã‚’é«˜é€Ÿã€å®‰ä¾¡ã«å®Ÿç¾ã€‚å¤§è¦æ¨¡åˆ†æ•£å‡¦ç†ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ ã®æ§‹ç¯‰æ”¯æ´ã‚’è¡Œã„ã¾ã™ã€‚ å‹é–“å’Œä»£ã•ã‚“ãŒã€ã€ŽåŠ¹çŽ‡ãŒ10å€ã‚¢ãƒƒãƒ—ã™ã‚‹æ–°ãƒ»çŸ¥çš„ç”Ÿç”£è¡“ã€ ã¨ã„ã†ãƒ™ã‚¹ãƒˆã‚»ãƒ©ãƒ¼ã‚’ã ã•ã‚Œã¾ã—ãŸãŒã€ãã®æœ¬ã« ã€Žâ€•è‡ªåˆ†ã‚’ã‚°ãƒ¼ã‚°ãƒ«åŒ–ã™ã‚‹æ–¹æ³•ã€ã¨ã„ã†ã‚µãƒ–ã‚¿ã‚¤ãƒˆãƒ«ãŒã‚ã‚Šã¾ã—ãŸã€‚ ã©ã‚Œã ã‘æƒ…å ±ã‚’ç„¡é§„ãªãã€åŠ¹çŽ‡ã‚ˆãå‡ºã—å…¥ã‚Œã§ãã‚‹ã‹ï¼Ÿ ã©ã‚Œã ã‘æœ‰åŠ¹ã«æƒ…å ±ã‚’æŠ½å‡ºã—æ˜Žæ—¥ã®æ¥ç¸¾ã«ã¤ãªã’ã‚‹ã‹ï¼Ÿ ã¨ã„ã£ãŸå†…å®¹ã®æœ¬ã§ã™ã€‚ ãã“ã§ã€ä¼šç¤¾ãã®ã‚‚ã®ã‚’ã€ŽgoogleåŒ–ã™ã‚‹æ–¹æ³•ã€ã‚’è€ƒãˆã¦ã¿ã¾ã›ã‚“ã‹ï¼Ÿ Webã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã€ä¾‹ãˆã°Eã‚³ãƒžãƒ¼ã‚¹ãªã©ã§ã¯ã€ã‚¢ã‚¯ã‚»ã‚¹ãƒ‡ãƒ¼ã‚¿ã ã‘ã§ãªãã€ ç”·å¥³ãƒ»å¹´é½¢ãªã©é¡§å®¢ãƒ‡ãƒ¼ã‚¿ã‚„ã€è³¼è²·å±¥æ´ã€æ™‚é–“ã€å•†å“ã€å›žæ•°ãªã©ã®è³¼è²·çŠ¶æ³ãªã©ã€ æ—¥ã€…è“„ç©ã•ã‚Œã‚‹è†¨å¤§ãªãƒ‡ãƒ¼ã‚¿ã‚„ãƒã‚°ãŒé›†ç©ã•ã‚Œã¦ã„ã‚‹ã“ã¨ã§ã—ã‚‡ã†ã€‚ å¤§é‡ã®ãƒ‡ãƒ¼ã‚¿ã¯ã‚ã£ã¦
shinichitomita 2009/06/15
hadoop

venture
ãƒªãƒ³ã‚¯
Sign in - Google Accounts
Not your computer? Use a private browsing window to sign in. Learn more about using Guest mode
shinichitomita 2009/05/12
via knsmr

hadoop

mapreduce

java
ãƒªãƒ³ã‚¯
ã¸ã€œãŸã®ã‚ã‚‚:Hadoop ã¨ RDBMS ã®æ€§èƒ½ã‚’æ¯”è¼ƒã—ã¦ã¿ãŸã€‚ã¨ã„ã†è«–æ–‡ã€‚ã®æ„Ÿæƒ³ã€‚ã®ç¿»è¨³ã€‚ - livedoor Blogï¼ˆãƒ–ãƒã‚°ï¼‰
2009å¹´04æœˆ19æ—¥ Hadoop ã¨ RDBMS ã®æ€§èƒ½ã‚’æ¯”è¼ƒã—ã¦ã¿ãŸã€‚ã¨ã„ã†è«–æ–‡ã€‚ã®æ„Ÿæƒ³ã€‚ã®ç¿»è¨³ã€‚ Stonebraker, DeWitt, et al. compare MapReduce to DBMS Map-Reduce (Hadoop)ã¨ Parallel DBMS (Vertica ã¨ DBMS-X) ã®æ€§èƒ½ã‚’æ¯”è¼ƒã—ãŸã¨ã“ã‚ã€ãŠãŠã‚€ã DB ã®æ–¹ãŒæ€§èƒ½ãŒã‚ˆãã€ã‚±ãƒ¼ã‚¹ã«ã‚ˆã£ã¦ã¯ 6.5å€ã‚‚æ€§èƒ½å·®ãŒã‚ã‚‹ã¨ã„ã†å®Ÿé¨“çµæžœãŒã€SIGMOD09 ã§ç™ºè¡¨ã•ã‚ŒãŸã‚‰ã—ã„ã€‚åŽŸè«–æ–‡ã¯ã“ã“ã‹ã‚‰ PDF ã§ãƒ€ã‚¦ãƒ³ãƒãƒ¼ãƒ‰ã§ãã‚‹ã€‚ ä»¥ä¸‹ã¯ DBMS2 ã¨ã„ã†ãƒ–ãƒã‚°ã«æ›¸ã‹ã‚ŒãŸã€ã“ã®è«–æ–‡ã‚’èªã‚“ã äººã®æ„Ÿæƒ³ã®ç¿»è¨³ã€‚ Stonebraker, DeWitt, et al. MapReduce ã¨ DBMS ã®æ¯”è¼ƒ 5äººã®å…±è‘—è€…(ä¸»ç†è€…ã¯ Andy Pavlo ã£ã½ã„)ã¨ã¨ã‚‚ã«ã€Map-Reduce å«Œã„ã§çŸ¥ã‚‰ã‚Œ
shinichitomita 2009/04/30
hadoop

database

distributed
ãƒªãƒ³ã‚¯
1