[B! hadoop] willnetã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

willnet id:willnet

hadoopã«é–¢ã™ã‚‹willnetã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (5)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

ç¬¬2å›žã€€ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰ã‚·ã‚¹ãƒ†ãƒ ã®å®Ÿè£…ã¨èª²é¡Œ | gihyo.jp
ä»Šå›žã¯ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰ã‚·ã‚¹ãƒ†ãƒ ã®å®Ÿè£…ã®å•é¡Œã¨ã€ãã®è§£æ±ºç–ã¨ã—ã¦åˆ©ç”¨ã™ã‚‹Hadoopã«ã¤ã„ã¦èª¬æ˜Žã—ã¾ã™ã€‚ ä»Šå›žã®ãƒã‚¤ãƒ³ãƒˆã¯ä»¥ä¸‹ã®é€šã‚Šã§ã™ã€‚ ã‚¹ã‚±ãƒ¼ãƒ«ã‚¢ãƒƒãƒ—ã®é™ç•Œ åˆ†æ•£å‡¦ç†ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã®æ°‘ä¸»åŒ– Hadoop Map Reduceã®æ¦‚è¦ å¢—ãˆç¶šã‘ã‚‹ãƒ‡ãƒ¼ã‚¿ã‚’ã©ã†æ‰±ã†ã‹ ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰ã‚·ã‚¹ãƒ†ãƒ ã‚’å”èª¿ãƒ•ã‚£ãƒ«ã‚¿ãƒªãƒ³ã‚°ã®ã†ã¡ã€ãƒ¦ãƒ¼ã‚¶ãƒ™ãƒ¼ã‚¹æ–¹å¼ã«ã‚ˆã‚Šå®Ÿè£…ã™ã‚‹ã“ã¨ã‚’è€ƒãˆã¾ã™ã€‚ã“ã®ã‚·ã‚¹ãƒ†ãƒ ã§ã¯æ¬¡ã®ã‚ˆã†ãªæ˜ ç”»ã®è©•ä¾¡å±¥æ´ã‚’ä½¿ã„ã€ã€Œâ ã“ã®æ˜ ç”»ã‚’è©•ä¾¡ã—ãŸäººã¯ã“ã®æ˜ ç”»ã‚‚è©•ä¾¡ã—ã¦ã„ã¾ã™ã€ã¨ã„ã†æ˜ ç”»ã®æŽ¨è–¦ã‚’è¡Œã„ã¾ã™ã€‚ é§„å³è¡›é–€ã€ã‚¶ã‚¹ãƒ¼ãƒ©ã€5ã€2009/12/21ã€ãƒ»ãƒ»ãƒ»ãƒ» é§„å³è¡›é–€ã€ã‚¸ãƒ¥ãƒžãƒ³ã‚¸ã€5ã€2009/12/28ã€ãƒ»ãƒ»ãƒ»ãƒ» èŠä¹‹åŠ©ã€ãƒ–ãƒ«ãƒ¼ã‚¹ãƒ–ãƒ©ã‚¶ãƒ¼ã‚ºã€5ã€2009/12/29ã€ãƒ»ãƒ»ãƒ»ãƒ» åˆ©å¹³ã€ã‚¶ã‚¹ãƒ¼ãƒ©ã€4ã€2010/01/01ã€ãƒ»ãƒ»ãƒ»ãƒ» åä¸‰ã€ãƒãƒ³ã‚°ã‚ªãƒ¼ãƒãƒ¼ã€4ã€2010/01/01ã€ãƒ»ãƒ»ãƒ»ãƒ» åˆ©å¹³ã€ã‚¸ãƒ¥ãƒžãƒ³ã‚¸ã€4ã€2010/
willnet 2011/11/18
ã‚³ã‚µã‚¤ãƒ³é–¢æ•°ãŒã‚ˆãã‚ã‹ã‚‰ãªã„ãªãƒ¼ï¼ãªã‚“ã å†…ç©ãªã®ã‹ã€‚çŸ¥ã‚‰ãªã„åå‰ã ã‹ã‚‰é›£ã—ã„å…¬å¼ãªã®ã‹ã¨ã€‚ id:mongrelP ã•ã‚“ã‚ã‚ŠãŒã¨ã†ã”ã–ã„ã¾ã™ï¼

hadoop
ãƒªãƒ³ã‚¯
Twitterã€åˆ†æ•£ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ å‡¦ç†ã‚·ã‚¹ãƒ†ãƒ ã€ŒStormã€å…¬é–‹ | gihyo.jp
2011å¹´9æœˆ18æ—¥ã€Twitterã¯åˆ†æ•£ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ å‡¦ç†ã‚·ã‚¹ãƒ†ãƒ ã€ŒStormã€ã‚’ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã§å…¬é–‹ã—ã¾ã—ãŸã€‚ã‚ˆã‚Šæ£ç¢ºã«è¨€ã†ã¨ã€2011å¹´7æœˆã«è²·åŽã•ã‚ŒãŸBackTypeã®ãƒ¡ãƒ³ãƒãƒ¼ãŒè²·åŽä»¥å‰ã‚ˆã‚Šé–‹ç™ºã‚’é€²ã‚ã¦ã„ãŸã‚‚ã®ã§ã™ã€‚BackTypeã¯Twitterã®ã¤ã¶ã‚„ãã‚’è§£æžã™ã‚‹ã‚·ã‚¹ãƒ†ãƒ ã‚’é–‹ç™ºã—ã¦ã„ãŸã‚¹ã‚¿ãƒ¼ãƒˆã‚¢ãƒƒãƒ—ä¼æ¥ã§Hadoopã®ãƒ˜ãƒ“ãƒ¼ãƒ¦ãƒ¼ã‚¶ã§ã‚‚ã‚ã‚Šã¾ã—ãŸãŒã€ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ æ€§ã®å•é¡Œç‚¹ã‚’è§£æ¶ˆã™ã¹ãStormã®é–‹ç™ºã«è‡³ã£ãŸã‚ˆã†ã§ã™ã€‚ã¡ãªã¿ã«åŒã˜ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã®åˆ†æ•£ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ å‡¦ç†ã‚·ã‚¹ãƒ†ãƒ ã¨ã—ã¦ç±³Yahoo! ãŒã€ŒS4ã€ã‚’å…¬é–‹ã—ã¦ã„ã¾ã™ãŒã€Stormã¯ãƒ‡ãƒ¼ã‚¿ã‚’å‡¦ç†ã™ã‚‹ä¿¡é ¼æ€§ã®é«˜ã•ã‚„APIã®ã‚·ãƒ³ãƒ—ãƒ«ã•ã«é•ã„ãŒã‚ã‚Šã¾ã™ã€‚ Stormã®ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ã¯Hadoopã¨ä¼¼é€šã£ã¦ãŠã‚Šã€ã‚¯ãƒ©ã‚¹ã‚¿ã¯Master-Workerãƒ‘ã‚¿ãƒ¼ãƒ³ã«åŸºã¥ã„ã¦ã„ã¾ã™ã€‚MasterãƒŽãƒ¼ãƒ‰ã«ã¯ã€ŒNimbusâ ã€â ã€WorkerãƒŽãƒ¼ãƒ‰ã«ã¯
willnet 2011/10/24
ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ãªhadoopã£ã¦ã“ã¨ã‹ãª

twitter

hadoop
ãƒªãƒ³ã‚¯
ã‚¦ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã‚ºã€æ¥ç•Œåˆã€åŸºå¹¹ãƒãƒƒãƒç”¨ã®Hadoopãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã€ŒAsakusaã€ ã‚’é–‹ç™ºã€ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹åŒ–ã—ã¦æä¾›é–‹å§‹ | ã‚¦ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã‚ºæ ªå¼ä¼šç¤¾
ã‚¦ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã‚ºã€æ¥ç•Œåˆã€åŸºå¹¹ãƒãƒƒãƒç”¨ã®Hadoopãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã€ŒAsakusaã€ ã‚’é–‹ç™ºã€ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹åŒ–ã—ã¦æä¾›é–‹å§‹ ã‚¦ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã‚ºæ ªå¼ä¼šç¤¾ï¼ˆæœ¬ç¤¾:æ±äº¬éƒ½ä¸å¤®åŒºã€ä»£è¡¨å–ç· å½¹ç¤¾é•·:æ¼†åŽŸ èŒ‚ã€ä»¥ä¸‹ ã‚¦ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã‚ºï¼‰ã¯ã€åŸºå¹¹æ¥å‹™ã‚·ã‚¹ãƒ†ãƒ ã®ãƒãƒƒãƒã‚’é«˜é€Ÿå‡¦ç†ã™ã‚‹ãŸã‚ã®ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ ã€ŒAsakusa FrameworkTMï¼ˆä»¥ä¸‹ã€Asakusaï¼‰ã€ã‚’æ¥ç•Œã§åˆã‚ã¦é–‹ç™ºã€ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹åŒ–ã—ã¦æä¾›ã™ã‚‹ã“ã¨ã‚’ç™ºè¡¨ã—ã¾ã™ã€‚ ã‚¯ãƒ©ã‚¦ãƒ‰æŠ€è¡“ã®æ™®åŠãŒé€²ã‚€ã¨å…±ã«ã€ä¼æ¥å†…ã‚·ã‚¹ãƒ†ãƒ ã¸ã®é©ç”¨ã®æ¤œè¨ŽãŒé€²ã‚“ã§ã„ã¾ã™ã€‚ç‰¹ã«Hadoopï¼ˆæ³¨1ï¼‰ã¯ã€ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã®åˆ†æ•£å‡¦ç†åŸºç›¤ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã¨ã—ã¦æ³¨ç›®ã‚’æµ´ã³ã¦ãŠã‚Šã€å¤§å®¹é‡ãƒ‡ãƒ¼ã‚¿ã‚’å¤šæ•°ã®ã‚µãƒ¼ãƒãƒ¼ã«åˆ†æ•£ã—ä¸¦åˆ—å‡¦ç†ã•ã›ã‚‹ã“ã¨ã§é«˜é€Ÿãªãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚’å®Ÿç¾ã§ãã¾ã™ã€‚ã—ã‹ã—ã“ã‚Œã¾ã§ã¯ã€Webãƒ‡ãƒ¼ã‚¿ã®åˆ†æžã‚„æ¶ˆè²»è€…ã®è¡Œå‹•è§£æžãªã©ã®B2Cåˆ†é‡Žã§ã®åˆ©ç”¨ãŒã»ã¨ã‚“ã©ã§ã‚ã‚Šã€ä¼æ¥ã®åŸºå¹¹æ¥å‹™ã‚·ã‚¹ãƒ†ãƒ ã«
willnet 2011/02/09
hadoopã®ãƒ©ãƒƒãƒ‘ï¼Ÿ

hadoop
ãƒªãƒ³ã‚¯
Hadoopã‚’æ¥å‹™ã§ä½¿ã£ã¦ã¿ãŸè©± - ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰é–‹ç™ºè€…ãƒ–ãƒã‚°
8æœˆã«å…¥ç¤¾ã—ãŸä½ã€…æœ¨ã§ã™ã€‚ã“ã‚“ã«ã¡ã‚ï¼ å…¥ç¤¾ã—ã¦ã‹ã‚‰ã¯Hadoopã‚’ä½¿ã†ã“ã¨ãŒå¤šãã€æ—¥ã€…ã€å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã¨æ ¼é—˜ã—ã¦ã„ã¾ã™ã€‚å¤§å¤‰ã§ã¯ã‚ã‚Šã¾ã™ãŒã€å€‹äººã§ã¯ãªã‹ãªã‹è§¦ã‚‹ã“ã¨ãŒå‡ºæ¥ãªã„ã‚ˆã†ãªå¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã‚’è§¦ã‚Œã‚‹ã®ã¯æ¥½ã—ã„ã§ã™ã€‚ ã•ã¦ã€Hadoopã¯æœ€è¿‘è‰²ã€…ãªã¨ã“ã‚ã§ä½¿ã‚ã‚Œå§‹ã‚ã¦ãã¦ã„ã‚‹ã¨æ€ã†ã‚“ã§ã™ãŒã€å®Ÿéš›ã«åˆ©ç”¨ã—ã¦ã¿ã¦å›°ã£ãŸäº‹ã‚„tipsãªã©ã€å®Ÿè·µçš„ãªæƒ…å ±ã¯ã¾ã ã‚ã¾ã‚Šå…¬é–‹ã•ã‚Œã¦ã„ã¾ã›ã‚“ã€‚ãã®è¾ºã®æƒ…å ±ã‚’ã¿ã‚“ãªæ±‚ã‚ã¦ã„ã‚‹ã¯ãšâ€¦ï¼ï¼ ãã“ã§ã€åƒ•ãŒå®Ÿéš›ã«è§¦ã£ã¦ã¿ã¦å›°ã£ãŸäº‹ã‚„Hadoopã‚’ä½¿ã†ä¸Šã§ãƒã‚¤ãƒ³ãƒˆã ã¨æ€ã£ãŸã“ã¨ãªã©ã‚’ç¤¾å†…å‹‰å¼·ä¼šã§ç™ºè¡¨ã—ãŸã®ã§å…¬é–‹ã—ã¦ã¿ã¾ã™ã€‚Hadoopã‚’ä½¿ã£ã¦ã„ã‚‹ï¼ˆä½¿ã„ãŸã„ã¨æ€ã£ã¦ã„ã‚‹ï¼‰æ–¹ã®å‚è€ƒã«ãªã‚Œã°å¹¸ã„ã§ã™ã€‚ [slideshare id=2711363&doc=20091214tech blog-091213183529-phpapp02] Hadoopã®åˆ©ç”¨ã¯ã¾ã ã¾ã è©¦è¡ŒéŒ¯èª¤ã®é€£ç¶š
willnet 2009/12/17
hadoop
ãƒªãƒ³ã‚¯
Rubyã§Hadoopã‚’ãƒ©ãƒƒãƒ—ã€åˆ†æ•£å‡¦ç†ãƒ„ãƒ¼ãƒ«ã‚ãƒƒãƒˆãŒç™»å ´ ï¼ ï¼ IT
2009/05/12 ç±³æ–°èžç¤¾å¤§æ‰‹ã®ãƒ‹ãƒ¥ãƒ¼ãƒ¨ãƒ¼ã‚¯ãƒ»ã‚¿ã‚¤ãƒ ã‚ºã¯5æœˆ11æ—¥ã€Rubyã«ã‚ˆã‚‹å¤§è¦æ¨¡åˆ†æ•£å‡¦ç†ã®ãƒ„ãƒ¼ãƒ«ã‚ãƒƒãƒˆã€ŒMap/Reduce Toolkitã€ï¼ˆMRToolkitï¼‰ã‚’GPLv3ã®ä¸‹ã«ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã§å…¬é–‹ã—ãŸã¨ç™ºè¡¨ã—ãŸã€‚MRToolkitã¯ã€ã™ã§ã«ç¨¼åƒã—ã¦ã„ã‚‹ã‚¯ãƒ©ã‚¹ã‚¿ä¸Šã®Hadoopã¨åˆã‚ã›ã¦ä½¿ã†ã“ã¨ã§Rubyã§å®¹æ˜“ã«Map/Reduceå‡¦ç†ã‚’è¨˜è¿°ã™ã‚‹ã“ã¨ãŒã§ãã‚‹ä¸€ç¨®ã®ãƒ©ãƒƒãƒ‘ãƒ¼ã€‚å‡¦ç†è‡ªä½“ã¯HadoopãŒè¡Œã†ã€‚ã™ã§ã«Hadoopã‚’ä½¿ã£ã¦ã„ã‚‹ãƒ¦ãƒ¼ã‚¶ãƒ¼ã§ã‚ã‚Œã°ã€ä¸å°è¦æ¨¡ã®ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã«å¯¾ã—ã¦ã€ã™ãã«MRToolkitã‚’é©ç”¨å¯èƒ½ã¨ã—ã¦ã„ã‚‹ã€‚ ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆã§æœ‰ç”¨ãªMapã€Reduceã®å‡¦ç†ãƒ¢ã‚¸ãƒ¥ãƒ¼ãƒ«ãŒå«ã¾ã‚Œã¦ã„ã¦ã€æ•°è¡Œã®Rubyã‚¹ã‚¯ãƒªãƒ—ãƒˆã‚’æ›¸ãã ã‘ã§ã€ä¾‹ãˆã°è†¨å¤§ãªApacheã®ãƒã‚°ã‹ã‚‰IPã‚¢ãƒ‰ãƒ¬ã‚¹åˆ¥ã®é–²è¦§å±¥æ´ã‚’ã¾ã¨ã‚ã‚‹ã¨ã„ã£ãŸå‡¦ç†ãŒå¯èƒ½ã¨ã„ã†ã€‚ç‹¬è‡ªã«Mapã‚„Reduceã®å‡¦ç†ã‚’å®šç¾©ã™ã‚‹ã“ã¨ã‚‚
willnet 2009/05/13
rubyã«ã‚ˆã‚‹Hadoopã®ãƒ©ãƒƒãƒ‘ãƒ¼ã€‚ãƒ‹ãƒ¥ãƒ¼ãƒ¨ãƒ¼ã‚¯ã‚¿ã‚¤ãƒ ã‚ºãŒã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã§å…¬é–‹ã—ãŸã£ã½ã„ã€‚

ruby

hadoop
ãƒªãƒ³ã‚¯
1