HadoopカンファレンスJapanã§è¡Œã£ãŸã‚¯ãƒãƒ¼ã‚¸ãƒ³ã‚°ã‚¹ãƒ”ーãƒã®ã‚¹ãƒ©ã‚¤ãƒ‰ã§ã™ã€‚ 日経BP社ã€æ—¥çµŒã‚³ãƒ³ãƒ”ュータ編集部ã®ä¸ç”°ãŒè¡Œã„ã¾ã—ãŸã€‚Read less
データセンター視点ã§è€ƒãˆã¦ã¿ã‚‹Hadoop
HadoopカンファレンスJapanã§è¡Œã£ãŸã‚¯ãƒãƒ¼ã‚¸ãƒ³ã‚°ã‚¹ãƒ”ーãƒã®ã‚¹ãƒ©ã‚¤ãƒ‰ã§ã™ã€‚ 日経BP社ã€æ—¥çµŒã‚³ãƒ³ãƒ”ュータ編集部ã®ä¸ç”°ãŒè¡Œã„ã¾ã—ãŸã€‚Read less
4å°æ§‹æˆã®Hadoopã‚’100分ã§è©¦ã—ã¦ã¿ã‚‹(CentOS + Cloudera)
(å‚考) Cloudera社ã®Hadoopパッケージã®æƒ…å ± http://archive.cloudera.com/docs/ å¿…è¦ãªã‚‚㮠・CentOS5.4ã®Linux環境4å°(CentOSã®ä»–ãƒãƒ¼ã‚¸ãƒ§ãƒ³ã‚„ã€Fedoraã€Redhatç‰ã§ã‚‚大丈夫ã ã¨æ€ã„ã¾ã™) ・インターãƒãƒƒãƒˆæŽ¥ç¶š ・Sun社Javaパッケージ(パッケージファイルをインターãƒãƒƒãƒˆã‹ã‚‰å–å¾—) ・Cloudera社ã®CDH3ã®Hadoopパッケージ(yumã§ã‚¤ãƒ³ã‚¿ãƒ¼ãƒãƒƒãƒˆã‹ã‚‰ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«) æ§‹æˆ 4å°ã®Hadoopクラスターã§ã™ã€‚ ・マスター1å°: nm01 ・スレーブ3å°: ns10, ns11, ns12 作æ¥æ‰‹é † 0. 準備 0-0. サーãƒãƒ¼å ã“ã“ã§åˆ©ç”¨ã™ã‚‹ã‚µãƒ¼ãƒãƒ¼/サーãƒãƒ¼åã¯ä»¥ä¸‹ã®4ã¤ã§ã™ã€‚ nm01 ns10 ns11 ns12 ã™ã¹ã¦ã®ã‚µãƒ¼ãƒãƒ¼(nm01, ns10, ns11, ns12)ã¨ä½œæ¥ã‚¯ãƒ©ã‚¤ã‚¢
Mapreduce & Hadoop Algorithms in Academic Papers | atbrox
The newest and most up-to-date version (May 2010) this blog post is available at http://mapreducebook.org An updated and extended version of this blog post can be found here. Motivation Learn from academic literature about how the mapreduce parallel model and hadoop implementation is used to solve algorithmic problems. Disclaimer: this is work in progress (look for updates) Input Data – Academic P
Mass-scale computing: Why Hadoop is hot but Java is not
With the massive amount of data proliferating the Web, companies such as Google and many others are building new technologies to sort it all. Core to that movement is something called MapReduce, a software technique that breaks down huge amounts of data into smaller bits. Operating on the smaller bits, and then piecing results together to form the big picture again has proven extremely successful.
HDFSã®èªã¿æ›¸ã - kikumotoã®ãƒ¡ãƒ¢å¸³
Hadoop ã‚’è¨å®šã§ããŸã®ã§ã€Java ã‹ã‚‰HDFSã«ãƒ•ã‚¡ã‚¤ãƒ«ã‚’ç½®ã„ãŸã‚Šã€HDFSã‹ã‚‰ãƒ•ã‚¡ã‚¤ãƒ«ã‚’èªã¿å‡ºã—ãŸã‚Šã™ã‚‹æ–¹æ³•ã‚’メモã£ã¦ãŠã。 以下ã§ã€å¤§ãã分ã‘ã¦ï¼’種類ã®æ–¹æ³•ã‚’書ã„ã¦ã„ã‚‹ã‘ã‚Œã©ã€å…±é€šãªç‚¹ã¨ã—ã¦ã¾ãšã€Configuration クラスã®ã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹ã‚’用æ„ã—ã¦ãŠãã¨ã„ã†ã®ãŒã‚る。ã“ã‚Œã¯ã‚¯ãƒ©ã‚¹ãƒ‘ス上ã‹ã‚‰ core-site.xml ã‚’èªã‚€è¾¼ã‚€ã‚ˆã†ã«ãªã£ã¦ã„る。 ã“れもå«ã‚ã¦ã€ã‚¯ãƒ©ã‚¹ãƒ‘スã«ã¯ core-site.xml ã‚’å«ã‚€ãƒ‡ã‚£ãƒ¬ã‚¯ãƒˆãƒªã¸ã®ãƒ‘ス hadoop-0.20.0-core.jar commons-logging-1.0.4.jar を指定ã™ã‚‹ã“ã¨ãŒãŒæœ€ä½Žé™å¿…è¦ãªã‚‚ã®ã®ã‚ˆã†ã§ã‚る。 ãƒãƒ¼ã‚«ãƒ«ãƒ•ã‚¡ã‚¤ãƒ«åを指定ã—ãŸèªã¿æ›¸ã HDFSã¸ã®æ›¸ã込㿠ãƒãƒ¼ã‚«ãƒ«ãƒ•ã‚¡ã‚¤ãƒ«åã¨HDFS上ã®ãƒ•ã‚¡ã‚¤ãƒ«åを指定ã—ã¦ã€HDFSã«æ›¸ãè¾¼ã‚€å ´åˆã¯ FileSystem.copyFromLocalFile
HDFSã®ãƒžã‚¦ãƒ³ãƒˆ - kikumotoã®ãƒ¡ãƒ¢å¸³
Hadoop Wiki ã« MountableHDFS ã¨ã„ã†ã®ãŒã‚ã‚Šã€HDFS をマウントã§ãるよã†ãªã“ã¨ãŒæ›¸ã„ã¦ã‚ã‚‹ã®ã§è©¦ã—ã¦ã¿ãŸã®ã§ã€ãれをメモã—ã¦ãŠã。 ã„ãã¤ã‹å®Ÿè£…ãŒã‚るよã†ã ã‘ã©ã€ä»Šå›žè©¦ã—ãŸã®ã¯ fuse-dfs ã¨ã„ã†ã‚‚ã®ã€‚ã“ã¡ã‚‰ã®ã‚µã‚¤ãƒˆã‚‚å‚考ã«ã•ã›ã¦ã„ãŸã ã„ãŸã€‚ gcc, gcc-c++ ã¨ã‹ ant ã¨ã‹ã„ã£ãŸã‚³ãƒ³ãƒ‘イル環境ã¯ã™ã§ã«æ•´ã£ã¦ã„ã‚‹ã‚‚ã®ã¨ã™ã‚‹ã€‚ã¨ã‚Šã‚ãˆãšã€/home/hadoop/.bashrc ã«ä»¥ä¸‹ã®ã‚ˆã†ãªè¨˜è¿°ã‚’ã—ã¦ã‚る。 export JAVA_HOME=/usr/java/latest export ANT_HOME=~/apache-ant-1.7.1 export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$PATH ã¾ãšã€FUSE ãŒå¿…è¦ãªã®ã§ã€ãƒ“ルド・インストールã—ã¦ãŠã。 $ tar xzf fuse-2.7.4.ta
1
ランã‚ング
ランã‚ング
ãŠçŸ¥ã‚‰ã›
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
How to use HBase with Hadoop
{{#tags}}- {{label}}
{{/tags}}