Spark shark
6. ãŠã¾ãˆã れよ？ Tsuyoshi Ozawa @oza_x86 OSS developer Apache Hadoop ã®é–‹ç™ºã‚’ã—ã¦ã¾ã™ github : oza â†ã®22ç« ã‚’æ›¸ã„ãŸã‚ˆï¼ Tuesday, October 22, 13
DiskSetup - HADOOP2 - Apache Software Foundation
Setting up Disks for Hadoop Here are some recommendations for setting up disks in a Hadoop cluster. What we have here is anecdotal -hard evidence is very welcome, and everyone should expect a bit of trial and error work. Key Points Goals for a Hadoop cluster are normally massive amounts of data with high I/O bandwidth. Your MapReduce jobs may be IO bound or CPU/Memory bound -if you know which one
[Hadoop]複数ディスクを使ã£ã¦åŠ¹çŽ‡ã®è‰¯ã„処ç†
Hadoop ã§ã¯ä¸€ã¤ã®ãƒŽãƒ¼ãƒ‰ã‚ãŸã‚Šè¤‡æ•°ãƒ‡ã‚£ã‚¹ã‚¯ã‚’使ã†ã“ã¨ãŒã§ãã¾ã™ãŒï¼Œãƒ‡ã‚£ã‚¹ã‚¯ã‚’増やã™ã“ã¨ã«ã‚ˆã£ã¦ã©ã‚Œãらã„性能ãŒå‘上ã™ã‚‹ã‹èª¿ã¹ã¾ã—ãŸï¼Ž HDFSã§ä½¿ç”¨ã™ã‚‹ãƒ‡ã‚£ã‚¹ã‚¯ã‚’dfs.data.dirã«ã‚³ãƒ³ãƒžåŒºåˆ‡ã‚Šã§è¨˜å…¥ã™ã‚‹ã“ã¨ã§è¤‡æ•°ä½¿ãˆã¾ã™ï¼Ž <property> <name>dfs.data.dir</name> <value>/data/local/${user.name}/hadoop/dfs/data, /data/local2/${user.name}/hadoop/dfs/data</value> </property> ã—ã‹ã—，ã“ã‚Œã ã‘ã§ã¯ã¾ã ダメã§ï¼Œmapタスク，reduceタスクãŒä¸é–“データを書ã込むディスクも複数指定ã—ãªã—ã¨Hadoopã®ã‚¸ãƒ§ãƒ–ã§è¤‡æ•°ãƒ‡ã‚£ã‚¹ã‚¯ã‚’効率良ã使ãˆã¾ã›ã‚“．mapred.local.dir ã§è¨å®šå¯èƒ½ã§ã™ï¼Ž <property> <name>mapre
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
処ç†ã‚’実行ä¸ã§ã™
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
Cloudera Blog
Configuring Heterogeneous Storage in HDFS | 6.3.x | Cloudera Documentation
{{#tags}}- {{label}}
{{/tags}}