The Hadoop Ecosystem Table
The Hadoop Distributed File System (HDFS) offers a way to store large files across multiple machines. Hadoop and HDFS was derived from Google File System (GFS) paper. Prior to Hadoop 2.0.0, the NameNode was a single point of failure (SPOF) in an HDFS cluster. With Zookeeper the HDFS High Availability feature addresses this problem by providing the option of running two redundant NameNodes in the s
èªè¨¼/èªå¯ãŒå®Ÿç¾ã™ã‚‹å®‰å…¨ã§é«˜é€Ÿåˆ†æžå¯èƒ½ãªåˆ†æžå‡¦ç†åŸºç›¤
分æžå‡¦ç†åŸºç›¤ã«ãŠã„ã¦ã€ãƒ‡ãƒ¼ã‚¿ã®æ°‘主化・マルãƒãƒ†ãƒŠãƒ³ãƒˆåŒ–を進ã‚ã‚‹ãŸã‚ã«ã¯ã„ãã¤ã‹ã®ãƒãƒ¼ãƒ‰ãƒ«ãŒã‚ã‚Šã¾ã™ã€‚ãã®ãƒãƒ¼ãƒ‰ãƒ«ã®ä¸€ã¤ã«ã€ãƒ‡ãƒ¼ã‚¿ãƒªã‚¹ã‚¯ã‚’低減ã™ã‚‹ãŸã‚ã®èªè¨¼ãƒ»èªå¯ã®å®Ÿç¾ãŒæŒ™ã’られã¾ã™ã€‚ 本講演ã§ã¯ã€ãƒ‰ãƒ¯ãƒ³ã‚´ãŒåˆ†æžå‡¦ç†åŸºç›¤ã®èªè¨¼ãƒ»èªå¯æ©Ÿæ§‹ã‚’一新ã—ãŸå†…容ã¨ãƒã‚¤ãƒ³ãƒˆã€ã¾ãŸãれらã®èªè¨¼ãƒ»èªå¯æ©Ÿæ§‹ã‚’ベースã¨ã—ã€åˆ†æžå‡¦ç†ã‚’æ•°å€ã‹ã‚‰æ•°åå€ã«é«˜é€ŸåŒ–ã—ãŸäº‹ä¾‹ã‚’紹介致ã—ã¾ã™ã€‚
Hadoopã¨ã‚¯ãƒ©ã‚¦ãƒ‰ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸
ã–ã£ã¨è¦ç´„ã™ã‚‹ã¨ オンデマンドã€ã‚¢ãƒ‰ãƒ›ãƒƒã‚¯ãªãƒ¯ãƒ¼ã‚¯ãƒãƒ¼ãƒ‰ã¯ã‚¯ãƒ©ã‚¦ãƒ‰ã§å‹•ä½œã•ã›ã‚‹ã¨çµŒæ¸ˆåˆç†æ€§ãŒé«˜ã„。必è¦ãªãƒªã‚½ãƒ¼ã‚¹ã‚’å¿…è¦ãªåˆ†ã ã‘利用ã—ã€çµ‚ã‚ã£ãŸã‚‰ç ´æ£„ã§ãã‚‹ã‹ã‚‰ã€‚（当ãŸã‚Šå‰ã®è©±ã€‚）ãã“ã§ãƒã‚¤ãƒ³ãƒˆã«ãªã£ã¦ãã‚‹ã®ãŒAmazon S3ã‚„Windows Azure Storage Blobã®ã‚ˆã†ãªã‚¯ãƒ©ã‚¦ãƒ‰ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã€‚ã“れらã®ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã¯é«˜ã„å¯ç”¨æ€§ã¨åœ°ç†çš„分散ãªã©ã«ã‚ˆã‚‹ãƒ‡ãƒ¼ã‚¿è€ä¹…性ã€ãƒžãƒ«ãƒãƒ†ãƒŠãƒ³ã‚·ãƒ¼ã‚’æŒã¡ãªãŒã‚‰ã‚‚経済的ãªä¾¡æ ¼ã§åˆ©ç”¨ã§ãる。クラウド上ã®ãƒ‡ãƒ¼ã‚¿ãƒ¬ã‚¤ã‚¯ã¯é–“é•ã„ãªãã“れらã®ã‚¯ãƒ©ã‚¦ãƒ‰ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã¨ãªã‚‹ã€‚ã¨ã„ã†ã“ã¨ã§ã€ã‚¯ãƒ©ã‚¦ãƒ‰ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã‚’ä¸å¿ƒã«ã€ãã®å‘¨ã‚Šã«ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã®ãŸã‚ã®ã‚¯ãƒ©ã‚¹ã‚¿ã‚’é…ç½®ã—ã¦ã„ã“ã†ã¨ã„ã†ã‚¢ãƒ¼ã‚テクãƒãƒ£ãŒèªžã‚‰ã‚Œã¦ã„る。ã—ã‹ã—ã€ã“ã“ã«ã¯ã„ãã¤ã‹è§£æ±ºã™ã¹ã課題ãŒã‚る。 クラウドストレージã¯File System APIã§ã¯ãªã REST APIã§åˆ©ç”¨ã™ã‚‹å¿…è¦ãŒã‚る。当然ãƒãƒ¼ã‚«ãƒ«ã«ã‚ã‚‹HDF
Hadoop入門ã¨ã‚¯ãƒ©ã‚¦ãƒ‰åˆ©ç”¨
This document discusses Hadoop and related technologies. It introduces Hadoop, its components MapReduce and HDFS, and how they work together. It also briefly mentions related Apache projects like Mahout and how companies like Amazon, Yahoo and Facebook use Hadoop in their systems. Finally, it covers Amazon's Elastic MapReduce service, which allows running Hadoop jobs in the cloud.
1
ランã‚ング
ランã‚ング
ãŠçŸ¥ã‚‰ã›
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
Hadoop and Kerberos
IBM Developer
{{#tags}}- {{label}}
{{/tags}}