[B! hadoop] micknã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

mickn id:mickn

hadoopã«é–¢ã™ã‚‹micknã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (17)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Hadoop Conference Japan 2013ã§è©±ã—ãŸã“ã¨ã¨æ€ã£ãŸã“ã¨ - æ€¥ãŒã°å›žã‚Œã€é¸ã¶ãªã‚‰è¿‘é“
Hadoop Conference Japan 2013 http://hcj2013w.eventbrite.com/ å…ˆé€±çµ‚äº†ã€‚ã‹ãªã‚Šã®ç››æ³ã§çµ‚ã‚ã£ãŸæ„Ÿã˜ã§ã™ã€‚ã¾ãšã¯é–‹å‚¬ã‚’ã‚µãƒãƒ¼ãƒˆã—ã¦é ‚ãã€ç›¸å½“ã®è² æ‹…ã¾ã§é ‚ã„ãŸãƒªã‚¯ãƒ«ãƒ¼ãƒˆãƒ»ãƒ†ã‚¯ãƒŽãƒã‚¸ãƒ¼æ§˜ã«æ„Ÿè¬ç”³ã—ä¸Šã’ã¾ã™ã€‚ã©ã†ã‚‚ã‚ã‚ŠãŒã¨ã†ã”ã–ã„ã¾ã—ãŸã€‚ ã•ã¦ã€ãˆã£ã¨ã€å‰å›žãŒãã‚‚ãã‚‚ã„ã¤ã ã£ãŸã®ã‹ã€è‰¯ãè¦šãˆã¦ãªã„ã‚ã‘ã§ã€‚2011 Fallã ã£ãŸã‚ˆã†ãªã€‚ http://hadoop-conference-japan-2011-fall.eventbrite.com/ 2011å¹´ã®9æœˆãªã®ã§ã€1å¹´4ãƒ¶æœˆã¶ã‚Šã¨ã„ã†æ„Ÿã˜ã§ã™ãã€‚Trackæ•°ãŒå¢—ãˆã¦2ã‹ã‚‰3ã§ã€ä¼šå ´ã‚‚ãƒ™ãƒ«ã‚µãƒ¼ãƒ«ã‹ã‚‰ãƒ“ãƒƒã‚°ã‚µã‚¤ãƒˆã«ãªã£ã¦ã„ã¾ã—ãŸã€‚äººæ•°ã‚‚1000äººè¶…ã«ãªã£ã¦ãŠã‚Šã¾ã™ã€‚ ä»¥ä¸‹ã€æ„Ÿæƒ³æ–‡ã§ã™ã€‚è¨˜éŒ²ã¨ã—ã¦ãŠã„ã¦ãŠãæ„Ÿã˜ã§ã€‚ ãƒ»å†…å®¹ã§å°è±¡ã«æ®‹ã£ãŸã‚‚ã® ãƒ»HBase~LINEã®ãƒãƒƒã‚¯ãƒœãƒ¼ãƒ³ã§ä½¿ã£ã¦
mickn 2013/01/30
hadoop

hbase
ãƒªãƒ³ã‚¯
Storage infrastructure using HBase behind LINE messages
The document discusses NHN Japan's use of HBase for the LINE messaging platform's storage infrastructure. Some key points: - HBase is used to store tens of billions of message rows per day for LINE, achieving sub-10ms response times and high availability through dual clusters. - The presentation covers their experience migrating HBase clusters between data centers online, handling NameNode failure
mickn 2013/01/23
hadoop

hbase
ãƒªãƒ³ã‚¯
ãƒãƒƒãƒè¨è¨ˆã¨å®Ÿè£…ã‚¬ã‚¤ãƒ‰ | Asakusa Framework
Asakusa Frameworkã§ã¯ã€åŸºå¹¹å‘ã‘ãƒãƒƒãƒã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã®ä½œæˆã«ä¸»çœ¼ã‚’ç½®ã„ãŸç‹¬è‡ªã®é ˜åŸŸç‰¹åŒ–è¨€èªž(Domain Specific Language, ä»¥ä¸‹ Asakusa DSL)ã‚’åˆ©ç”¨ã—ã¦ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã‚’ä½œæˆã—ã¾ã™ã€‚ ã‚±ãƒ¼ã‚¹ãƒ»ã‚¹ã‚¿ãƒ‡ã‚£ã‚’é€šã˜ã¦ã€Asakusa DSLã‚’å‰æã¨ã—ãŸè¨è¨ˆæ‰‹æ³•ã¨å…·ä½“çš„ãªå®Ÿè£…æ–¹æ³•ã‚’ç†è§£ã™ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ ã“ã“ã§è§£èª¬ã™ã‚‹è¨è¨ˆæ‰‹æ³•ã¨å®Ÿè£…ã‚¬ã‚¤ãƒ‰ã¯ã€Asakusa DSLã®æ¦‚è¦ã‚’ç†è§£ã—ã¦ã„ã‚‹ã“ã¨ã‚’å‰æã¨ã—ã¦ã„ã¾ã™ã€‚ Asakusa DSLã«ã¤ã„ã¦ã¯ã€ä»¥ä¸‹ã®ãƒ‰ã‚ãƒ¥ãƒ¡ãƒ³ãƒˆã‚’å‚ç…§ã—ã¦ãã ã•ã„ã€‚ Asakusa Framework å…¥é–€ Asakusaã§ã¯ã€è¨è¨ˆæ‰‹æ³•ã¨ã—ã¦DAGï¼ˆæœ‰å‘éžå¾ªç’°ã‚°ãƒ©ãƒ•ï¼‰ã‚’åˆ©ç”¨ã—ãŸè¨˜è¿°æ³•ã§è¨è¨ˆã‚’è¡Œã„ã¾ã™ã€‚ä»¥ä¸‹ã®ãƒ‰ã‚ãƒ¥ãƒ¡ãƒ³ãƒˆã§ã¯Asakusa DSLã®è¨è¨ˆæ‰‹æ³•ã¨å®Ÿè£…æ‰‹æ³•ã‚’è§£èª¬ã—ã¾ã™ã€‚ AsakusaDSLè¨è¨ˆæ‰‹æ³•ï¼ˆPDFï¼‰ AsakusaD
mickn 2012/01/24
hadoop
ãƒªãƒ³ã‚¯
Facebook has the world's largest Hadoop cluster!
It is not a secret anymore! The Datawarehouse Hadoop cluster at Facebook has become the largest known Hadoop storage cluster in the world. Here are some of the details about this single HDFS cluster:21 PB of storage in a single HDFS cluster2000 machines12 TB per machine (a few machines have 24 TB each)1200 machines with 8 cores each + 800 machines with 16 cores each32 GB of RAM per machine15 map-r
mickn 2010/07/07
hadoop

facebook

hdfs
ãƒªãƒ³ã‚¯
Scalability of the Hadoop Distributed File System (Hadoop and Distributed Computing at Yahoo!)
The bottom line is that we achieved the target in Petabytes and got close to the target in the number of files. But this is done with a smaller number of nodes and the need to support a workload close to 100,000 clients has not yet materialized. The question now is whether the goals are feasible with the current system architecture. Namespace Limitations HDFS is based on an architecture where the
mickn 2010/05/13
hadoop

hdfs
ãƒªãƒ³ã‚¯
Legacy Communities - IBM Community
If youâ€™re looking for a developerWorks forum â€” Don't panic! You are in the right place. You are here because specific IBM developerWorks forums, blogs and other Connections content have been decommissioned. This page will help you find the content you are looking for, get answers to your questions, and find a new community to call home. Where am I? You are on the IBM Community area, a collection o
mickn 2010/05/10
hadoop
ãƒªãƒ³ã‚¯
åƒ•ãŒ Amazon Elastic MapReduce ã‚’ä½¿ã‚ãªã„ï¼“ã¤ã®ç†ç”± - (ï¾Ÿâˆ€ï¾Ÿ)oå½¡ sasata299's blog
2010å¹´04æœˆ20æ—¥22:46 Hadoop åƒ•ãŒ Amazon Elastic MapReduce ã‚’ä½¿ã‚ãªã„ï¼“ã¤ã®ç†ç”± ã€è¿½è¨˜ã€‘ ã“ã®å†…å®¹ã¯å¤ã„ã§ã™ã€‚æœ€è¿‘ã¯EMRã‚’åˆ©ç”¨ã—ã¦ã¾ã™ã€‚ã¤ 961ä¸‡äººã®é£Ÿå“ã‚’æ”¯ãˆã‚‹ãƒ‡ãƒ¼ã‚¿è§£æž åƒ•ã¯æ—¥é ƒã‹ã‚‰ Hadoop ã•ã‚“ã‚’ä½¿ã£ã¦ï¼ˆã‚ã¨ EC2 ã¨ S3 ã‚‚ä½¿ã£ã¦ã¾ã™ï¼ï¼‰è‰²ã€…ã”ã«ã‚‡ã”ã«ã‚‡ã™ã‚‹ã“ã¨ãŒå¤šã„ã‚“ã§ã™ãŒã€EC2 ä¸Šã§ Ruby ã§ã‚¹ã‚¯ãƒªãƒ—ãƒˆæ›¸ã„ã¦ä½¿ã£ã¦ã‚‹ã‚ˆãƒ¼ã¨è¨€ã†ã¨ã‚ˆãè¨€ã‚ã‚Œã‚‹ã®ãŒã€ ã€Œä½•ã§ Amazon Elastic MapReduce ä½¿ã‚ãªã„ã®ï¼Ÿã€ ã¨ã„ã†è³ªå•ã§ã™ã€‚ä»Šã¾ã§ã“ã‚Œã«å€‹åˆ¥ã«ç”ãˆã¦ãŸã‚Šã—ãŸã‚“ã§ã™ãŒã€ã‚ã‚“ã©ãã•ã„ã®ã§ä¸€åº¦ã“ã“ã«ã¾ã¨ã‚ã¦ãŠãã¾ã™ã€‚ä»¥å‰ã€Elastic MapReduce ã‚’å®Ÿéš›ã«è©¦ã—ã¦ã¿ãŸã‚“ã§ã™ãŒã€ãã®ã¨ãæ„Ÿã˜ãŸãƒ¡ãƒªãƒƒãƒˆãƒ»ãƒ‡ãƒ¡ãƒªãƒƒãƒˆã¯ã“ã‚“ãªæ„Ÿã˜ã§ã™ã€‚ä»Šã¯ãã†ã˜ã‚ƒãªã„ã‚ˆï¼ã¨ã‹ã‚ã£ãŸã‚‰ã”ã‚ã‚“ãªã•ã„ï¼žï¼œ â˜… ãƒ¡ãƒªãƒƒãƒˆ ãƒ»Hado
mickn 2010/04/21
hadoop

amazonaws
ãƒªãƒ³ã‚¯
Hadoop Hack Night â€¦ æŠ€è¡“è©•è«–ç¤¾
ç¾åœ¨ï¼Œç±³å›½Yahoo! Hadoopãƒãƒ¼ãƒ ã«ã¦ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒˆã‚’å‹™ã‚ã‚‹Owen Oâ€™MalleyãŒ3æœˆã«æ¥æ—¥ã„ãŸã—ã¾ã™ã€‚ãã‚Œã«åˆã‚ã›ã¦Hadoopã®ç´¹ä»‹ï¼Œç±³å›½Yahoo!ã§ã®æ´»ç”¨äº‹ä¾‹ãªã©ã”ç´¹ä»‹ã™ã‚‹ã‚»ãƒŸãƒŠãƒ¼ã‚’é–‹å‚¬ã„ãŸã—ã¾ã™ã€‚å½“æ—¥ã¯å‚åŠ è€…ã®æ–¹ã€…ã¨ã®ãƒˆãƒ¼ã‚¯ã‚»ãƒƒã‚·ãƒ§ãƒ³ã‚’è¨ã‘ï¼ŒHadoopã‚³ãƒŸãƒ¥ãƒ‹ãƒ†ã‚£ã«æ‰€å±žã—ã¦ã„ã‚‹Owenæ°ã¨ã®äº¤æµã®å ´ã‚‚æä¾›ã„ãŸã—ã¾ã™ã€‚ æœ¬ã‚¤ãƒ™ãƒ³ãƒˆã®Twitterãƒãƒƒã‚·ãƒ¥ã‚¿ã‚°ã¯ã€Ž#hadoophnã€ã§ã™ã€‚ ãªãŠï¼Œå½“æ—¥ã®æ¨¡æ§˜ã¯ä»¥ä¸‹ã®URLã§Ustreamä¸ç¶™ã‚’äºˆå®šã—ã¦ãŠã‚Šã¾ã™ã€‚ http://www.ustream.tv/channel/hadoophn ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯å›žç·šã®éƒ½åˆã«ã‚ˆã‚Šï¼Œ3Gå›žç·šã§ã®ä¸ç¶™ã¨ãªã‚‹ã“ã¨ãŒäºˆæƒ³ã•ã‚Œã¾ã™ã€‚å®‰å®šã—ãŸé…ä¿¡ãŒã§ããªã„å¯èƒ½æ€§ãŒã‚ã‚Šã¾ã™ã®ã§ï¼Œä¼šå ´ã¾ã§ãŠè¶Šã—ã„ãŸã ãã“ã¨ã‚’ãŠã™ã™ã‚ã„ãŸã—ã¾ã™ã€‚ ã€ŒHadoop Hack Nightã€ã«ç”³ã—è¾¼ã‚€ ãŠç”³ã—è¾¼ã¿æœŸé–“ï¼š2æœˆ
mickn 2010/02/25
hadoop
ãƒªãƒ³ã‚¯
Hadoopã‚’ä½¿ã„ã“ãªã™(1)
ã¾ãšã€ 1 ã®å…¥åŠ›ãƒ•ã‚¡ã‚¤ãƒ«ã‚’åˆ†å‰²ã™ã‚‹æ–¹æ³•ã¯ã€InputFormatã‚¯ãƒ©ã‚¹ã®ã€getSplitsé–¢æ•°ã‚’ä¸Šæ›¸ãã™ã‚‹ã“ã¨ã§ã€ã‚«ã‚¹ã‚¿ãƒžã‚¤ã‚ºã§ãã¾ã™ã€‚ ã¾ãŸã€ 3 ã®InputSplitã‹ã‚‰ã€Keyã¨Valueã‚’æŠ½å‡ºã™ã‚‹å‡¦ç†ã‚‚ã€InputFormatã‚¯ãƒ©ã‚¹ã‚’é€šã˜ã¦ã‚«ã‚¹ã‚¿ãƒžã‚¤ã‚ºã§ãã¾ã™ã€‚ InputFormatã®getRecordReaderé–¢æ•°ã‚’é€šã˜ã¦ã€RecordReaderã‚¯ãƒ©ã‚¹ã‚’ç”Ÿæˆã™ã‚‹ã®ã§ã™ãŒã€ã“ã‚Œã«ä»»æ„ã®RecordReaderã‚¯ãƒ©ã‚¹ã‚’æŒ‡å®šã™ã‚Œã°OKã§ã™ã€‚ 2 ã®Mapå‡¦ç†ã§ã™ãŒã€ãƒ¦ãƒ¼ã‚¶ãŒæŒ‡å®šã—ãŸMapperã‚¯ãƒ©ã‚¹ã®å‡¦ç†ã‚’å®Ÿè¡Œã—ã¾ã™ã€‚ Mapperã‚¯ãƒ©ã‚¹ã¯ã€MapRunnerã‚¯ãƒ©ã‚¹ã‚’é€šã˜ã¦ã€åˆæœŸåŒ–å‡¦ç†ã€mapé–¢æ•°ã‚’ç¹°ã‚Šè¿”ã™éŽç¨‹ã€çµ‚äº†å‡¦ç†ã¨ã„ã£ãŸä¸€é€£ã®æµã‚Œã‚’å®Ÿè¡Œã—ã¾ã™ã€‚ MapRunnerã‚¯ãƒ©ã‚¹ã‚’ã‚«ã‚¹ã‚¿ãƒžã‚¤ã‚ºã™ã‚Œã°ã€ã“ã†ã—ãŸæµã‚Œã‚’åˆ¶å¾¡ã™ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ 0.20.0ã‹ã‚‰ã®æ–°ã—ã„MapRed
mickn 2010/01/27
hadoop

mapreduce
ãƒªãƒ³ã‚¯
ç¬¬28å›žã€€Rubyã¨Hadoopã§åˆ†æ•£å‡¦ç†ã€€Hadoop Streamingã®ä»•çµ„ã¿ | gihyo.jp
ã¯ã˜ã‚ã« Hadoopã¨ã¯ã€Googleã®åŸºç›¤æŠ€è¡“ã§ã‚ã‚‹MapReduceã‚’Javaã§ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹å®Ÿè£…ã—ãŸã‚‚ã®ã§ã€åˆ†æ•£å‡¦ç†ã®ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã§ã™ã€‚Hadoopã‚’ä½¿ã†ã¨ã€1å°ã®ã‚µãƒ¼ãƒã§ã¯æ™‚é–“ã®æŽ›ã‹ã‚‹ã‚ˆã†ãªå‡¦ç†ã‚’ã€è¤‡æ•°ã®ã‚µãƒ¼ãƒã§åˆ†æ•£å‡¦ç†ã•ã›ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ã€Œâ å‡¦ç†ã‚’å‰²ã‚ŠæŒ¯ã£ãŸã‚µãƒ¼ãƒãŒå£Šã‚ŒãŸå ´åˆã©ã†ã™ã‚‹ã‹ã€ãªã©ã®è€éšœå®³æ€§ã®å•é¡Œã‚‚HadoopãŒç®¡ç†ã—ã¦ãã‚Œã‚‹ãŸã‚ã€åˆ©ç”¨è€…ã¯å‡¦ç†ã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã®ã¿ã«é›†ä¸ã™ã‚‹ã“ã¨ãŒã§ãã‚‹ã®ã§ã™ã€‚ç´ æ™´ã‚‰ã—ã„ã§ã™ãã€‚æœ€è¿‘ã§ã¯Yahoo!ã‚„ã¯ã¦ãªãªã©ã€æ§˜ã€…ãªä¼æ¥ã§ã‚‚åˆ©ç”¨ã•ã‚Œã‚‹ã‚ˆã†ã«ãªã£ã¦ãã¦ã„ã¾ã™ã€‚ Hadoopå°Žå…¥ã®èƒŒæ™¯ ç†è€…ã¯ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰æ ªå¼ä¼šç¤¾ã«å‹¤ã‚ã¦ã„ã¾ã™ã€‚open('http://cookpad.com'); return false;">ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã¨ã„ã†ã‚µã‚¤ãƒˆãŒæœ‰åã ã¨æ€ã„ã¾ã™ãŒã€ä»–ã«ã‚‚æºå¸¯ç‰ˆã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã§ã‚ã‚‹open('http://m.cookpad.co
mickn 2010/01/18
hadoop
ãƒªãƒ³ã‚¯
Scala on Hadoop: Hadoop Conference - stanaka's blog
å…ˆæ—¥ã€Hadoop Conferenceã§Scala on Hadoopã¨ã„ã†ã‚¿ã‚¤ãƒˆãƒ«ã§ç™ºè¡¨ã—ã¦ãã¾ã—ãŸã€‚ã‚¹ãƒ©ã‚¤ãƒ‰ã‚’ä»¥ä¸‹ã«ç½®ã„ã¦ãŠãã¾ã™ã€‚ Scala on HadoopView more presentations from Shinji Tanaka. ãƒ€ã‚¤ã‚¸ã‚§ã‚¹ãƒˆã¨ã—ã¦ã€Scalaã‚’Hadoopã§å‹•ã‹ã™ãŸã‚ã®æ–¹æ³•ã‚’æ›¸ã„ã¦ãŠãã¾ã™ã€‚ ã¾ãšã€Hadoopä¸Šã§Scalaã‚’å®Ÿè¡Œã•ã›ã‚‹ãŸã‚ã«ã¯ã€Javaã¨Scalaã‚’æŽ¥ç¶šã™ã‚‹ãƒ©ã‚¤ãƒ–ãƒ©ãƒªãŒå¿…è¦ã¨ãªã‚Šã¾ã™ã€‚ã“ã“ã§ã¯ã€SHadoop( http://code.google.com/p/jweslley/source/browse/#svn/trunk/scala/shadoop )ã‚’ä½¿ç”¨ã—ã¾ã™ã€‚SHadoopã¯ã€åž‹å¤‰æ›ã‚’è¡Œã†ã‚·ãƒ³ãƒ—ãƒ«ãªãƒ©ã‚¤ãƒ–ãƒ©ãƒªã§ã™ã€‚ ã‚ˆãã‚ã‚‹WordCountã®ã‚µãƒ³ãƒ—ãƒ«ã€WordCount.scala (http://blog.jo
mickn 2009/11/25
scala

hadoop
ãƒªãƒ³ã‚¯
ãƒ‡ãƒ¼ã‚¿ã‚»ãƒ³ã‚¿ãƒ¼è¦–ç‚¹ã§è€ƒãˆã¦ã¿ã‚‹Hadoop
Hadoopã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹Japanã§è¡Œã£ãŸã‚¯ãƒãƒ¼ã‚¸ãƒ³ã‚°ã‚¹ãƒ”ãƒ¼ãƒã®ã‚¹ãƒ©ã‚¤ãƒ‰ã§ã™ã€‚ æ—¥çµŒBPç¤¾ã€æ—¥çµŒã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ã‚¿ç·¨é›†éƒ¨ã®ä¸ç”°ãŒè¡Œã„ã¾ã—ãŸã€‚Read less
mickn 2009/11/25
hadoop
ãƒªãƒ³ã‚¯
HadoopãŒç§˜ã‚ã‚‹å¯èƒ½æ€§ï¼šã‚ªãƒ³ãƒ—ãƒ¬ãƒŸã‚¹ã§ã‚‚ã‚¯ãƒ©ã‚¦ãƒ‰ã§ã‚‚ä½¿ãˆã‚‹ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ ã®é…åŠ›
å°åˆ·ã™ã‚‹ ãƒ¡ãƒ¼ãƒ«ã§é€ã‚‹ ãƒ†ã‚ã‚¹ãƒˆ HTML é›»åæ›¸ç± PDF ãƒ€ã‚¦ãƒ³ãƒãƒ¼ãƒ‰ ãƒ†ã‚ã‚¹ãƒˆ é›»åæ›¸ç± PDF ã‚¯ãƒªãƒƒãƒ—ã—ãŸè¨˜äº‹ã‚’Myãƒšãƒ¼ã‚¸ã‹ã‚‰èªã‚€ã“ã¨ãŒã§ãã¾ã™ ãƒ‹ãƒ¥ãƒ¼ãƒ¨ãƒ¼ã‚¯ã§10æœˆ2æ—¥ã€ã€ŒHadoop World:NYC 2009ã€ãŒé–‹å‚¬ã•ã‚ŒãŸã€‚ Hadoopã¨ã¯ã€Googleã®MapReduceã¨Google File Systemï¼ˆGFSï¼‰ã®ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹å®Ÿè£…ã€‚Yahooã‚„Amazonãªã©å„ç¨®ã®ã‚µãƒ¼ãƒ“ã‚¹ãƒ—ãƒãƒã‚¤ãƒ€ãƒ¼ã§åˆ©ç”¨ã•ã‚Œã¦ãŠã‚Šã€ãƒšã‚¿ãƒã‚¤ãƒˆä¸–ç•Œã«å¯¾å¿œã™ã‚‹ãŸã‚ã®ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ ã ã€‚ æ¤œç´¢ä»¥å¤–ã§ã®åˆ©ç”¨ã«å¤§ããªå¯èƒ½æ€§ã‚’ç§˜ã‚ã‚‹Hadoop æ¤œç´¢ã‚¨ãƒ³ã‚¸ãƒ³ã®ãŸã‚ã®ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ãŒã“ã‚Œã»ã©ã¾ã§ã«æ³¨ç›®ã•ã‚Œã‚‹ç†ç”±ã¯ã€ãã®å®ˆå‚™ç¯„å›²ã®åºƒãŒã‚Šã«å¤§ããªå¯èƒ½æ€§ãŒè¦‹å‡ºã•ã‚Œã¦ã„ã‚‹ã‹ã‚‰ã ã€‚ ãŸã¨ãˆã°ã€ã‚¦ã‚§ãƒ–ã‹ã‚‰é›†ã‚ã¦ããŸãƒ‡ãƒ¼ã‚¿ã‚’ã€æ¤œç´¢ã¨ã¯åˆ¥ã®æ–¹å‘æ€§ã§åˆ†æžã™ã‚‹ã“ã¨ãŒå¯èƒ½ã€‚ã¾ãŸã€å„ç¨®ãƒã‚°ãƒ‡ãƒ¼ã‚¿ã®è§£æžã€éºä¼åå·¥å¦ã«ãŠã‘ã‚‹ã‚²ãƒŽãƒ
mickn 2009/11/06
hadoop
ãƒªãƒ³ã‚¯
Facebook, Hadoop, and Hive | DBMSÂ 2 : DataBase Management System Services
I few weeks ago, I posted about a conversation I had with Jeff Hammerbacher of Cloudera, in which he discussed a Hadoop-based effort at Facebook he previously directed. Subsequently, Ashish Thusoo and Joydeep Sarma of Facebook contacted me to expand upon and in a couple of instances correct what Jeff had said. They also filled me in on Hive, a data-manipulation add-on to Hadoop that they developed
mickn 2009/05/12
hadoop

cloud
ãƒªãƒ³ã‚¯
Amazon Elastic MapReduceã‚’ä½¿ã£ã¦ã¿ãŸ - moratorium
Amazon Elastic MapReduceã‚’ä½¿ã£ã¦ã¿ãŸ 2009-04-03 (Fri) 3:06 Amazon EC2 é€£æ—¥ã®EC2ãƒã‚¿ã§ã™ã€‚æœ¬æ—¥ã€Amazonã‹ã‚‰Elastic MapReduceã¨ã„ã†ã‚µãƒ¼ãƒ“ã‚¹ãŒãƒªãƒªãƒ¼ã‚¹ã•ã‚Œã¾ã—ãŸã€‚å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿å‡¦ç†æŠ€è¡“ãŒä¸€æ°—ã«æ°‘é–“ã®æ‰‹ã«ä¸‹ã‚Šã¦ãã‚‹ã€ã¾ã•ã«é©å‘½çš„ãªã‚µãƒ¼ãƒ“ã‚¹ã ã¨æ€ã„ã¾ã™ã€‚ Amazon Elastic MapReduce Amazon ElasticMapReduce ç´¹ä»‹ãƒ“ãƒ‡ã‚ª With Hadoop, Amazon Adds A Web-Scale Data Processing Engine To Its Cloud Computer by techcrunch.com Elastic MapReduceã¯ã€Googleã®åŸºç›¤æŠ€è¡“ã®ä¸€ã¤ã§ã‚ã‚‹MapReduceã‚’æ™‚é–“å˜ä½èª²é‡‘ã§å®Ÿè¡Œã§ãã‚‹ã‚µãƒ¼ãƒ“ã‚¹ã§ã™ã€‚MapReduceã«ã¤ã„ã¦ã¯ä»¥
mickn 2009/04/03
hadoop
ãƒªãƒ³ã‚¯
ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ã®é–¢é€£ã‚¨ãƒ³ãƒˆãƒªãƒ¼æ©Ÿèƒ½é–‹ç™ºã€PFI ã•ã‚“ã¨ã®åˆå®¿ - naoyaã®ã¯ã¦ãªãƒ€ã‚¤ã‚¢ãƒªãƒ¼
ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ã«é–¢é€£ã‚¨ãƒ³ãƒˆãƒªãƒ¼ã‚’é…ä¿¡ã™ã‚‹æ©Ÿèƒ½ã‚’è¿½åŠ ã—ã¾ã—ãŸã€‚è©³ã—ãã¯ å‘ŠçŸ¥æ—¥è¨˜ã§ã€‚ ã“ã®é–¢é€£ã‚¨ãƒ³ãƒˆãƒªãƒ¼ã¯ã€æ ªå¼ä¼šç¤¾ãƒ—ãƒªãƒ•ã‚¡ãƒ¼ãƒ‰ã‚¤ãƒ³ãƒ•ãƒ©ã‚¹ãƒˆãƒ©ã‚¯ãƒãƒ£ãƒ¼ (ä»¥ä¸‹ PFI) ã®æŠ€è¡“è€…ã®ã¿ãªã•ã‚“ã¨ä¸€ç·’ã«é–‹ç™ºã—ã¾ã—ãŸã€‚é€±æœ«ã«2æ³Š3æ—¥ã§äº¬éƒ½ã§åˆå®¿ã‚’ã—ã¦ã‚³ã‚¢éƒ¨åˆ†ã‚’ä½œã‚Šã€ãã®å¾Œäº¬éƒ½ã¨æ±äº¬ã«åˆ†ã‹ã‚Œã¦ã‚ªãƒ³ãƒ©ã‚¤ãƒ³ã§é€£çµ¡ã‚’å–ã‚ŠãªãŒã‚‰2é€±é–“ã»ã©ä½œã‚Šè¾¼ã¿ã‚’ã—ã¦ã€ä»Šæ—¥ãƒªãƒªãƒ¼ã‚¹ã§ã™ã€‚ ã“ã®åˆå®¿ã§ã¯ä½•ãƒãƒ¼ãƒ ã‹ã«åˆ†ã‹ã‚Œã¦ã€ä»Šå›žã®é–¢é€£ã‚¨ãƒ³ãƒˆãƒªãƒ¼ã®æ©Ÿèƒ½ä»¥å¤–ã®é–‹ç™ºã‚‚è¡Œã£ã¦ã„ã¾ã™ã€‚ãã®è¾ºã®æˆæžœã¯ã¾ãŸå¾Œæ—¥ã«ãƒªãƒªãƒ¼ã‚¹ã§ãã‚‹ã®ã§ã¯ãªã„ã‹ã¨æ€ã„ã¾ã™ã€‚ ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ã®ä¸€ã¤ã®å•é¡Œã¨ã—ã¦ã€æ˜”ã®ã‚¨ãƒ³ãƒˆãƒªãƒ¼ãŒãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã«åŸ‹ã‚‚ã‚Œã¦ã—ã¾ã†ã¨ã„ã†ç‚¹ãŒã‚ã‚Šã¾ã—ãŸã€‚ãã®å•é¡Œã®è§£æ±ºç–ã¨ã—ã¦ã®é¡žä¼¼è¨˜äº‹æŠ½å‡ºã€ãã‚Œã‹ã‚‰æ¤œç´¢æ©Ÿèƒ½ã®å¼·åŒ–ã‚’ä»¥å‰ã‹ã‚‰è€ƒãˆã¦ã„ã¾ã—ãŸã€‚PFI ã®ãƒ¡ãƒ³ãƒãƒ¼ã®ã¿ãªã•ã‚“ã¯æƒ…å ±æ¤œç´¢æŠ€è¡“ã®ã‚¹ãƒšã‚·ãƒ£ãƒªã‚¹ãƒˆã§ã™ã€‚ã‚¢ã‚«ãƒ‡ãƒŸãƒƒã‚¯ãªç ”ç©¶ã®æˆæžœã‚’è£½å“åŒ–ã‚’é€š
mickn 2009/04/03
hadoop
ãƒªãƒ³ã‚¯
https://jp.techcrunch.com/2009/04/03/20090402with-hadoop-amazon-adds-a-web-scale-file-system-to-its-cloud-computer/
mickn 2009/04/03
cloud

hadoop
ãƒªãƒ³ã‚¯
1