Apache Twill – Home
This project has retired. For details please refer to its Attic page. What is Apache Twill? Apache Twill is an abstraction over Apache Hadoop® YARN that reduces the complexity of developing distributed applications, allowing developers to focus instead on their application logic. Apache Twill allows you to use YARN’s distributed capabilities with a programming model that is similar to running thre
Hadoop and Solid State Drives
Is there a story for the Hadoop Storage Stack (HDFS+HBase) on Solid State Drive (SSD)? This is a question that I have been asked by quite a few people in the last two days, mostly by people at the OpenComputeSummit. This piece discusses the possible use cases of using SSD with Hadoop or HBase. Use Case Currently, there are two primary use cases for HDFS: data warehousing using map-reduce and a key
Brown University - Data Management Group » A Comparison of Approaches to Large-Scale Data Analysis: MapReduce vs. DBMS Benchmarks
Overview The following information is meant to provide documentation on how others can recreate the benchmark trials used in our SIGMOD 2009 paper. Our experiments were conducted on a 100-node cluster at the University of Wisconsin-Madison; each node had a single 2.40 GHz Intel Core 2 Duo processor running 64-bit Red Hat Enterprise Linux 5 (kernel version 2.6.18) with 4GB RAM and two 250GB SATA-I
分散データベース「HBaseã€ã®å®‰å®šé‹ç”¨ã‚’目指ã—㦠- Preferred Networks Research & Development
1年経ã£ã¦iPhone4ã®é›»æ± ãŒãƒ˜ã‚¿ã£ã¦ããŸã€å¤ªç”°ã§ã™ã€‚ 指数関数的ã«ã‚¨ãƒ³ãƒˆãƒªæ•°ãŒå°‘ãªããªã£ã¦ããŸæœ¬ãƒ–ãƒã‚°ã§ã™ãŒwã€æ™¯æ°—付ã‘ã®ãŸã‚ã«ã‚¨ãƒ³ãƒˆãƒªã‚’投稿ã—ãŸã„ã¨æ€ã„ã¾ã™ï¼æœ¬æ—¥ã¯HBaseã«ã¤ã„ã¦ã§ã™ã€‚ Linux 㨠Hadoop 㨠HBase 㨠ZooKeeper ã«è©³ã—ã„ã‚ãªãŸï¼ã‚ãªãŸãŒã‚¿ãƒ¼ã‚²ãƒƒãƒˆã§ã™ã€‚ HBaseã¨ã¯? HBaseã¨ã¯ã€HDFS (Hadoop Distributed File System)上ã«æ§‹ç¯‰ã•ã‚ŒãŸåˆ†æ•£ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã§ã™ã€‚大é‡ã®éžå¸¸ã«ç´°ã‹ã„データをリアルタイムã«èªã¿æ›¸ã出æ¥ã‚‹ã®ãŒç‰¹å¾´ã§ã™ã€‚最近ã§ã¯Facebook Messageã®åŸºç›¤æŠ€è¡“ã¨ã—ã¦ä½¿ç”¨ã•ã‚ŒãŸäº‹ã§æ³¨ç›®ã‚’集ã‚ã¦ã„ã¾ã™ã€‚ HBaseå…¬å¼ã‚µã‚¤ãƒˆ Apache HBase ブック ä¿å˜ã•ã‚ŒãŸãƒ‡ãƒ¼ã‚¿ã¯HDFS上ã«ä¿å˜ã•ã‚Œã€HDFSã®ä»•çµ„ã¿ã«ã‚ˆã£ã¦ãƒ¬ãƒ—リケーションã•ã‚Œã‚‹ãŸã‚安全ã«ãƒ‡ãƒ¼ã‚¿ã‚’ä¿æŒã™ã‚‹ã“ã¨ãŒå‡ºæ¥ã¾ã™ã€‚ ã¾
NTTデータã®Hadoopå ±å‘Šæ›¸ãŒã™ã”ã‹ã£ãŸ - 科å¦ã¨éžç§‘å¦ã®è¿·å®®
æ¥ç•Œãƒˆãƒƒãƒ— ã®ã‚¨ãƒ³ã‚¿ãƒ¼ãƒ—ライズ Hadoop ä¼æ¥ Cloudera ã«å…¥ç¤¾ã—ã¾ã—㟠http://www.cloudera.co.jp/ 今年ã®6月ã«ã€ã€Œå¹³æˆï¼’１年度 産å¦é€£æºã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢å·¥å¦å®Ÿè·µäº‹æ¥å ±å‘Šæ›¸ã€ã¨ã„ã†ãƒ‰ã‚ュメント群ãŒçµŒç”£çœã‹ã‚‰å…¬è¡¨ã•ã‚Œã¾ã—ãŸã€‚ ãã®ã†ã¡ã®ä¸€ã¤ã«ã€NTTデータã«å§”託ã•ã‚ŒãŸHadoopã«é–¢ã™ã‚‹å®Ÿè¨¼å®Ÿé¨“ã®å ±å‘Šæ›¸ãŒã‚ã‚Šã¾ã—ãŸã®ã§ã€ä»Šæ›´ãªãŒã‚‰èªã‚“ã§ã¿ã‚‹ã“ã¨ã«ã—ã¾ã—ãŸã€‚ Hadoop界隈ã®äººã¯ã‚‚ã†ã¿ã‚“ãªã¨ã£ãã«èªã‚“ã§ã‚‹ã®ã‹ã‚‚ã—ã‚Œã¾ã›ã‚“ã‘ã©ã€‚ http://www.meti.go.jp/policy/mono_info_service/joho/downloadfiles/2010software_research/clou_dist_software.pdf ã€Œé«˜ä¿¡é ¼ã‚¯ãƒ©ã‚¦ãƒ‰å®Ÿç¾ç”¨ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢é–‹ç™ºï¼ˆåˆ†æ•£åˆ¶å¾¡å‡¦ç†æŠ€è¡“ç‰ã«ä¿‚ã‚‹ãƒ‡ãƒ¼ã‚¿ã‚»ãƒ³ã‚¿ãƒ¼é«˜ä¿¡é ¼åŒ–ã«å‘ã‘ãŸå®Ÿè¨¼äº‹æ¥ï¼‰ã€ã¨ã„ã†
HadoopDB Project
HadoopDB An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads. HadoopDB is: A hybrid of DBMS and MapReduce technologies that targets analytical workloads Designed to run on a shared-nothing cluster of commodity machines, or in the cloud An attempt to fill the gap in the market for a free and open source parallel DBMS Much more scalable than currently available parall
Data Applications and Infrastructure at LinkedIn__HadoopSummit2010
Hadoop Summit 2010 - application track Data Applications and Infrastructure at LinkedIn Jay Kreps, LinkedInRead less
10 Common Hadoop-able Problems Webinar
3. Today’s speaker - Jeff Hammerbacher • [email protected] • Studied Mathematics at Harvard • Worked as a Quant on Wall Street • Conceived, built, and led Data team at Facebook • Nearly 30 amazing engineers and data scientists • Several open source projects and research papers • Founder of Cloudera • Chief Scientist • Also, check out the book “Beautiful Data†Copyright 2010 Cloudera Inc. All rig
マイクãƒã‚½ãƒ•ãƒˆã®Hadoop対抗技術「Dryadã€ã€ã„よã„よ始動ã‹ï¼Ÿ
大è¦æ¨¡åˆ†æ•£å‡¦ç†ã®ãƒ•ãƒ¬ãƒ¼ãƒ ワークã¨ã—ã¦ã‚°ãƒ¼ã‚°ãƒ«ãŒé–‹ç™ºã—ãŸMapReduce処ç†ã‚„ã€ãã®ã‚ªãƒ¼ãƒ—ンソース実装ã§ã‚ã‚‹HadoopãŒæ€¥æˆé•·ã—ã€ãƒ“ジãƒã‚¹ã®åˆ†é‡Žã§ã®å•†æ¥åˆ©ç”¨ãŒç«‹ã¡ä¸ŠãŒã‚Šå§‹ã‚ã¦ã„ã‚‹ã“ã¨ã¯ã€Publickeyã§ã‚‚何度ã‹è¨˜äº‹ã§ç´¹ä»‹ã—ã¦ãã¾ã—ãŸã€‚ Hadoopを表計算ã®ã‚ˆã†ã«ä½¿ãˆã‚‹ã€ŒInfoSphere BigInsightsã€ã€IBMãŒç™ºè¡¨ グーグルã«ã‚ˆã‚‹MapReduceサービス「BigQueryã€ãŒç™»å ´ã€‚SQLライクãªå‘½ä»¤ã§å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿æ“作 Hadoopã¯ä¼æ¥ã®ãŸã‚ã®æ–°ãŸãªæƒ…å ±åˆ†æžãƒ—ラットフォームã¨ãªã‚‹ã€ã¨Cloudera グーグルãŒBigQueryã®é–‹å§‹ã‚’発表ã—ã€IBMも大è¦æ¨¡å‡¦ç†ã®ã‚¨ãƒ³ã‚¸ãƒ³ã¨ã—ã¦Hadoopを採用ã€Amazonクラウドã§ã‚‚Hadoop処ç†ã‚’è¡Œã†ã€ŒAmazon Elastic MapReduceã€ã‚µãƒ¼ãƒ“スをæä¾›ã—ã¦ã„ã‚‹ã“ã¨ã‹ã‚‰åˆ†ã‹ã‚‹ã‚ˆã†ã«ã€Hadoopã¯ã‚¯ãƒ©ã‚¦ãƒ‰ã§ã®å¤§è¦æ¨¡
Hadoop座談会ã«è¡Œã£ã¦ãã¾ã—㟠- nokunoã®æ—¥è¨˜
大谷ã•ã‚“：Hadoopã¨NoSQLã¨RDBMS Hadoopã®ç¾çŠ¶ï¼šã‚‚ã†ã™ãæ™®åŠæœŸ NoSQLã®ç¾çŠ¶ï¼šã¾ã 枯れã¦ãªã„ RDBMSã®ç¾çŠ¶ï¼šå¸‚å ´ãŒã‚·ãƒ¥ãƒªãƒ³ã‚¯ã—ã¦ã㟠Hadoopã®ç‰¹æ€§ï¼šå·¨å¤§ãƒ‡ãƒ¼ã‚¿ã®ãƒãƒƒãƒå‡¦ç†ã«å¼·ã„ NoSQLã®ç‰¹æ€§ï¼ˆCassandraより）：スケーラビリティ NoSQLã¨CAP：Cassandraã¯å¤šæ§˜æ€§ã€HBaseã¯ä¸€è²«æ€§é‡è¦– RDBMSã®ç‰¹æ€§ï¼šå„ªã‚ŒãŸãƒ‡ãƒ¼ã‚¿ä¸€è²«æ€§(ACID)ã¨ã‚¯ã‚¨ãƒªãƒ¼è¨€èªžã‚’も㤠CQRSパターン：更新系ã¨å‚照系ã¯æ˜Žç¢ºã«åˆ†ã‘ã¾ã—ょㆠ一貫性ä¿è·ã‚’アプリ開発者ã«ä»»ã›ã‚‹ã®ã¯ä¸å¯èƒ½ï¼šã‚³ãƒ³ã‚·ã‚¹ãƒ†ãƒ³ã‚·ãƒ¼ãƒ¬ãƒ™ãƒ«ã‚’アプリã‹ã‚‰ã‚‚らã†ã®ã¯å¿…é ˆ アーã‚テクãƒãƒ£å›³ï¼šHadoopã§Viewã®æ§‹ç¯‰ã‚’行ㆠãƒã‚¤ãƒ³ãƒˆï¼šä¸€è²«æ€§ã¯RDBMSã€é€Ÿåº¦é‡è¦–ã¯NoSQLã«ã€‚力技ã¯Hadoopã§ã€‚ 課題1：RDBMSã‹ã‚‰Hadoop：ãƒã‚¤ãƒ†ã‚£ãƒ–ã«æŒã£ã¦ãã‚‹ Hadoopã‹ã‚‰NoSQQL：HDFSã‹ã‚‰ãƒ€
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
s4.io - 
IBM Developer
Hadoop Summit 2010 - Agenda
UCI ISG Lecture Series on Scalable Data Management
{{#tags}}- {{label}}
{{/tags}}