Powering Flickr’s Magic view by fusing bulk and real-time compute | code.flickr.com
Hbaseã¨HadoopMR - 急ãŒã°å›žã‚Œã€é¸ã¶ãªã‚‰è¿‘é“
Hbase勉強会ã®ã¾ã¨ã‚ã®å»¶é•·ã¨ã—㦠今後ã®è€ƒãˆæ–¹ã‚’ã¾ã¨ã‚ã¦ãŠãã¾ã™ã€‚ ã¾ãšã¯å‰æã¨ã—㦠<一般論> Hbaseã«ã‹ãŽã‚‰ãšã€NoSQL系一般ã«è¨€ãˆã‚‹ã“ã¨ã§ã¯ã‚る㌠Usecaseã‚’æ„è˜ã—ã¦åˆ©ç”¨ã™ã‚‹äº‹ãŒå¿…è¦ã ã€ã¨ã„ã†ã“ã¨ã ã¨æ€ã†ã€‚ 最近ã®å‚¾å‘ã¨ã—ã¦ã¯ã€Googleã§ã‚‚é¡•è‘—ã ã‘ã©ã€ 一定ã®ç”¨é€”をターゲットã«ã—㦠特定ã®ãƒŸãƒ‰ãƒ«ã‚’開発ã™ã‚‹ã¨ã„ã†æ–¹æ³•ãŒçµæ§‹å¤šã„。 Hbaseã‚‚ãã®æµã‚Œã¯ã‚ã‚‹ã®ã§ã€ ãã®ã‚ãŸã‚Šã¯æ„è˜ã™ã‚‹å¿…è¦ã¯ã‚ã‚‹ã‹ã‚‚ã—ã‚Œãªã„。 Hbaseã¤ã„ã¦ã¯ã€æ³¨ç›®ã™ã‚‹ã¨ã™ã‚Œã°Facebookã«ãªã‚‹ã‹ãªã€‚ http://www.cloudera.com/resource/hw10_hbase_in_production_at_facebook ã„ãšã‚Œã«ã—ã¦ã‚‚ã€å‰²ã¨ã†ã¾ãã„ã£ã¦ã„ã‚‹Usecaseã®æƒ…å ±ã®æœ‰ç”¨æ€§ã¯ ä»–ã®æŠ€è¡“よりも高ã„ã¨æ€ã†ã€‚ 基本的ã«å˜ç´”ã«åˆ†æ•£KVSを使ã„ãŸã„ãªã‚‰Hbaseã«ã“ã ã‚ã‚‹å¿…è¦
Facebookã®æ–°ã—ã„リアルタイム解æžã‚·ã‚¹ãƒ†ãƒ ã¨ã¯ï¼Ÿ - nokunoã®æ—¥è¨˜
Facebookã®æ–°ã—ã„リアルタイム解æžã®ã‚·ã‚¹ãƒ†ãƒ ã§ã¯ã€HBaseã§1æ—¥200億件ã®ã‚¤ãƒ™ãƒ³ãƒˆã‚’処ç†ã—ã¦ã„ã‚‹ãã†ã§ã™ã€‚以下ã®è¨˜äº‹ã®ç¿»è¨³ã§ã™ã€‚High Scalability - High Scalability - Facebook’s New Realtime Analytics System: HBase to Process 20 Billion Events Per DayFacebookãŒã¾ãŸã‚„ã£ã¦ãã‚ŒãŸã€‚彼らã¯å·¨å¤§ãªãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ データã®ã‚¹ãƒˆãƒªãƒ¼ãƒ を処ç†ã™ã‚‹ã‚‚ã†1ã¤ã®ã‚·ã‚¹ãƒ†ãƒ を構築ã—ãŸã®ã 。以å‰ã«ã‚‚Facebookã¯ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ãªãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã‚·ã‚¹ãƒ†ãƒ ã‚’HBaseã§æ§‹ç¯‰ã—ã¦ã„ã‚‹(http://highscalability.com/blog/2010/11/16/facebooks-new-real-time-messaging-system-hbase-to-store-135.ht
第1回HBaseã¨Cassandraã®è¨Žè«–会ã®ãƒ¡ãƒ¢ - ã²ã—ã ã¾ã®å¤‰æ›´å±¥æ´
ã²ã—ã ã¾ï¼¨ï¼°ã®æ›´æ–°å±¥æ´ã€‚ 主ã«ï¼´ï¼²ï¼°ï¼§ãƒªãƒ—レイã®å…ƒãƒã‚¿é›†ã€ãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°æŠ€è¡“メモã¨è‡ªä½œã‚½ãƒ•ãƒˆã€å¥½ããªã‚²ãƒ¼ãƒ や音楽ã§ã™ã€‚ HBaseã¨Cassandraã®è¨Žè«–会　第1回（Togetter：第1回HBaseã¨Cassandra討論会）ã«å‚åŠ ã—ãŸã®ã§ã€ãã®ãƒ¡ãƒ¢ã§ã™ã€‚ …例ã«ã‚ˆã£ã¦ã€èžã逃ã—・ç†è§£ä¸è¶³ãƒ»èª¤è§£èª¤èªã¯ã‚ã‚‹ã¨ã¯æ€ã„ã¾ã™ãŒ(æ±—) 最åˆã«kimteaã•ã‚“ã®2010-02-15ã®åˆ†æ•£ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã®ãƒšãƒ¼ã‚¸ã‚’見ãªãŒã‚‰è©±ã‚’èžã。 第1世代…Google BigTable 第2世代…Amazon Dynamo 第3世代…Microsoft SQL Azure（→リレーショナルモデルã«å¤‰æ›´ã•ã‚ŒãŸã€‚ä¸ã§ä½¿ã‚ã‚Œã¦ã„ã‚‹Azure Tableã¯KVS） 第4世代…Cassandra KVSã¯ã‚ªãƒ–ジェクト指å‘ã§ã€æ¥å‹™ã‚·ã‚¹ãƒ†ãƒ を作るã®ãŒé›£ã—ã„。（◆â—オブジェクト指å‘ã¨ã„ã†ç†ç”±ã§é›£ã—ã„ã®ï¼Ÿï¼‰ GAE/Jã®Slim3ã¯
Cloudera Blog
We are excited to announce the acquisition of Octopai, a leading data lineage and catalog platform that provides data discovery and governance for enterprises to enhance their data-driven decision making. Cloudera’s mission since its inception has been to empower organizations to transform all their data to deliver trusted, valuable, and predictive insights. With AI and […] Read blog post
Facebook on Hadoop, Hive, HBase, and A/B Testing
Beat the Plan: Probabilistic Strategies for Successful Software Delivery at Scale Large-scale software delivery demands managing complexity across teams and organizations. Similarly to betting strategies in Vegas, embracing probabilistic thinking helps tackle uncertainty, shifting from rigid plans to adaptive systems. By making informed bets and designing for change, leaders can control volatility
Pig, Cascalog & HBase Among Highlights of May Hadoop Meet-Up (Hadoop and Distributed Computing at Yahoo!)
YDN Hadoop and Distributed Computing at Yahoo! Pig, Cascalog & HBase Among Highlights of May Hadoop Meet-Up Hi Hadoopers Thanks to close to 300 developers who came this week to Yahoo! for our monthly Hadoop User Group meeting. The energy in the packed room was phenomenal and conversations continued long after the formal sessions. Hundreds of Hadoop Fans Flock to Yahoo! for the May Hadoop User Grou
Riak and Cassandra and HBase, oh my! « Blog of Data
We are marching along in our integration of HBase with the Socorro Crash Stats project, but I wanted to take a minute away from that to talk about a separate project the Metrics team has also been involved with. Mozilla Labs Test Pilot is a project to experiment and analyze data from real world Firefox users to discover quantifiable ways to improve our user experience. I was very interested and e
1 Billion Reasons Why Adobe Chose HBase - High Scalability -
Cosmin Lehene wrote two excellent articles on Adobe's experiences with HBase: Why we’re using HBase: Part 1 and Why we’re using HBase: Part 2. Adobe needed a generic, real-time, structured data storage and processing system that could handle any data volume, with access times under 50ms, with no downtime and no data loss. The article goes into great detail about their experiences with HBase and t
From Shared-All to Shared-Nothing(PDF)
-Patterns From Shared-All to Shared-Nothing Successfully used Patterns in application and table design with Hbase Bob Schulze, eCircle AG March 2010 @ Berlin Apache Hadoop Get Together -Patterns Audience âž² You have Big Data âž² Your Organization needs predictable scaling options âž² You need to be flexible with your Data âž² You are a Techie Person -Patterns Content âž² What is shared? âž² Recap RDBMS vs HB
Pig on Hadoop - kuangueã®æ—¥è¨˜
Pigã£ã¦ã®ã¯ï¼Œgoogleã§è¨€ã†ã¨ã“ã‚ã®sawzallã«å¯¾å¿œã™ã‚‹ã‚ˆã†ã§ã™ï¼ŽãŒï¼Œã¡ã‚‡ã£ã¨è¦‹ãŸã¨ã“ã‚ã§ã¯ï¼ŒSawzallã©ã“ã‚ã§ã¯ãªãã¦ï¼Œã‚‚ã£ã¨æ„欲的ã§ã™ï¼ŽSawzallã¯ï¼ŒMapReduce処ç†ãƒ¢ãƒ‡ãƒ«ã«æ€ã„切り引ã£å¼µã‚‰ã‚Œã¦ã„ã‚‹ã‘ã©ï¼ŒPigã¯ï¼Œãƒªãƒ¬ãƒ¼ã‚·ãƒ§ãƒŠãƒ«æ¼”ç®—ã‚’Hadoop::MapReduce上ã®å‡¦ç†ã«å¤‰æ›ã—よã†ã¨ã„ã†å‰²ã¨å£®å¤§ãªè©¦ã¿ï¼ŽHadoopã¯åˆ©ç”¨ã—ã¦ã„ã‚‹ã‘ã©ã‚‚，完全ã«åˆ¥ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã§ã‚„ã£ã¦ã„ã¾ã™ï¼Žyahooã§ä½œã‚‰ã‚Œã¦ã„ãŸã‚‚ã®ã‚’オープンソースã«ã—ã¾ã—ãŸã¨ã„ã†ã“ã¨ã§ã™ã． ãŸã¨ãˆã°ï¼Œä¸‹ã®ã‚ˆã†ã«æ›¸ãã“ã¨ãŒã§ãるよã†ãªè¨€èªžã«ãªã£ã¦ã„ã¾ã™ï¼Ž VISITS = load '/visits' as (user, url, time); USER_VISITS = group VISITS by user; USER_COUNTS = foreach USER_VISITS generate gr
1
ランã‚ング
ランã‚ング
ランã‚ング
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
NoSQL Week in Review 25
Good night, Posterous
{{#tags}}- {{label}}
{{/tags}}