Caching and Processing 2TB Mozilla Crash Reports in memory with Hazelcast - High Scalability -
« Sponsored Post: Gazillion, Edmunds, OPOWER, ClearStone, deviantART, ScaleOut, aiCache, WAPT, Karmasphere, Kabam, Newrelic, Cloudkick, Membase, Joyent, CloudSigma, ManageEngine, Site24x7 | Main | Stuff The Internet Says On Scalability For April 8, 2011 » Mozilla processes TB's of Firefox crash reports daily using HBase, Hadoop, Python and Thrift protocol. The project is called Socorro, a system f
BLOG::broomie.net: 多é‡åŒ–ã—ã¦Thriftを使ã£ã¦ã¿ãŸ
ã“ã“ã¾ã§ã®æµã‚Œ Thriftã®èª¿æŸ»ã®ç¶šãã§ã™ã€‚å‰å›žã®ã‚¨ãƒ³ãƒˆãƒªãƒ¼(Thriftã®ã‚¹ãƒ”ードãŒæ”¹å–„ã—ã¾ã—ãŸ)ã§ã¯ã€Thriftã®perlクライアントãŒé…ã„ã®ã¯NagleアルゴリズムãŒåŽŸå› ã§ã¯ãªã„ã‹ã¨ã„ã†ä»®èª¬ã‚’ãŸã¦ã€ãã—ã¦Nagleアルゴリズムをオフã«ã—ãŸçŠ¶æ…‹ã§é€Ÿåº¦ãŒå‘上ã—ãŸã“ã¨ã‚’書ãã¾ã—ãŸã€‚ ãã‚Œã«å¯¾ã—ã€tokuhiromã•ã‚“ã«ã‚ˆã‚ŠæŠ•ç¨¿ã•ã‚ŒãŸã‚¨ãƒ³ãƒˆãƒªãƒ¼(Thriftã¯Thrift::BufferedTransport ã‚’ã¤ã‹ã„ã‚ã™ã‚Œã‚‹ã¨ 147 å€é…ããªã£ã¦ã¤ã‹ã„ã‚‚ã®ã«ãªã‚‰ãªã„)ã§ã¯ã€send(2)ã®ãƒãƒƒãƒ•ã‚¡ãƒªãƒ³ã‚°ã‚’ã—ã¦ã„ãªã„ã“ã¨ãŒåŽŸå› ã§ã€ThriftãŒåã出ã™ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆã®perlクライアントã®ã‚¹ã‚±ãƒ«ãƒˆãƒ³ã§ã¯ä½¿ã‚ã‚Œã¦ã„ãªã„〠Thrift::BufferedTransport を使ã†ã“ã¨ã«ã‚ˆã£ã¦send(2)ãŒãƒãƒƒãƒ•ã‚¡ãƒªãƒ³ã‚°ã•ã‚Œã¦ã€é«˜é€Ÿã«ãªã‚‹ã“ã¨ã‚’示ã—ã¦ãã‚Œã¾ã—ãŸã€‚åŠ ãˆã¦kazuhookuã•ã‚“ã®
Rubyã‹ã‚‰HiveサーãƒçµŒç”±ã§Hiveを使ㆠ- brfrn169ã®æ—¥è¨˜
Hiveサーãƒã¯Thriftã§ã§ãã¦ã‚‹ã‚“ã§å½“然Rubyã‹ã‚‰ã‚‚使ãˆã¾ã™ã€‚ ãŒã€ã²ã©ãã¯ã¾ã£ãŸã‚“ã§ãƒ¡ãƒ¢ã—ã¨ãã¾ã™ã€‚ 環境ã¯ã€CentOS 5.5ã§ã™ã€‚ Thriftã®ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ« id:brfrn169:20101010:1286699822 Hiveサーãƒã®ç«‹ã¡ä¸Šã’æ–¹ãªã© id:brfrn169:20100312:1270530644 クライアントコードã®ç”Ÿæˆ ã¾ãšã€http://archive.cloudera.com/cdh/3/ ã‹ã‚‰ã‚½ãƒ¼ã‚¹ã‚’è½ã¨ã—ã¦ãã¾ã™ã€‚ è½ã¨ã—ã¦ããŸã‚‰ã€IDLã‹ã‚‰ã‚¯ãƒ©ã‚¤ã‚¢ãƒ³ãƒˆã‚³ãƒ¼ãƒ‰ã‚’生æˆã—ã¾ã™ã€‚ tar xvzf hive-0.5.0+20.tar.gz thrift -gen rb hive-0.5.0+20/src/service/include/thrift/fb303/if/fb303.thrift thrift -gen rb hive-0.5.0+20
The Cult of Gary » Blog Archive » Making PHP talk to Hive through Thrift
Hive is a data warehouse system build ontop of Hadoop. I’ve been experimenting with it for the past few days. Using the thrift service, I’ve been able to drive it from PHP. Here’s what I’ve done to get it going: Launching a Cluster Using the EC2 scripts, I launched a cluster of Hadoop servers on EC2. It’s straight forward to get up and running. It takes me about 5 minutes to get a cluster going, i
Erlang, Thrift and HBase | (defun side of kungfooguru)
Since documentation is lacking for Thrift and HBase (and Erlang), though the examples in Thrift’s code are pretty good, I figured as I got things going I’d post snippets. I assume you have Erlang installed, but I will suggest checking out Erlware for setting up Erlang and installing releases and applications. First, download Thrift and HBase. The Thrift instructions are below. However, note you ne
ãƒã‚¿ã•ã‚“ã®ãƒ–ãƒã‚°(復刻版) : HDFS ã® HadoopThriftServer ã‚’ãªã‚“ã¨ã‹ã™ã‚‹
hadoop ã®è©±é¡Œã€‚ãã®2 hadoop を支ãˆã‚‹ HDFS ã«ã¯ HDFS-APIを通ã™ã“ã¨ã§ã€ãƒ—ãƒã‚°ãƒ©ãƒ ä¸ã‹ã‚‰ HDFS ã®èªã¿æ›¸ããŒå‡ºãるよã†ã«ãªã‚Šã¾ã™ã€‚(ãŸã¶ã‚“ã€hdfs-s3 ãªã‚“ã‹ã‚‚ã“ã®API経由(? ソースèªã‚“ã§ãªã„)) (ä¸ç•¥) ã‚“ã§ã€ã“ã® HDFS-API ã®ãªã‹ã«ã€Thrift を使ã£ã¦ リモート上ã‹ã‚‰ HDFS ã®èªã¿æ›¸ãã‚’ã§ãるよã†ã«ã—ã¦ã„ã‚‹ HadoopThriftServer(theiftfs) ãŒã‚ã‚Šã¾ã™ã€‚ ã“ã® thriftfs ã®èµ·å‹•ã¯ ã«æ›¸ã‹ã‚Œã¦ã„ã‚‹ã®ã§ã™ãŒã€shellã‚’æ¡ã£ã¦ã—ã¾ã†ã®ã§ã“ã‚“ãªæ„Ÿã˜ã«ã—ã¾ã—ãŸã€‚ #!/usr/bin/env bash THRIFTFS_PID_FILE=$HADOOP_PID_DIR/thrift.pid THRIFTFS_LOG_FILE=$HADOOP_LOG_DIR/thrift.log if [ -f $THR
Apache Thrift - Home
The Apache Thrift software framework, for scalable cross-language services development, combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, OCaml and Delphi and other languages. Getting Started Download Apache Thrift To get started, down
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
{{#tags}}- {{label}}
{{/tags}}