Spark at Twitter - Seattle Spark Meetup, April 2014Evaluation of Spark (on YARN) at Twitter, and lessons learnt from our ongoing experiments. Read less
Pig TPC-H Benchmark and Performance Tuning
For a class project we developed a whole set of Pig scripts for TPC-H. Our goals are: 1) identifying the bottlenecks of Pig's performance especially of its relational operators, 2) studying how to write efficient scripts by making full use of Pig Latin's features, 3) comparing with Hive's TPC-H results for verifying both 1) and 2)Read less
Flexible In-Situ Indexing for Hadoop via Elephant Twin
This document discusses flexible indexing in Hadoop. It describes how Twitter uses Elephant-Twin, an open source library they developed, to create indexes at the block level or record level in Hadoop. Elephant-Twin allows minimal changes to jobs/scripts, indexes data without copying it, supports post-factum indexing, and indexes can be used to efficiently retrieve relevant data through an IndexedI
Pig and Hive at Yahoo! · Yahoo! Hadoop Blog
Yahoo! has begun evaluating Hive for use as part of its Hadoop stack. Since, in many peoples' minds, Hive and Pig are roughly equivalent and Pig Latin is very close to SQL, this has led to some confusion. Why are we interested in using both technologies? As we have looked at our workloads and analyzed our use cases, we have come to the conclusion that the different use cases require different tool
Pig | Yahoo! Research
We are creating infrastructure to support ad-hoc analysis of very large data sets. Parallel processing is the name of the game. Our system runs on a cluster computing architecture, on top of which sit several layers of abstraction that ultimately bring the power of parallel computing into the hands of ordinary users. The layers in between automatically translate user queries into efficient paralle
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
処ç†ã‚’実行ä¸ã§ã™
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
Pig Latin: A Not-So-Foreign Language for Data Processing Christopher Olston ∗ Yahoo! Research Benjamin Reed †Yahoo! Research Utkarsh Srivastava ‡ Yahoo! Research Ravi Kumar § Yahoo! Research Andrew Tomkins ¶ Yahoo! Research ABSTRACT There is a g
{{#tags}}- {{label}}
{{/tags}}