Engineering Data Analytics with Presto and Parquet at Uber
Data / MLEngineering Data Analytics with Presto and Apache Parquet at UberJuly 11, 2017 / Global From determining the most convenient rider pickup points to predicting the fastest routes, Uber uses data-driven analytics to create seamless trip experiences. Within engineering, analytics inform decision-making processes across the board. As we expand to new markets, the ability to accurately and qui
Hadoop Query Performance Smackdown
Are you using the fastest query tool for Hadoop? Provide and discuss the latest performance results of the industry standard TPC_H benchmarks executed across an assortment of open source query tools such as Hive (using MR, TEZ, LLAP, SPARK), SparkSQL, Presto, and Drill. Additionally, the performance tests will utilize a variety of data sizes and popular storage formats such as ORC, Parquet and Tex
Prestoã§å®Ÿç¾ã™ã‚‹ã‚¤ãƒ³ã‚¿ãƒ©ã‚¯ãƒ†ã‚£ãƒ–クエリ - dbtech showcase 2014 Tokyo
Treasure Dataã§ã¯Fluentdãªã©ã§åŽé›†ã—ãŸãƒ‡ãƒ¼ã‚¿ã«å¯¾ã—ã€Prestoã«ã‚ˆã‚‹ä½Žãƒ¬ã‚¤ãƒ†ãƒ³ã‚·ã‚¯ã‚¨ãƒªã‚µãƒ¼ãƒ“スをæä¾›ã—ã¦ã„ã¾ã™ã€‚ã“ã‚Œã«ã‚ˆã‚Šãƒ¦ãƒ¼ã‚¶ãƒ¼ã¯ã™ã°ã‚„ãデータã«é–¢ã™ã‚‹çŸ¥è¦‹ã‚’å¾—ã‚‹ã“ã¨ãŒã§ãã€ãƒ‡ãƒ¼ã‚¿åˆ†æžã®ç”Ÿç”£æ€§ã‚’å‘上ã§ãã¾ã™ã€‚ã“ã®ã‚¹ãƒ©ã‚¤ãƒ‰ã§ã¯åˆ†æ•£SQLエンジンã§ã‚ã‚‹Prestoã®ç‰¹å¾´ã¨ãã®å®Ÿè£…ã«ã¤ã„ã¦ç´¹ä»‹ã—ã¾ã™ã€‚ ã“ã®å†…容ã¯dbtech showcase 2014 Tokyo @秋葉原UDX ã§ç´¹ä»‹ã—ã¾ã—ãŸã€‚ http://www.insight-tec.com/dbts-tokyo-2014.htmlRead less
Python + Hive on AWS EMR ã§è²§è€…ã®ãƒã‚°ã‚µãƒžãƒª
1. Akira Chiku is an engineer who works on an engineering team. Their requirements include collecting between 10-20GB of data per day from various sources like Hadoop and Hive. 2. Data is collected from sources like Fluentd and parsed using Query String and stored in Hive. It is then processed and visualized. 3. Data can be stored in S3, processed using services like AWS EMR, and visualized in das
MPP on Hadoop, Redshift, BigQuery - Go ahead!
Twitterã§ã€Œæ—©ã今æµè¡Œã®MPPã®å¤§ã¾ã‹ãªä½¿ã„æ–¹ã®é•ã„書ã‘よï¼ã€ã¨ã„ã†ãƒ—レッシャーãŒåŠç«¯ãªã„ã®ã§ã¦ãã¨ã†ã«æ›¸ãã¾ã™ï¼Žã“ã®è¨˜äº‹ã¯ä¿ºã®çµŒé¨“ã¨å‹‰å¼·ä¼šãªã©ã§ãƒ¦ãƒ¼ã‚¶ã‹ã‚‰èžã„ãŸè©±ã‚’ã‚‚ã¨ã«æ›¸ã„ã¦ã„ã‚‹ã®ã§ï¼Œã™ã¹ã¦ãŒä¿ºã®çµŒé¨“ã§ã¯ã‚ã‚Šã¾ã›ã‚“(特ã«BigQuery)．å„社ã®SAã®äººã¨ã‹ã«èžã‘ã°ï¼Œã‚‚ã£ã¨è‰¯ã„アプãƒãƒ¼ãƒã¨ã‹è©³ç´°ã‚’æ•™ãˆã¦ãれるã‹ã‚‚ã—ã‚Œã¾ã›ã‚“． オンプレミスã®å•†ç”¨MPPã¯ä½¿ã£ãŸã“ã¨ãªã„ã®ã§ãƒŽãƒ¼ã‚³ãƒ¡ãƒ³ãƒˆã§ã™ï¼Ž MPP on Hadoopã§PrestoãŒãƒ¡ã‚¤ãƒ³ãªã®ã¯ä»Šä¸€ç•ªä½¿ã£ã¦ã„ã‚‹ã‹ã‚‰ã§ï¼ŒImpalaãªã©ä»–ã®MPP on Hadoopçš„ãªã‚‚ã®ã‚‚ä¼¼ãŸã‚ˆã†ãªæ„Ÿã˜ã‹ãªã¨æ€ã£ã¦ã„ã¾ã™ï¼Ž ã‚‚ã¡ã‚ん実装ã®é•ã„ãªã©ãŒã‚ã‚‹ã®ã§ï¼Œãã®è¾ºã¯é©å®œè‡ªåˆ†ã§è£œé–“ã—ã¦ãã ã•ã„． å‰æ アプリケーションを開発ã—ã¦ã„ã¦ï¼Œãã®ãŸã‚ã®è§£æžåŸºç›¤ã‚’一ã‹ã‚‰ä½œã‚‹ï¼Ž ç°¡å˜ãªã¾ã¨ã‚ データを貯ã‚る所ãŒä½œã‚Œã‚‹ã®ã§ã‚ã‚Œã°ï¼Œãã“ã«ç›´æŽ¥ã‚¯ã‚¨ãƒªã‚’投ã’られるPre
What are the main differences between Facebook, Presto, and Amplab Shark?
Answer (1 of 2): 1. Primary Use Case: While both are intended for analytics, Shark's primary use case is providing SQL to an (extremely fast) in-memory database, with support also for on-disk (or abstract) data sources. Presto is designed to be a fast SQL engine for the latter, and does not have ...
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
Dynamic filtering for highly-selective join optimization
GitHub - Netflix/iceberg: Iceberg is a table format for large, slow-moving tabular data
Prestoソースコードリーディング
{{#tags}}- {{label}}
{{/tags}}