Dremel made simple with Parquet
Columnar storage is a popular technique to optimize analytical workloads in parallel RDBMs. The performance and compression benefits for storing and processing large amounts of data are well documented in academic literature as well as several commercial analytical databases. The goal is to keep I/O to a minimum by reading from a disk only the data required for the query. Using Parquet at Twitter,
'An Inside Look at Google BigQuery'ã‚’èªã‚“ã - blue_field
Google BigQueryを使ã£ã¦ã¿ã‚ˆã†ã¨æ€ã£ã¦ã€æœ€è¿‘å°‘ã—勉強ã—ã¦ã„る。GoogleãŒãƒ›ãƒ¯ã‚¤ãƒˆãƒšãƒ¼ãƒ‘ーを出ã—ã¦ã„ãŸã®ã§ã€èªã‚“ã§ã¿ãŸã€‚（※2012å¹´ã®æ–‡çŒ®ï¼‰ BigQuery ã«ã¤ã„ã¦ã®ãƒ›ãƒ¯ã‚¤ãƒˆãƒšãƒ¼ãƒ‘ーを公開ã—ã¾ã—㟠- Google Developer Relations Japan Blog 以下ã€å†…容ã®ç°¡å˜ãªãƒ¡ãƒ¢ã€‚ ã‚‚ã¨ã‚‚ã¨Google社内ã§åˆ©ç”¨ã•ã‚Œã¦ã„㟠Google社内ã§åˆ©ç”¨ã•ã‚Œã¦ããŸ'Dremel'ã¨ã„ã†ã‚µãƒ¼ãƒ“スãŒã‚る。巨大ãªãƒ‡ãƒ¼ã‚¿ã«å¯¾ã—ã¦SQLライクãªã‚¯ã‚¨ãƒªã‚’実行ã™ã‚‹ã¨ã€æ•°ç§’ã§çµæžœãŒè¿”ã£ã¦ãる。Googleã§ã¯ã€ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã ã‘ã§ãªãアナリストãªã©éžã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã®äººã‚‚利用ã—ã¦ã„る。 DremelãŒãƒ™ãƒ¼ã‚¹ã¨ãªã‚Šã€å¤–部ã«å…¬é–‹ã•ã‚ŒãŸã®ãŒBig Query。フルマãƒãƒ¼ã‚¸ãƒ‰ãªã‚¯ãƒ©ã‚¦ãƒ‰ã‚µãƒ¼ãƒ“ス。サードパーティã®é–‹ç™ºè€…ã¯ã€REST APIã‚„CLI, Web UIãªã©ã‚’利用ã—ã¦ã“ã®ã‚µãƒ¼ãƒ“スã«ã‚¢
Developer's Summit 2013 å‚åŠ ãƒ¡ãƒ¢ï¼ˆï¼‘ï¼‰
★å—講ä¸ã«æ›¸ã„ã¦ã„ãŸãƒ¡ãƒ¢ã‚’ã€æŽ¨æ•²ç„¡ã—ã§ãã®ã¾ã¾ä¸Šã’ã¦ã„ã¾ã™ã€‚ ★誤å—脱å—ã€å†…容ã®æ¼ã‚Œãªã©ã‚ã‚‹ã‹ã¨æ€ã„ã¾ã™ãŒã”了承下ã•ã„。。 600億件を数å秒ã§æ¤œç´¢ã™ã‚‹ã‚¯ãƒ©ã‚¦ãƒ‰æ¤œç´¢ã‚¯ã‚¨ãƒªã‚µãƒ¼ãƒ“スBigQuery / ä½è—¤ä¸€æ†²æ°@google å°Žå…¥ BigQueryã®ãƒ—レゼン 自己紹介 @kazunori_279/#gaeja/#gcloudja クラウドソリューションãƒãƒ¼ãƒ ソリューションズアーã‚テクト appengine ja night管ç†äºº(23回ãらã„) AppEngine技術者ã®ãŸã‚ã®æƒ…å ±äº¤æ›ã‚¤ãƒ™ãƒ³ãƒˆ Agenda ビッグデータをGoogleスピード㧠「Googleスピードã€ã¯ç¤¾å†…用語ã€ã™ã”ã早ㄠデモ＆事例紹介 WhitePaper ãªãœæ—©ã„？ MapReduceã¨GoogleBigQueryã®é©æé©æ‰€ ビッグデータをGoogleスピード㧠Googleã§ã¯ã‚³ãƒ¼ãƒ‰ã‚’書ã時ã«æœ€åˆã«ã‚¹ã‚±ãƒ¼ãƒ©ãƒ“リ
ビッグデータをドリルã™ã‚‹
Hadoop関連ã®ã‚らゆる技術ãŒæ³¨ç›®ã‚’集ã‚ã‚‹ç¾åœ¨ã€Hadoopã®ã‚¨ã‚³ã‚·ã‚¹ãƒ†ãƒ ã®ä¸ã‹ã‚‰æ–°ã—ã„プãƒã‚¸ã‚§ã‚¯ãƒˆ/実装ãŒç”Ÿã¾ã‚Œç¶šã‘ã¦ã„ã‚‹ã®ã¯é©šãã¹ãã“ã¨ã§ã¯ãªã„。Apache Drillã¯å¤§è¦æ¨¡ãªãƒ‡ãƒ¼ã‚¿ã‚»ãƒƒãƒˆã‚’インタラクティブã«åˆ†æžã§ãる分散システムを開発ã™ã‚‹ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã§ã‚ã‚Šã€Googleã®Dremelã‹ã‚‰ç€æƒ³ã‚’å¾—ã¦ã„る。Hadoop MapReduceã®ã‚ˆã†ãªæ—¢å˜ã®ãƒ“ッグデータãƒãƒƒãƒå‡¦ç†ãƒ•ãƒ¬ãƒ¼ãƒ ワークやS4ã‚„Stormã®ã‚ˆã†ãªã‚¹ãƒˆãƒªãƒ¼ãƒ プãƒã‚»ã‚¹ãƒ•ãƒ¬ãƒ¼ãƒ ワークã®ä»£æ›¿ã§ã¯ãªãã€å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã®ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ã§ã‚¤ãƒ³ã‚¿ãƒ©ã‚¯ãƒ†ã‚£ãƒ–ãªåˆ†æžã‚’実ç¾ã™ã‚‹ã€ä»Šã¾ã§ãªã‹ã£ãŸè£½å“ã 。 Dremelã¨åŒã˜ã‚ˆã†ã«Drillã®å®Ÿè£…ã¯å…¥ã‚Œåã«ãªã£ã¦ã„ã‚‹ãƒ„ãƒªãƒ¼æ§‹é€ ã«ä¼¼ãŸãƒ‡ãƒ¼ã‚¿ã®å‡¦ç†ã‚’基本ã«ã—ã¦ã„る。Dremelã§ã¯ã€ãƒ‡ãƒ¼ã‚¿ã¯å…¥ã‚Œåã®ã‚¹ã‚ーマベースデータモデルã§ã‚るプãƒãƒˆã‚³ãƒ«ãƒãƒƒãƒ•ã‚¡ãŒåŸºæœ¬ã«ãªã£ã¦ã„る。Drillã¯ã“ã®ãƒ‡ãƒ¼ã‚¿ãƒ¢ãƒ‡ãƒ«ã‚’æ‹¡
IBM Cloudant
The data layer for hyperscale, resilient and globally available applications—based on open-source Apache CouchDB IBM® Cloudant® is a fully managed NoSQL database service that offers scalability capabilities to for developers and businesses. It allows users to effortlessly handle large volumes of data and high traffic loads with minimal maintenance. Built on CouchDB technology with compatible API a
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
Apache Drill Links
Dremel: Interactive Analysis of Web-Scale Datasets
Drill at the Chug 9-19-12
{{#tags}}- {{label}}
{{/tags}}