[B! hive] mogwaingã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

mogwaing id:mogwaing

hiveã«é–¢ã™ã‚‹mogwaingã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (53)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

ç¬¬10å›žã€€ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã®æœ€é©åŒ– | gihyo.jp
ã¯ã˜ã‚ã« å‰å›žã¯ã€ã“ã‚Œã¾ã§èª¬æ˜Žã—ã¦ããŸã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã®æ€§èƒ½ã‚’å®šé‡çš„ã«è¦‹ç©ã‚Šã€æ¯”è¼ƒã—ã¾ã—ãŸã€‚ä»Šå›žã¯ã€ã“ã‚Œã‚‰ã®æ€§èƒ½è¦‹ç©ã‚Šã‚’ç”¨ã„ã¦è¡Œã†ãƒ‡ãƒ¼ã‚¿å‡¦ç†ï¼ˆå•ã„åˆã‚ã›ï¼‰ã®æœ€é©åŒ–æ–¹æ³•ã«ã¤ã„ã¦èª¬æ˜Žã—ã¾ã™ã€‚ ãƒ‡ãƒ¼ã‚¿å‡¦ç†ï¼ˆå•ã„åˆã‚ã›ï¼‰ã®æœ€é©åŒ– ç¬¬4å›žã§è¿°ã¹ãŸã‚ˆã†ã«ã€Hadoopã®SQLå‡¦ç†ç³»ã§ã‚ã‚‹Hiveã‚’ã¯ã˜ã‚ã¨ã—ã€Impalaã‚„Prestoãªã©ã®å®£è¨€åž‹è¨€èªžã‚’ç”¨ã„ã‚‹ãƒ‡ãƒ¼ã‚¿å‡¦ç†ç³»ã«ãŠã„ã¦ã¯ã€åˆ©ç”¨è€…ã¯ä½•ã‚’ï¼ˆWhatï¼‰å‡¦ç†ã—ã¦ã»ã—ã„ã‹ã‚’å‡¦ç†ç³»ã«æŒ‡ç¤ºã™ã‚‹ã®ã¿ã§ã‚ã‚Šã€ã©ã®ã‚ˆã†ã«ï¼ˆHowï¼‰å‡¦ç†ã‚’ã—ã¦ã»ã—ã„ã‹ã¯æŒ‡å®šã—ã¾ã›ã‚“ã€‚ã™ãªã‚ã¡ã€å½“è©²å‡¦ç†ç³»ã«ãŠã„ã¦ã¯ã€ã©ã®ã‚ˆã†ã«å‡¦ç†ã‚’ã™ã‚‹ã‹ã¯å‡¦ç†ç³»è‡ªä½“ãŒæ±ºã‚ã‚‹å¿…è¦ãŒã‚ã‚Šã€ä¸Žãˆã‚‰ã‚ŒãŸå•ã„åˆã‚ã›ï¼ˆã‚¯ã‚¨ãƒªï¼‰ã‚’æœ€ã‚‚è‰¯ã„ã¨æ€ã‚ã‚Œã‚‹æ–¹æ³•ã§å‡¦ç†ã—ã¾ã™ã€‚ã“ã®ã‚ˆã†ã«ã€å•ã„åˆã‚ã›ã«ãŠã„ã¦æœ€è‰¯ã¨æ€ã‚ã‚Œã‚‹ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã®æ–¹æ³•ã‚’è¦‹ã¤ã‘ã‚‹ã“ã¨ã‚’ã€Œâ ï¼ˆâ å•ã„åˆã‚ã›ï¼‰æœ€é©åŒ–ã€ã¨å‘¼ã³ã¾ã™ã€‚ æœ€é©åŒ–ã«ãŠã„ã¦ã¯ã€å•ã„åˆã‚ã›ã‚’å®Ÿè¡Œã™
mogwaing 2015/09/16
hadoop

parallel db

parallel processing

distributed system

query optimizer

impala

hive
ãƒªãƒ³ã‚¯
SQLæ„Ÿè¦šã§HiveQLã‚’æ›¸ãã¨ç—›ã„ç›®ã«ã‚ã†ä¾‹ - still deeper
Hadoop Advent Calendar 2013 4æ—¥ç›®ã®è¨˜äº‹ã§ã™ tl;dr explainã¨job historyã‚’èªã‚ 1 reducerã¯æ‚ª data skewã¯æ‚ª å‰æ›¸ã ã¿ã‚“ãªå¤§å¥½ãSQLã§Hadoopä¸Šã§ã®å‡¦ç†ã‚’å®Ÿè¡Œã§ãã‚‹Hiveã«ã¯ã¿ãªã•ã‚“æ™®æ®µã‹ã‚‰ãŠä¸–è©±ã«ãªã£ã¦ã„ã‚‹ã“ã¨ã§ã—ã‚‡ã†ã€‚ã¡ã‚‡ã£ã¨èª¿ã¹ç‰©ã§ã‚°ã‚°ã‚‹åº¦ã«ç›®ã«å…¥ã‚‹æ„›ã‚‰ã„ã—ã„ãƒžã‚¹ã‚³ãƒƒãƒˆãŒã€è’ã‚“ã å¿ƒã«æ¸…æ¶¼ãªé¢¨ã‚’ã¯ã“ã‚“ã§ãã‚Œã¾ã™ã€‚ ã§ã™ãŒHiveã®ã‚¯ã‚¨ãƒªè¨€èªžã¯SQLã§ã¯ãªãHiveQLã§ã™ã—ã€å®Ÿè¡Œã‚¨ãƒ³ã‚¸ãƒ³ã‚‚RDBã®ãã‚Œã¨ã¯å…¨ãç•°ãªã‚‹MapReduceã§ã™ã€‚SQLã®ã¤ã‚‚ã‚Šã§HiveQLã‚’æ›¸ã„ã¦ã„ã‚‹ã¨åœ°é›·ã‚’è¸ã‚“ã§ã—ã¾ã†ã“ã¨ãŒã¾ã‚Œã«ã‚ˆãã‚ã‚Šã¾ã™ã€‚æœ¬ã‚¨ãƒ³ãƒˆãƒªã§ã¯é™¥ã‚ŠãŒã¡ãªHiveQLã®è½ã¨ã—ç©´ã‚’2ã¤ç´¹ä»‹ã—ã¾ã™ã€‚ ä¾‹1 SELECT count(DISTINCT user_id) FROM access_log SQLã«æ…£ã‚ŒãŸæ–¹ã§ã‚ã‚Œ
mogwaing 2014/08/15
hadoop

hive

tips
ãƒªãƒ³ã‚¯
A perfect Hive query for a perfect meeting (Hadoop Summit 2014)
A perfect Hive query for a perfect meeting (Hadoop Summit 2014) During one of our epic parties, Martin Lorentzon (chairman of Spotify) agreed to help me to arrange a dinner for me and Timbuktu (my favourite Swedish rap and reggae artist), if I prove somehow that I am the biggest fan of Timbuktu in my home country. Because at Spotify we attack all probl ems using data-driven approaches, I decided to
mogwaing 2014/06/10
hive

hadoop
ãƒªãƒ³ã‚¯
Blog | Cloudera
For the inaugural episode of Women Leaders in Techno logy on The AI Forecast, we welcomed Kari Briski â€“ Vice President AI Software Product Management at NVIDIA. Kari shared the stories and strategies that inform her leadership style (like GSD or â€œgetting stuff doneâ€), what it means to trust your instinct, and the advice she gives to young women embarking on a career in techno logy and to women furth
mogwaing 2014/04/22
hive

tez

stinger

hadoop
ãƒªãƒ³ã‚¯
Blog | Cloudera
For the inaugural episode of Women Leaders in Techno logy on The AI Forecast, we welcomed Kari Briski â€“ Vice President AI Software Product Management at NVIDIA. Kari shared the stories and strategies that inform her leadership style (like GSD or â€œgetting stuff doneâ€), what it means to trust your instinct, and the advice she gives to young women embarking on a career in techno logy and to women furth
mogwaing 2013/10/14
hive

database

acid
ãƒªãƒ³ã‚¯
Hadoop vs. Aster: A Quick Comparison | Gilt Tech
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.
mogwaing 2013/10/14
aster data

database

hadoop

hive
ãƒªãƒ³ã‚¯
Cloudera Blog
Enterprise IT leaders across industries are tasked with preparing their organizations for the techno logies of the future â€“ which is no simple task. With the use of AI exploding, Cloudera, in partnership with Researchscape, surveyed 600 IT leaders who work at companies with over 1,000 employees in the U.S., EMEA and APAC regions. The survey, [â€¦] Read blog post
mogwaing 2013/02/21
hive

hadoop
ãƒªãƒ³ã‚¯
GitHub - intel-hadoop/hive-0.9-panthera: The repository of this project is moved to https://github.com/intel-hadoop/project-panthera-ase
Dismiss Join GitHub today GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together. Sign up
mogwaing 2012/12/19
It uses an open source SQL parser (https://github.com/porcelli/plsql-parser) to generate AST for the SQL query, then transforms the SQL AST to MapReduce jobs through a series of context-aware analyses and optimizations

hadoop

hive

sql
ãƒªãƒ³ã‚¯
Hive Installation - Cloudera Support
Note To see which version of Hive is shipping in the latest CDH3 release, check the Version and Packaging Information. For important information on new and changed components, see the Release Notes. If you used Hive in versions prior to CDH3 Beta 4, read this section. CDH3 includes changes in the Hive MetaStore schema that are part of the Hive 0.7 release. If you want to use Hive in CDH3, it is im
mogwaing 2012/08/29
hive metastore

hive
ãƒªãƒ³ã‚¯
Cloudera Blog
Enterprise IT leaders across industries are tasked with preparing their organizations for the techno logies of the future â€“ which is no simple task. With the use of AI exploding, Cloudera, in partnership with Researchscape, surveyed 600 IT leaders who work at companies with over 1,000 employees in the U.S., EMEA and APAC regions. The survey, [â€¦] Read blog post
mogwaing 2012/08/29
hive

statistics

query optimizer
ãƒªãƒ³ã‚¯
Hiveã®çµ±è¨ˆæƒ…å ± - wyukawa's diary
Hive 0.7ã‹ã‚‰analyzeã‚³ãƒžãƒ³ãƒ‰ã§çµ±è¨ˆæƒ…å ±ãŒå–ã‚Œã‚‹ã‚ˆã†ã«ãªã‚Šã¾ã—ãŸã€‚ [HIVE-1361] table/partition level statistics - ASF JIRA ãƒ¡ã‚¿ã‚¹ãƒˆã‚¢ã‚’Derbyä»¥å¤–ã«ã—ã¦ã‚‹ã®ã«derby.logã¨ã‹TempStatsStoreã¨ã‹ã‚ã£ã¦ä½•ã ã‚ã†ã¨æ€ã£ãŸã‚‰çµ±è¨ˆæƒ…å ±ã‚’ä¿å˜ã—ã¦ã„ã‚‹ã‚ˆã†ã§ã™ã€‚ ãƒ‰ã‚ãƒ¥ãƒ¡ãƒ³ãƒˆã‚‚ã‚ã‚Šã¾ã—ãŸã€‚ã¦ã„ã†ã‹ã“ã®ãƒšãƒ¼ã‚¸ã£ã¦ãƒˆãƒƒãƒ—ã‹ã‚‰ãƒªãƒ³ã‚¯ã•ã‚Œã¦ãªã„ã‚ˆã†ãªã€‚ã€‚ã€‚ StatsDev - Apache Hive - Apache Software Foundation çµ±è¨ˆæƒ…å ±ã®ä¿å˜å…ˆã¯Derbyä»¥å¤–ã«ã¯MySQLã¨HBaseãŒä½¿ãˆã‚‹ã‚ˆã†ã§ã™ã€‚HBaseã®æ„å‘³ãŒã‚ˆãã‚ã‹ã‚‰ãªã„ã§ã™ãŒã€Hiveã¨Hbaseã‚’çµ±åˆã—ãŸå ´åˆã®è©±ã‹ãªã€‚ Hiveã¯ã„ã¾ã¾ã§ã¯ãƒ«ãƒ¼ãƒ«ãƒ™ãƒ¼ã‚¹ã‚ªãƒ—ãƒ†ã‚£ãƒžã‚¤ã‚¶ã ã£ãŸã®ã§ã™ãŒã€ä»Šå¾Œã¯ã‚³ã‚¹ãƒˆãƒ™ãƒ¼ã‚¹ã‚ªãƒ—ãƒ†ã‚£ãƒžã‚¤ã‚¶å¯¾å¿œã‚‚é€²ã‚
mogwaing 2012/08/28
statistics

hive
ãƒªãƒ³ã‚¯
[HIVE-1938] Cost Based Query optimization for Joins in Hive - ASF JIRA
mogwaing 2012/08/28
hive

query optimizer
ãƒªãƒ³ã‚¯
[HIVE-3027] The optimizer architecture of Hive is terrible, need code refactoring - ASF JIRA
Now I want to add a complete cost-based optimization for hive. but when I begin the work, I found it very difficult to do using current hive optimization framework. The current code of hive, optimizations are all done after generating DAG of operators. It is a awful design and makes me mad. For example, the map-side optimization, it scans the whole operators' DAG and try to find the operators that
mogwaing 2012/08/28
hive

query optimizer
ãƒªãƒ³ã‚¯
mixi ã®è§£æžåŸºç›¤ã¨Apache Hive ã§ã® JSON ãƒ‘ãƒ¼ã‚µã®æ´»ç”¨ã®ç´¹ä»‹ - mixi engineer blog
ã“ã‚“ã«ã¡ã¯ï¼Žæœ€è¿‘ãƒ”ã‚¯ãƒ«ã‚¹ä½œã‚Šã§ç²¾ç¥žçµ±ä¸€ã‚’ã—ã¦ã„ã‚‹ï¼ŒãŸã‚“ã½ã½ã‚°ãƒ«ãƒ¼ãƒ—è§£æžãƒãƒ¼ãƒ ã®çŸ³å·æœ‰ã§ã™ï¼Ž ã“ã®ãƒ–ãƒã‚°ã§ã¯ãŠé¦´æŸ“ã¿ã®ãŸã‚“ã½ã½ã‚°ãƒ«ãƒ¼ãƒ—ã§ã™ãŒï¼Œ"No More ã€Œåˆºèº«ã®ä¸Šã«ã‚¿ãƒ³ãƒãƒã‚’ã®ã›ã‚‹ä»•äº‹ã€ - å˜ç´”ä½œæ¥ã®ç¹°ã‚Šè¿”ã—ã§é–‹ç™ºè€…ã®æ™‚é–“ã‚’æµªè²»ã—ãªã„ãŸã‚ã«ã€‚"ã¨ã„ã†ãƒŸãƒƒã‚·ãƒ§ãƒ³ã‚’æŒã£ã¦ã„ã¾ã™ï¼Žãã®ä¸ã§è§£æžãƒãƒ¼ãƒ ã¯ï¼Œãƒ‡ãƒ¼ã‚¿è§£æžåŸºç›¤ã®æ§‹ç¯‰ï¼Œãƒ‡ãƒ¼ã‚¿ãƒžã‚¤ãƒ‹ãƒ³ã‚°ï¼Œãƒ‡ãƒ¼ã‚¿è§£æžã®ç¤¾å†…ã‚³ãƒ³ã‚µãƒ«ãƒ†ã‚£ãƒ³ã‚°ã‚’è¡Œãªã„æŠ€è¡“ã‹ã‚‰ã®æ”¹å–„ã‚’æ‹…å½“ã—ã¦ã„ã¾ã™ï¼Ž ä»Šå›žã®è¨˜äº‹ã§ã¯ï¼Œmixi ã«ãŠã‘ã‚‹è§£æžåŸºç›¤ã«ã¤ã„ã¦ç°¡å˜ã«è§¦ã‚ŒãŸã‚ã¨ï¼Œãã®åŸºç›¤ã«ãŠã‘ã‚‹ã€Œåˆºèº«ã®ä¸Šã«ã‚¿ãƒ³ãƒãƒã‚’ã®ã›ã‚‹ä»•äº‹ã€ã‚’ã©ã†æ¸›ã‚‰ã™ã‹ã®ï¼’ç‚¹ã«ã¤ã„ã¦æ›¸ãã¾ã™ï¼Ž mixi ã®è§£æžåŸºç›¤ ã¾ãšã¯è§£æžç’°å¢ƒã«ã¤ã„ã¦ï¼Œç°¡å˜ã«ãŠè©±ã—ã¾ã™ï¼Ž2012-08 ç¾åœ¨ mixi ã§ã¯ï¼Œä¸»ãªè§£æžç”¨ã®ãƒ„ãƒ¼ãƒ«ã¨ã—ã¦ã¯ï¼ŒApache Hadoop, Hive ã‚’åˆ©ç”¨ã—ã¦ã„ã¾ã™ï¼Žã¾ãŸã‚ã‚ã›ã¦ï¼Œè‡ªåˆ†ãªã©ä¸€éƒ¨ã®äººã¯ï¼Œ
mogwaing 2012/08/10
mixi

hive

hadoop
ãƒªãƒ³ã‚¯
Pig TPC-H Benchmark and Performance Tuning
For a class project we developed a whole set of Pig scripts for TPC-H. Our goals are: 1) identifying the bottlenecks of Pig's performance especially of its relational operators, 2) studying how to write efficient scripts by making full use of Pig Latin's features, 3) comparing with Hive's TPC-H results for verifying both 1) and 2)Read less
mogwaing 2012/08/09
hadoop

pig

hive
ãƒªãƒ³ã‚¯
Data analytics with hadoop hive on multiple data centers
This document discusses GMO Internet's data analytics system for analyzing social game data from over 500 game titles across multiple data centers in Japan and the US. It summarizes the system's architecture, which uses Hadoop/Hive to process logging data from game servers into hourly, daily, weekly, and monthly reports on key performance indicators. The system partitions and stores large volumes
mogwaing 2012/07/31
hadoop

hive

analytics

gmo
ãƒªãƒ³ã‚¯
Cloudera Blog
In an era where artificial intelligence (AI) is reshaping enterprises across the globeâ€”be it in healthcare, finance, or manufacturingâ€”itâ€™s hard to overstate the transf ormation that AI has had on businesses, regardless of industry or size. At Cloudera, we recognize the urgent need for bold steps to harness this potential and dramatically accelerate the time to [â€¦] Read blog post
mogwaing 2012/07/26
hadoop

hive

hbase
ãƒªãƒ³ã‚¯
[HIVE-1642] Convert join queries to map-join based on size of table/row - ASF JIRA
mogwaing 2012/07/24
hive 0.7.0 hive.auto.convert.join

hive
ãƒªãƒ³ã‚¯
ã“ã®ãƒšãƒ¼ã‚¸ã‚’è¦‹ã‚‹ã«ã¯ã€ãƒã‚°ã‚¤ãƒ³ã¾ãŸã¯ç™»éŒ²ã—ã¦ãã ã•ã„
Facebookã§æŠ•ç¨¿ã‚„å†™çœŸãªã©ã‚’ãƒã‚§ãƒƒã‚¯ã§ãã¾ã™ã€‚
mogwaing 2012/07/02
hive

join

hadoop
ãƒªãƒ³ã‚¯
Replacing Telco DB/DW to Hadoop and Hive
The document discusses replacing a telecommunications company's traditional data warehouse with Hadoop and Hive. It provides an overview of how KT Cloud has used Hadoop and Hive for cloud computing platforms, massive em ail archiving, log archiving, data analysis, and data platforms. It also includes sections on Hive architecture, how Hive queries are processed, and Hive's internal components like
mogwaing 2012/05/11
oracle

hive
ãƒªãƒ³ã‚¯
1 2 3 æ¬¡ã®ãƒšãƒ¼ã‚¸