[B! hive] ICHIROã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

ICHIRO id:ICHIRO

hiveã«é–¢ã™ã‚‹ICHIROã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (27)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

HadoopãŒæ‰±ã†æ©Ÿå¯†ãƒ‡ãƒ¼ã‚¿ã®ãƒžã‚¹ã‚ãƒ³ã‚°ã«ã¤ã„ã¦ - Qiita
CDH 5.4 ã‹ã‚‰å°Žå…¥ã•ã‚ŒãŸã€Sensitive Data Redaction (æ©Ÿå¯†ãƒ‡ãƒ¼ã‚¿ã®ãƒžã‚¹ã‚ãƒ³ã‚°) æ©Ÿèƒ½ã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ ã§ãã‚‹ã“ã¨ Hadoopã‚¯ãƒ©ã‚¹ã‚¿ã®ãƒã‚°ãƒ•ã‚¡ã‚¤ãƒ«ã€Hive/Impalaã‚¯ã‚¨ãƒªã«å«ã¾ã‚Œã‚‹ä»»æ„ã®æ©Ÿå¯†ãƒ‡ãƒ¼ã‚¿ã®ãƒžã‚¹ã‚ãƒ³ã‚°ãŒå¯èƒ½ã§ã™ã€‚ å¿…è¦ãªã‚‚ã® CDH 5.4 / Cloudera Manager 5.4 æ‰‹é † Cloudera Managerã«ãƒã‚°ã‚¤ãƒ³ã—ã€HDFSã‚µãƒ¼ãƒ“ã‚¹ã‚’é¸æŠžã—ã¾ã™ã€‚ 2. HDFSã®è¨å®šç”»é¢ã§ã€ã€Œredactionã€ã§æ¤œç´¢ã—ã¾ã™ã€‚ 3. ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆã§ã¯ã€Œã‚¯ãƒ¬ã‚¸ãƒƒãƒˆã‚«ãƒ¼ãƒ‰æƒ…å ±ã€ã€ã€Œç¤¾ä¼šä¿éšœç•ªå·ã€ã€ã€Œãƒ›ã‚¹ãƒˆåã€ã€ã€Œãƒ¡ãƒ¼ãƒ«ã‚¢ãƒ‰ãƒ¬ã‚¹ã€ã®ãƒžã‚¹ã‚ãƒ³ã‚°ãƒ†ãƒ³ãƒ—ãƒ¬ãƒ¼ãƒˆãŒç”¨æ„ã•ã‚Œã¦ã„ã¾ã™ã€‚ã‚«ã‚¹ã‚¿ãƒ ã®ãƒžã‚¹ã‚ãƒ³ã‚°ã‚’å®šç¾©ã™ã‚‹ã“ã¨ã‚‚å¯èƒ½ã§ã™ã€‚ã“ã“ã§ã¯ã‚¯ãƒ¬ã‚¸ãƒƒãƒˆã‚«ãƒ¼ãƒ‰æƒ…å ±ã‚’ãƒžã‚¹ã‚ãƒ³ã‚°ã—ã¾ã™ã€‚ 4. è¨å®šç”»é¢å†…ã§ã€ãƒžã‚¹ã‚ãƒ³ã‚°ãŒã©ã®ã‚ˆã†ã«å‹•ä½œã™ã‚‹ã®ã‹ã€ãƒ†ã‚¹ãƒˆã™ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚
ICHIRO 2015/07/23
hadoop

hive

security
ãƒªãƒ³ã‚¯
A perfect Hive query for a perfect meeting (Hadoop Summit 2014)
A perfect Hive query for a perfect meeting (Hadoop Summit 2014) During one of our epic parties, Martin Lorentzon (chairman of Spotify) agreed to help me to arrange a dinner for me and Timbuktu (my favourite Swedish rap and reggae artist), if I prove somehow that I am the biggest fan of Timbuktu in my home country. Because at Spotify we attack all probl ems using data-driven approaches, I decided to
ICHIRO 2014/06/10
hive
ãƒªãƒ³ã‚¯
Doesn't work with HA Namenodes (CDH 4.3) Â· Issue #845 Â· prestodb/presto
ICHIRO 2013/11/20
Presto with HA NN

hive

hadoop

facebook
ãƒªãƒ³ã‚¯
Presto: Free, Open-Source SQL Query Engine for any Data
Calling our Presto community speakers â€“ we want to hear from you! Fill out out community call for papers to speak at upcoming meetups andÂ conferences. What is Presto?Presto is an open source SQL query engine thatâ€™s fast, reliable, and efficient at scale. Use Presto to run interactive/ad hoc queries at sub-second performance for your high volume apps.
ICHIRO 2013/11/13
hadoop

hive

facebook
ãƒªãƒ³ã‚¯
Cloudera Blog
The ongoing progress in Artificial Intelligence is constantly expanding the realms of possibility, revolutionizing industries and societies on a global scale. The release of LLMs surged by 136% in 2023 compared to 2022, and this upward trend is projected to continue in 2024. Today, 44% of organizations are experimenting with generative AI, with 10% having [â€¦] Read blog post
ICHIRO 2013/10/17
hive

hadoop
ãƒªãƒ³ã‚¯
ä»Šæ—¥ã‹ã‚‰ã¯ã˜ã‚ã‚‹ãŠæ‰‹è»½ Hive ãƒ‡ãƒ¼ã‚¿ç§»è¡Œ - mixi engineer blog
ã“ã‚“ã«ã¡ã¯ï¼Ž æ˜¨å¹´çŸ¥äººã®ã‚ªãƒ¼ã‚±ã‚¹ãƒˆãƒ©æ¼”å¥ä¼šã§è´ã„ãŸãƒ¡ãƒ³ãƒ‡ãƒ«ã‚¹ã‚¾ãƒ¼ãƒ³äº¤éŸ¿æ›²ç¬¬ï¼”ç•ªã€Œã‚¤ã‚¿ãƒªã‚¢ã€ãŒå¤§å¥½ããªï¼ŒæŠ€è¡“éƒ¨ã®çŸ³å·æœ‰ã§ã™ï¼Ž ãã‚“ãªã€Œã‚¤ã‚¿ãƒªã‚¢ã€å¤§å¥½ããªç§ã§ã™ãŒ Hive ã®ãƒ‡ãƒ¼ã‚¿ç§»è¡Œã‚‚å¤§å¥½ãã¨ã„ã†ä½“ã§ï¼Œ ä»Šå›žã®è¨˜äº‹ã§ã¯ã‚ªãƒ³ãƒ—ãƒ¬ãƒŸã‚¹ã§ Hive ã‚’é ‘å¼µã£ã¦é‹ç”¨ã—ã¦ã„ã‚‹æ–¹ãŸã¡ã«å‘ã‘ã¦ï¼Œã©ã†ã—ã¦ã‚‚ã¨ã„ã†ã¨ãã® Hive ã®ãƒ‡ãƒ¼ã‚¿ç§»è¡Œæ–¹æ³•ã‚’ç´¹ä»‹ã—ãŸã„ã¨æ€ã„ã¾ã™ï¼Ž å¼Šç¤¾ã§ã¯æœ€è¿‘ï¼ŒCloudera's Distribution including Apache Hadoop 3 update 2 (CDH3u2) ã‹ã‚‰ CDH 4.1.2 ã«ãƒãƒ¼ã‚¸ãƒ§ãƒ³ã‚¢ãƒƒãƒ—ã‚’è¡Œãªã£ã¦ã„ã¾ã™ï¼Ž ã“ã®ã‚ˆã†ã«ã‚ªãƒ³ãƒ—ãƒ¬ãƒŸã‚¹ã§ Hive ã‚’é‹ç”¨ã—ã¦ã„ã‚‹æ–¹ãŸã¡ã®ä¸ã«ã¯ï¼Œã™ã§ã«æŒã£ã¦ã„ã‚‹ãƒ‡ãƒ¼ã‚¿ã‚’ã†ã¾ãç§»è¡Œã—ãŸã„ã¨ã„ã†æ–¹ã‚‚å¤šã„ã‹ã¨æ€ã„ã¾ã™ï¼Ž Hive ã«ã¯ç¾çŠ¶ MySQL ã® mysqldump ã®ã‚ˆã†ãªæ©Ÿèƒ½ãŒæä¾›ã•ã‚Œã¦ã„ãªã„ã®ã§ãƒ
ICHIRO 2013/01/29
hive

cdh
ãƒªãƒ³ã‚¯
Performance evaluation of cloudera impala (with Comparison to Hive)
ICHIRO 2012/12/20
hadoop

Impala

hive
ãƒªãƒ³ã‚¯
Hive undocumented feature
2. ï‚¡ Name: tamtam (ã‚¿ãƒ ã‚¿ãƒ ) ï‚¡ Twitter : @tamtam180 ï‚¡ Blog: http://mt.orz.at/ ï‚¡ Works: Software Enginner ï‚§ SIer -> Video Game, Web Developer -> ??? 3. ï‚¡ HiveCLIã‚³ãƒžãƒ³ãƒ‰ ï‚¡ Hookæ©Ÿèƒ½ ï‚¡ PerfLog ï‚¡ PDK ï‚¡ è‡ªå‹•ã§UDFç™»éŒ² ï‚¡ HTTP-RESTç‹¬è‡ªã‚µãƒ¼ãƒ ï‚¡ ãŠã¾ã‘ 5. ï‚¡ Hadoop DFSã‚³ãƒžãƒ³ãƒ‰ãŒä½¿ãˆã¾ã™ hive> dfs -ls /; Found 3 it ems drwxr-xr-x - tamtam supergroup 0 2012-04-28 23:03 /home drwxr-xr-x - tamtam supergroup 0 2012-05-20 14:17 /tmp drwxr-xr-x -
ICHIRO 2012/08/09
hive
ãƒªãƒ³ã‚¯
HiveServerã§ãƒ•ã‚¡ã‚¤ãƒ«ãƒ‡ã‚£ã‚¹ã‚¯ãƒªãƒ—ã‚¿ãŒãƒªãƒ¼ã‚¯ã™ã‚‹ä»¶ã«ã¤ã„ã¦
@tagomorisã•ã‚“ãŒã“ã‚“ãªäº‹ã‚’è¨€ã£ã¦ã„ãŸã®ã§ã€èª¿ã¹ã¦ã¿ã¾ã—ãŸã€‚ ã¾ãšã¯ã€.pipeoutãƒ•ã‚¡ã‚¤ãƒ«ã«ã¤ã„ã¦ã€‚ çµè«–ã‹ã‚‰è¨€ã†ã¨ã€Hiveã®ä¸å…·åˆã§ã™ã€‚ ï¼ˆï¼ˆè¿½è¨˜ 2012/06/23 14:00ï¼‰Hiveã®ãƒ—ãƒã‚°ãƒ©ãƒ ã®æ›¸ãæ–¹ã¯è‰¯ããªã„ã‚“ã ã‘ã©ã€Leakã¯ã—ã¦ã„ãªã„ã¨æ€ã‚ã‚Œã¾ã™.ï¼‰ ã“ã®.pipeoutãƒ•ã‚¡ã‚¤ãƒ«ã¯SessionStateã®startãƒ¡ã‚½ãƒƒãƒ‰ã§Openã•ã‚Œã¾ã™ã€‚startãƒ¡ã‚½ãƒƒãƒ‰ã¯HiveServerã®å ´åˆã€ã‚¯ãƒ©ã‚¤ã‚¢ãƒ³ãƒˆãŒã‚³ãƒã‚¯ã‚·ãƒ§ãƒ³ã‚’ã¯ã£ãŸæ™‚ã«ã‚³ãƒ¼ãƒ«ã•ã‚Œã¾ã™ã€‚(HiveSourceCodeReadingã®ç™ºè¡¨è³‡æ–™ã«ã‚‚ã¡ã‚‡ã£ã¨ã ã‘æ›¸ã„ã¦ã‚ã‚Šã¾ã™ã€‚) .pipeoutãƒ•ã‚¡ã‚¤ãƒ«ã¯HiveServerã§ã¯pipeInã¨ã„ã†å¤‰æ•°ã«ã‚ˆã£ã¦ã€BufferedReader -> FileReaderã‚’é€šã—ã¦å‚ç…§ã•ã‚Œã¾ã™ã€‚ â†“ã®ã‚ˆã†ãªæ„Ÿã˜ã€‚ pipeIn = new BufferedRead
ICHIRO 2012/06/22
hive
ãƒªãƒ³ã‚¯
Industryâ€™s First Auto-Scaling Hadoop Clusters | Qubole
BackgroundIn 2009 I first started playing around with Hive and EC2/S3. I was blown away by the potential of the cloud. But it bothered me that the burden of sizing the cluster was put on the user. How would an analyst know how many machines were required for a given query or job? To make it worse â€“ one had to even decide whether to add map-reduce or HDFS nodes. Within a single user session â€“ diffe
ICHIRO 2012/06/19
hive

hadoop
ãƒªãƒ³ã‚¯
Calculating unique visitors in Hadoop and Hive | DataMine Lab
ICHIRO 2011/09/07
hive
ãƒªãƒ³ã‚¯
Hive: Queries on Table Data
SELECT [ALL | DISTINCT] select_expr, select_expr, ... FROM table_reference [WHERE where_condition] [GROUP BY col_list] [ CLUSTER BY col_list | [DISTRIBUTE BY col_list] [SORT BY col_list] ] [LIMIT number] A SELECT statement can be part of a unionÂ query or a subqueryÂ of another query. table_reference indicates the input to the query. It can be a regular table, a view, a join constructÂ or a subquery
ICHIRO 2011/01/26
hive

Reference
ãƒªãƒ³ã‚¯
Hive Evolution: ApacheCon NA 2010
Hive is a data warehouse system for querying large datasets using SQL. Version 0.6 added views, multiple databases, dynamic partitioning, and storage handlers. Version 0.7 will focus on concurrency control, statistics collection, indexing, and performance improvements. Hive has become a top-level Apache project and aims to improve security, testing, and integration with other Hadoop components in
ICHIRO 2010/11/09
hadoop

hive
ãƒªãƒ³ã‚¯
Facebookã«ã‚ˆã‚‹Hadoop, Hive, HBaseãã—ã¦ A/B Testingã¸ã®å–çµ„ã¿
Spring Bootã«ã‚ˆã‚‹APIãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰æ§‹ç¯‰å®Ÿè·µã‚¬ã‚¤ãƒ‰ ç¬¬2ç‰ˆ ä½•åƒäººã‚‚ã®é–‹ç™ºè€…ãŒã€InfoQã®ãƒŸãƒ‹ãƒ–ãƒƒã‚¯ã€ŒPractical Guide to Building an API Back End with Spring Bootã€ã‹ã‚‰ã€Spring Bootã‚’ä½¿ã£ãŸREST APIæ§‹ç¯‰ã®åŸºç¤Žã‚’å¦ã‚“ã ã€‚ã“ã®æœ¬ã§ã¯ã€å‡ºç‰ˆæ™‚ã«æ–°ã—ããƒªãƒªãƒ¼ã‚¹ã•ã‚ŒãŸãƒãƒ¼ã‚¸ãƒ§ãƒ³ã§ã‚ã‚‹ Spring Boot 2 ã‚’ä½¿ç”¨ã—ã¦ã„ã‚‹ã€‚ã—ã‹ã—ã€Spring Boot3ãŒæœ€è¿‘ãƒªãƒªãƒ¼ã‚¹ã•ã‚Œã€é‡è¦ãªå¤‰...
ICHIRO 2010/09/29
hadoop

hive
ãƒªãƒ³ã‚¯
ã€ŽHadoop/Hiveã‚’ç”¨ã„ãŸãƒã‚°è§£æžåŸºç›¤ã®æ§‹ç¯‰ã€
ã“ã‚“ã«ã¡ã¯ã€‚Amebaã®ãƒã‚°è§£æžåŸºç›¤ã‚’æ‹…å½“ã—ã¦ã„ã‚‹ICHIROã§ã™ã€‚ ä»Šå›žã¯æ§‹ç¯‰ãƒ»é‹ç”¨ä¸ã®ãƒã‚°è§£æžåŸºç›¤ã€ŒPatriotã€ã«ã¤ã„ã¦æ›¸ããŸã„ã¨æ€ã„ã¾ã™ã€‚ Webã‚µãƒ¼ãƒ“ã‚¹ã‚’é‹å–¶ã—ã¦ã„ã‚‹ã¨æ—¥ã€…å¤§é‡ã®ãƒã‚°ãƒ‡ãƒ¼ã‚¿ã‚„ãƒ¦ãƒ¼ã‚¶ãƒ‡ãƒ¼ã‚¿ãŒè“„ç©ã•ã‚Œã¾ã™ã€‚ ä»Šã¾ã§ã¯PVï¼ˆãƒšãƒ¼ã‚¸ãƒ“ãƒ¥ãƒ¼ï¼‰ã‚„UUï¼ˆãƒ¦ãƒ‹ãƒ¼ã‚¯ãƒ¦ãƒ¼ã‚¶ï¼‰ãªã©ã‚¢ã‚¯ã‚»ã‚¹ãƒã‚°ãªã©ã‹ã‚‰å–ã‚Œã‚‹ã€å¤§æž ã®æŒ‡æ¨™ã®ã¿ã‚’è¦‹ã‚‹ã“ã¨ãŒã»ã¨ã‚“ã©ã§ã—ãŸã€‚ ãƒšãƒ¼ã‚¸ãƒ“ãƒ¥ãƒ¼ã«åˆã‚ã›ã¦ã‚·ã‚¹ãƒ†ãƒ ã‚’å¢—å¼·ã™ã‚‹ãªã©ã€ã‚·ã‚¹ãƒ†ãƒ å´ã¨ã—ã¦ã¯ååˆ†ã¨ã‚‚è¨€ãˆã‚‹æŒ‡æ¨™ã§ã™ãŒã€åºƒå‘Šã‚„èª²é‡‘ã‚µãƒ¼ãƒ“ã‚¹ã¨ã„ã†è¦–ç‚¹ã‹ã‚‰è¦‹ã‚‹ã¨ååˆ†ã¨ã¯è¨€ãˆã¾ã›ã‚“ã€‚ ä»Šã¾ã§ã¯Amebaå†…ã®å€‹ã€…ã®ã‚µãƒ¼ãƒ“ã‚¹æ‹…å½“è€…ãŒå¿…è¦ã¨ã™ã‚‹æŒ‡æ¨™ã‚’å‡ºã™ãŸã‚ã«ã‚¢ãƒ—ãƒªã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã‚„DB ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã«éƒ½åº¦ä¾é ¼ã‚’ã™ã‚‹å½¢ã§ãƒ‡ãƒ¼ã‚¿ã‚’æŠ½å‡ºã—ã¦ã„ã¾ã—ãŸã€‚ ä»Šå¾Œã®èª²é‡‘ã‚µãƒ¼ãƒ“ã‚¹ã®ç™ºå±•ãŒè¦‹è¾¼ã¾ã‚Œã€ã‚ˆã‚Šãƒ‡ãƒ¼ã‚¿åˆ†æžã®é‡è¦æ€§ãŒé«˜ã¾ã‚‹ã¨è€ƒãˆãŸç§ã¯ã€ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã§ãªã„ã‚µãƒ¼ãƒ“ã‚¹æ‹…å½“è€…ï¼ˆä¸»ã«ãƒ—ãƒãƒ‡ãƒ¥ãƒ¼ã‚µï¼‰ãŒã‚ˆã‚Šç°¡å˜
ICHIRO 2010/09/24
æ›¸ãã¾ã—ãŸâŠ‚(ï¾Ÿãƒ¼ï¾Ÿ*âŠ‚âŒ’ï½€ã¤â‰¡â‰¡â‰¡ã‚ºã‚µãƒ¼

hive

hadoop

CA
ãƒªãƒ³ã‚¯
Cloudera Blog
ICHIRO 2010/09/09
hadoop

hive
ãƒªãƒ³ã‚¯
Hive/StatisticsAndDataMining - Hadoop Wiki
Statistics and Data Mining in Hive This page is the secondary documentation for the slightly more advanced statistical and data mining functions that are being integrated into Hive, and especially the functions that warrant more than one-line descriptions. ngrams() and context_ngrams(): N-gram frequency estimation N-grams are subsequences of length N drawn from a longer sequence. The purpose of t
ICHIRO 2010/08/24
hive
ãƒªãƒ³ã‚¯
Hive/GenericUDAFCaseStudy - Hadoop Wiki
ICHIRO 2010/08/24
hive
ãƒªãƒ³ã‚¯
Inside Flume
Henry Robinson works at Cloudera on distributed data collection tools like Flume and ZooKeeper. Cloudera provides support for Hadoop and open source projects like Flume. Flume is a scala ble and configurable system for collecting large amounts of log and event data into Hadoop from diverse sources. It allows defining flexible data flows that can reliably move data between collection agents and stor
ICHIRO 2010/08/18
hadoop

Data

hive

Cloud
ãƒªãƒ³ã‚¯
Cloudera Blog
ICHIRO 2010/07/29
hive

hadoop
ãƒªãƒ³ã‚¯
1 2 æ¬¡ã®ãƒšãƒ¼ã‚¸