[B! hadoop] wata88ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

wata88 id:wata88

hadoopã«é–¢ã™ã‚‹wata88ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (14)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Apache Hadoopã‚¨ã‚³ã‚·ã‚¹ãƒ†ãƒ ã«ãŠã‘ã‚‹ã€ç•°ãªã‚‹ãƒ•ã‚¡ã‚¤ãƒ«å½¢å¼ã¨ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã‚¨ãƒ³ã‚¸ãƒ³ã®ãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹æ¯”è¼ƒ
è‘—è€…/Author: Zbigniew Branowski (Cern) åŽŸæ–‡/Originalï¼šhttp://blog.cloudera.com/blog/2017/02/performance-comparing-of-different-file-formats-and-storage-engines-in-hadoop-file-system/ Zbigniew Baranowskiã¯ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã‚·ã‚¹ãƒ†ãƒ ã®å°‚é–€å®¶ã§ã‚ã‚Šã€CERNã§ã‚»ãƒ³ãƒˆãƒ©ãƒ«ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã¨Hadoopãƒ™ãƒ¼ã‚¹ã®ã‚µãƒ¼ãƒ“ã‚¹ã‚’æä¾›ã€ã‚µãƒãƒ¼ãƒˆã—ã¦ã„ã‚‹ã‚°ãƒ«ãƒ¼ãƒ—ã®ãƒ¡ãƒ³ãƒãƒ¼ã§ã™ã€‚ ã“ã®ãƒ–ãƒã‚°ã¯ã‚‚ã¨ã‚‚ã¨CERNã®ã€ŒDatabases at CERNã€ãƒ–ãƒã‚°ã§å…¬é–‹ã•ã‚Œã¦ãŠã‚Šã€CERNã®è¨±å¯ã‚’å¾—ã¦ã“ã“ã§å…¬é–‹ã•ã‚Œã¦ã„ã¾ã™ã€‚ ãƒˆãƒ”ãƒƒã‚¯ã“ã®è¨˜äº‹ã§ã¯ã€Apache Hadoopã‚¨ã‚³ã‚·ã‚¹ãƒ†ãƒ ã§åˆ©ç”¨å¯èƒ½ãªã„ãã¤ã‹ã®ä¸€èˆ¬çš„ãªãƒ‡ãƒ¼ã‚¿ãƒ•ã‚©ãƒ¼ãƒžãƒƒãƒˆã¨ã‚¹ãƒˆãƒ¬ãƒ¼
wata88 2018/07/25
benchmark

hadoop

performance
ãƒªãƒ³ã‚¯
ã€ŒHadoopã®æ™‚ä»£ã¯çµ‚ã‚ã£ãŸã€ã®æ„å‘³ã‚’æ£ã—ãç†è§£ã™ã‚‹ - ç§‘å¦ã¨éžç§‘å¦ã®è¿·å®®
Hadoopã®æ™‚ä»£ã¯çµ‚ã‚ã£ãŸã€ã¨ã„ã†è¨€èª¬ã‚’ãŸã¾ã«è¦‹ã‹ã‘ã‚‹ã‚ˆã†ã«ãªã‚Šã¾ã—ãŸã€‚ ã‚‚ã¡ã‚ã‚“çµ‚ã‚ã£ã¦ãªã©ã„ã¾ã›ã‚“ã€‚ã—ã‹ã—ã€Hadoopã¨ãã®å–ã‚Šå·»ãç’°å¢ƒãŒå¤‰åŒ–ã—ãŸã®ã¯äº‹å®Ÿã§ã™ã€‚ æœ¬è¨˜äº‹ã§ã¯ã€ã“ã®å¤‰åŒ–ãŒä½•ãªã®ã‹ã‚’æ˜Žã‚‰ã‹ã«ã—ã€ãã®ä¸Šã§ã€ãªãœHadoopã®æ™‚ä»£ã¯çµ‚ã‚ã£ãŸã¨ã„ã†ä¸»å¼µãŒå®Ÿæ…‹ã‚’æ£ã—ãè¡¨ã—ã¦ã„ãªã„ã®ã‹ã‚’èª¬æ˜Žã—ã¦ã„ãã¾ã™ã€‚ DISCLAIMER ç§ã¯Hadoopã‚’ä¸å¿ƒã¨ã—ãŸãƒ‡ãƒ¼ã‚¿åŸºç›¤ã‚’å–ã‚Šæ‰±ã†ãƒ™ãƒ³ãƒ€ãƒ¼ã€Clouderaã®ç¤¾å“¡ã§ã™ã€‚ ä¸ç«‹çš„ã«æ›¸ãã‚ˆã†åŠªã‚ã¾ã™ãŒã€æ‰€å±žçµ„ç¹”ã«ã‚ˆã£ã¦ç™ºç”Ÿã™ã‚‹ãƒã‚¤ã‚¢ã‚¹ã®å®Œå…¨ãªæŽ’é™¤ã‚’ä¿è¨¼ã™ã‚‹ã“ã¨ã¯ã§ãã¾ã›ã‚“ã€‚ ä»¥ä¸Šã‚’ã”äº†æ‰¿ã®ä¸Šã€èªã¿é€²ã‚ã¦ãã ã•ã„ã€‚ è¦ç´„ ãƒ‡ãƒ¼ã‚¿åŸºç›¤ã¯ã€Hadoopã®ç™»å ´ã«ã‚ˆã‚Šéžå¸¸ã«å®‰ä¾¡ã¨ãªã‚Šã€ä»Šã¾ã§ã§ã¯ä¸å¯èƒ½ã ã£ãŸå¤§é‡ã®ãƒ‡ãƒ¼ã‚¿ã‚’å–ã‚Šæ‰±ãˆã‚‹ã‚ˆã†ã«ãªã‚Šã¾ã—ãŸã€‚ Hadoopã¯ã€NoSQLãƒ–ãƒ¼ãƒ ã®ä¸ã€å‡¦ç†ã‚¨ãƒ³ã‚¸ãƒ³ã§ã‚ã‚‹MapReduceã¨ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã§ã‚ã‚‹HDFSãŒ
wata88 2017/07/10
ã‚ã‹ã‚Šã‚„ã™ã„è§£èª¬

hadoop
ãƒªãƒ³ã‚¯
Apache Kudu - Fast Analytics on Fast Data
Home Overview Documentation Releases Blog Community GET IN TOUCH Mailing Lists Slack Channel Events and Meetups Project Committers Ecosystem How to Contribute DEVELOPER RESOURCES GitHub Gerr it Code Review JIRA Issue Tracker SOCIAL MEDIA Twitter Reddit APACHE SOFTWARE FOUNDATION Security Sponsorship Thanks License FAQ Apache Kudu is an open source distributed data storage engine that makes fast ana
wata88 2015/09/30
ãƒã‚´ãŒã‹ã‚ã„ã„

analytics

hadoop

hbase

hdfs

cloudera
ãƒªãƒ³ã‚¯
Kuduãƒªãƒªãƒ¼ã‚¹ï¼
ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã®æ–°ã—ã„ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã‚¨ãƒ³ã‚¸ãƒ³ã€Kudu ã¤ã„ã«æœ¬æ—¥ã‚¢ãƒŠã‚¦ãƒ³ã‚¹ã•ã‚ŒãŸ Kudu (ã‚¯ãƒ‰ã‚¥ï¼‰ã€æ¬¡ä¸–ä»£ãƒãƒ¼ãƒ‰ã‚¦ã‚§ã‚¢ã«ã‚‚å¯¾å¿œã™ã‚‹ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã®ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã‚¨ãƒ³ã‚¸ãƒ³ã§ã™ã€‚ã¾ã ãƒ™ãƒ¼ã‚¿ç‰ˆã§ã™ãŒã€3å¹´ã‚‚ã®é–‹ç™ºæœŸé–“ã‚’çµŒã¦ã¤ã„ã«å…¬é–‹ã•ã‚Œã¾ã—ãŸï¼ Super excited to finally talk about what I've been working on the last 3 years: Kudu! http://t.co/1W4sqFBcyH http://t.co/1mZCwgdOO5 â€” Todd Lipcon (@tlipcon) September 28, 2015 æ•°æ—¥å‰ã«ãƒªãƒ¼ã‚¯ã•ã‚ŒãŸè¨˜äº‹ã«ã€ŒKuduã¯HDFSã‚„HBaseã‚’ç½®ãæ›ãˆã‚‹ã‚‚ã®ã ã€ã¨æ›¸ã‹ã‚Œã¦ã„ã¾ã—ãŸãŒã€ãã‚Œã¯é–“é•ã„ã§ã™ã€‚è©³ç´°ã¯FAQãªã©ã«ã‚‚æ›¸ã‹ã‚Œã¦ã„ã¾ã™ãŒã€HDFSã¨HBaseã‚’è£œã†æ–°ã—ã„ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã‚¨ãƒ³ã‚¸ãƒ³ã§ã™
wata88 2015/09/30
hdfs

hadoop

HBase

analytics
ãƒªãƒ³ã‚¯
MapReduceã¯ã€Œã‚†ã£ãã‚Šæ»ã‚“ã§ã„ãã€ã€Hadoopé–‹ç™ºè€…ã®ã‚«ãƒƒãƒ†ã‚£ãƒ³ã‚°æ°
ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã‚½ãƒ•ãƒˆã‚¦ã‚¨ã‚¢ï¼ˆOSSï¼‰ã®åˆ†æ•£ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚½ãƒ•ãƒˆã€ŒHadoopã€ãŒã€å¤§ããªå¤‰è²Œã‚’é‚ã’å§‹ã‚ã¦ã„ã‚‹ã€‚ãƒãƒƒãƒå‡¦ç†ã ã‘ã§ãªãã€SQLã‚¯ã‚¨ãƒªãƒ¼ã‚’ä½¿ã£ãŸã‚¤ãƒ³ã‚¿ãƒ©ã‚¯ãƒ†ã‚£ãƒ–åˆ†æžã‚„ã‚¤ãƒ³ãƒ¡ãƒ¢ãƒªãƒ¼å‡¦ç†ã€æ©Ÿæ¢°å¦ç¿’å‡¦ç†ãªã©ã¸ã¨é©ç”¨é ˜åŸŸã‚’åºƒã’ã¦ã„ã‚‹ã‹ã‚‰ã ã€‚Hadoopã®ã‚ªãƒªã‚¸ãƒŠãƒ«é–‹ç™ºè€…ã§ã‚ã‚‹ãƒ€ã‚°ãƒ»ã‚«ãƒƒãƒ†ã‚£ãƒ³ã‚°æ°ã¯ã€ã€Œæ–°ã—ã„æ™‚ä»£ãŒå§‹ã¾ã£ãŸã€ã¨èªžã‚‹ã€‚ ç¾åœ¨ã®Hadoopã®çŠ¶æ³ã‚’ã©ã®ã‚ˆã†ã«è¦‹ã¦ã„ã‚‹ã‹ï¼Ÿ åŒã˜ã‚ˆã†ãªæ©Ÿèƒ½ã‚’å‚™ãˆãŸã‚³ãƒ³ãƒãƒ¼ãƒãƒ³ãƒˆãŒè¤‡æ•°ææ¡ˆã•ã‚Œã‚‹ãªã©é‡è¤‡ã¯ã‚ã‚‹ãŒã€æŠ€è¡“é–‹ç™ºã¯éžå¸¸ã«é«˜é€Ÿã«é€²ã‚“ã§ã„ã‚‹ã€‚ï¼ˆç±³Apacheã‚½ãƒ•ãƒˆã‚¦ã‚¨ã‚¢è²¡å›£ãŒ2013å¹´10æœˆã«æ£å¼ç‰ˆã‚’ãƒªãƒªãƒ¼ã‚¹ã—ãŸï¼‰ã‚¹ã‚±ã‚¸ãƒ¥ãƒ¼ãƒ©ãƒ¼ã®ã€ŒYARNã€ã«ã‚ˆã£ã¦ã€ï¼ˆHadoopã®ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã‚·ã‚¹ãƒ†ãƒ ã§ã‚ã‚‹ï¼‰ã€ŒHDFSã€ã®ä¸Šã«ã€æ§˜ã€…ãªãƒ†ã‚¯ãƒŽãƒã‚¸ãƒ¼ã‚’å…±å˜ã§ãã‚‹ã‚ˆã†ã«ãªã£ãŸã“ã¨ãŒå¤§ãã„ã€‚ç§ãŒç‰¹ã«é©šã„ã¦ã„ã‚‹ã®ã¯ã€ï¼ˆDAGï¼šDirected Acyclic Graphï¼æœ‰å‘
wata88 2014/08/05
Sparkã®ã»ã†ãŒæ±Žç”¨çš„ã§å°†æ¥æ€§ã‚ã‚“ã˜ã‚ƒãï¼Ÿãã‚Œã¨ã‚‚MR2ã‚‚ã¨ã„YARNã¡ã‚ƒã‚“ã“ãæœ¬å‘½ã£ã¦ã“ã¨ï¼Ÿã£ã¦æ˜”èª°ã‹ã«èžã„ãŸã‘ã©ï¼Œç…™ã«å·»ã‹ã‚Œã¦ãƒ¢ãƒ¤ã£ã¨ã—ãŸã®æ€ã„å‡ºã—ãŸ

Spark

hadoop
ãƒªãƒ³ã‚¯
ã€ŽPrestoã¨ã¯ä½•ã‹ï¼ŒPrestoã§ä½•ãŒã§ãã‚‹ã‹ã€ - ãƒˆãƒ¬ã‚¸ãƒ£ãƒ¼ãƒ‡ãƒ¼ã‚¿ï¼ˆTreasure Dataï¼‰ãƒ–ãƒã‚°
ãƒˆãƒ¬ã‚¸ãƒ£ãƒ¼ãƒ‡ãƒ¼ã‚¿ã¯ã‚¯ãƒ©ã‚¦ãƒ‰ã§ãƒ‡ãƒ¼ã‚¿ãƒžãƒãƒ¼ã‚¸ãƒ¡ãƒ³ãƒˆã‚µãƒ¼ãƒ“ã‚¹ã‚’æä¾›ã—ã¦ã„ã¾ã™ã€‚ Hadoop Conference Japan 2014 ä»¥å‰ã«å‘ŠçŸ¥ã—ãŸHadoop Conference Japan 2014ã§ï¼Œå¼Šç¤¾Software Architectã®å¤æ©‹ãŒç™ºè¡¨ã—ã¾ã—ãŸã€‚ ãƒ†ãƒ¼ãƒžã¯ï¼ŒFacebookãŒå…¬é–‹ã—ãŸæ–°ã—ã„åˆ†æ•£å‡¦ç†åŸºç›¤ï¼ŒPrestoã€‚å®Ÿã¯FacebookãŒå½¼ã‚‰ã®è¶…å¤§è¦æ¨¡ãªãƒ‡ãƒ¼ã‚¿ã‚»ãƒƒãƒˆã«å¯¾ã—ã¦ã‚¤ãƒ³ã‚¿ãƒ©ã‚¯ãƒ†ã‚£ãƒ–ã«çµæžœã‚’è¿”ã›ã‚‹ã‚ˆã†ã«ã¨é–‹ç™ºã•ã‚ŒãŸã‚‚ã®ã§ã™ã€‚é–‹ç™ºãŒå§‹ã¾ã£ã¦ã¾ã 2å¹´ã‚‚çµŒã£ã¦ãŠã‚Šã¾ã›ã‚“ãŒï¼Œä»Šã§ã¯ãƒˆãƒ¬ã‚¸ãƒ£ãƒ¼ãƒ‡ãƒ¼ã‚¿ã‚’åˆã‚ã¨ã—ã¦å¤šãã®ãƒãƒƒã‚«ãƒ¼é”ãŒã‚³ãƒŸãƒƒã‚¿ãƒ¼ã¨ã—ã¦å‚åŠ ã™ã‚‹æ´»ç™ºçš„ãªãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã«æˆé•·ã—ã¦ã„ã¾ã™ã€‚ Prestoã¯Hiveã‚„Impalaã¨åŒã˜ã€ŒSQL Query Engineã€ã§ã‚ã‚Šï¼Œç‰¹ã«æ•°ç™¾GBã‚’è¶…ãˆã‚‹å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã«å¯¾ã—ã¦ã‚‚ã‚¤ãƒ³ã‚¿ãƒ©ã‚¯ãƒ†ã‚£ãƒ–ãªãƒ¬ã‚¹ãƒãƒ³ã‚¹ã‚’ï¼ˆã‚³ãƒ³ãƒž0ç§’ä»¥ä¸‹ï¼Œé…ãã¦
wata88 2014/07/13
hadoop

presto
ãƒªãƒ³ã‚¯
cdh-twitter-exampleã§éŠã¶
ã“ã®è¨˜äº‹ã¯Hadoop Advent Calendar 2013, 13æ—¥ç›®ã®ã‚¨ãƒ³ãƒˆãƒªã§ã™ã€‚ãƒ–ãƒã‚°ãªã‚“ã–æ›¸ã„ãŸã“ã¨ãªã„ã®ã§å‹æ‰‹ãŒè‰¯ãåˆ†ã‹ã‚Šã¾ã›ã‚“ãŒ@sudabonã•ã‚“@kernel023ã•ã‚“ã®ãŠå‰ã‚‚ä½•ã‹æ›¸ã‘ã‚„ã‚³ãƒ©ã¨ã„ã†åœ§åŠ›ã«è² ã‘ã¦Bloggerã«ãƒšãƒ¼ã‚¸ã‚’ç”¨æ„ã—ã¦ã¿ã¾ã—ãŸã€‚ Hadoopã®åˆ©ç”¨ç›®çš„ã¨ã—ã¦ãƒ‡ãƒ¼ã‚¿ã®åˆ†æžã«å–ã‚Šçµ„ã‚“ã§ã„ã‚‹ã€ã‚‚ã—ãã¯ã“ã‚Œã‹ã‚‰å–ã‚Šçµ„ã‚‚ã†ã¨ã—ã¦ã„ã‚‹ãƒ¦ãƒ¼ã‚¶ã¯å¤šã„ã®ã§ã¯ãªã„ã‹ã¨æ€ã„ã¾ã™ã€‚ä¸ã§ã‚‚ãƒã‚°åˆ†æžã¨ä¸¦ã‚“ã§ã‚½ãƒ¼ã‚·ãƒ£ãƒ«ãƒ¡ãƒ‡ã‚£ã‚¢ã®åˆ†æžã¯å„ç¤¾ãŒç©æ¥µçš„ã«å–ã‚Šçµ„ã‚‚ã†ã¨ã—ã¦ã„ã‚‹ãƒ†ãƒ¼ãƒžã§ã‚ã‚Šã€ç§ãŒå‹æ‰‹ã«Twitteråˆ†æž3éƒ¨ä½œã¨å‘¼ã‚“ã§ã„ã‚‹ä»¥ä¸‹ã®ãƒ–ãƒã‚°ã‚’èªã¾ã‚ŒãŸæ–¹ã‚‚å¤šã„ã®ã§ã¯ãªã„ã§ã—ã‚‡ã†ã‹ï¼š http://blog.cloudera.com/blog/2012/09/analyzing-twitter-data-with-hadoop/ http://blog.cloudera.com/b
wata88 2014/05/20
cdh-twitter-example

hadoop

cloudera

HBase

twitter
ãƒªãƒ³ã‚¯
Hadoopã‚’10åˆ†ã§è©¦ã™ï¼ˆï¼–ï¼‰Hueã‹ã‚‰Solrã‚’ä½¿ã†
ï¼Šï¼Šã“ã®è¨˜äº‹ã®å†…å®¹ã¯è‹¥å¹²å¤ããªã£ã¦ã„ã¾ã™ã€‚ã¾ã¨ã‚ãƒšãƒ¼ã‚¸ã‚‚ã”è¦§ä¸‹ã•ã„ï¼Šï¼Š ãã‚Œã§ã¯ã€Clouderaã®ãƒ–ãƒã‚°ã«å¾“ã„ã€Solrã‚’ä½¿ã£ã¦ã¿ã¾ã—ã‚‡ã†ã€‚ MapReduceã«ã‚ˆã‚‹ãƒãƒƒãƒã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ ã¾ãšã¯ã€ã‚¿ãƒ¼ãƒŸãƒŠãƒ«ã‹ã‚‰ç–‘ä¼¼ãƒ„ã‚¤ãƒ¼ãƒˆã‚’ã‚»ãƒƒãƒˆã‚¢ãƒƒãƒ—ã—ã¾ã™ã€‚ã“ã®ä»®æƒ³ãƒžã‚·ãƒ³ã«ã¯ã‚µãƒ³ãƒ—ãƒ«ãƒ„ã‚¤ãƒ¼ãƒˆã‚’ç”Ÿæˆã™ã‚‹ã‚¹ã‚¯ãƒªãƒ—ãƒˆãŒç”¨æ„ã•ã‚Œã¦ã„ã¾ã™ã€‚ä»¥ä¸‹ã®ã‚³ãƒžãƒ³ãƒ‰ã‚’å®Ÿè¡Œã—ã¾ã—ã‚‡ã†ã€‚ $ ~/datasets/batch-tweets.sh [cloudera@localhost ~]$ ~/datasets/batch-tweets.sh Uploading configs from /home/cloudera/batch_tweets_configs/conf to localhost:2181/solr. This may take up to a minute. rm: `/user/cloudera/batch_
wata88 2014/05/20
apache

hadoop

solr

hue
ãƒªãƒ³ã‚¯
Cloudera Standard ã®ã”æ¡ˆå†… ~ ç„¡å„Ÿç‰ˆ å¤§å¹…æ©Ÿèƒ½å¼·åŒ–ã®ãŠçŸ¥ã‚‰ã› | Cloudera Japan
Cloudera ã¯ã€æœ€å¤§è¦æ¨¡ã®ä¼æ¥ãŒã‚ã‚‰ã‚†ã‚‹å ´æ‰€ã«å˜åœ¨ã™ã‚‹ã™ã¹ã¦ã®ãƒ‡ãƒ¼ã‚¿ã‚’ä¿¡é ¼æ€§ã®é«˜ã„æœ‰æ„ç¾©ãªæ´žå¯Ÿã«å¤‰æ›ã§ãã‚‹ã‚ˆã†æ”¯æ´ã—ã¾ã™ã€‚ Trusted Data Today for Tomorrowâ€™s AI (ä»Šæ—¥ã®ä¿¡é ¼ã§ãã‚‹ãƒ‡ãƒ¼ã‚¿ã‚’ã€æ˜Žæ—¥ã® AI ã«è¨—ã™) ãƒ‡ãƒ¼ã‚¿ã¨ AI ãŒãƒ†ãƒ¼ãƒžã®ãƒ—ãƒ¬ãƒŸã‚¢ã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ã§ã€æ¥ç•Œæœ€å‰ç·šã®æ˜Žç¢ºãªãƒ“ã‚¸ãƒ§ãƒ³ã‚’æŒã£ãŸæ–¹ã€…ã‚„ã‚½ãƒ¼ãƒˆãƒªãƒ¼ãƒ€ãƒ¼ã‹ã‚‰è²´é‡ãªæ´žå¯Ÿã‚’å¾—ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚
wata88 2014/04/04
hadoop
ãƒªãƒ³ã‚¯
hadoopã®è©±ã¨pythonã§ãƒ‡ãƒ¼ã‚¿ãƒžã‚¤ãƒ‹ãƒ³ã‚°ã‚’ã™ã‚‹è©± - gumi Engineerâ€™s Blog
gumiã®ç²Ÿé£¯åŽŸã§ã™ã€‚ ãƒ‡ãƒ¼ã‚¿ãƒžã‚¤ãƒ‹ãƒ³ã‚°ã‚„ã£ã¦ã¾ã™ã€‚ã—ã‹ã—ãªãŒã‚‰æœ€è¿‘ã¯ã‚ã¾ã‚Šãƒ‡ãƒ¼ã‚¿ã‚’ã„ã˜ã‚‹æ™‚é–“ãŒãªãç¤¾å†…ã§ãƒ—ãƒ¬ã‚¼ãƒ³ã°ã‹ã‚Šã™ã‚‹æ—¥ã€…ã§ã€ç¤¾å†…ã§ã‚‚ç§ãŒãªã«ã‚„ã£ã¦ã‚‹ã®ã‹ã‚ã‹ã‚‰ãªã„ã¨ã„ã†ã‚‚ã£ã±ã‚‰ã®è©•åˆ¤ã€‚ãã“ã§ä»Šå›žã¯ä¸€å¿œãƒ‡ãƒ¼ã‚¿è§£æžã‚‚ã‚„ã£ã¦ã¯ã„ã‚‹ã‚“ã ã‚ˆã¨ã„ã†ã“ã¨ã‚’å†…å¤–ã«ä¸»å¼µã™ã‚‹ãŸã‚ã«ã‚‚ãƒ‡ãƒ¼ã‚¿ãƒžã‚¤ãƒ‹ãƒ³ã‚°ã®è©±ã‚’ã—ã‚ˆã†ã¨æ€ã„ã¾ã™ã€‚ ã‚¢ãƒ—ãƒªã®åŸºæœ¬çš„ãªæ—¥ã€…ã®çµ±è¨ˆãƒ‡ãƒ¼ã‚¿å–å¾—ãªã©ã¯åˆ¥ã®æ–¹ã€…ã¯ã‚„ã‚‰ã‚Œã¦ã„ã‚‹ã®ã§ã€ç§ã‹ã‚‰ã¯hadoopã‚’ä½¿ã£ãŸå¤§è¦æ¨¡è§£æžã®è©±ã‚„ã€ãã“ã§å¾—ã‚‰ã‚ŒãŸãƒ‡ãƒ¼ã‚¿ã‚’åˆ†æžã™ã‚‹ç’°å¢ƒã®è©±ã‚’ã—ãŸã„ã¨æ€ã„ã¾ã™ã€‚ã‚³ãƒ¼ãƒ‰ã‚’ä½µè¨˜ã—ãŸå…·ä½“ä¾‹ãªã©ã¯ä»Šå›žè¼‰ã›ã‚‰ã‚Œãªã„ã®ã§ã™ãŒã€ä»Šå¾Œã¾ãŸç´¹ä»‹ã—ã¦ã„ã‘ã‚Œã°ã¨æ€ã„ã¾ã™ã€‚ å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã®è§£æž æ—¥ã€…ã®ãƒã‚°è§£æžã‚„DBè§£æžã¯cronã«ã‚ˆã‚‹å‡¦ç†ã§æ¯Žæœãƒ¬ãƒãƒ¼ãƒ†ã‚£ãƒ³ã‚°ã‚’è¡Œã£ã¦ã„ã‚‹ã®ã§ã™ãŒã€æ–°ã—ã„æƒ…å ±ã‚’éŽåŽ»ã®ã‚¢ã‚¯ã‚»ã‚¹ãƒã‚°ã‹ã‚‰ã¾ã¨ã‚ã¦ã©ã‹ã‚“ã¨å–å¾—ã—ãŸã„ã¨è¨€ã†æ™‚ã¯Hadoopã«ã‚ˆã‚‹å¤§è¦æ¨¡è§£æžã‚’å®Ÿè¡Œã—ã¦ã„ã¾ã™
wata88 2013/10/29
Python

hadoop
ãƒªãƒ³ã‚¯
livedoor Techãƒ–ãƒã‚° : ç¬¬2å›ž NHNãƒ†ã‚¯ãƒŽãƒã‚¸ãƒ¼ã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ é–‹å‚¬ãƒã‚°
æ ªå¼ä¼šç¤¾ãƒ‡ãƒ¼ã‚¿ãƒ›ãƒ†ãƒ«ã®ä¼Šå‹¢ã§ã™ã€‚ 2012å¹´8æœˆ18æ—¥ï¼ˆåœŸï¼‰ã«é–‹å‚¬ã•ã‚Œã¾ã—ãŸ ç¬¬2å›ž NHN ãƒ†ã‚¯ãƒŽãƒã‚¸ãƒ¼ã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ ã®ç™ºè¡¨è³‡æ–™ã¨å‹•ç”»ã‚’å…¬é–‹è‡´ã—ã¾ã™ã€‚ ã”ç™»å£‡é ‚ãã¾ã—ãŸçš†æ§˜ã€ã”å‚åŠ é ‚ãã¾ã—ãŸçš†æ§˜ã€ã©ã†ã‚‚ã‚ã‚ŠãŒã¨ã†ã”ã–ã„ã¾ã—ãŸã€‚ ã¾ãŸã€ä»Šå›žã€ŒHã€ãªæŠ€è¡“ã¨è¨€ã†äº‹ã§ã€ã‚ªãƒ©ã‚¤ãƒªãƒ¼ãƒ»ã‚¸ãƒ£ãƒ‘ãƒ³æ§˜ã‚ˆã‚Šã€ŒHBaseã€ã®æ›¸ç±ã‚’ãƒ—ãƒ¬ã‚¼ãƒ³ãƒˆé ‚ãã¾ã—ãŸã€‚ã‚¢ãƒ¬ãƒ³ã‚¸ã—ã¦ã„ãŸã ã„ãŸç¿»è¨³è€…ã®çŽ‰å·ã•ã‚“ã€ã‚ªãƒ©ã‚¤ãƒªãƒ¼ãƒ»ã‚¸ãƒ£ãƒ‘ãƒ³æ§˜ã‚ã‚ŠãŒã¨ã†ã”ã–ã„ã¾ã™ã€‚ ã€ŒHã€æœ¬å½“ãŸã£ãŸçš†æ§˜ã€ãŠã‚ã§ã¨ã†ã”ã–ã„ã¾ã—ãŸã€‚ ãã‚Œã§ã¯ã€ä»¥ä¸‹ ç¬¬2å›žãƒ†ã‚¯ãƒŽãƒã‚¸ãƒ¼ã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ã®é–‹å‚¬ãƒã‚°ã¨ãªã‚Šã¾ã™ã€‚ â€» ç™»å£‡è€…ã®çš†æ§˜ã¨ã€‚å·¦ã‹ã‚‰ ç”°ç± æ°ã€äº•ä¸Šæ°ã€ä¸æ‘æ°ã€æ¿±é‡Žæ°ã€æ²ˆ æ°ã€ä¼Šå‹¢ã§ã™ã€‚ ã€ŒHTML5 Animation in Mobile Web Gamesã€(æ²ˆ ç›¸æ—» æ° NHN Koreaã€ Mobile Ajax ãƒãƒ¼ãƒ ) ã€Œæ—¥ã€…é€²åŒ–ã™ã‚‹Hadoopã®ã€Žä»Š
wata88 2013/10/21
hadoop

ã¾ã¨ã‚
ãƒªãƒ³ã‚¯
æ—¥ã€…é€²åŒ–ã™ã‚‹Hadoopã€‚ã“ã‚Œã¾ã§ã®ãŠã•ã‚‰ã„ã¨æœ€è¿‘ã®å‹•å‘ï¼ˆå‰ç·¨ï¼‰
å¤§è¦æ¨¡ãªåˆ†æ•£å‡¦ç†ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã¨ã—ã¦HadoopãŒç™»å ´ã—ãŸã“ã¨ã«ã‚ˆã‚Šã€ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã®ãƒ–ãƒ¼ãƒ ã‚„ã€å¤§è¦æ¨¡ãªã‚½ãƒ¼ã‚·ãƒ£ãƒ«ã‚²ãƒ¼ãƒ ã§ã®ãƒã‚°è§£æžã«ã‚ˆã‚‹æ”¹å–„ã€ã‚³ãƒžãƒ¼ã‚¹ã‚µã‚¤ãƒˆã§ã®æ©Ÿæ¢°å¦ç¿’ã«ã‚ˆã‚‹ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‡ãƒ¼ã‚·ãƒ§ãƒ³ãªã©ã€å¤šãã®å¤‰åŒ–ãŒå¼•ãèµ·ã“ã•ã‚Œã¦ãã¾ã—ãŸã€‚ ãã—ã¦ãã®Hadoopè‡ªä½“ã‚‚ã€æ—¥ã€…é€²åŒ–ã—ç¶šã‘ã¦ã„ã¾ã™ã€‚ Hadoopã¨ã¯ã©ã†ã„ã†ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã§ã‚ã‚Šã€ã„ã¾ã©ã®ã‚ˆã†ãªçŠ¶æ³ã«ãªã£ã¦ã„ã‚‹ã®ã‹ã€‚NTTãƒ‡ãƒ¼ã‚¿ã®æ¿±é‡Žè³¢ä¸€æœ—æ°ãŒã€å…ˆæ—¥è¡Œã‚ã‚ŒãŸç¬¬2å›ž NHNãƒ†ã‚¯ãƒŽãƒã‚¸ãƒ¼ã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ã§è¡Œã£ãŸã‚»ãƒƒã‚·ãƒ§ãƒ³ã€Œæ—¥ã€…é€²åŒ–ã™ã‚‹Hadoopã®ã€Žã„ã¾ã€ã€ã§åˆ†ã‹ã‚Šã‚„ã™ãè§£èª¬ã—ã¦ã„ã¾ã™ã€‚ ã“ã®è¨˜äº‹ã§ã¯ãã®ã‚»ãƒƒã‚·ãƒ§ãƒ³ã®å†…å®¹ã‚’ãƒ€ã‚¤ã‚¸ã‚§ã‚¹ãƒˆã§ç´¹ä»‹ã—ã¾ã—ã‚‡ã†ã€‚ æ—¥ã€…é€²åŒ–ã™ã‚‹Hadoopã®ã€Œã„ã¾ã€ NTTãƒ‡ãƒ¼ã‚¿ åŸºç›¤ã‚·ã‚¹ãƒ†ãƒ äº‹æ¥éƒ¨ æ¿±é‡Žè³¢ä¸€æœ—æ°ã€‚ NTTãƒ‡ãƒ¼ã‚¿ã¨ã„ã†ã¨ã“ã‚ã§ä»•äº‹ã‚’ã—ã¦ã„ã¾ã™ã€‚NTTãƒ‡ãƒ¼ã‚¿è‡ªä½“ã¯ã‚‚ã†5å¹´ãã‚‰ã„Hadoopã‚’ã‚„ã£ã¦ã¾ã—ã¦ã€ã
wata88 2013/10/21
hadoop

æŠ€è¡“

ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯
ãƒªãƒ³ã‚¯
Pig, Hiveãªã©8ç¨®é¡žã®MapReduceè¨€èªžã«ã¤ã„ã¦ã®æ¯”è¼ƒ - nokunoã®æ—¥è¨˜
ä»¥ä¸‹ã®è¨˜äº‹ã§ã¯Pig, Hiveãªã©8ç¨®é¡žã®è¨€èªžã§MapReduceã®ä¸€ç•ªå˜ç´”ãªä¾‹ã€WordCountã‚’è¨˜è¿°ã—ãŸå ´åˆã®ã‚¹ã‚¯ãƒªãƒ—ãƒˆã®æ¯”è¼ƒã‚’è¡Œã£ã¦ã„ã¾ã™ã€‚ã¨ã¦ã‚‚èˆˆå‘³æ·±ã‹ã£ãŸã®ã§ç´¹ä»‹ã—ãŸã„ã¨æ€ã„ã¾ã™ã€‚Pigs, Bees, and Elephants: A Comparison of Eight MapReduce Languages Â« Dataspora ãªãŠã€ä»Šå›žã¯RSSè³¼èªã—ã¦ã„ã‚‹DataSporaã®ãƒ–ãƒã‚°ã‹ã‚‰ç™ºè¦‹ã—ã¾ã—ãŸãŒã€ã‚¨ãƒ³ãƒˆãƒªã‚’å¯„ç¨¿ã—ãŸã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã®å€‹äººãƒ–ãƒã‚°ã‚‚å¤§å¤‰èˆˆå‘³æ·±ã„ã®ã§ã‚ªã‚¹ã‚¹ãƒ¡ã§ã™ã€‚ ã¯ã˜ã‚ã«MapReduceã‚’ç¾Žã—ãåŠ¹çŽ‡çš„ã«æ›¸ããŸã‚ã«ã€ç§ã¯æ§˜ã€…ãªè¨€èªžã‚’æ¯”è¼ƒã—ã¾ã—ãŸã€‚æžœãŸã—ã¦ãã®å‹è€…ã¯ï¼ï¼Ÿç§ã®å€‹äººãƒ–ãƒã‚°ã§ã¯çµ±è¨ˆã‚„ã‚°ãƒ©ãƒ•ã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã‚’MapReduceã§è¨˜è¿°ã™ã‚‹æ–¹æ³•ã‚’ç´¹ä»‹ã—ã€æ“¬ä¼¼ã‚³ãƒ¼ãƒ‰ã«ã‚ˆã‚‹å®Ÿè£…ã‚’ç¤ºã—ã¾ã—ãŸã€‚æ“¬ä¼¼ã‚³ãƒ¼ãƒ‰ã«ã¯2ã¤ã®å•é¡Œç‚¹ãŒã‚ã‚Šã¾ã™ï¼šèª°ã‚‚ãŒãã®å‘½ä»¤ã‚’ç†è§£ã§ãã‚‹ã¨
wata88 2013/10/18
ãƒ—ãƒã‚°ãƒ©ãƒ

mapreduce

hadoop

programming
ãƒªãƒ³ã‚¯
Springã®Cache Abstractionã«ã¤ã„ã¦
â•â”â”â”³â•®â•â”â•®â•â”â”â”â”³â”â•®â•â”â•® â•°â”«â”£â”«â”ƒâ”ƒâ•â•¯â”ƒâ•â”â•®â”ƒâ”ƒâ•°â•¯â”ƒâ”ƒ â•±â”ƒâ”ƒâ”ƒâ•°â•¯â•¯â•±â”ƒâ”ƒâ•±â”ƒâ”ƒâ•â•®â•â•®â”ƒ â•±â”ƒâ”ƒâ”ƒâ•â•®â”ƒâ•±â”ƒâ•°â”â•¯â”ƒâ”ƒâ”ƒâ”ƒâ”ƒâ”ƒ â•â”«â”£â”«â”ƒâ”ƒâ•°â”³â”«â•â”â•®â”ƒâ”ƒâ”ƒâ”ƒâ”ƒâ”ƒ â•°â”â”â”»â•¯â•°â”â”»â”»â•¯â•±â•°â”»â•¯â•°â•¯â•°â•¯ @making's tech note HomeEntriesCategoriesTagsNoteAbout HomeLatest EntriesTanzu Application Platform 1.9 (Full Profile) ã‚’EKSã«ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã™ã‚‹ãƒ¡ãƒ¢Â ðŸ—“ Updated at 2024-04-19T04:20:37Zllama-cpp-pythonã‚’ä½¿ã£ã¦Gemmaãƒ¢ãƒ‡ãƒ«ã‚’ä½¿ã£ãŸOpenAIäº’æ›ã‚µãƒ¼ãƒãƒ¼ã‚’èµ·å‹•ã—Spring AIã‹ã‚‰ã‚¢ã‚¯ã‚»ã‚¹ã™ã‚‹Â ðŸ—“ Updated at 2024-02-25T09:05:48ZKubernetesã‚¯ãƒ©ã‚¹ã‚¿å†…ã‹ã‚‰
wata88 2013/10/18
Clojure

Hadoop
ãƒªãƒ³ã‚¯
1