[B! Hadoop] kamatama_41ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

kamatama_41 id:kamatama_41

Hadoopã«é–¢ã™ã‚‹kamatama_41ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (18)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Apache SparkãŒã‚¹ãƒ«ãƒ¼ãƒ—ãƒƒãƒˆã¨ãƒ¬ã‚¤ãƒ†ãƒ³ã‚·ã‚’ä¸¡ç«‹ã•ã›ãŸä»•çµ„ã¿ã¨æœ€æ–°å‹•å‘ã‚’ã€Sparkã‚³ãƒŸãƒƒã‚¿ã¨ãªã£ãŸNTTãƒ‡ãƒ¼ã‚¿çŒ¿ç”°æ°ã«èžã„ãŸï¼ˆå‰ç·¨ï¼‰
Apache SparkãŒã‚¹ãƒ«ãƒ¼ãƒ—ãƒƒãƒˆã¨ãƒ¬ã‚¤ãƒ†ãƒ³ã‚·ã‚’ä¸¡ç«‹ã•ã›ãŸä»•çµ„ã¿ã¨æœ€æ–°å‹•å‘ã‚’ã€Sparkã‚³ãƒŸãƒƒã‚¿ã¨ãªã£ãŸNTTãƒ‡ãƒ¼ã‚¿çŒ¿ç”°æ°ã«èžã„ãŸï¼ˆå‰ç·¨ï¼‰ æœ€è¿‘ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿å‡¦ç†åŸºç›¤ã¨ã—ã¦æ€¥é€Ÿã«æ³¨ç›®ã‚’é›†ã‚ã¦ã„ã‚‹ã®ãŒã€ŒApache Sparkã€ã§ã™ã€‚ Sparkã¯ã€Hadoopã¨æ¯”è¼ƒã•ã‚Œã‚‹ã“ã¨ã‚‚å¤šãã€Hadoopã‚ˆã‚Šã‚‚é«˜é€Ÿã‹ã¤é«˜æ©Ÿèƒ½ãªåˆ†æ•£å‡¦ç†åŸºç›¤ã ã¨è¨€ã‚ã‚Œã¦ã„ã¾ã™ã€‚Sparkã¨ã¯ã„ã£ãŸã„ã€ã©ã®ã‚ˆã†ãªã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ãªã®ã§ã—ã‚‡ã†ã‹ï¼Ÿ ä»Šå¹´6æœˆã«Sparkã®ã‚³ãƒŸãƒƒã‚¿ã«å°±ä»»ã—ãŸNTTãƒ‡ãƒ¼ã‚¿ã®çŒ¿ç”°æµ©è¼”æ°ã«èžãã¾ã—ãŸã€‚ ä»¥ä¸‹ã¯çŒ¿ç”°æ°ã‹ã‚‰ä¼ºã£ãŸSparkã®ç´¹ä»‹ã‚’ã¾ã¨ã‚ãŸã‚‚ã®ã§ã™ã€‚ã¾ãŸã€å¾Œç·¨ã§ã¯çŒ¿ç”°æ°ãŒã‚³ãƒŸãƒƒã‚¿ã«ãªã£ãŸçµŒç·¯ãªã©ã‚‚ã‚¤ãƒ³ã‚¿ãƒ“ãƒ¥ãƒ¼ã—ã¾ã—ãŸã€‚ Hadoopã§ã¯è¤‡é›‘ãªå‡¦ç†ã«æ™‚é–“ãŒã‹ã‹ã‚‹ Sparkã¨ã¯ãªã«ã‹ã®å‰ã«ã€ã¾ãšã¯Hadoopã®è©±ã‹ã‚‰å§‹ã‚ã•ã›ã¦ãã ã•ã„ã€‚ Hadoopã¨ã¯ã€ã–ã£ãã‚Šè¨€ã†ã¨åˆ†æ•£å‡¦ç†ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã€Œ
kamatama_41 2015/08/05
Hadoop

spark
ãƒªãƒ³ã‚¯
Sparkã‚’ç”¨ã„ãŸãƒ’ã‚™ãƒƒã‚¯ã‚™ãƒ†ã‚™ãƒ¼ã‚¿è§£æž ã€œ å‰ç·¨ ã€œ
9. MapReduceã§Word Count 9 Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua Ut enim ad minim veniam, quis nostrud exercitation ullamco lorem 2 lorem 1 ipsum 1 dolor 1 lorem 1 lorem 1 sit 1 amet 1 consect.. 1 do 1 eiusmod 1 tempor 1 adipisic.. 1 elit 1 sed 1 ipsum 1 ipsum 1 ipsum 1 dolor 1 ipsum 3 sit 1 sit 1 sit 1 dolor 1 si
kamatama_41 2015/04/24
spark

hadoop
ãƒªãƒ³ã‚¯
Building Hadoop Source Jars
kamatama_41 2015/01/27
Hadoop

maven

java
ãƒªãƒ³ã‚¯
DMM inside
ãªã›ã‚™DMMã‹ã‚™web3ã«å‚å…¥ã—ãŸã®ã‹ã€‚Seamoon Protocolã‹ã‚™ç›®æŒ‡ã™æ–°ãŸãªã‚¨ãƒ³ã‚¿ãƒ¡ä½“é¨“ã®æœªæ¥ã¨ã¯
kamatama_41 2015/01/07
Hadoop

ã‚ã¨ã§èªã‚€
ãƒªãƒ³ã‚¯
Eclipseã«Hadoopã®é–‹ç™ºç’°å¢ƒã‚’ä½œã‚‹ã€‚ - tetsuya_odakaã®æ—¥è¨˜
2014/6/13 æ–°ã—ã„è¨˜äº‹ãŒã‚ã‚Šã¾ã™ã€‚ ã€ŒEclipseã§Hadoop2.4ã®é–‹ç™ºç’°å¢ƒã‚’ä½œã‚‹ï¼†ãƒ¯ãƒ¼ãƒ‰ã‚«ã‚¦ãƒ³ãƒˆã®ãƒ—ãƒã‚°ãƒ©ãƒ ã‚’ä½œæˆã™ã‚‹ã€‚ã€ - Hadoopã‚’å®Ÿé¨“ã™ã‚‹ã«ã‚ãŸã£ã¦ã€ãƒ—ãƒã‚°ãƒ©ãƒ ãŒä½œã‚Œã‚‹ç’°å¢ƒã‚’æ§‹ç¯‰ã—ãŸã€‚ æ§‹ç¯‰ã«ã¯ã€ã€Œ30 åˆ†ã§ NetBeans ã‚’ä½¿ã£ã¦ Hadoop ã®ãƒ—ãƒã‚°ãƒ©ãƒ ã‚’ä½œã£ã¦ã¿ã‚‹ã€ ã‚’å‚è€ƒã«ã•ã›ã¦ã„ãŸã ã„ãŸã€‚ãŸã ã—ã€æ…£ã‚Œã¦ã„ã‚‹Eclipseï¼ˆJunoï¼‰ã‚’ä½¿ã†ã“ã¨ã«ã—ãŸã€‚ é–‹ç™ºæ©Ÿã¯Mac Book Pro (OSX Lion)ã€‚ ã¾ãšã€Eclipseã®æœ¬å®¶ãƒšãƒ¼ã‚¸ã‚ˆã‚Šã€Eclipse Juno JEEã‚’ãƒ€ã‚¦ãƒ³ãƒãƒ¼ãƒ‰ã—ã¦ã€æŠµå½“ãªå ´æ‰€ã«è§£å‡ã—ã€wordbenchã‚’é–‹ãã¨ã“ã‚ã¾ã§ã‚„ã‚‹ã€‚ æ¬¡ã«hadoopã®æœ¬å®¶ã‚µã‚¤ãƒˆã‚ˆã‚Šã€hadoop-1.1.2ã‚’ãƒ€ã‚¦ãƒ³ãƒãƒ¼ãƒ‰ã—ã€ãƒ›ãƒ¼ãƒ ãƒ‡ã‚£ãƒ¬ã‚¯ãƒˆãƒªã«è§£å‡ã€ä»¥ä¸‹ã®ã‚ˆã†ãªã‚·ãƒ³ãƒœãƒªãƒƒã‚¯ãƒªãƒ³ã‚¯ã‚’å¼µã£ãŸã€‚ Java ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã®ä½œæˆ javaãƒ‘ãƒ¼ã‚¹ãƒšã‚¯ãƒ†
kamatama_41 2014/12/17
Hadoop

apache

eclipse

é–‹ç™ºç’°å¢ƒ
ãƒªãƒ³ã‚¯
Apache Sparkã§ã‚«ã‚¹ã‚¿ãƒ Streamingã™ã‚‹ - ã‚‚ã‚‡ã‚‚ã¨ã®æŠ€è¡“ãƒŽãƒ¼ãƒˆ
ã¯ã˜ã‚ã« Spark, SQL on Hadoop etc. Advent Calendar 2014 - Qiita 3æ—¥ç›®ã®è¨˜äº‹ã§ã™ã€‚ Sparkã§ã‚«ã‚¹ã‚¿ãƒ Streamingã™ã‚‹æ–¹æ³•ã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ Twitterã‚„Flumeãªã©ã®Spark Streamingã®æ´»ç”¨ä¾‹ãŒä¸‹è¨˜ã«ã‚ã‚Šã¾ã™ã€‚ spark/examples/src/main/scala/org/apache/spark/examples/streaming at master Â· apache/spark Â· GitHub spark/external at master Â· apache/spark Â· GitHub ã“ã‚Œã‚‰ã¯ã€ã„ã‚ã„ã‚åˆ©ç”¨ã§ããã†ã§ã™ãã€‚ ä¸€æ–¹ã§ã€ã‚ªãƒªã‚¸ãƒŠãƒ«ã®Streamingå‡¦ç†ã‚’è¡Œã„ãŸã„å ´åˆã«ã¯ã€ SparkãŒæä¾›ã™ã‚‹Receiverã‚¯ãƒ©ã‚¹ã‚’æ‹¡å¼µã™ã‚‹å¿…è¦ãŒã‚ã‚Šã¾ã™ã€‚ ã“ã®è¨˜äº‹ã§ã¯ã€Receiverã‚¯ãƒ©ã‚¹
kamatama_41 2014/12/13
hadoop

spark

netty
ãƒªãƒ³ã‚¯
MRUnitã§Hadoop MapReduceã®è©¦é¨“ã‚’ - Taste of Tech Topics
ã“ã‚“ã«ã¡ã¯ã€‚ è½åˆã§ã™ã€‚ MRUnit ã‚’ä½¿ã£ãŸã€ Hadoop MapReduce ã®ãƒ†ã‚¹ãƒˆæ–¹æ³•ã‚’ã”ç´¹ä»‹ã—ã¾ã™ã€‚ MRUnitã¨ã¯ MRUnitã¯ã€Hadoop MapReduce ã®ãƒ†ã‚¹ãƒˆã®ãŸã‚ã®ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã§ã™ã€‚ ã“ã‚Œã‚’ä½¿ã†ã¨ã€Hadoop MapReduce ã® JUnitãƒ†ã‚¹ãƒˆã‚’è¡Œã†ã“ã¨ãŒã§ãã¾ã™ã€‚ Context ã‚’è‡ªåˆ†ã§ä½œã‚‹å¿…è¦ã‚‚ãªã„ãŸã‚ã€ Mapper ã‚„ Reducer ã® in ã¨ out ã®ç¢ºèªãŒç°¡å˜ã«ã§ãã¾ã™ã€‚ é–‹ç™ºç’°å¢ƒã‚’ä½œã‚‹ æœ¬ãƒ–ãƒã‚°åŸ·ç†æ™‚ç‚¹ã§æœ€æ–°ã®ã€Cloudera ã®ãƒ‡ã‚£ã‚¹ãƒˆãƒªãƒ“ãƒ¥ãƒ¼ã‚·ãƒ§ãƒ³ CDH3u1 ã‚’ä½¿ã„ã¾ã—ãŸã€‚ ä»¥ä¸‹ã®ã‚µã‚¤ãƒˆã§Tarballï¼ˆhadoop-0.20.2-cdh3u1.tar.gzï¼‰ã‚’ãƒ€ã‚¦ãƒ³ãƒãƒ¼ãƒ‰ã—è§£å‡ã—ã¾ã™ã€‚ https://ccp.cloudera.com/display/SUPPORT/Downloads è§£å‡ã—ãŸã‚‰ã€libã®ä¸‹ã¨ã€h
kamatama_41 2014/12/10
Hadoop

MapReduce

java
ãƒªãƒ³ã‚¯
HDFS HA ã‚»ãƒŸãƒŠãƒ¼ #hadoop
2013/05/30ã«é–‹å‚¬ã—ãŸã€HDFS HA(High Availability: é«˜å¯ç”¨æ€§)ã‚»ãƒŸãƒŠãƒ¼ã®è³‡æ–™ã§ã™ã€‚åŒã˜ãã”ç™»å£‡é ‚ã„ãŸã€æ ªå¼ä¼šç¤¾ã‚µã‚¤ãƒãƒ¼ã‚¨ãƒ¼ã‚¸ã‚§ãƒ³ãƒˆã®ä¸ŠåŽŸèª æ§˜ã®è³‡æ–™ã¯â†“ã§ã™ã€‚ http://www.slideshare.net/makotouehara39/cl-st-20130530nnhaRead less
kamatama_41 2014/12/10
hadoop

HA
ãƒªãƒ³ã‚¯
SSSSLIDE
kamatama_41 2014/11/14
bigquery

hadoop

ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿

ãƒ‡ãƒ¼ã‚¿ãƒžã‚¤ãƒ‹ãƒ³ã‚°
ãƒªãƒ³ã‚¯
Googleã®è™Žã®åã€ŒBigQueryã€ã‚’Fluentdãƒ¦ãƒ¼ã‚¶ãƒ¼ãŒä½¿ã‚ãªã„ç†ç”±ãŒãªããªã£ãŸç†ç”± #gcpja - Qiita
From Fluentd Meetupã«è¡Œã£ã¦ãã¾ã—ãŸ ã“ã‚Œã‚’èªã‚“ã æ™‚ã€BigQueryã®æ¤œç´¢ã‚¹ãƒ”ãƒ¼ãƒ‰ã«ã¤ã„ã¦ã¡ã‚‡ã£ã¨è£œè¶³ã—ãŸããªã£ãŸã€‚ç¢ºã‹ã«Fluentd Meetupã®ãƒ‡ãƒ¢ã§ã¯9å„„ä»¶ã‚’7ç§’ç¨‹åº¦ã§æ¤œç´¢ã—ã¦ã„ãŸãŒã€BigQueryã®çœŸã®å®ŸåŠ›ã¯ã“ã‚Œã‚ˆã‚Š1ã€œ2ã‚±ã‚¿ä¸Šã ã‹ã‚‰ã ã€‚ã¡ã‚‡ã£ã¨æ‰‹å…ƒã§å°‘ã—å¤§ãã‚ã®ãƒ†ãƒ¼ãƒ–ãƒ«ã§è©¦ã—ã¦ã¿ãŸã‚‰ã€120å„„è¡Œã®æ£è¦è¡¨ç¾ãƒžãƒƒãƒä»˜ãé›†è¨ˆãŒ5ç§’ã§å®Œäº†ã—ãŸã€‚è«–ã‚ˆã‚Šè¨¼æ‹ ã§ã€ãƒ‡ãƒ¢ãƒ“ãƒ‡ã‚ªï¼ˆ1åˆ†16ç§’ï¼‰ã‚’ä½œã£ã¦ã¿ãŸï¼š From The Speed of Google BigQuery ã“ã‚Œã¯é€Ÿã™ãŽã‚‹ã€‚ä½•ã‹ã®ã‚¤ãƒ³ãƒã‚ã§ã‚ã‚‹ï¼ˆæœ€åˆã«ãƒ‡ãƒ¢ã‚’è¦‹ãŸæ™‚ãã†æ€ã£ãŸï¼‰ã€‚æ£è¦è¡¨ç¾ã‚’ã„ã‚ã„ã‚å¤‰ãˆã¦ã¿ã¦ã‚‚ã‚¹ãƒ”ãƒ¼ãƒ‰ã¯å¤‰ã‚ã‚‰ãªã„ã€‚ã¤ã¾ã‚Šã€ã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ã‚’äº‹å‰æ§‹ç¯‰ã§ããªã„ã‚¯ã‚¨ãƒªã«å¯¾ã—ã¦ã“ã®ã‚¹ãƒ”ãƒ¼ãƒ‰ãªã®ã§ã‚ã‚‹ã€‚ ä¾¡æ ¼ã‚‚å®‰ã„ã€‚ã•ã™ãŒã«120å„„è¡Œã®ã‚¯ã‚¨ãƒªã¯1å›žã§200å††ã‚‚ã‹ã‹ã£ã¦æ°—è»½ã«å®Ÿè¡Œã§ããªã•ãã†ã§ã‚ã‚‹ãŒã€1.2å„„
kamatama_41 2014/05/19
fluentd

google

bigquery

bigdata

hadoop
ãƒªãƒ³ã‚¯
Apache Mahoutã®ä½¿ã„æ–¹ï¼šãƒ†ã‚ã‚¹ãƒˆåˆ†é¡žã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã‚’æ´»ç”¨ã™ã‚‹
ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿æ´»ç”¨ï¼šãã®åˆ†æžå®Ÿè£…ã¨ã—ã¦æ³¨ç›®ã•ã‚Œã‚‹Mahout é•·å¹´è“„ç©ã—ãŸä¼æ¥å†…ãƒ‡ãƒ¼ã‚¿ã‚„ã€ã‚½ãƒ¼ã‚·ãƒ£ãƒ«ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã‚µãƒ¼ãƒ“ã‚¹ã€ã‚»ãƒ³ã‚µç«¯æœ«ã‹ã‚‰é›†ã‚ã‚‰ã‚Œã‚‹è†¨å¤§ãªãƒ‡ãƒ¼ã‚¿ã‚’æ´»ç”¨ã—ã€ä¼æ¥ã«ãŠã‘ã‚‹åˆ©ç›Šå‘ä¸Šã‚„ã‚³ã‚¹ãƒˆå‰Šæ¸›ãªã©ã«æ´»ç”¨ã™ã‚‹å‹•ããŒæ´»ç™ºã«ãªã£ã¦ãã¾ã—ãŸã€‚ ãƒ‡ãƒ¼ã‚¿ã®åˆ†æžæ‰‹æ®µã¨ã—ã¦æœ€è¿‘ã¨ã¿ã«æ³¨ç›®ã•ã‚Œã¦ã„ã‚‹æŠ€è¡“ã¨ã—ã¦ã€Œæ©Ÿæ¢°å¦ç¿’ã€ãŒã‚ã‚Šã¾ã™ã€‚å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã®å‡¦ç†ã‚’å¾—æ„ã¨ã™ã‚‹å¤§è¦æ¨¡åˆ†æ•£å‡¦ç†åŸºç›¤ã€ŒApache Hadoopã€ã®å¼·ã¿ã‚’ç”Ÿã‹ã—ã€ç°¡å˜ã«æ©Ÿæ¢°å¦ç¿’ã‚’è¡Œã†ãŸã‚ã®ãƒ©ã‚¤ãƒ–ãƒ©ãƒªãŒã€ã€ŒApache Mahoutã€ï¼ˆä»¥ä¸‹ã€Mahoutï¼‰ã§ã™ã€‚ æœ¬ç¨¿ã§ã¯Mahoutã‚’ç”¨ã„ãŸãƒ‡ãƒ¼ã‚¿åˆ†æžã®ä¾‹ã¨ã—ã¦ã€Œæ–‡æ›¸åˆ†é¡žã€ã‚’å–ã‚Šä¸Šã’ã¾ã™ã€‚ãƒžã‚·ãƒ³ã‚’ç”¨ã„ã¦åˆ†æžå®Ÿè¡Œã™ã‚‹éš›ã®æ‰‹é †ã‚„é™¥ã‚ŠãŒã¡ãªãƒã‚¤ãƒ³ãƒˆã€ãƒãƒ¥ãƒ¼ãƒ‹ãƒ³ã‚°æ–¹æ³•ã®ä¸€ä¾‹ã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ Mahoutã¨ã¯ï¼Ÿ Mahoutã¨ã¯Apache Software FoundationãŒå…¬é–‹ã—ã¦ã„ã‚‹OSSã®æ©Ÿæ¢°
kamatama_41 2014/01/03
Mahout

hadoop

æ©Ÿæ¢°å¦ç¿’

ãƒ†ã‚ã‚¹ãƒˆãƒžã‚¤ãƒ‹ãƒ³ã‚°
ãƒªãƒ³ã‚¯
Treasure Data - naoyaã®ã¯ã¦ãªãƒ€ã‚¤ã‚¢ãƒªãƒ¼
å°‘ã—å‰ã«ãƒã‚°ã®è©±ã‚’æ›¸ã„ãŸ http://d.hatena.ne.jp/naoya/20130219/1361262854 ã¨ãã«ã€Treasure Data ã«ã¤ã„ã¦ã¯å¾Œæ—¥ã«ã‚‚ã†å°‘ã—è©³ç´°ã«æ›¸ãã¨è¨€ã£ãŸã®ã§æ›¸ãã¨ã—ã‚ˆã†ã€‚ è¿‘é ƒ Treasure Data (ä»¥ä¸‹ã€æ™‚æŠ˜ TD) ã¨ã„ã†åå‰ã‚’ã¡ã‚‰ã»ã‚‰èžã„ãŸã“ã¨ãŒã‚ã‚‹äººã¯å¤šã„ã®ã§ã¯ãªã„ã‹ã¨æ€ã„ã¾ã™ã€‚ã€Œãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã®ã‚¯ãƒ©ã‚¦ãƒ‰ã‚µãƒ¼ãƒ“ã‚¹ã§ã‚ã‚‹ã€ã¨ã‹ã€Œæ—¥æœ¬äººãŒå‰µæ¥ã—ãŸã‚·ãƒªã‚³ãƒ³ãƒãƒ¬ãƒ¼ã®ãƒ™ãƒ³ãƒãƒ£ãƒ¼ã€ã€ã‚ã‚‹ã„ã¯ Yahoo! å‰µæ¥è€…ã® Jerry Yang ãŒæŠ•è³‡ã—ãŸã¨ã‹ã€Fluentd ã¨ä½•ã‹é–¢ä¿‚ãŒã‚ã‚‹ã¨ã„ã£ãŸæ–‡è„ˆãªã©ãªã©ã€‚ ã‘ã©ã€å…·ä½“çš„ã« Treasure Data ãŒã©ã†ã„ã†ã‚µãƒ¼ãƒ“ã‚¹ã§ã€ã©ã†ã„ã†æ©Ÿèƒ½ã‚’æŒã£ã¦ã„ã¦ã€ã©ã‚“ãªå ´é¢ã§åˆ©ç”¨ã•ã‚Œã‚‹ã‚‚ã®ãªã®ã‹ã¯ã¾ã ã‚ã¾ã‚Šè‰¯ãçŸ¥ã‚‰ã‚Œã¦ã„ãªã„ã‹ã‚‚ã—ã‚Œãªã„ï½¥ï½¥ï½¥ã‚ˆã†ã«ã‚‚è¦‹ãˆã‚‹ã€‚ä»Šæ—¥ã¯ãã®è¾ºã‹ã‚‰å°‘ã—ç´¹ä»‹ã—ã¦ã„ã“ã†ã‹ãªã¨æ€ã†ã€‚
kamatama_41 2013/03/23
ãƒã‚°

hadoop

bigdata

fluentd

MapReduce

Treasure Data
ãƒªãƒ³ã‚¯
HBase at Ameba
1. Developers Summit HBase at Ameba ï½žAmebaã®ãŠã‘ã‚‹HBaseã®å°Žå…¥äº‹ä¾‹ï½ž 14-A-2 éˆ´æœ¨ä¿Šè£• #devsumiA æ ªå¼ä¼šç¤¾ã‚µã‚¤ãƒãƒ¼ã‚¨ãƒ¼ã‚¸ã‚§ãƒ³ãƒˆ ã‚¢ãƒ¡ãƒ¼ãƒäº‹æ¥æœ¬éƒ¨ ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ éƒ¨é–€ Developers Summit 2013 Action ! Ameba Techno logy Laboratory 2. Developers Summit è‡ªå·±ç´¹ä»‹ â€¢â€¯ éˆ´æœ¨ ä¿Šè£•(ã™ãšã ã¨ã—ã²ã‚) â€¢â€¯ ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ â€¢â€¯ æ ªå¼ä¼šç¤¾ã‚µã‚¤ãƒãƒ¼ã‚¨ãƒ¼ã‚¸ã‚§ãƒ³ãƒˆ â€“â€¯ ã‚¢ãƒ¡ãƒ¼ãƒäº‹æ¥æœ¬éƒ¨ â€“â€¯ ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ éƒ¨é–€ â€“â€¯ Ameba Techno logy Laboratory â€¢â€¯ Hadoopã‚’ä½¿ã£ãŸãƒã‚°è§£æžåŸºç›¤ã®é–‹ç™ºãƒ»é‹ç”¨ â€¢â€¯ HBaseã‚’ã¤ã‹ã£ãŸåŸºç›¤ã‚·ã‚¹ãƒ†ãƒ ã®é–‹ç™ºãƒ»é‹ç”¨ â€¢â€¯ Twi;er @brfrn169 Developers Summit
kamatama_41 2013/02/25
ameba

ã‚µã‚¤ãƒãƒ¼ã‚¨ãƒ¼ã‚¸ã‚§ãƒ³ãƒˆ

hBase

hadoop
ãƒªãƒ³ã‚¯
è©¦ã™ã®ãŒé›£ã—ã„â€•æ©Ÿæ¢°å¦ç¿’ã®å¸¸è˜ã¯Mahoutã§å¤‰ã‚ã‚‹
ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿æ™‚ä»£â€•ãªãœã€ã„ã¾æ©Ÿæ¢°å¦ç¿’ãªã®ã‹ Apache Hadoopï¼ˆä»¥ä¸‹ã€Hadoopï¼‰ã®ç™»å ´ã§ã€ä»Šã¾ã§æ¨ã¦ã¦ã„ãŸãƒ‡ãƒ¼ã‚¿ã€è²¯ã‚ã‚‹ã ã‘ã§å‡¦ç†ã—ãã‚Œãªã‹ã£ãŸãƒ‡ãƒ¼ã‚¿ã‚’æ´»ç”¨ã§ãã‚‹ã‚ˆã†ã«ãªã‚Šã¾ã—ãŸã€‚ æ´»ç”¨æ‰‹æ®µã¨ã—ã¦æœ€è¿‘ã¨ã¿ã«æ³¨ç›®ã•ã‚Œã¦ã„ã‚‹æŠ€è¡“ãŒã€Œæ©Ÿæ¢°å¦ç¿’ã€ã§ã‚ã‚Šã€Hadoopã®å¼·ã¿ã‚’ç”Ÿã‹ã—ç°¡å˜ã«æ©Ÿæ¢°å¦ç¿’ã‚’è¡Œã†ãŸã‚ã®ãƒ©ã‚¤ãƒ–ãƒ©ãƒªãŒã€ã€ŒApache Mahoutã€ï¼ˆä»¥ä¸‹ã€Mahoutï¼‰ã§ã™ã€‚ æœ¬ç¨¿ã§ã¯Mahoutã‚’å‹•ã‹ã—ã¦ã¿ã‚‹ã“ã¨ã§ã€æ©Ÿæ¢°å¦ç¿’ã®å¸¸è˜ã‚’èº«ã«ä»˜ã‘ã¾ã™ã€‚ ãã‚‚ãã‚‚ã€æ©Ÿæ¢°å¦ç¿’ã¨ã¯ï¼Ÿ æ©Ÿæ¢°å¦ç¿’ã¨ã¯ã€ä¸€å®šã®ãƒ‡ãƒ¼ã‚¿ã‚’ã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ã‚¿ãƒ»ãƒ—ãƒã‚°ãƒ©ãƒ ã«ã€Œå¦ç¿’ã€ã•ã›ï¼ˆã™ãªã‚ã¡ã€ãã®ãƒ‡ãƒ¼ã‚¿ã«æ½œã‚€ãƒ‘ã‚¿ãƒ¼ãƒ³ã‚„è¦å‰‡æ€§ã‚’è¡¨ã™ã€Œãƒ¢ãƒ‡ãƒ«ã€ã‚’è‡ªå‹•çš„ã«æ§‹ç¯‰ã•ã›ï¼‰ã€ä»–ã®ãƒ‡ãƒ¼ã‚¿ã«ãã®ãƒ¢ãƒ‡ãƒ«ã‚’é©ç”¨ã™ã‚Œã°ã€ã‚ãŸã‹ã‚‚äººé–“ã®ã‚ˆã†ã«è¤‡é›‘ã§æŸ”è»Ÿãªåˆ¤æ–ãŒè¡Œãˆã‚‹ã‚ˆã†ã«ã™ã‚‹ã¨ã„ã†è©¦ã¿ã§ã™ã€‚ æ©Ÿæ¢°å¦ç¿’ã‚’ãƒ“ã‚¸ãƒã‚¹ã«æ´»ç”¨ã—ãŸä¾‹ã¯ã€ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰ï¼ˆãƒ¦ãƒ¼ã‚¶ãƒ¼ã‚„å•†å“
kamatama_41 2012/08/03
ã‚ã¨ã§èªã‚€

æ©Ÿæ¢°å¦ç¿’

Hadoop

Mahout

bigdata

Java

ã‚ã¨ã§èªã‚€
ãƒªãƒ³ã‚¯
å°Žå…¥äº‹ä¾‹ã‹ã‚‰ITè£½å“ãƒ»ã‚µãƒ¼ãƒ“ã‚¹ã‚’æŽ¢ã™ï½œã‚ãƒ¼ãƒžãƒ³ã‚ºãƒãƒƒãƒˆ
åŸºå¹¹ç³»ã‚·ã‚¹ãƒ†ãƒ ERP ä¼šè¨ˆã‚·ã‚¹ãƒ†ãƒ é›»åå¸³ç¥¨ã‚·ã‚¹ãƒ†ãƒ ãƒ¯ãƒ¼ã‚¯ãƒ•ãƒãƒ¼ å‹¤æ€ ç®¡ç†ã‚·ã‚¹ãƒ†ãƒ ã‚‚ã£ã¨è¦‹ã‚‹ æƒ…å ±å…±æœ‰ã‚·ã‚¹ãƒ†ãƒ ãƒ»ã‚³ãƒŸãƒ¥ãƒ‹ã‚±ãƒ¼ã‚·ãƒ§ãƒ³ãƒ„ãƒ¼ãƒ« ã‚°ãƒ«ãƒ¼ãƒ—ã‚¦ã‚§ã‚¢ Webä¼šè° ãƒ†ãƒ¬ãƒ“ä¼šè°/ãƒ“ãƒ‡ã‚ªä¼šè° ãƒ•ã‚¡ã‚¤ãƒ«å…±æœ‰ æ–‡æ›¸ç®¡ç† ã‚‚ã£ã¨è¦‹ã‚‹ æƒ…å ±ã‚·ã‚¹ãƒ†ãƒ SFA CRM ã‚³ãƒ¼ãƒ«ã‚»ãƒ³ã‚¿ãƒ¼/CTI BPM PLM ã‚‚ã£ã¨è¦‹ã‚‹ ãƒ¡ãƒ¼ãƒ« é›»åãƒ¡ãƒ¼ãƒ« ãƒ¡ãƒ¼ãƒ«ã‚»ã‚ãƒ¥ãƒªãƒ†ã‚£ ãƒ¡ãƒ¼ãƒ«ã‚¢ãƒ¼ã‚«ã‚¤ãƒ– ãã®ä»–ãƒ¡ãƒ¼ãƒ«é–¢é€£ ã‚‚ã£ã¨è¦‹ã‚‹ ã‚¨ãƒ³ãƒ‰ãƒã‚¤ãƒ³ãƒˆã‚»ã‚ãƒ¥ãƒªãƒ†ã‚£ ã‚¢ãƒ³ãƒã‚¦ã‚¤ãƒ«ã‚¹ æš—å·åŒ– èªè¨¼ IDç®¡ç† ãƒ¡ãƒ¼ãƒ«ã‚»ã‚ãƒ¥ãƒªãƒ†ã‚£ ã‚‚ã£ã¨è¦‹ã‚‹ ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã‚»ã‚ãƒ¥ãƒªãƒ†ã‚£ ãƒ•ã‚¡ã‚¤ã‚¢ã‚¦ã‚©ãƒ¼ãƒ« WAF IPS UTM ã‚»ã‚ãƒ¥ãƒªãƒ†ã‚£è¨ºæ– ã‚‚ã£ã¨è¦‹ã‚‹ é‹ç”¨ç®¡ç† çµ±åˆé‹ç”¨ç®¡ç† ITè³‡ç”£ç®¡ç† ã‚µãƒ¼ãƒãƒ¼ç®¡ç† ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ç®¡ç† çµ±åˆãƒã‚°ç®¡ç† ã‚‚ã£ã¨è¦‹ã‚‹ ãƒãƒƒã‚¯ã‚¢ãƒƒãƒ— ãƒãƒƒã‚¯ã‚¢ãƒƒãƒ—ãƒ„ãƒ¼ãƒ« ãƒãƒƒã‚¯ã‚¢ãƒƒãƒ—ã‚µãƒ¼ãƒ“ã‚¹ ãƒ†ãƒ¼ãƒ—ãƒãƒƒã‚¯ã‚¢ãƒƒãƒ— ãã®ä»–ãƒãƒƒã‚¯ã‚¢ãƒƒãƒ—é–¢é€£ ã‚‚ã£
kamatama_41 2012/05/21
jubatus

hadoop
ãƒªãƒ³ã‚¯
NTTãƒ‡ãƒ¼ã‚¿ã®Hadoopå ±å‘Šæ›¸ãŒã™ã”ã‹ã£ãŸ - ç§‘å¦ã¨éžç§‘å¦ã®è¿·å®®
æ¥ç•Œãƒˆãƒƒãƒ— ã®ã‚¨ãƒ³ã‚¿ãƒ¼ãƒ—ãƒ©ã‚¤ã‚º Hadoop ä¼æ¥ Cloudera ã«å…¥ç¤¾ã—ã¾ã—ãŸ http://www.cloudera.co.jp/ ä»Šå¹´ã®6æœˆã«ã€ã€Œå¹³æˆï¼’ï¼‘å¹´åº¦ ç”£å¦é€£æºã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢å·¥å¦å®Ÿè·µäº‹æ¥å ±å‘Šæ›¸ã€ã¨ã„ã†ãƒ‰ã‚ãƒ¥ãƒ¡ãƒ³ãƒˆç¾¤ãŒçµŒç”£çœã‹ã‚‰å…¬è¡¨ã•ã‚Œã¾ã—ãŸã€‚ ãã®ã†ã¡ã®ä¸€ã¤ã«ã€NTTãƒ‡ãƒ¼ã‚¿ã«å§”è¨—ã•ã‚ŒãŸHadoopã«é–¢ã™ã‚‹å®Ÿè¨¼å®Ÿé¨“ã®å ±å‘Šæ›¸ãŒã‚ã‚Šã¾ã—ãŸã®ã§ã€ä»Šæ›´ãªãŒã‚‰èªã‚“ã§ã¿ã‚‹ã“ã¨ã«ã—ã¾ã—ãŸã€‚ Hadoopç•Œéšˆã®äººã¯ã‚‚ã†ã¿ã‚“ãªã¨ã£ãã«èªã‚“ã§ã‚‹ã®ã‹ã‚‚ã—ã‚Œã¾ã›ã‚“ã‘ã©ã€‚ http://www.meti.go.jp/policy/mono_info_service/joho/downloadfiles/2010software_research/clou_dist_software.pdf ã€Œé«˜ä¿¡é ¼ã‚¯ãƒ©ã‚¦ãƒ‰å®Ÿç¾ç”¨ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢é–‹ç™ºï¼ˆåˆ†æ•£åˆ¶å¾¡å‡¦ç†æŠ€è¡“ç‰ã«ä¿‚ã‚‹ãƒ‡ãƒ¼ã‚¿ã‚»ãƒ³ã‚¿ãƒ¼é«˜ä¿¡é ¼åŒ–ã«å‘ã‘ãŸå®Ÿè¨¼äº‹æ¥ï¼‰ã€ã¨ã„ã†
kamatama_41 2012/01/08
Hadoop

ã‚ã¨ã§èªã‚€
ãƒªãƒ³ã‚¯
"BigData"ã§ã¯ä½•ãŒå•é¡Œãªã®ã‹ï¼Ÿ - æ€¥ãŒã°å›žã‚Œã€é¸ã¶ãªã‚‰è¿‘é“
â€ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã§å¥‡è·¡ãŒèµ·ã“ã‚‹â€ ã¯ã„ã©ã†ã‚‚ã€‚ã¾ãšã€å€‹äººçš„ã«ã¯æ¥½å¤©çš„ãªé€²æ©å²è¦³ã«ã¯ã€ã¾ã£ãŸãçµ„ã—ãªã„ã€‚ å¾“ã£ã¦ã€çªç„¶ã«æ–°æŠ€è¡“ãŒã§ãã¦ã€ãªã‚“ã‹å‡„ã„äº‹ã«ãªã‚‹ã€ã¨ã„ã†ã®ã¯ã•ã‚‰ã«ã¾ã£ãŸãåŒæ„ã—ãªã„ã€‚ã™ã¹ã‹ã‚‰ããƒ–ãƒ¬ã‚¤ã‚¯ã‚¹ãƒ«ãƒ¼ã¯èª²é¡Œè§£æ±ºã«ã‚ˆã‚Šèµ·ã“ã‚‹ã¨æ€ã£ã¦ã„ã‚‹ã®ã§ã€å•é¡Œæ„è˜ã®ãªã„ã¨ã“ã‚ã«ã€ã“ã‚“ãªã‚‚ã®ã§ãã¾ã—ãŸçš„ãªç™ºæƒ³ã¯ã€åŸºæœ¬çš„ã«ãƒ—ãƒ©ã‚¹ã«ãªã‚‰ãªã„ã“ã¨ãŒå¤šã„ã¨æ€ã£ã¦ã„ã‚‹ã€‚ç¾çŠ¶ã®ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ãƒ–ãƒ¼ãƒ ã¯2011å¹´ã®ç§‹å£ç¾åœ¨ã¯å®Œå…¨ã«ãƒã‚¤ãƒ—ã«ãªã£ã¦ãŠã‚Šã€ãƒãƒ–ãƒ«ã¨è¨€ã£ã¦ã‚‚ã„ã„ã¨æ€ã†ã€‚å°è±¡ã¨ã—ã¦ã€åæ•°å¹´å‰ã®ãƒŠãƒŽãƒ†ã‚¯ãƒ»ãƒ–ãƒ¼ãƒ ã«ä¼¼ã¦ã„ã‚‹ã€‚ ã¨ã¯ã„ãˆã€éŽåº¦ã®æœŸå¾…ã¨ã„ã†å´é¢ã‚’é™¤ã‘ã°ã€ãã‚Œãªã‚ŠåŠ¹æžœã‚‚ã‚ã‚‹éƒ¨åˆ†ã‚‚ã‚ã‚Šã€â€ãã“ã ã‘â€ã‚’è¦‹ã¦ã„ã‘ã°ãã‚Œãªã‚Šã«åŠ¹æžœã¯ã‚ã‚‹ï¼ˆã¨æ€ã†ï¼‰ã€‚å¤§ä½“ã«ãŠã„ã¦ã€ä»Šå¾Œã¯ä»¥ä¸‹ã®äºŒã¤ã®ãƒ¦ãƒ¼ã‚¹ã‚±ãƒ¼ã‚¹ãƒ»ã‚«ãƒ†ã‚´ãƒªãƒ¼ã«é›†ç´„ã•ã‚Œã‚‹ã¨æ€ã†ã€‚ã™ãªã‚ã¡ã€ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã®æ‹ ã‚Šæ‰€ã¯ã¾ãšã‚‚ã£ã¦ä»¥ä¸‹ã®ï¼’ç‚¹ã ã€‚ 1 Webã®ãƒã‚°è§£æž ã¨ã„ã†ã‹
kamatama_41 2011/12/11
hadoop

bigdata
ãƒªãƒ³ã‚¯
Hadoopå…¥é–€
2. ã¨ã¯ï¼Ÿ â€¢ Googleã®åŸºç›¤ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã®ã‚¯ãƒãƒ¼ãƒ³ â€“ Google File System, MapReduce â€¢ Yahoo Research ã® Doug Cuttingæ°ãŒé–‹ç™º â€“ å…ƒã€…ã¯Luceneã®ã‚µãƒ–ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆ â€“ Apache Project â€¢ Javaã§è¨˜è¿°! 3. Googleé–¢é€£ å‚è€ƒè«–æ–‡ & ã‚¹ãƒ©ã‚¤ãƒ‰ â€¢ The Google File System â€“ Sanjay Ghemawat, Howard Gobioff, and Shu-Tak Leong, SOSP 2003 â€¢ MapReduce: Simplified Data Processing on Large Clusters â€“ Jeffrey Dean and Sanjay Ghemawat, SOSP 2004 â€¢ Parallel Architectures and Compila
kamatama_41 2011/04/22
ã‚ã¨ã§èªã‚€

Hadoop

å…¥é–€
ãƒªãƒ³ã‚¯
1