[B! hadoop] nantanã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

nantan id:nantan

hadoopã«é–¢ã™ã‚‹nantanã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (40)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

ä½•ç•ªç…Žã˜ã‹åˆ†ã‹ã‚‰ãªã„ã‘ã©é›†åˆçŸ¥ãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°ã‚’PHPã§ã‚„ã£ã¦ã¿ãŸ ãã®7ã€ŒéšŽå±¤çš„ã‚¯ãƒ©ã‚¹ã‚¿ãƒªãƒ³ã‚°ã«ã‚ˆã‚Šã‚°ãƒ«ãƒ¼ãƒ—ã‚’è¦‹ã¤ã‘å‡ºã™ã€ - Stellaqua - ï¼´ï¼¯ï¼ã®æŠ€è¡“æ—¥è¨˜
å‰å›žã¯è©±ã ã‘ã§çµ‚ã‚ã£ã¦ã—ã¾ã£ãŸã®ã§ã€ä»Šå›žã¯ã‚½ãƒ¼ã‚¹ã‚³ãƒ¼ãƒ‰ä¸å¿ƒã§ã™ã€‚ ã‚¢ã‚¤ãƒ†ãƒ åŒå£«ã®è·é›¢ã®è¨ˆç®—ã«å¿…è¦ãªæƒ…å ±ã‚’å‡ºåŠ›ã™ã‚‹Reducerã‚’å®Ÿè£…ã™ã‚‹ ã¨ã„ã†è¨³ã§æ—©é€Ÿã§ã™ãŒã€å‰å›žå»¶ã€…ã¨è©±ã‚’ã—ã¦ã„ãŸäº‹ã‚’Reducerã«å®Ÿè£…ã—ã¾ã™ã€‚ #!/usr/bin/php <?php require_once(dirname(dirname(__FILE__)).'/lib/HadoopStreaming/Reducer.php'); class Reducer extends HadoopStreaming_Reducer { public function reduce ( $key, $values ) { $wordcount = array(); while ( $values->has_next_value ) { list($id, $count) = explode(':', $values->cur
nantan 2012/08/20
hadoop

php
ãƒªãƒ³ã‚¯
blog.katsuma.tv
å‰å›žã€JavaScriptã§Map Reduceã®ã‚³ãƒ¼ãƒ‰ãŒæ›¸ã‘ã‚‹Hadoop Streamingã«ã¤ã„ã¦ç´¹ä»‹ã—ã¾ã—ãŸã€‚ æ¨™æº–å…¥å‡ºåŠ›ã•ãˆã‚µãƒãƒ¼ãƒˆã•ã‚Œã¦ã‚ã‚Œã°ã€ä»»æ„ã®ã‚³ãƒ¼ãƒ‰ã§Map Reduuceã®å‡¦ç†ãŒæ›¸ã‘ã‚‹ã€ã¨ã„ã†ã‚‚ã®ã§ã—ãŸãŒã€ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã¯ãã‚‚ãã‚‚é¢å€’ãã•ãŒã‚Šã€‚ã‚³ãƒ¼ãƒ‰ã‚‚æ›¸ãã®ã‚‚é¢å€’ã§ã™ã€‚ ã¨ã€ã„ã†ã‚ã‘ã§ã€ä»Šå›žã¯ã‚‚ã†ã‚³ãƒ¼ãƒ‰ã™ã‚‰æ›¸ã‹ãšã«SQLãƒ©ã‚¤ã‚¯ã§Map Reduceã§ãã‚‹Hiveã¨ã„ã†ãƒ—ãƒãƒ€ã‚¯ãƒˆã«ã¤ã„ã¦ã€ã¾ã¨ã‚ãŸã„ã¨æ€ã„ã¾ã™ã€‚ Hive Hiveã¨ã¯ã€ç°¡å˜ã«è¨€ã†ã¨Hadoopä¸Šã§å‹•ä½œã™ã‚‹RDBã®ã‚ˆã†ãªã‚‚ã®ã§ã™ã€‚ HDFSãªã©ã®åˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ä¸Šã«å˜åœ¨ã™ã‚‹ãƒ‡ãƒ¼ã‚¿ã«å¯¾ã—ã¦ã€HiveQLã¨ã„ã†SQLãƒ©ã‚¤ã‚¯ãªè¨€èªžã§æ“ä½œã§ãã¾ã™ã€‚ ã§ã€é¢ç™½ã„ã®ãŒHiveQLã®æ“ä½œã¯åŸºæœ¬çš„ã«Map Reduceã®ãƒ©ãƒƒãƒ‘ãƒ¼ã«ãªã£ã¦ã„ã‚‹ã“ã¨ã€‚ è¦ã™ã‚‹ã«ã€SELECTæ–‡å®Ÿè¡Œã™ã‚‹ã¨è£ã§Map&Reduceã®ã‚¿ã‚¹ã‚¯ãŒèµ°ã‚Šå‡º
nantan 2012/07/11
hadoop

hive

mapreduce
ãƒªãƒ³ã‚¯
Hadoop and Hive
- Facebook relies heavily on open source software like Apache Hadoop and Hive for its data warehousing and analytics needs. Hive provides an SQL-like interface to query large datasets stored in Hadoop's HDFS distributed file system using MapReduce. - Hive optimizes SQL queries by merging joins, sharing common operations between queries, and load balancing MapReduce jobs. It uses a metastore to man
nantan 2012/07/11
Facebook

hadoop

hive
ãƒªãƒ³ã‚¯
Hadoop Combinerãƒ¡ãƒ¢(Hishidama's Hadoop Combiner Memo)
Hadoopã®Combinerã‚¯ãƒ©ã‚¹ï¼ˆé›†ç´„é–¢æ•°ï¼‰ã«ã¤ã„ã¦ã€‚ ï¼ˆcombineã¯ã€Œã‚³ãƒ³ãƒã‚¤ãƒ³ã€ã¨èªã‚€ãŒã€combinerã¯ã€Œã‚³ãƒ³ãƒã‚¤ãƒŠãƒ¼ã€ã§ã„ã„ã®ã‹ãªï¼‰ Combinerã®æ¦‚è¦ Combinerã¯Mapperã®å‡ºåŠ›çµæžœã‚’ï¼ˆReducerã«æ¸¡ã™å‰ã«ï¼‰ä¸é–“é›†è¨ˆã™ã‚‹ã‚‚ã®ã€‚ Combinerã®ç›®çš„ã¯ã€â€œMapperã®å‡ºåŠ›ãƒ¬ã‚³ãƒ¼ãƒ‰ãŒå¤šããªã£ã¦ãƒãƒƒãƒ•ã‚¡ãƒ¼ã«å…¥ã‚Šåˆ‡ã‚‰ãªããªã£ãŸéš›ã«ã€ï¼ˆReducerã¨åŒæ§˜ã®ï¼‰é›†è¨ˆã‚’è¡Œã£ã¦ãƒ¬ã‚³ãƒ¼ãƒ‰æ•°ã‚’æ¸›ã‚‰ã™â€ã¨ã„ã†ã‚‚ã®ã€‚ ã—ãŸãŒã£ã¦ã€ï¼ˆMapperã‚„Reducerã«ã¯å…¨ãƒ‡ãƒ¼ã‚¿ãŒå¿…ãšæ¸¡ã•ã‚Œã‚‹ãŒã€ï¼‰Combinerã«ã¯å…¨ã¦ã®ãƒ¬ã‚³ãƒ¼ãƒ‰ãŒæ¸¡ã•ã‚Œã‚‹ã¨ã¯é™ã‚‰ãªã„ã€‚ ï¼ˆCombinerãŒå¸¸ã«å‘¼ã°ã‚Œã‚‹ã¨ã‚‚é™ã‚‰ãªã„ã—ã€åŒä¸€ã‚ãƒ¼ã§ã‚ã£ã¦ã‚‚Mapperå‡ºåŠ›ã®ä¸€éƒ¨åˆ†ã—ã‹æ¸¡ã£ã¦ã“ãªã„ã“ã¨ã‚‚ã‚ã‚‹ï¼ˆã¨ã„ã†ã‹ä¸é–“é›†è¨ˆãªã®ã§ã€ä¸€éƒ¨åˆ†ã—ã‹æ¥ãªã„ã®ãŒæ™®é€šã¨è€ƒãˆã‚‹ã¹ãï¼Ÿï¼‰ï¼‰ Mapperå‡ºåŠ›ãŒãƒ¡ãƒ¢ãƒªãƒ¼ä¸Šã®ãƒãƒƒãƒ•ã‚¡ãƒ¼
nantan 2012/07/04
hadoop

combiner
ãƒªãƒ³ã‚¯
VersionEye - Notification System for Software Packages
nantan 2012/07/04
hadoop
ãƒªãƒ³ã‚¯
Apache Hadoop MapReduce (3) â€” MapReduce ã‚’å¤šæ®µã§å‹•ã‹ã™ : MYH
MapReduce ã§å°‘ã—è¤‡é›‘ãªã“ã¨ã‚’ã—ã‚ˆã†ã¨æ€ã†ã¨ã€MapReduce ã‚’è¤‡æ•°å›žé©ç”¨ã—ãªã‘ã‚Œã°ãªã‚‰ãªã„ã“ã¨ãŒã‚ã‚Šã¾ã™ã€‚ã“ã‚Œã«ã¯ã„ãã¤ã‹ã®æ–¹æ³•ãŒã‚ã‚Šã¾ã™ã€‚ ã‚‚ã—ã€Map+ Reduce Map* (ï¼‘å›žä»¥ä¸Šã® Map, Reduce, ï¼å›žä»¥ä¸Šã® Map) ã¨ã„ã†å½¢ã§è¨˜è¿°ã™ã‚‹ã“ã¨ãŒå‡ºæ¥ã‚‹ãªã‚‰ã°ã€ChainMapper, ChainReducer ã¨ã„ã†ã‚¯ãƒ©ã‚¹ãŒç”¨æ„ã•ã‚Œã¦ãŠã‚Šã€ã“ã‚Œã‚’ Mapper, Reducer ã¨ã—ã¦åˆ©ç”¨ã™ã‚‹ã®ãŒç°¡å˜ã§ã™ã€‚ ã‚‚ã—ã€ã‚·ãƒ¼ã‚±ãƒ³ã‚·ãƒ£ãƒ«ã« MapReduce ã‚’é©ç”¨ã™ã‚‹ã®ã§ã‚ã‚Œã°ã€Job ã®çµ‚äº†ã‚’å¾…ã£ã¦ã‹ã‚‰æ¬¡ã® Job ã‚’æŠ•å…¥ã™ã‚Œã°æ¸ˆã¿ã¾ã™ã€‚ ChainMapper / ChainReducer ã“ã®ã‚¯ãƒ©ã‚¹ã¯æ—¢å˜ã® Mapper ã‚’ç°¡å˜ã«ã¤ãªã’ã‚‰ã‚Œã‚‹ã“ã¨ã‚’ç›®çš„ã¨ã—ã¦ã„ã¾ã™ã€‚ ä»®ã« ChainMapper, ChainReducer ã¨ã„ã†ã‚¯ãƒ©ã‚¹ãŒãŒãªã‹ã£ãŸã¨ã—ã¾
nantan 2012/07/03
hadoop

MapReduce
ãƒªãƒ³ã‚¯
Amebaã‚µãƒ¼ãƒ“ã‚¹ã®ãƒã‚°è§£æžåŸºç›¤
This document summarizes the Ameba and Neutral Techno logy Group's Patriot and Stinger big data platforms. Patriot is based on Hadoop and Hive and allows analyzing log data through a web UI and Hue. It loads data from MySQL into HDFS and analyzes it with HiveQL. Stinger collects log data with Flume agents and stores it incrementally in HBase for real-time analytics of hourly and daily metrics throu
nantan 2012/07/02
hadoop

hive
ãƒªãƒ³ã‚¯
Hadoopã‚’ã‚ˆã‚Šä¾¿åˆ©ã«ä½¿ã†ï¼Hiveã§ã®MapReduceã¾ã¨ã‚ - Y's note
ç›®æ¬¡ Hiveã¨ã¯ Hiveã®è¨å®š HiveQLæ§‹æ–‡(DDL) DataBase/SCHEMAã®ä½œæˆ Database/SCHEMAã®å‰Šé™¤ Tableã®ä½œæˆ Tableã®å‰Šé™¤ Tableåå¤‰æ›´ Partitionä½œæˆ Partitionã®å‰Šé™¤ Columnã®å¤‰æ›´ Columnã®è¿½åŠ /ç½®ãæ›ãˆ Tableã®Propertyå¤‰æ›´ SerDe Propertyã®è¿½åŠ HiveQLæ§‹æ–‡(SQL) ãƒ†ãƒ¼ãƒ–ãƒ«ä¸€è¦§è¡¨ç¤º ãƒ†ãƒ¼ãƒ–ãƒ«ã®å†…å®¹ã‚’è¡¨ç¤º åŸºæœ¬çš„ãªSELECTæ–‡ WHEREå¥(æ¡ä»¶æŒ‡å®š) DISTINCT(é‡è¤‡å‰Šé™¤) ORDER BY / SORT BYå¥(ã‚½ãƒ¼ãƒˆ) GROUP BYå¥(ã‚°ãƒ«ãƒ¼ãƒ—åŒ–) HAVINGå¥(ã‚°ãƒ«ãƒ¼ãƒ—åŒ–å¾Œã®æ¡ä»¶) LIMITå¥ JOIN(ãƒ†ãƒ¼ãƒ–ãƒ«çµåˆ) æŠ½å‡ºã‚«ãƒ©ãƒ ã‚’æ£è¦è¡¨ç¾ã§æŒ‡å®š UNION(çµæžœã®çµåˆ) SUBQUERY LOAD(ãƒ‡ãƒ¼ã‚¿ã®èªã¿è¾¼ã¿) Hiveã‚’ä½¿ã† äº‹å‰æº–å‚™
nantan 2012/07/02
hadoop

hive
ãƒªãƒ³ã‚¯
Hadoopï¼‹Hiveæ¤œè¨¼ç’°å¢ƒã‚’æ§‹ç¯‰ã—ã¦ã¿ã‚‹
Hadoopï¼‹Hiveæ¤œè¨¼ç’°å¢ƒã‚’æ§‹ç¯‰ã—ã¦ã¿ã‚‹ï¼šHiveâ€•â€•RDBä½¿ã„ã®ãŸã‚ã®Hadoopã‚¬ã‚¤ãƒ‰ï¼ˆå‰ç·¨ï¼‰ï¼ˆ1/3 ãƒšãƒ¼ã‚¸ï¼‰ Hadoop Hiveã¯Hadoopä¸Šã§SQLãƒ©ã‚¤ã‚¯ãªã‚¯ã‚¨ãƒªæ“ä½œãŒå¯èƒ½ãªDWHå‘ã‘ã®ãƒ—ãƒãƒ€ã‚¯ãƒˆã§ã™ã€‚SQLã«è¿‘ã„æ“ä½œãŒå¯èƒ½ãªãŸã‚ã€HBaseã‚ˆã‚Šã‚‚ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã«æ…£ã‚Œè¦ªã—ã‚“ã ã¿ãªã•ã‚“ã«ã¯ä½¿ã„å‹æ‰‹ãŒã„ã„ã‹ã‚‚ã—ã‚Œã¾ã›ã‚“ã€‚æœ¬ç¨¿ã§ã¯ã“ã®Hiveã®ä½¿ã„æ–¹ã¨ãƒ¬ãƒ“ãƒ¥ãƒ¼ã‚’è¡Œã£ã¦ã„ãã¾ã™ã€‚
nantan 2012/07/02
hadoop

hive
ãƒªãƒ³ã‚¯
SQLãƒ©ã‚¤ã‚¯ã«Hadoop Hiveã‚’ä½¿ã„å€’ã™!
ãƒ‘ãƒ¼ãƒ†ã‚£ã‚·ãƒ§ãƒ³ã‚’åˆ©ç”¨ã™ã‚‹ ä»Šå›žã¯å°‘ã—å‡ã£ãŸãƒ†ãƒ¼ãƒ–ãƒ«ã‚’å®šç¾©ã‚’ã—ã¦ã¿ã¾ã—ã‚‡ã†ã€‚ éƒµä¾¿ç•ªå·ãƒ‡ãƒ¼ã‚¿ã¯æ¯Žæœˆæ›´æ–°ã•ã‚Œã‚‹ã®ã§ã€ãƒ†ãƒ¼ãƒ–ãƒ«æŒ‡å®šæ™‚ã«ãƒãƒ¼ã‚¸ãƒ§ãƒ³ã‚‚æŒ‡å®šã§ãã‚‹ã‚ˆã†ã«ã—ã¾ã™ã€‚ã“ã®ã‚ˆã†ãªå ´åˆã€Hiveã§ã¯ãƒ‘ãƒ¼ãƒ†ã‚£ã‚·ãƒ§ãƒ³ã‚’ä½¿ã„ã¾ã™ã€‚ ä»¥ä¸‹ã«éƒµä¾¿ç•ªå·ã‚’ä¿å˜ã™ã‚‹ãƒ†ãƒ¼ãƒ–ãƒ«ã€Œzipã€ã‚’å®šç¾©ã—ã¾ã™ãŒã€æ—¥ä»˜åž‹DATEã®ãƒ‘ãƒ¼ãƒ†ã‚£ã‚·ãƒ§ãƒ³verã‚’è¨å®šã™ã‚‹ã‚ˆã†ã«ã—ã¾ã™ã€‚ hive> CREATE TABLE zip (zip STRING, pref INT, city STRING, town STRING) > PARTITIONED BY (ver DATE) > ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' > LINES TERMINATED BY '\n'; OK Time taken: 0.128 seconds
nantan 2012/07/02
hadoop

hive
ãƒªãƒ³ã‚¯
ä¸¦åˆ—åˆ†æ•£å‡¦ç†ã®å¸¸è˜ã‚’Hadoopãƒ•ã‚¡ãƒŸãƒªã‹ã‚‰å¦ã¶
ä¸¦åˆ—åˆ†æ•£å‡¦ç†ã®å¸¸è˜ã‚’Hadoopãƒ•ã‚¡ãƒŸãƒªã‹ã‚‰å¦ã¶ï¼šãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã®å¸¸è˜ã‚’Javaã§èº«ã«ã¤ã‘ã‚‹ï¼ˆ2ï¼‰ï¼ˆ3/3 ãƒšãƒ¼ã‚¸ï¼‰ 3ã¤ã®èª²é¡Œã«å¿œãˆã‚‹ã€ŒHadoopãƒ•ã‚¡ãƒŸãƒªã€ é–¢é€£ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã¯ã€HadoopãŒæŠ±ãˆã‚‹ä»¥ä¸‹ã®3ã¤ã®èª²é¡Œã«å¿œãˆãŸã‚‚ã®ã§ã™ã€‚ ã€èª²é¡Œ1ã€‘ã€Œä¸¦åˆ—åˆ†æ•£å‡¦ç†ã‚’ã‚‚ã£ã¨ç°¡å˜ã«æ›¸ããŸã„ã€ MapReduceã‚’Javaã§æ›¸ãå‡¦ç†ã¯ã‚ã‚‹ç¨®ã®ã²ã‚‰ã‚ããŒå¿…è¦ã¨ã‚‚ã„ã‚ã‚Œã¾ã™ã€‚ç‰¹ã«ã€SQLã§æ›¸ã‹ã‚Œã¦ã„ãŸå‡¦ç†ã‚’MapReduceã§æ›¸ãæ›¿ãˆã‚‹å ´åˆã€æ€è€ƒã®åˆ‡ã‚Šæ›¿ãˆãŒå¿…è¦ã§è‹¦åŠ´ã™ã‚‹ã“ã¨ã‚‚å¤šã„ã‚ˆã†ã§ã™ã€‚ãŸã¨ãˆæ…£ã‚Œã¦ã„ã¦ã‚‚ã€Hadoopã§ç°¡å˜ãªãƒ‡ãƒ¼ã‚¿åŠ å·¥ã—ãŸã„ã€ãã®ãŸã‚ã ã‘ã«å¤šãã®ã‚¯ãƒ©ã‚¹å®šç¾©ã‚’ã™ã‚‹ã®ã¯é¢å€’ã§ã™ã€‚ ã“ã®é›£æ˜“åº¦ã‚’ä¸‹ã’ã‚‹ãƒã‚¤ãƒ¬ãƒ™ãƒ«è¨€èªžã¨ã—ã¦ã€ŒHiveã€ã€ŒPigã€ãŒå˜åœ¨ã—ã¾ã™ã€‚ã©ã¡ã‚‰ã®è¨€èªžã‚‚SQLã‚„ã‚¹ãƒˆã‚¢ãƒ‰ãƒ—ãƒã‚·ã‚¸ãƒ£ã®ã‚ˆã†ãªã‚³ãƒ¼ãƒ‰ã‚’æ›¸ã„ã¦ã€MapReduceã‚’å‹•ã‹ã›ã¾ã™ã€‚å¯¾è©±åž‹ã®ã‚·ã‚§ãƒ«ãƒ¦ãƒ¼ãƒ†ã‚£ãƒªãƒ†ã‚£ã‚‚ã‚ã‚Š
nantan 2012/07/02
hadoop

hive

MapReduce
ãƒªãƒ³ã‚¯
Hadoopå®Ÿè¡Œç’°å¢ƒã‚’Windowsä¸Šã«ç°¡å˜ã«æ§‹ç¯‰ã™ã‚‹æ–¹æ³•: TBãƒ˜ãƒƒãƒ‰ãƒ©ã‚¤ãƒ³
ã“ã‚“ã«ã¡ã¯ã€ã‚·ã‚¹ãƒ†ãƒ é–‹ç™ºéƒ¨ã®KTã§ã™ã€‚ ä»Šå›žã¯ã€Windowsä¸Šã«Hadoopã®å®Ÿè¡Œç’°å¢ƒã‚’æ§‹ç¯‰ã™ã‚‹ãŸã‚ã®ãŠæ‰‹è»½ãªæ–¹æ³•ã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ Hadoopã¯å†…éƒ¨ã§Linuxã‚³ãƒžãƒ³ãƒ‰ã‚’ä½¿ç”¨ã—ã¦ã„ã‚‹ãŸã‚ã€æœ¬æ ¼çš„ã«ä½¿ç”¨ã™ã‚‹ãŸã‚ã«ã¯Linuxç’°å¢ƒã‚’æº–å‚™ã™ã‚‹å¿…è¦ãŒã‚ã‚Šã¾ã™ã€‚ä¸€å¿œWindowsç’°å¢ƒã¸ã®ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã‚‚å¯èƒ½ã§ã™ãŒã€ãã®å ´åˆã¯åˆ¥é€”Cygwinç‰ã‚’ä½¿ç”¨ã™ã‚‹å¿…è¦ãŒã‚ã‚Šã€ã„ãšã‚Œã«ã—ã¦ã‚‚å€‹äººãƒ™ãƒ¼ã‚¹ã§è¡Œã†ã«ã¯ç’°å¢ƒã‚’æ•´ãˆã‚‹ã¾ã§ã«ã‚ã‚‹ç¨‹åº¦ã®æ‰‹é–“ãŒã‹ã‹ã£ã¦ã—ã¾ã„ã¾ã™ã€‚ ãã®ãŸã‚ã€ã€Œå··ã§è©±é¡Œã®Hadoopã«ã¡ã‚‡ã£ã¨ã ã‘è§¦ã‚Œã¦ã¿ãŸã„ï¼ã€ã¨ã„ã£ãŸæ¯”è¼ƒçš„è»½ã„æ°—æŒã¡ã®äººé”ã«ã¨ã£ã¦ã¯ã€ãªã‹ãªã‹æ‰‹ã‚’å‡ºã—è¾›ã„ä»£ç‰©ã«ãªã£ã¦ã—ã¾ã£ã¦ã„ã‚‹ã®ã§ã¯ãªã„ã§ã—ã‚‡ã†ã‹ã€‚ ã“ã“ã§ã¯ãã‚“ãªæ–¹ã€…ã®ãŸã‚ã«ã€Windowsç’°å¢ƒã«ç°¡å˜ç°¡æ½”ã«Hadoopã®å®Ÿè¡Œç’°å¢ƒã‚’æ§‹ç¯‰ã™ã‚‹æ‰‹é †ã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ ä»Šå›žã¯ã€VMPlayerã¨Clouderaç¤¾ãŒå…¬é–‹ã—ã¦ã„ã‚‹VMã‚¤ãƒ¡ãƒ¼ã‚¸
nantan 2012/06/28
windows

hadoop
ãƒªãƒ³ã‚¯
800ä¸‡äººã®"é£Ÿã¹ãŸã„"ã‚’Hadoopã§åˆ†æ•£å‡¦ç†
Hadoop is an open-source software framework for distributed storage and processing of large datasets across clusters of computers. It allows for the distributed processing of large datasets across clusters of nodes using simple programming models. Hadoop is highly scala ble, running on thousands of nodes, and is designed to reliably handle failures at the hardware or software level.Read less
nantan 2012/06/26
hadoop

slideshare
ãƒªãƒ³ã‚¯
Hadoopäº‹å§‹ã‚
http://www.wankuma.com/seminar/20101016nagoya15/ ä»Šã¯ã‚„ã‚Šã®Hadoopã«ã¤ã„ã¦ã€MapReduceã‚„åˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã¨ã„ã£ãŸåŸºç›¤æŠ€è¡“ã‹ã‚‰ç´¹ä»‹ã—ãŸã„ã¨æ€ã„ã¾ã™ã€‚ Read less
nantan 2012/06/26
hadoop

slideshare
ãƒªãƒ³ã‚¯
Hadoopã®ç´¹ä»‹
3. è¬è¾ž ä»¥ä¸‹ã®ãƒ›ãƒ¼ãƒ ãƒšãƒ¼ã‚¸ã‚’å‚ç…§ï¼†è³‡æ–™ã‚’è²¸ã—ã¦ã„ãŸã ãã¾ã—ãŸ http://hadoop.apache.org http://developer.yahoo.net/blogs/hadoop/2008/02/yahoo-worlds-largest-production-hadoop.html http://www.techcrunch.com/2008/01/09/google-processing-20000-terabytes-a-day-and-growing/ http://codezine.jp/article/detail/2448?p=1 http://www.slideshare.net/kakuda/hadoop/ 4. outline what is Hadoop? Hadoop distributed file system Hadoop mapreduce H
nantan 2012/06/26
hadoop

slideshare
ãƒªãƒ³ã‚¯
Cassandra Hadoopã‚µãƒ³ãƒ—ãƒ«(Hishidama's Apache Cassandra Hadoop Memo)
Hadoopã®Map/Reduceã‚’ä½¿ã£ã¦Cassandraã§é›†è¨ˆã™ã‚‹ã‚µãƒ³ãƒ—ãƒ«ã€‚ WordCountã‚µãƒ³ãƒ—ãƒ« æ¦‚è¦ WordCountSetup WordCount [/2010-07-31] WordCountã‚µãƒ³ãƒ—ãƒ«ã®æ¦‚è¦ Cassandraã¯0.6ã§Hadoopã«å¯¾å¿œã—ãŸã‚‰ã—ã„ã€‚ Map/Reduceç”¨ã®InputFormatï¼ˆãŠã‚ˆã³ãã‚Œã«é–¢ã™ã‚‹Readerç‰ï¼‰ãŒæä¾›ã•ã‚Œã¦ã„ã‚‹ã€‚ ï¼ˆOutputFormatç³»ã¯æœªå¯¾å¿œï¼‰ ãã—ã¦ã€apache-cassandra-0.6.x-src/contribã®ä¸ã«word_countãŒæä¾›ã•ã‚Œã¦ã„ã‚‹ã€‚ ã“ã‚ŒãŒCassandraã‚’ä½¿ã£ã¦WordCountã‚’è¡Œã†ã‚µãƒ³ãƒ—ãƒ«ãªã®ã§ã€ã“ã‚Œã‚’å®Ÿè¡Œã—ã¦ã¿ã‚‹ã€‚ ã“ã®word_countã«ã¯ã€WordCountã¨WordCountSetupã¨ã„ã†2ã¤ã®ã‚¯ãƒ©ã‚¹ï¼ˆJavaã®ã‚½ãƒ¼ã‚¹ãƒ•ã‚¡ã‚¤ãƒ«ï¼‰ãŒã‚ã‚‹ã€‚ WordCountS
nantan 2012/06/26
hadoop

cassandra
ãƒªãƒ³ã‚¯
Mahout JP ã‚’ç«‹ã¡ä¸Šã’ã¾ã—ãŸ #MahoutJP - hamadakoichi blog
Hadoopä¸Šã§å‹•ä½œã™ã‚‹ å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ãƒžã‚¤ãƒ‹ãƒ³ã‚°ãƒ»æ©Ÿæ¢°å¦ç¿’ãƒ©ã‚¤ãƒ–ãƒ©ãƒª Apache Mahout ã«é–¢ã—ã€æŠ€è¡“æƒ…å ±ã¾ã¨ã‚ãƒ»ç™ºä¿¡ã‚ˆã‚‹æ´»ç”¨ã®è£¾é‡Žã‚’åºƒã’ã‚‹ã“ã¨ã‚’ç›®çš„ã¨ã—Mahout JPã‚’ç«‹ã¡ä¸Šã’ã¾ã—ãŸã€‚ ç§ã‚‚å«ã‚ TokyoWebminingã§Mahoutã«é–¢ã™ã‚‹å„ç¨®è¬›å¸«ã‚’ã—ã¦ã„ãŸãƒ¡ãƒ³ãƒãƒ¼ã‚„ã€Tokyo.Rã€PRMLä¼šã®ãƒ¡ãƒ³ãƒãƒ¼å«ã‚ã€å„æ¥ç•Œã®ãƒ‡ãƒ¼ã‚¿ãƒžã‚¤ãƒ‹ãƒ³ã‚°ãƒ»æ©Ÿæ¢°å¦ç¿’ã§æ´»å‹•ã—ã¦ããŸãƒ¡ãƒ³ãƒãƒ¼ã§é›†ã¾ã‚Šã€Mahoutã«é–¢ã™ã‚‹æƒ…å ±ã¾ã¨ã‚ãƒ»ç™ºä¿¡ã‚’ã—ã¦ã„ãã¾ã™ã€‚ Mahout JP -Effective Applications of Apache Mahout in Japan- #MahoutJP ç¾åœ¨ã€Mahout ã¯ãƒ‰ã‚ãƒ¥ãƒ¡ãƒ³ãƒˆãŒã¾ã æ•´å‚™ã•ã‚Œã¦ã„ãªãã€å”¯ä¸€ã‚ã‚‹æ›¸ç± Mahout in Actionã§ã‚‚æƒ…å ±ãŒé™ã‚‰ã‚Œã¦ã„ã‚‹ãŸã‚ã€å®Ÿéš›ã«æ´»ç”¨ã—ã‚ˆã†ã¨ã™ã‚‹ã¨ã‚½ãƒ¼ã‚¹ã‚³ãƒ¼ãƒ‰ã‹ã‚‰èªã¿è¾¼ã‚€å¿…è¦ãŒã‚ã‚‹çŠ¶æ…‹ã§ã™ã€‚ä»Šå›žã€
nantan 2012/06/26
hadoop

Mahout

æ©Ÿæ¢°å¦ç¿’
ãƒªãƒ³ã‚¯
IBM Developer
IBM Developer is your one-stop location for getting hands-on training and learning in-demand skills on relevant techno logies such as generative AI, data science, AI, and open source.
nantan 2012/06/26
Hadoop

Mahout
ãƒªãƒ³ã‚¯
Apache Mahoutã§æ©Ÿæ¢°å¦ç¿’ã—ã¦ã¿ã‚‹ã¹ - éƒ½å…ƒãƒ€ã‚¤ã‚¹ã‚± IT-PRESS
Mahoutã‚·ãƒªãƒ¼ã‚ºç›®æ¬¡ï¼ˆéšæ™‚æ›´æ–°ï¼‰ éžåˆ†æ•£ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‡ãƒ¼ã‚·ãƒ§ãƒ³ Apache Mahoutã§æ©Ÿæ¢°å¦ç¿’ã—ã¦ã¿ã‚‹ã¹ - éƒ½å…ƒãƒ€ã‚¤ã‚¹ã‚± IT-PRESS ï¼ˆã“ã‚Œï¼‰ ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‡ãƒ¼ã‚·ãƒ§ãƒ³ã®ç°¡å˜ãªåŽŸç†ã‚’è¦–è¦šçš„ã«æŠŠæ¡ã—ã¦ã‹ã‚‰å®Ÿéš›ã«è¨ˆç®—ã—ã¦ã¿ã‚‹ - éƒ½å…ƒãƒ€ã‚¤ã‚¹ã‚± IT-PRESS æ©Ÿæ¢°å¦ç¿’ã«ãŠã‘ã‚‹é‡å¤§ãª"ä»®å®š"ã¨ã€ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã®è©•ä¾¡ - éƒ½å…ƒãƒ€ã‚¤ã‚¹ã‚± IT-PRESS åˆ†æ•£ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‡ãƒ¼ã‚·ãƒ§ãƒ³ Mahoutã§åˆ†æ•£ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰(1) - éƒ½å…ƒãƒ€ã‚¤ã‚¹ã‚± IT-PRESS Mahoutã§åˆ†æ•£ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰(2) - éƒ½å…ƒãƒ€ã‚¤ã‚¹ã‚± IT-PRESS Mahoutã§åˆ†æ•£ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰(3) - éƒ½å…ƒãƒ€ã‚¤ã‚¹ã‚± IT-PRESS ã‚¯ãƒ©ã‚¹ã‚¿ãƒªãƒ³ã‚° ä»Šåº¦ã¯Mahoutã§ã‚¯ãƒ©ã‚¹ã‚¿ãƒªãƒ³ã‚° - éƒ½å…ƒãƒ€ã‚¤ã‚¹ã‚± IT-PRESS ä»Šåº¦ã¯Mahoutã§ã‚¯ãƒ©ã‚¹ã‚¿ãƒªãƒ³ã‚°(ã‚½ãƒ¼ã‚¹ç·¨) - éƒ½å…ƒãƒ€ã‚¤ã‚¹ã‚± IT-PRESS ã§ã¯ã€æœ¬æ–‡ã„ãã¾ã™ã€‚ Apach
nantan 2012/06/26
hadoop

Mahout
ãƒªãƒ³ã‚¯
Hadoopã®å…¨ä½“åƒ | å¤§è¦æ¨¡è¨ˆç®—ãƒ‰ãƒƒãƒˆã‚³ãƒ
nantan 2012/06/25
hadoop
ãƒªãƒ³ã‚¯
1 2 æ¬¡ã®ãƒšãƒ¼ã‚¸