[B! hadoop] maxyã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

maxy id:maxy

hadoopã«é–¢ã™ã‚‹maxyã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (24)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ Web æ¤œç´¢ã‚¨ãƒ³ã‚¸ãƒ³ Apache Nutch ã®æ¦‚è¦ - æœ¨æ›œä¸è¶³
Apache Nutch ã¯ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã® Web æ¤œç´¢ã‚¨ãƒ³ã‚¸ãƒ³ã€‚ ä»Šå¹´ã«ãªã£ã¦ãƒãƒ¼ã‚¸ãƒ§ãƒ³ 1.0 ãŒå‡ºã¦ã€Apache ã® TLP(ãƒˆãƒƒãƒ—ãƒ¬ãƒ™ãƒ«ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆ) ã«ãªã£ãŸã€‚ç¾åœ¨æœ€æ–°ã¯ v1.2ã€‚ Apache Nutch http://nutch.apache.org/ ä¹±æš´ã«è¨€ã†ã¨ã€ã€Œæ¤œç´¢ã‚¨ãƒ³ã‚¸ãƒ³ Lucene ï¼‹å…¨æ–‡æ¤œç´¢ Solr ï¼‹ Web ã‚¯ãƒãƒ¼ãƒ©ãƒ¼ï¼‹ã‚¹ã‚³ã‚¢ãƒªãƒ³ã‚°(Page Rank)ï¼‹åˆ†æ•£å®Ÿè¡Œã®ä»•çµ„ã¿ã€ãŒ Nutchã€‚ ã¡ãªã¿ã«æ¤œç´¢ã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ã‚’ä½œæˆï¼†æ ¼ç´ã™ã‚‹ãŸã‚ã® mapper / reducer ãŠã‚ˆã³åˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ãŒ Nutch ã‹ã‚‰ spin out ã—ãŸã®ãŒè©±é¡Œã® Hadoopã€‚ ã®å‰²ã‚Šã«ã¯çŸ¥ååº¦ãŒä½Žã„æ°—ãŒã™ã‚‹ã®ã¯ã€Web æ¤œç´¢ã‚¨ãƒ³ã‚¸ãƒ³ã‚’ä½œã‚ŠãŸã„ã¨ã„ã†è¦æ±‚ãŒå°‘ãªã„ã›ã„ï¼Ÿ(ã‚µã‚¤ãƒˆå†…å…¨æ–‡æ¤œç´¢ãªã‚‰ Solr ã§ã„ã„) ãƒ—ãƒ©ã‚°ã‚¤ãƒ³ã®çµ„ã¿åˆã‚ã›ã§ virtical sea
maxy 2010/12/14
apache

db

hadoop

solr

lucene

search

nutch
ãƒªãƒ³ã‚¯
å¤§è¦æ¨¡ãªãƒ‡ãƒ¼ã‚¿ã‚»ãƒƒãƒˆã‚’åŠ¹çŽ‡çš„ã«æ‰±ã†ãŸã‚ã® Pig è¶…å…¥é–€ - (ï¾Ÿâˆ€ï¾Ÿ)oå½¡ sasata299's blog
2010å¹´07æœˆ07æ—¥22:38 Hadoop å¤§è¦æ¨¡ãªãƒ‡ãƒ¼ã‚¿ã‚»ãƒƒãƒˆã‚’åŠ¹çŽ‡çš„ã«æ‰±ã†ãŸã‚ã® Pig è¶…å…¥é–€ Pig ã‚ã‚‹ã˜ã‚ƒãªã„ã§ã™ã‹ã€‚Hadoop ã®ãƒ©ãƒƒãƒ‘ãƒ¼ã§ã€DSL ã§æ›¸ã‘ã‚‹ã¨ã„ã†ã‚¢ãƒ¬ã§ã™ã€‚ æœ€è¿‘ã¯ Twitter ã‚„ Yahoo! ãªã©ã§ä½¿ã‚ã‚Œã¦ã„ã‚‹ã‚‰ã—ã Pigã€‚Hadoop ã‚’æ‰±ã†å ´åˆã€mapper ã¨ reducer ã‚’ãã‚Œãžã‚Œè¨˜è¿°ã™ã‚‹å¿…è¦ãŒã‚ã‚Šã¾ã™ãŒã€ã“ã® Pig ã‚’ä½¿ã†ã¨ DSL ã‚’æ›¸ãã ã‘ã§å†…éƒ¨çš„ã«å‡¦ç†ã‚’ mapper, reducer ã¨ã—ã¦å®Ÿè¡Œã—ã¦ãã‚Œã¾ã™ã€‚ãã®çµæžœã€è¨˜è¿°é‡ãŒæ¸›ã£ã¦é–‹ç™ºæ™‚é–“ãŒçŸç¸®ã§ãã‚‹ã¨ã„ã†ãƒ¡ãƒªãƒƒãƒˆãŒã‚ã‚Šã¾ã™ã€‚ã‚¢ã‚¤ã‚³ãƒ³ãŒã‚‚ã†å°‘ã—å¯æ„›ã‘ã‚Œã°ã€ã€ã¨æ€ã†ã¨æ®‹å¿µã§ãªã‚Šã¾ã›ã‚“ã€‚ãªã‚“ã ã“ã®ãƒ‰ãƒ¤é¡”ã¯ãƒ»ãƒ» ä»Šã¾ã§ã€Œè‰¯ã•ãã†ã ãªãƒ¼ã€ã¨æ€ã„ã¤ã¤è§¦ã‚Œã¦ã„ãªã‹ã£ãŸã®ã§ã™ãŒã€ä»Šå›žè§¦ã£ã¦ã¿ã‚‹æ©Ÿä¼šãŒã‚ã£ãŸã®ã§è»½ãã¾ã¨ã‚ã¦ãŠãã¾ã™ã€‚â€»Pig ã‚’å‹•ã‹ã™ç’°å¢ƒã«ã¤ã„ã¦ã¯å‡ºæ¥ã¦ã„ã‚‹å‰æã§ã™ã€‚
maxy 2010/11/04
hadoop

pig

dsl
ãƒªãƒ³ã‚¯
800ä¸‡äººã®"é£Ÿã¹ãŸã„"ã‚’Hadoopã§åˆ†æ•£å‡¦ç†
Hadoop is an open-source software framework for distributed storage and processing of large datasets across clusters of computers. It allows for the distributed processing of large datasets across clusters of nodes using simple programming models. Hadoop is highly scala ble, running on thousands of nodes, and is designed to reliably handle failures at the hardware or software level.Read less
maxy 2010/08/13
ruby

presentation

s3

hadoop
ãƒªãƒ³ã‚¯
Hadoop at Twitter (Hadoop Summit 2010)
Kevin Weil presented on Hadoop at Twitter. He discussed Twitter's data lifecycle including data input via Scribe and Crane, storage in HDFS and HBase, analysis using Pig and Oink, and data products like Birdbrain. He described how tools like Scribe, Crane, Elephant Bird, Pig, and HBase were developed and used at Twitter to handle large volumes of log and tabular data at petabyte scale.Read less
maxy 2010/07/05
twitter

hadoop

rails

hbase
ãƒªãƒ³ã‚¯
Hadoop, Pig, and Twitter (NoSQL East 2009)
A talk on the use of Hadoop and Pig inside Twitter, focusing on the flexibility and simplicity of Pig, and the benefits of that for solving real-world big data probl ems.Read less
maxy 2010/02/22
twitter

hadoop

data
ãƒªãƒ³ã‚¯
ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã¨Hadoop - ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰é–‹ç™ºè€…ãƒ–ãƒã‚°
ã¯ã˜ã‚ã¾ã—ã¦ã€‚ä»Šå¹´ã®5æœˆã«å…¥ç¤¾ã—ãŸå‹é–“@ã•ãŒã™ãƒãƒ¼ãƒ ã§ã™ã€‚ å…¥ç¤¾ã—ã¦ã‹ã‚‰ã¯ã€ãªã‹ãªã‹å¤§å¤‰ãªã“ã¨ã‚‚å¤šã„ã§ã™ãŒã€æœ€è¿‘ã¯ãŠé…’å¥½ããŒé›†ã¾ã£ã¦æœˆæ›œã‹ã‚‰é£²ã¿åˆã† ã€Œå‹é–“ä¼šã€ãªã‚‹ã‚‚ã®ã‚‚ç™ºè¶³ã—ã¦ã€ä»•äº‹é¢ã§ã‚‚ä»•äº‹ä»¥å¤–ã®é¢ã§ã‚‚å¯†åº¦ã®é«˜ã„æ¯Žæ—¥ã‚’éŽã”ã—ã¦ã„ã¾ã™ï¼ ã•ã¦ã€åƒ•ã¯ã€Œã•ãŒã™ã€ãƒãƒ¼ãƒ æ‰€å±žã¨ã„ã†ã“ã¨ã§ã€æ™®æ®µã¯ãƒ¬ã‚·ãƒ”ã‚’ã€Œã•ãŒã™ã€ãƒ¦ãƒ¼ã‚¶ã®æº€è¶³åº¦ã‚’ä¸Šã’ã‚‹ãŸã‚ã«ã€ ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã®æ¤œç´¢ã¾ã‚ã‚Šã«ã¤ã„ã¦ã€ã„ã‚ã„ã‚ãªé–‹ç™ºã‚’è¡Œã£ã¦ã„ã¾ã™ã€‚ ä¸€æ–¹ã§ã€ãƒ¦ãƒ¼ã‚¶ã®ã€Œã•ãŒã™æ¬²æ±‚ã€ã«ã¤ã„ã¦æ·±ãçŸ¥ã‚‹ãŸã‚ã«ã€å¤§è¦æ¨¡ãªãƒ‡ãƒ¼ã‚¿è§£æžã‚’è¡Œã„ã€æ¬²æ±‚ã®åˆ†æžã‚’è¡Œã†æ©Ÿä¼šã‚‚å¢—ãˆã¦ãã¾ã—ãŸã€‚ ã¨ã“ã‚ãŒã€ã‚¯ãƒƒã‚¯ãƒ‘ãƒƒãƒ‰ã®ãƒã‚°ã¯è†¨å¤§ãªæ•°ãŒã‚ã‚‹ã®ã§ã€ä¸€å£ã®ãƒ‡ãƒ¼ã‚¿è§£æžã¨è¨€ã£ã¦ã‚‚é€šå¸¸ã®ãƒãƒƒãƒå‡¦ç†ã ã¨é–“ã«åˆã‚ãªã„ãŸã‚ã€ åˆ†æ•£å‡¦ç†ç’°å¢ƒã®å¿…è¦æ€§ãŒé«˜ã¾ã£ã¦ãã¾ã—ãŸã€‚ ãã“ã§ã€ã¾ãšã¯æ‰‹è»½ã«è©¦ã›ã‚‹åˆ†æ•£å‡¦ç†ã®çŽ‹é“ã¨ã„ã†ã“ã¨ã§ã€æœ€è¿‘ã§ã¯Hadoopã‚’ä½¿ã£ãŸãƒ‡ãƒ¼ã‚¿è§£æžç’°å¢ƒã‚’æ•´å‚™ã—ã¦ã„ã¾ã™ã€‚
maxy 2009/09/16
db

hadoop

ruby

mapreduce

log
ãƒªãƒ³ã‚¯
AmazonEC2 - HADOOP2 - Apache Software Foundation
Running Hadoop on Amazon EC2 Amazon EC2 (Elastic Compute Cloud) is a computing service. One allocates a set of hosts, and runs one's application on them, then, when done, de-allocates the hosts. Billing is hourly per host. Thus EC2 permits one to deploy Hadoop on a cluster without having to own and operate that cluster, but rather renting it on an hourly basis. If you run Hadoop on EC2 you might c
maxy 2009/09/15
amazon

amazon ec2

hadoop

mapreduce
ãƒªãƒ³ã‚¯
ec2ã§hadoopã—ã¦ã¿ãŸã‚ˆ - id:lopnor
ã¨ã„ã†ã‹http://codezine.jp/article/detail/2841ã‚’ã¾ã‚“ã¾ã‚³ãƒ”ãƒšã—ãŸã ã‘ãªã‚“ã§ã™ã‘ã©ãã€‚ ec2ã®ã‚¢ã‚«ã‚¦ãƒ³ãƒˆæŒã£ã¦ã‚‹ã—ä½¿ã„ã¾ãã£ã¦ã‚‹ã‚ˆãƒ¼ã€ã¨ã„ã†å‰æã§ã€‚ é£›ã°ã—ãªãŒã‚‰èªã‚“ã§å®Ÿè¡Œã—ãŸã‚‰ã“ã‘ã¦æ°—ã¥ã„ãŸã‚“ã ã‘ã©ã€ ec2-add-keypair gsg-keypairã¯gsg-keypairã¨ã„ã†åå‰æ±ºã‚æ‰“ã¡ãªã®ã§ã¡ã‚ƒã‚“ã¨ã‚„ã‚Šã¾ã—ã‚‡ã†ã€‚ EC2_KEYDIR=`dirname "$EC2_PRIVATE_KEY"`ã£ã¦ãªã£ã¦ã‚‹ã®ã§ãã“ã«id_rsa-gsg-keypairã¨ã„ã†åå‰ã§ä¿å˜ ./hadoop-ec2 launch-cluster test-cluster 2ã¨ã™ã‚‹ã¨ã¡ã‚ƒã‚“ã¨ç«‹ã¡ä¸ŠãŒã£ãŸ ./hadoop-ec2 login test-clusterã§ãƒã‚°ã‚¤ãƒ³ã€‚ ãŠã‚Œjavaè©³ã—ããªã„ã‚“ã§ã‚ˆãã‚ã‹ã‚“ãªã‹ã£ãŸã‚“ã ã‘ã© è¨˜äº‹ã«æ›¸ã„ã¦ã‚ã‚‹ jar hado
maxy 2009/09/15
amazon

amazon ec2

hadoop

mapreduce
ãƒªãƒ³ã‚¯
blogeyeã®å®Ÿè£…ã«å¦ã¶ã€Amazon EC2/S3ã§ã®Hadoopæ´»ç”¨è¡“
ã¯ã˜ã‚ã« ã“ã‚“ã«ã¡ã¯ã€‚Hadoopé€£è¼‰ ç¬¬4å›žã¯å¤ªç”°ã•ã‚“ã«ä»£ã‚ã£ã¦å¤§å€‰ãŒæ‹…å½“ã—ã¾ã™ã€‚ ã“ã‚Œã¾ã§ã®é€£è¼‰ã§ã€Hadoopã«ã‚ˆã‚‹ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã®æ¦‚ç•¥ã«ã¤ã„ã¦ã¯ç†è§£ã•ã‚Œã¦ã„ã‚‹ã¨æ€ã„ã¾ã™ã€‚ä»Šå›žã¯Hadoopã‚’åˆ©ç”¨ã—ãŸã‚·ã‚¹ãƒ†ãƒ ã®å®Ÿä¾‹ã¨ã„ã†ã“ã¨ã§ã€ãƒ–ãƒã‚°åˆ†æžã‚’è¡Œã†ã€Œblogeyeã€ã‚·ã‚¹ãƒ†ãƒ ã®æ¦‚ç•¥ã¨ã€ãã®ä¸ã§ã®Hadoopåˆ©ç”¨æ³•ã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ ã¾ãŸã€blogeyeã¯AmazonãŒæä¾›ã—ã¦ã„ã‚‹EC2ï¼ˆãƒ¬ãƒ³ã‚¿ãƒ«ã‚µãƒ¼ãƒï¼‰ã€S3ï¼ˆã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ï¼‰ã‚’Hadoopã¨çµ„ã¿åˆã‚ã›ã¦åˆ©ç”¨ã—ã¦ã„ã‚‹ã®ã§ã€ãã®è¾ºã‚Šã®å°Žå…¥æ–¹æ³•ã«ã¤ã„ã¦ã‚‚ç´¹ä»‹ã—ã¾ã™ã€‚ ã“ã‚Œã¾ã§ã®é€£è¼‰ Hadoopã€hBaseã§æ§‹ç¯‰ã™ã‚‹å¤§è¦æ¨¡åˆ†æ•£ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚·ã‚¹ãƒ†ãƒ Hadoopã®ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã¨ã‚µãƒ³ãƒ—ãƒ«ãƒ—ãƒã‚°ãƒ©ãƒ ã®å®Ÿè¡Œ è¤‡æ•°ãƒžã‚·ãƒ³ã¸Hadoopã‚’ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã™ã‚‹ blogeyeã¨ã¯ ã€Œblogeyeã€ï¼ˆãƒ–ãƒã‚°ã‚¢ã‚¤ï¼‰ã¯æ—¥æœ¬èªžã®ãƒ–ãƒã‚°ã‚’ã‚¯ãƒãƒ¼ãƒ«ã€ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ã«åˆ†æžã—ã¦ã€æµè¡Œèªžã¨æ€ã‚
maxy 2009/09/15
amazon

amazon ec2

hadoop

mapreduce
ãƒªãƒ³ã‚¯
hadoop-ec2ã§ã‚¢ã‚¯ã‚»ã‚¹è§£æžã—ã¦ã¿ãŸã‚ˆï¼
å‰å›žï¼ˆhttp://d.hatena.ne.jp/lopnor/20080831/1220183688ï¼‰ã‚µãƒ³ãƒ—ãƒ«ãƒ—ãƒã‚°ãƒ©ãƒ ã‚’å‹•ã‹ã™ã¨ã“ã‚ã¾ã§è¡Œãã¾ã—ãŸã‘ã©ã€ä»Šå›žã¯ä¾‹ã®id:naoyaã®hadoop streamingã§ã‚¢ã‚¯ã‚»ã‚¹è§£æžï¼ˆhttp://d.hatena.ne.jp/naoya/20080513/1210684438ï¼‰ã™ã‚‹ã®ã‚’ã‚„ã£ã¦ã¿ã¾ã™ã€‚ã¨ã„ã†ã‹ãªã‚“ã¨ã‹ãŸã©ã‚Šç€ã„ãŸã®ã§ã¾ã¨ã‚ã‚’æ›¸ãã¾ã™ã€‚ *ec2ã®ä½¿ã„æ–¹ id:rx7ã•ã‚“ãŒã¨ã¦ã‚‚ä¸å¯§ã«èª¬æ˜Žã•ã‚Œã¦ã„ã‚‹è³‡æ–™ï¼ˆhttp://d.hatena.ne.jp/rx7/20080528/p1ï¼‰ãŒã‚ã‚‹ã®ã§ã€ãã¡ã‚‰ã‚’èªã‚ã°å®Œç’§ã ã¨æ€ã„ã¾ã™ã€‚åƒ•ã‚‚ã“ã‚Œã§ec2ãŒä½¿ãˆã‚‹ã‚ˆã†ã«ãªã‚Šã¾ã—ãŸã€‚ *hadoop-ec2ã®ä½¿ã„æ–¹ https://codezine.jp/article/detail/2841ãŒã‚¤ãƒ³ãƒˆãƒãƒ€ã‚¯ã‚·ãƒ§ãƒ³ã€‚http://d.hatena.n
maxy 2009/09/15
amazon

amazon ec2

hadoop

mapreduce
ãƒªãƒ³ã‚¯
mrtoolkit - Google Code
Code Archive Skip to content Google About Google Privacy Terms
maxy 2009/05/13
ruby

mapreduce

hadoop
ãƒªãƒ³ã‚¯
Cloudera Support Center
maxy 2009/04/06
hadoop

cloud
ãƒªãƒ³ã‚¯
Cloudera's Distribution for Hadoop on Amazon EC2 | Cloudera
Support We're always trying to improve our distribution for Hadoop. If you have trouble finding information you need on the following pages, check out our community support. We also offer commercial support for Hadoop. Running Hadoop on EC2 is easy. We've taken Cloudera's Distribution for Hadoop, and packaged it up in an easy-to-use AMI and associated launch scripts. Once you download and install
maxy 2009/03/16
cloud

amazon ec2

hadoop
ãƒªãƒ³ã‚¯
æ¥½å¤©ç‰ˆMapReduceãƒ»Hadoopã¯Rubyã‚’æ´»ç”¨ ï¼ ï¼ IT
2008/12/01 æ¥½å¤©ã¯11æœˆ29æ—¥ã€æ±äº¬ãƒ»å“å·ã®æœ¬ç¤¾ã§é–‹å‚¬ã—ãŸæŠ€è¡“ç³»ã‚¤ãƒ™ãƒ³ãƒˆã€Œæ¥½å¤©ãƒ†ã‚¯ãƒŽãƒã‚¸ãƒ¼ã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹2008ã€ã«ãŠã„ã¦ã€è¿‘ã„å°†æ¥ã«åŒç¤¾ã®Eã‚³ãƒžãƒ¼ã‚¹ã‚µãƒ¼ãƒ“ã‚¹ã€Œæ¥½å¤©å¸‚å ´ã€ã‚’æ”¯ãˆã‚‹è¨ˆç”»ãŒã‚ã‚‹Rubyãƒ™ãƒ¼ã‚¹ã®å¤§è¦æ¨¡åˆ†æ•£å‡¦ç†æŠ€è¡“ã€ŒROMAã€ï¼ˆãƒãƒ¼ãƒžï¼‰ã¨ã€Œfairyã€ï¼ˆãƒ•ã‚§ã‚¢ãƒªãƒ¼ï¼‰ã«ã¤ã„ã¦ã€ãã®æ¦‚è¦ã‚’æ˜Žã‚‰ã‹ã«ã—ãŸã€‚ ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‡ãƒ¼ã‚·ãƒ§ãƒ³ã®å‡¦ç†è‡ªä½“ã¯ã‚·ãƒ³ãƒ—ãƒ« æ¥½å¤©å¸‚å ´ã§ã¯ç¾åœ¨ã€2600ä¸‡ç‚¹ã®å•†å“ã‚’å–ã‚Šæ‰±ã„ã€4200ä¸‡äººã®ä¼šå“¡ã«å¯¾ã—ã¦ã‚µãƒ¼ãƒ“ã‚¹ã‚’æä¾›ã—ã¦ã„ã‚‹ã€‚ã“ã®è¦æ¨¡ã®ä¼šå“¡æ•°ãƒ»å•†å“ç‚¹æ•°ã§ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‡ãƒ¼ã‚·ãƒ§ãƒ³ï¼ˆå•†å“ã®æŽ¨è–¦ï¼‰ã‚’è¡Œã†ã®ã¯å®¹æ˜“ã§ã¯ãªã„ã€‚ â€»è¨˜äº‹åˆå‡ºæ™‚ã«æ¥½å¤©å¸‚å ´ã®ä¼šå“¡æ•°ã‚’4800ä¸‡äººã¨ã—ã¦ã‚ã‚Šã¾ã—ãŸãŒã€ã“ã‚Œã¯æ¥½å¤©ã‚°ãƒ«ãƒ¼ãƒ—ã®ã‚µãƒ¼ãƒ“ã‚¹åˆ©ç”¨è€…å…¨ä½“ã®æ•°å—ã§ã—ãŸã€‚æ¥½å¤©å¸‚å ´ã®ä¼šå“¡æ•°ã¯æ£ã—ãã¯4200ä¸‡äººã¨ã®ã“ã¨ã§ã™ã€‚ãŠè©«ã³ã—ã¦è¨‚æ£ã„ãŸã—ã¾ã™ã€‚ ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‡ãƒ¼ã‚·ãƒ§ãƒ³ã®ä»•çµ„ã¿ã¨ã—ã¦åŒç¤¾ã¯ã€ä¸€èˆ¬çš„ã§ã‚·ãƒ³ãƒ—ãƒ«ãªã‚¢
maxy 2009/02/24
roma

scalability

ruby

hadoop

mapreduce

programming
ãƒªãƒ³ã‚¯
Hadoop Streaming
Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows you to create and run map/reduce jobs with any executable or script as the mapper and/or the reducer. For example: $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/hadoop-streaming.jar \ -input myInputDirs \ -output myOutputDir \ -mapper /bin/cat \ -reducer /bin/wc In the above example, both the mapper and the reduce
maxy 2008/11/30
hadoop

streaming
ãƒªãƒ³ã‚¯
HadoopStreaming - HADOOP2 - Apache Software Foundation
Hadoop Streaming is a utility which allows users to create and run jobs with any executables (e.g. shell utilities) as the mapper and/or the reducer. Usage: $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/mapred/contrib/streaming/hadoop-streaming.jar [options] Options: -input <path> DFS input file(s) for the Map step -output <path> DFS output directory for the Reduce step -mapper <cmd|JavaClassName> The
maxy 2008/11/30
hadoop

streaming
ãƒªãƒ³ã‚¯
InfoQ: Cascading -- Hadoop MapReduceå‘ã‘ãƒ‡ãƒ¼ã‚¿å‡¦ç†ç³»API
Spring Bootã«ã‚ˆã‚‹APIãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰æ§‹ç¯‰å®Ÿè·µã‚¬ã‚¤ãƒ‰ ç¬¬2ç‰ˆ ä½•åƒäººã‚‚ã®é–‹ç™ºè€…ãŒã€InfoQã®ãƒŸãƒ‹ãƒ–ãƒƒã‚¯ã€ŒPractical Guide to Building an API Back End with Spring Bootã€ã‹ã‚‰ã€Spring Bootã‚’ä½¿ã£ãŸREST APIæ§‹ç¯‰ã®åŸºç¤Žã‚’å¦ã‚“ã ã€‚ã“ã®æœ¬ã§ã¯ã€å‡ºç‰ˆæ™‚ã«æ–°ã—ããƒªãƒªãƒ¼ã‚¹ã•ã‚ŒãŸãƒãƒ¼ã‚¸ãƒ§ãƒ³ã§ã‚ã‚‹ Spring Boot 2 ã‚’ä½¿ç”¨ã—ã¦ã„ã‚‹ã€‚ã—ã‹ã—ã€Spring Boot3ãŒæœ€è¿‘ãƒªãƒªãƒ¼ã‚¹ã•ã‚Œã€é‡è¦ãªå¤‰...
maxy 2008/10/24
hadoop

mapreduce
ãƒªãƒ³ã‚¯
MapReduceã¨Hadoopã®å°†æ¥ã«ã¤ã„ã¦ã€Yahooã®Doug Cuttingã«ã‚¤ãƒ³ã‚¿ãƒ“ãƒ¥ãƒ¼
Spring Bootã«ã‚ˆã‚‹APIãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰æ§‹ç¯‰å®Ÿè·µã‚¬ã‚¤ãƒ‰ ç¬¬2ç‰ˆ ä½•åƒäººã‚‚ã®é–‹ç™ºè€…ãŒã€InfoQã®ãƒŸãƒ‹ãƒ–ãƒƒã‚¯ã€ŒPractical Guide to Building an API Back End with Spring Bootã€ã‹ã‚‰ã€Spring Bootã‚’ä½¿ã£ãŸREST APIæ§‹ç¯‰ã®åŸºç¤Žã‚’å¦ã‚“ã ã€‚ã“ã®æœ¬ã§ã¯ã€å‡ºç‰ˆæ™‚ã«æ–°ã—ããƒªãƒªãƒ¼ã‚¹ã•ã‚ŒãŸãƒãƒ¼ã‚¸ãƒ§ãƒ³ã§ã‚ã‚‹ Spring Boot 2 ã‚’ä½¿ç”¨ã—ã¦ã„ã‚‹ã€‚ã—ã‹ã—ã€Spring Boot3ãŒæœ€è¿‘ãƒªãƒªãƒ¼ã‚¹ã•ã‚Œã€é‡è¦ãªå¤‰...
maxy 2008/03/14
mapre

hadoop

interview
ãƒªãƒ³ã‚¯
HBase - HADOOP2 - Apache Software Foundation
HBase: Bigtable-like structured storage for Hadoop HDFS HBase has an awesome community resource in our online book. It's maintained and looked after with diligence. In contrast, the below Wiki content has not been updated since 2012. For that reason, the Wiki has been deprecated (see HBASE-14481). Please update your links. You can find historical pages (which have been updated to give the new link
maxy 2008/03/14
bigtable

hadoop
ãƒªãƒ³ã‚¯
Yahoo!'s bet on Hadoop - O'Reilly Radar
One of the most important announcements at Oscon last week was Yahoo!'s commitment to support Hadoop. We've been writing about Hadoop on radar for a while, so it's probably not news to you that we think Hadoop is important. Yahoo's involvement wasn't actually news either, because Yahoo! had hired Doug Cutting, the creator of hadoop, back in January. But Doug's talk at Oscon was kind of a coming ou
maxy 2008/03/14
yahoo

hadoop
ãƒªãƒ³ã‚¯
1 2 æ¬¡ã®ãƒšãƒ¼ã‚¸