[B! S3] e-kurodaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

e-kuroda id:e-kuroda

S3ã«é–¢ã™ã‚‹e-kurodaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (10)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

blogeyeã®å®Ÿè£…ã«å¦ã¶ã€Amazon EC2/S3ã§ã®Hadoopæ´»ç”¨è¡“
ã¯ã˜ã‚ã« ã“ã‚“ã«ã¡ã¯ã€‚Hadoopé€£è¼‰ ç¬¬4å›žã¯å¤ªç”°ã•ã‚“ã«ä»£ã‚ã£ã¦å¤§å€‰ãŒæ‹…å½“ã—ã¾ã™ã€‚ ã“ã‚Œã¾ã§ã®é€£è¼‰ã§ã€Hadoopã«ã‚ˆã‚‹ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã®æ¦‚ç•¥ã«ã¤ã„ã¦ã¯ç†è§£ã•ã‚Œã¦ã„ã‚‹ã¨æ€ã„ã¾ã™ã€‚ä»Šå›žã¯Hadoopã‚’åˆ©ç”¨ã—ãŸã‚·ã‚¹ãƒ†ãƒ ã®å®Ÿä¾‹ã¨ã„ã†ã“ã¨ã§ã€ãƒ–ãƒã‚°åˆ†æžã‚’è¡Œã†ã€Œblogeyeã€ã‚·ã‚¹ãƒ†ãƒ ã®æ¦‚ç•¥ã¨ã€ãã®ä¸ã§ã®Hadoopåˆ©ç”¨æ³•ã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ ã¾ãŸã€blogeyeã¯AmazonãŒæä¾›ã—ã¦ã„ã‚‹EC2ï¼ˆãƒ¬ãƒ³ã‚¿ãƒ«ã‚µãƒ¼ãƒï¼‰ã€S3ï¼ˆã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ï¼‰ã‚’Hadoopã¨çµ„ã¿åˆã‚ã›ã¦åˆ©ç”¨ã—ã¦ã„ã‚‹ã®ã§ã€ãã®è¾ºã‚Šã®å°Žå…¥æ–¹æ³•ã«ã¤ã„ã¦ã‚‚ç´¹ä»‹ã—ã¾ã™ã€‚ ã“ã‚Œã¾ã§ã®é€£è¼‰ Hadoopã€hBaseã§æ§‹ç¯‰ã™ã‚‹å¤§è¦æ¨¡åˆ†æ•£ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚·ã‚¹ãƒ†ãƒ Hadoopã®ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã¨ã‚µãƒ³ãƒ—ãƒ«ãƒ—ãƒã‚°ãƒ©ãƒ ã®å®Ÿè¡Œ è¤‡æ•°ãƒžã‚·ãƒ³ã¸Hadoopã‚’ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã™ã‚‹ blogeyeã¨ã¯ ã€Œblogeyeã€ï¼ˆãƒ–ãƒã‚°ã‚¢ã‚¤ï¼‰ã¯æ—¥æœ¬èªžã®ãƒ–ãƒã‚°ã‚’ã‚¯ãƒãƒ¼ãƒ«ã€ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ã«åˆ†æžã—ã¦ã€æµè¡Œèªžã¨æ€ã‚
e-kuroda 2009/04/02
Amazon

EC2

S3

Hadoop

MapReduce
ãƒªãƒ³ã‚¯
blogeyeã®å®Ÿè£…ã«å¦ã¶ã€Amazon EC2/S3ã§ã®Hadoopæ´»ç”¨è¡“ï¼šCodeZine
ã¯ã˜ã‚ã«ã€€ã“ã‚“ã«ã¡ã¯ã€‚Hadoopé€£è¼‰ ç¬¬4å›žã¯å¤ªç”°ã•ã‚“ã«ä»£ã‚ã£ã¦å¤§å€‰ãŒæ‹…å½“ã—ã¾ã™ã€‚ã€€ã“ã‚Œã¾ã§ã®é€£è¼‰ã§ã€Hadoopã«ã‚ˆã‚‹ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã®æ¦‚ç•¥ã«ã¤ã„ã¦ã¯ç†è§£ã•ã‚Œã¦ã„ã‚‹ã¨æ€ã„ã¾ã™ã€‚ä»Šå›žã¯Hadoopã‚’åˆ©ç”¨ã—ãŸã‚·ã‚¹ãƒ†ãƒ ã®å®Ÿä¾‹ã¨ã„ã†ã“ã¨ã§ã€ãƒ–ãƒã‚°åˆ†æžã‚’è¡Œã†ã€Œblogeyeã€ã‚·ã‚¹ãƒ†ãƒ ã®æ¦‚ç•¥ã¨ã€ãã®ä¸ã§ã®Hadoopåˆ©ç”¨æ³•ã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ã€€ã¾ãŸã€blogeyeã¯AmazonãŒæä¾›ã—ã¦ã„ã‚‹EC2ï¼ˆãƒ¬ãƒ³ã‚¿ãƒ«ã‚µãƒ¼ãƒï¼‰ã€S3ï¼ˆã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ï¼‰ã‚’Hadoopã¨çµ„ã¿åˆã‚ã›ã¦åˆ©ç”¨ã—ã¦ã„ã‚‹ã®ã§ã€ãã®è¾ºã‚Šã®å°Žå…¥æ–¹æ³•ã«ã¤ã„ã¦ã‚‚ç´¹ä»‹ã—ã¾ã™ã€‚ã“ã‚Œã¾ã§ã®é€£è¼‰Hadoopã€hBaseã§æ§‹ç¯‰ã™ã‚‹å¤§è¦æ¨¡åˆ†æ•£ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚·ã‚¹ãƒ†ãƒ Hadoopã®ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã¨ã‚µãƒ³ãƒ—ãƒ«ãƒ—ãƒã‚°ãƒ©ãƒ ã®å®Ÿè¡Œè¤‡æ•°ãƒžã‚·ãƒ³ã¸Hadoopã‚’ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã™ã‚‹ blogeyeã¨ã¯ã€€ã€Œblogeyeã€ï¼ˆãƒ–ãƒã‚°ã‚¢ã‚¤ï¼‰ã¯æ—¥æœ¬èªžã®ãƒ–ãƒã‚°ã‚’ã‚¯ãƒãƒ¼ãƒ«ã€ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ã«åˆ†æžã—ã¦ã€æµè¡Œèªžã¨æ€ã‚ã‚Œã‚‹ã‚‚ã®
e-kuroda 2009/04/02
EC2

S3

Amazon

Hadoop
ãƒªãƒ³ã‚¯
AWS::S3 - Ruby Library for Amazon Simple Storage Service (S3)
A Ruby Library for Amazon's Simple Storage Service's (S3) REST API. Browse code | Docs | Downloads | Bugs & Patches | Mailing list Download with RubyGems sudo gem i aws-s3 Clone from the master git repository git clone git://github.com/marcel/aws-s3.git
e-kuroda 2009/04/02
Amazon

AWS

S3

Ruby
ãƒªãƒ³ã‚¯
Amazon EC2/S3ã‚’ä½¿ã£ã¦ã¿ãŸ - 5.S3ã‚’ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã¨ã—ã¦æ‰±ã† - å…ƒRX-7ä¹—ã‚Šã®é©å½“ãªæ—¥ã€…
ã•ã¦ã€ãªã‚“ã ã‹ã™ã£ã‹ã‚Šã‚·ãƒªãƒ¼ã‚ºåŒ–ã—ã¦ãã¦ã—ã¾ã„ã¾ã—ãŸã€‚ ä»Šæ—¥ã¯ã€Amazon EC2ã‹ã‚‰S3ã‚’ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã¨ã—ã¦ãƒžã‚¦ãƒ³ãƒˆã™ã‚‹æ–¹æ³•ã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ Amazon EC2ã§ã¯ã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹ã‚’åœæ¢ã•ã›ã¦ã—ã¾ã†ã¨ã€ä¿å˜ã—ãŸãƒ‡ã‚£ã‚¹ã‚¯ã®å†…å®¹ãŒå¤±ã‚ã‚Œã¦ã—ã¾ã†ã“ã¨ã¯å…ˆæ—¥ã®ã‚¨ãƒ³ãƒˆãƒª(d:id:rx7:20080425:p3)ã§æ›¸ã„ãŸé€šã‚Šã§ã€ä¸€èˆ¬çš„ã«ã¯S3ã¨ã„ã£ãŸã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã«å¿…è¦ãªãƒ‡ãƒ¼ã‚¿ã‚’é‹ç”¨ã§ä¿å˜ã•ã›ã‚‹å¿…è¦ãŒå‡ºã¦ãã¾ã™ã€‚ ã§ã€ä¸–ã«ã¯è‰²ã€…ã¨S3ã¸ã®ãƒãƒƒã‚¯ã‚¢ãƒƒãƒ—ãƒ„ãƒ¼ãƒ«ãŒã‚ã‚Šã€ä»¥ä¸‹ã®ãƒšãƒ¼ã‚¸ã®ãƒªã‚¹ãƒˆãŒå‚è€ƒã«ãªã‚Šã¾ã™ãŒã€ä»Šã¯"S3Sync"ã‚ãŸã‚ŠãŒãƒ¡ã‚¸ãƒ£ãƒ¼ãªã¨ã“ã‚ã§ã—ã‚‡ã†ã‹ã€‚ s3sync.rb is written in Ruby as a sort of rsync clone to replace the perl script s3sync which is now abandonware. Given th
e-kuroda 2009/04/02
Amazon

EC2

S3

backup
ãƒªãƒ³ã‚¯
Google Code Archive - Long-term storage for Google Code Project Hosting.
Code Archive Skip to content Google About Google Privacy Terms
e-kuroda 2009/04/02
Amazon

AWS

S3

tools
ãƒªãƒ³ã‚¯
é€£è¼‰ï¼šã¯ã˜ã‚ã¦ã®Amazon EC2&S3ã€€ã€œã“ã‚Œã‹ã‚‰ã®æ–°ã‚µãƒ¼ãƒ“ã‚¹ã®å…¬é–‹ã®å½¢ã€œï½œgihyo.jp â€¦ æŠ€è¡“è©•è«–ç¤¾
ç¬¬1å›žä»®æƒ³ãƒžã‚·ãƒ³ãƒ¬ãƒ³ã‚¿ãƒ«ã‚µãƒ¼ãƒ“ã‚¹Amazon EC2ã€å¤§å®¹é‡ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã‚µãƒ¼ãƒ“ã‚¹Amazon S3ã‚’ä½¿ã£ã¦ã¿ã‚ˆã† å¢—äº•é›„ä¸€éƒŽ 2008-05-19
e-kuroda 2009/04/02
Amazon

AWS

EC2

S3

tutorial
ãƒªãƒ³ã‚¯
ã¯ã˜ã‚ã¦ã®Amazon EC2&S3ã€€ã€œã“ã‚Œã‹ã‚‰ã®æ–°ã‚µãƒ¼ãƒ“ã‚¹ã®å…¬é–‹ã®å½¢ã€œï¼šç¬¬2å›žã€€Amazon S3ã‚’ãƒãƒƒã‚¯ã‚¢ãƒƒãƒ—ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã¨ã—ã¦ä½¿ãŠã†ï½œgihyo.jp â€¦ æŠ€è¡“è©•è«–ç¤¾
Amazon S3ã‚’ãƒãƒƒã‚¯ã‚¢ãƒƒãƒ—ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã¨ã—ã¦ä½¿ãŠã† S3ã¯ã€ãã®ä¿¡é ¼æ€§ã®é«˜ã•ã‹ã‚‰ãƒãƒƒã‚¯ã‚¢ãƒƒãƒ—ç”¨ã®ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã¨ã—ã¦æœ€é©ãªã‚µãƒ¼ãƒ“ã‚¹ã§ã™ã€‚ã¾ãŸã€S3ã¯å„ç¨®è¨€èªžç”¨ã®ãƒ©ã‚¤ãƒ–ãƒ©ãƒªãŒãã‚ã£ã¦ã„ã‚‹ã®ã§ã€è‡ªåˆ†ã§ç°¡å˜ã«ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã‚’ä½œã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ ãã“ã§ä»Šå›žã¯ã€Rubyã‚’ä½¿ã£ã¦ç°¡å˜ãªãƒãƒƒã‚¯ã‚¢ãƒƒãƒ—ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã‚’ä½œã£ã¦ã¿ã¾ã—ã‚‡ã†ã€‚ ãªãŠã€å„ç¨®è¨€èªžç”¨ã®ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã®æƒ…å ±ã‚‚AWSãƒˆãƒƒãƒ—ãƒšãƒ¼ã‚¸ã‹ã‚‰å…¥æ‰‹ã§ãã¾ã™ã€‚ Rubyãƒ©ã‚¤ãƒ–ãƒ©ãƒªaws-s3ã®å°Žå…¥ Rubyã‹ã‚‰S3ã‚’æ“ä½œã™ã‚‹ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã¯è¤‡æ•°ç”¨æ„ã•ã‚Œã¦ã„ã¾ã™ãŒã€ä»Šå›žã¯ã€Œaws-s3ã€ã‚’ä½¿ã£ã¦ã¿ã¾ã™ã€‚ aws-s3ã¯RubyGemsã§æä¾›ã•ã‚Œã¦ã„ã¾ã™ã®ã§ã€gemã‚³ãƒžãƒ³ãƒ‰ã§ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã—ã¾ã™ã€‚ # gem install aws-s3 ãƒã‚±ãƒƒãƒˆä¸€è¦§ã®å–å¾— ã¾ãšã¯ç·´ç¿’ã¨ã—ã¦ã€ãƒã‚±ãƒƒãƒˆä¸€è¦§ã‚’å–å¾—ã™ã‚‹ã‚³ãƒ¼ãƒ‰ã‚’æ›¸ã„ã¦ã¿ã¾ã™ï¼ˆãƒªã‚¹ãƒˆ1â ï¼‰â ã€‚AWS::S3::Base
e-kuroda 2009/04/02
Amazon

S3
ãƒªãƒ³ã‚¯
http://news.tokyo.wakhok.ac.jp/x/modules/tinyd1/content/newtech2007/Amazon_S3_EC2-2/siframes.html
e-kuroda 2009/04/02
Amazon

S3

EC2

hosting
ãƒªãƒ³ã‚¯
Amazon Web Services Developer Community : Using Amazon S3, EC2, SQS, Lucene, and Ruby for Web Spidering
Robert Dempsey guides you through the steps of building a web spidering application using Amazon EC2, Amazon SQS, and Amazon S3. AWS Products Used: By Robert Dempsey of Atlantic Dominion Solutions, LLC Each Amazon Web Services (AWS) product is, in and of itself, very powerful. By combining them, you can produce true enterprise-class web applications. This article guides you through building on
e-kuroda 2009/04/02
Amazon

S3

EC2

Ruby

Lucene
ãƒªãƒ³ã‚¯
MOONGIFT: ? Amazon S3ã‚¯ãƒãƒ¼ãƒ³ã‚’ä½œã‚‹ã€ŒPark Placeã€:ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã‚’æ¯Žæ—¥ç´¹ä»‹
Amazon S3ã¯å‰ã€…ã‹ã‚‰å°‘ã—ãšã¤è§¦ã£ã¦ã„ãŸã®ã ãŒã€ã“ã‚Œã¯ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã‚ˆã‚Šã‚‚ãƒ—ãƒã‚°ãƒ©ãƒ ã‹ã‚‰è§¦ã£ãŸæ–¹ãŒæ¥½ã—ã„ã‹ã‚‚çŸ¥ã‚Œãªã„ã€‚ç°¡å˜ã«ã‚¢ãƒƒãƒ—ãƒãƒ¼ãƒ‰ã§ãã€ãƒ¡ã‚¿ãƒ‡ãƒ¼ã‚¿ã‚„ã‚³ãƒ³ãƒ†ãƒ³ãƒ„ã‚¿ã‚¤ãƒ—ã‚’è‡ªåˆ†ã§è¿½åŠ ã§ãã€ã•ã‚‰ã«ã‚¢ã‚¯ã‚»ã‚¹æ¨©é™ã‚‚å‹•çš„ã«è¨å®šã§ãã‚‹ã€‚ éžå¸¸ã«ãƒ¦ãƒ‹ãƒ¼ã‚¯ã§é¢ç™½ã„ã‚µãƒ¼ãƒ“ã‚¹ã§ã¯ã‚ã‚‹ã®ã ãŒã€ä¸€ã¤æ¬ ç‚¹ãŒã‚ã‚‹ã€‚ãã‚Œã¯é–‹ç™ºä¸ã‚‚Amazon S3ã‚’ä½¿ã‚ãã°ãªã‚‰ãšã€ãã®ãŸã‚ã«ä½™è¨ˆãªã‚³ã‚¹ãƒˆï¼ˆæã‚‰ãä½Žã‚³ã‚¹ãƒˆã ãŒï¼‰ãŒç™ºç”Ÿã—ãŸã‚Šã€ã‚¤ãƒ³ã‚¿ãƒ¼ãƒãƒƒãƒˆã«æŽ¥ç¶šã•ã‚Œã¦ã„ã‚‹å¿…è¦æ€§ãŒç”Ÿã˜ã‚‹ã€‚ãã‚Œã§ã¯ã„ã¤ã§ã‚‚ã©ã“ã§ã‚‚é–‹ç™ºã¨ã„ã†ã‚ã‘ã«ã¯ã„ã‹ãªã„ã€‚ãã“ã§ã“ã‚Œã ã€‚ ä»Šå›žç´¹ä»‹ã™ã‚‹ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ãƒ»ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã¯Park Placeã€Amazon S3ã‚¯ãƒãƒ¼ãƒ³ã ã€‚ Park Placeã¯Rubyã§ä½œã‚‰ã‚ŒãŸã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã§ã€Mongrelã‚’ä½¿ã£ãŸHTTPã‚µãƒ¼ãƒã‚’åˆ©ç”¨ã—ã¦Amazon S3ã®çœŸä¼¼ã‚’ã™ã‚‹ã€‚SOAPã‚¤ãƒ³ã‚¿ãƒ•ã‚§ãƒ¼ã‚¹ãŒåˆ©ç”¨ã§ãã‚‹ã‚ˆã†ã«ãªã£ã¦ã„ã‚‹ã€‚
e-kuroda 2009/04/02
Amazon

S3

opensource

development

tools
ãƒªãƒ³ã‚¯
1