[B! s3][elastic map reduce] kanbayashiã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

kanbayashi id:kanbayashi

s3ã¨elastic map reduceã«é–¢ã™ã‚‹kanbayashiã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (5)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Forums
kanbayashi 2011/05/06
elastic map reduce

hdfs

hadoop

emr

s3

amazon s3
ãƒªãƒ³ã‚¯
Amazon S3 Tools: Command Line S3 Client and S3 Backup for Windows, Linux: s3cmd, s3express
Amazon S3 Tools: Command Line S3 Client Software and S3 Backup AWS S3 Command Line Clients for Windows, Linux, Mac. Backup to S3, upload, retrieve, query data on Amazon S3. S3cmd : Command Line S3 Client and Backup for Linux and Mac Amazon S3 is a reasonably priced data storage service. Ideal for off-site file backups, file archiving, web hosting and other data storage needs. It is generally more
kanbayashi 2011/05/06
s3

elastic map reduce

storage

cloud

amazon s3

api

command line

s3cmd
ãƒªãƒ³ã‚¯
CUIã§Amazon S3ã‚’æ“ä½œã™ã‚‹Â·s3cmd MOONGIFT
Amazon EC2ã‚’ä½¿ã£ã¦ã„ã‚‹äººã§ã‚ã‚Œã°ã€æ·»ä»˜ãƒ•ã‚¡ã‚¤ãƒ«ã‚„ç”»åƒã®ç½®ãå ´ã¨ã—ã¦Amazon S3ã‚’ä½¿ã£ã¦ã„ã‚‹äººã¯å¤šã„ã¯ãšã ã€‚ä»–ã«ã‚‚ãƒãƒƒã‚¯ã‚¢ãƒƒãƒ—ç”¨é€”ã‚„ã€å¤§é‡ã®ãƒ‡ãƒ¼ã‚¿ã‚’ä¿å˜ã—ã¦ãŠãå ´æ‰€ã¨ã—ã¦Amazon S3ã¯ä¾¿åˆ©ãªWebã‚µãƒ¼ãƒ“ã‚¹ã ã€‚ ãƒ˜ãƒ«ãƒ— ã ãŒAmazon S3ã¯Web APIã«ç‰¹åŒ–ã—ã¦ãŠã‚Šãã®æ“ä½œã«ã¯ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã‚’ä»‹ã—ã¦è¡Œã†ã®ãŒä¸€èˆ¬çš„ã ã€‚ã ãŒã“ã‚Œã§ã¯è‡ªå‹•åŒ–ã™ã‚‹éš›ã«ã¯ãƒ—ãƒã‚°ãƒ©ãƒ ã‚’çµ„ã‚€å¿…è¦ãŒã‚ã‚Šé¢å€’ã ã€‚ãã“ã§Amazon S3ã‚’ã‚¿ãƒ¼ãƒŸãƒŠãƒ«ã‹ã‚‰æ“ä½œã™ã‚‹s3cmdã‚’ä½¿ãŠã†ã€‚ ä»Šå›žç´¹ä»‹ã™ã‚‹ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ãƒ»ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã¯s3cmdã€Amazon S3ã®CUI ãƒ•ãƒãƒ³ãƒˆã‚¨ãƒ³ãƒ‰ã ã€‚ s3cmdã¯Pythonã§ä½œã‚‰ã‚ŒãŸã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã§ã€Amazon S3ã®æ“ä½œã‚’ã‚¿ãƒ¼ãƒŸãƒŠãƒ«ã‹ã‚‰è¡Œã†ã“ã¨ãŒã§ãã‚‹ã€‚ã“ã‚Œã§ã‚ã‚Œã°ãƒ•ã‚¡ã‚¤ãƒ«ã®é€ä¿¡ã‚„å—ä¿¡ãªã©ã‚’ã‚³ãƒžãƒ³ãƒ‰ã§è¡Œãˆã‚‹ã‚ˆã†ã«ãªã‚‹ã®ã§è‡ªå‹•åŒ–ã‚‚å®¹æ˜“ã«ãªã‚‹ã ã‚ã†ã€‚ ãƒªã‚¹ãƒˆã‚³ãƒžãƒ³ãƒ‰ã‚’å®Ÿè¡Œã—ãŸæ‰€
kanbayashi 2011/05/06
amazon s3

s3

cloud

ruby

storage

api

command line

elastic map reduce
ãƒªãƒ³ã‚¯
Ruby/Amazon S3ã‚’ä½¿ã† - ä¿ºã®åŸºåœ°
ãƒãƒ¼ã‚¸ãƒ§ãƒ³ã¨è£½é€ å¹´æœˆæ—¥ 2009å¹´09æœˆ08æ—¥ Ruby 1.8.6 aws-s3 0.6.2 WindowsXP SP3 â–²Â â–¼ ç’°å¢ƒæ§‹ç¯‰ Amazonã®ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã‚µãƒ¼ãƒ“ã‚¹ã‚’S3ã‹ã‚‰ä½¿ã†ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã€€aws-s3ã€€ãŒã‚ã‚‹ã®ã§ã“ã„ã¤ã‚’ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã™ã‚‹ gem install aws-s3 ã“ã„ã¤ã«ã¯3ã¤ã®ä¾å˜æ€§ãŒã‚ã‚Šgemãªã‚‰è‡ªå‹•çš„ã«ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã•ã‚Œã‚‹ã‚“ã ãŒã€ä¸€å¿œä¹—ã£ã‘ã¦ãŠã xml-simple http://rubyforge.org/projects/xml-simple/ builder http://rubyforge.org/projects/builder/ mime-types http://rubyforge.org/projects/mime-types/ â–²Â â–¼ æŽ¥ç¶šè¨å®š ã¾ãšæŽ¥ç¶šè¨å®šã‚’è¡Œã†ã€‚ã“ã„ã¤ã¯Amazonã®ã‚µãƒ¼ãƒ“ã‚¹ã§æ¯Žåº¦ä½¿ã†ã€ç”Ÿæˆidã¨keyã®
kanbayashi 2011/05/06
ruby

aws

s3

ec2

elastic map reduce

mapreduce
ãƒªãƒ³ã‚¯
ã‚¯ãƒ©ã‚¦ãƒ‰åž‹ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã€ŒAmazon S3ã€ã¯å®‰ã„ã‹ï¼Ÿ
Amazon S3ã¯å®‰ã„ã‹é«˜ã„ã‹ã¨ã„ã†è°è«–ã‚’ã€ã“ã®2æ—¥é–“ã§ç«‹ã¦ç¶šã‘ã«3å€‹æ‰€ã§è€³ï¼ˆç›®ï¼‰ã«ã—ãŸã€‚ 1äººã¯ãƒãƒƒãƒˆã‚µãƒ¼ãƒ“ã‚¹ãƒ™ãƒ³ãƒãƒ£ãƒ¼ã®CTOã§ã€ã‚¯ãƒ©ã‚¦ãƒ‰ã¸ã®ç§»è¡Œã‚’æ¤œè¨Žã—ã¦ã„ã‚‹ã‚‚ã®ã®ã€Amazon S3ã‚„EC2ã®å®Ÿéš›ã®ä¾¡æ ¼ã‚„ä½¿ã„å‹æ‰‹ãŒã‚ˆãåˆ†ã‹ã‚‰ãªã„ã¨ã„ã†è©±ã ã£ãŸã€‚ã‚³ã‚¹ãƒˆçš„ã«è¦‹åˆã†ãªã‚‰ã€ã‚‚ã†ã‚µãƒ¼ãƒé‹ç”¨ã«ç…©ã‚ã•ã‚ŒãŸããªã„ã¨æ¼ã‚‰ã—ã¦ã„ãŸã€‚ ã‚‚ã†1ã¤ã¯ã€ãƒ™ãƒ³ãƒãƒ£ãƒ¼ã‚ãƒ£ãƒ”ã‚¿ãƒ«ã¨ã—ã¦çŸ¥ã‚‰ã‚Œã‚‹Yã‚³ãƒ³ãƒ“ãƒãƒ¼ã‚¿ãƒ¼ã®æŽ²ç¤ºæ¿ã§è¦‹ã‹ã‘ãŸã€ŒDoes Amazon S3 really save moneyï¼Ÿã€ã§å§‹ã¾ã‚‹è°è«–ã€‚1GBå½“ãŸã‚Š1ã‚«æœˆã§0.15ãƒ‰ãƒ«ã¨èžãã¨å®‰ã„ã‚ˆã†ã«ã‚‚æ€ãˆã‚‹ãŒã€1TBã®æœˆé¡ã¯150ãƒ‰ãƒ«ã€2å¹´ã«ã™ã‚‹ã¨3600ãƒ‰ãƒ«ï¼ˆ1ãƒ‰ãƒ«92å††ã¨ã—ã¦ç´„33ä¸‡å††ï¼‰ã«ã‚‚ãªã‚‹ã€‚ã—ã‹ã‚‚è»¢é€é‡ã«å¿œã˜ãŸèª²é‡‘ã‚‚ã‚ã‚‹ãŸã‚ã€å®Ÿéš›ã«ã¯ã“ã‚Œä»¥ä¸Šã«ãªã‚‹ã€‚ä»Šã‚„1TBã®ãƒ‰ãƒ©ã‚¤ãƒ–å˜ä¾¡ã¯1ä¸‡å††ã‚’å‰²ã£ã¦ã„ã¦ã€2å¹´ã§3600ãƒ‰ãƒ«ã¨ã¯æ¯”è¼ƒã«ã‚‚ãªã‚‰ãªã„ã€‚ã€Œã‚¯ãƒ©ã‚¦ãƒ‰
kanbayashi 2011/04/10
amzon s3

s3

aws

storage

cloud

cost

elastic map reduce
ãƒªãƒ³ã‚¯
1