[B! s3] msyktã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

msykt id:msykt

s3ã«é–¢ã™ã‚‹msyktã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (21)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Amazon S3ã®ãƒ‡ãƒ¼ã‚¿ã‚’ç›´æŽ¥æ¤œç´¢ã§ãã‚‹ã€ŒMongoDB Atlas Data Lakeã€æ£å¼ãƒªãƒªãƒ¼ã‚¹ã€‚ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã¸ã®è»¢é€ä¸è¦ã€MongoDBã®ã‚¯ã‚¨ãƒªè¨€èªžã‚’ç›´æŽ¥å®Ÿè¡Œ
Amazon S3ã®ãƒ‡ãƒ¼ã‚¿ã‚’ç›´æŽ¥æ¤œç´¢ã§ãã‚‹ã€ŒMongo DB Atlas Data Lakeã€æ£å¼ãƒªãƒªãƒ¼ã‚¹ã€‚ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã¸ã®è»¢é€ä¸è¦ã€Mongo DBã®ã‚¯ã‚¨ãƒªè¨€èªžã‚’ç›´æŽ¥å®Ÿè¡Œ Mongo DBã¯ã€Amazon S3ã«ä¿å˜ã•ã‚Œã¦ã„ã‚‹ãƒ‡ãƒ¼ã‚¿ã‚’ãã®ã¾ã¾ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã¸è»¢é€ã™ã‚‹ã“ã¨ãªãã€ç›´æŽ¥Mongo DBã®ã‚¯ã‚¨ãƒªè¨€èªžã§æ¤œç´¢å¯èƒ½ã«ã™ã‚‹ã‚¯ãƒ©ã‚¦ãƒ‰ã‚µãƒ¼ãƒ“ã‚¹ã€ŒMongo DB Atlas Data Lakeã€ã®æ£å¼ãƒªãƒªãƒ¼ã‚¹ã‚’ç™ºè¡¨ã—ã¾ã—ãŸã€‚ ç™ºè¡¨ã¯åŒç¤¾ãŒã‚ªãƒ³ãƒ©ã‚¤ãƒ³ã§é–‹å‚¬ã—ãŸã‚¤ãƒ™ãƒ³ãƒˆã€ŒMongo DB.Live 2020ã€ã§è¡Œã‚ã‚Œã¾ã—ãŸã€‚ é€šå¸¸ã€ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã§æ¤œç´¢ã‚„åˆ†æžã®å¯¾è±¡ã¨ãªã‚‹ãƒ‡ãƒ¼ã‚¿ã¯ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹å†…ã«ä¿å˜ã•ã‚Œã¦ã„ã‚‹ãƒ‡ãƒ¼ã‚¿ã§ã‚ã‚‹ãŸã‚ã€ãƒ•ã‚¡ã‚¤ãƒ«ã‚µãƒ¼ãƒã‚„ã‚ªãƒ–ã‚¸ã‚§ã‚¯ãƒˆã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ãªã©ã«ä¿å˜ã•ã‚Œã¦ã„ã‚‹JSONãƒ‡ãƒ¼ã‚¿ã‚„CSVãƒ•ã‚¡ã‚¤ãƒ«ãªã©ã¯ã€ä½•ã‚‰ã‹ã®æ–¹æ³•ã§ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã¸ã‚¤ãƒ³ãƒãƒ¼ãƒˆã™ã‚‹å¿…è¦ãŒã‚ã‚Šã¾ã™ã€‚ Mongo DB Atlas Data
msykt 2020/06/12
S3 ã®ãƒ‡ãƒ¼ã‚¿ã‚’æ¤œç´¢ã§ãã‚‹ã“ã¨ã‚ˆã‚Šã‚‚ã€MongoDB ãŒ Avro ã‚„ Parquet ã‚’ã‚µãƒãƒ¼ãƒˆã—ã¦ã„ãŸã®çŸ¥ã‚‰ãªã‹ã£ãŸã€‚ç¢ºã‹ã« MQL ã¨ãƒã‚¹ãƒˆæ§‹é€ ã®ãƒ‡ãƒ¼ã‚¿ã®ç›¸æ€§ã¯è‰¯ã•ãã†ã ã‘ã©â€¦

mongodb

s3
ãƒªãƒ³ã‚¯
https://docs.confluent.io/current/connect/kafka-connect-s3/index.html
msykt 2020/02/11
Kafka ã® S3 Sink Connectorã€‚Kafka Connect ã® Sink ã¯ å‡ºåŠ›å…ˆ / Partitioner / Format ã¨ã„ã†è¤‡æ•°ã®è¦ç´ ã‚’ 1 ã¤ã® Sink ã§æ‰±ã†ã“ã¨ã«ãªã‚‹ã®ã§å¤§å¤‰ã ãªâ€¦

kafka

s3
ãƒªãƒ³ã‚¯
ã‚µãƒ¼ãƒãƒ¬ã‚¹æ™‚ä»£ã«ãŠã‘ã‚‹ãƒ˜ãƒ†ãƒã‚¸ãƒ‹ã‚¢ã‚¹æ™‚ç³»åˆ—ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ - ã‚†ã†ã†ããƒ–ãƒã‚°
ã“ã®è¨˜äº‹ã¯ã€ç¬¬2å›žã‚¦ã‚§ãƒ–ã‚·ã‚¹ãƒ†ãƒ ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ç ”ç©¶ä¼šã®äºˆç¨¿ã§ã™ã€‚ ã‚¦ã‚§ãƒ–ã‚·ã‚¹ãƒ†ãƒ ã‚’ãƒ¢ãƒ‹ã‚¿ãƒªãƒ³ã‚°ã™ã‚‹ãŸã‚ã«ã€é«˜å¯ç”¨æ€§ã€é«˜æ›¸ãè¾¼ã¿ã‚¹ã‚±ãƒ¼ãƒ©ãƒ“ãƒªãƒ†ã‚£ã€ãƒ¡ãƒˆãƒªãƒƒã‚¯ã®é•·æœŸä¿å˜ãŒå¯èƒ½ãªæ™‚ç³»åˆ—ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ãŒæ±‚ã‚ã‚‰ã‚Œã¦ã„ã‚‹ã€‚ ã“ã‚Œã‚‰ã‚’å®Ÿç¾ã™ã‚‹ãŸã‚ã«ã€æ€§èƒ½ç‰¹æ€§ã®ç•°ãªã‚‹æ±Žç”¨Key-Value Store(ä»¥ä¸‹KVS)ã‚’çµ„ã¿åˆã‚ã›ã€é€éŽçš„ã«å•ã„åˆã‚ã›å¯èƒ½ãªã€ãƒ˜ãƒ†ãƒã‚¸ãƒ‹ã‚¢ã‚¹æ™‚ç³»åˆ—ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã§ã‚ã‚‹Diamondã‚’é–‹ç™ºã—ãŸã€‚ ã“ã®è¨˜äº‹ã§ã¯ã€Diamondã‚’åˆ†æ•£ã‚·ã‚¹ãƒ†ãƒ ã®è¦³ç‚¹ã§æ‰ãˆã€ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ã€ãƒ‡ãƒ¼ã‚¿æ§‹é€ ã€å®Ÿè£…ã‚’ç´¹ä»‹ã—ã€è€ƒå¯Ÿã«ã‚ˆã‚ŠFuture Workã‚’è°è«–ã™ã‚‹ã€‚ 1. ã¯ã˜ã‚ã« 2. ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£æ¦‚è¦ å‹•ä½œãƒ•ãƒãƒ¼ ãƒ‡ãƒ¼ã‚¿æ§‹é€ KVSã®æ©Ÿèƒ½è¦ä»¶ 3. å®Ÿè£… å®Ÿè£…æ¦‚è¦ KVSé–“ã®ãƒ‡ãƒ¼ã‚¿ç§»å‹• ãƒ‡ãƒ¼ã‚¿ä½ç½®ã®è§£æ±º è²»ç”¨ç‰¹æ€§ 4. è€ƒå¯Ÿã¨ä»Šå¾Œã®èª²é¡Œ Diamondã®æ¬ ç‚¹ å°†æ¥æ©Ÿèƒ½ 5. ã¾ã¨ã‚ ã‚¹ãƒ©ã‚¤ãƒ‰
msykt 2018/05/19
LSM-Treeæ–¹å¼ã®KVSã‚’ã€ãƒ—ãƒã‚»ã‚¹ã§ã¯ãªãã‚µãƒ¼ãƒ“ã‚¹ã‚’çµ„ã¿åˆã‚ã›ã¦å®Ÿç¾ã—ãŸã€ã¨ã„ã†å°è±¡ã‚’æŒã£ãŸã€‚æœ¬å½“ã«å‡„ã„ã€‚

dynamodb

redis

kvs

s3
ãƒªãƒ³ã‚¯
Building a Database on S3
msykt 2016/07/27
s3
ãƒªãƒ³ã‚¯
S3+Lambda+CloudFormationã§ã‚µãƒ¼ãƒãƒ¬ã‚¹yumãƒªãƒã‚¸ãƒˆãƒª - so what
å…ˆæ—¥ã®è¨˜äº‹ã§ã€å¤§ãã‚ã®å‡¦ç†ã‚’Lambda+CloudFormationã§å®Ÿè¡Œã™ã‚‹ã‚ã©ãŒã¤ã„ãŸã®ã§ã€S3+Lambda+CloudFormationã§ã‚µãƒ¼ãƒãƒ¬ã‚¹yumãƒªãƒã‚¸ãƒˆãƒªã‚’ä½œã£ã¦ã¿ãŸã€‚ å‡¦ç†ã®æ¦‚è¦ S3ã«rpmã‚’è¿½åŠ ãƒ»æ›´æ–°ãƒ»å‰Šé™¤ LambdaãŒã‚¤ãƒ™ãƒ³ãƒˆã‚’ãƒ•ãƒƒã‚¯â†’cfnã‚¹ã‚¿ãƒƒã‚¯ã‚’ä½œæˆ cfnã‚¹ã‚¿ãƒƒã‚¯ãŒEC2ã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹ã‚’èµ·å‹•â†’S3ãƒªãƒã‚¸ãƒˆãƒªã‚’ãƒ€ã‚¦ãƒ³ãƒãƒ¼ãƒ‰ createrepoã‚’å®Ÿè¡Œã—ã¦ã€ã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ã‚’S3ã«ã‚¢ãƒƒãƒ—ãƒãƒ¼ãƒ‰ cfnã‚¹ã‚¿ãƒƒã‚¯ã¯å‡¦ç†å®Œäº†å¾Œã«è‡ªå‹•çš„ã«å‰Šé™¤ Lambda Function ã“ã‚“ãªæ„Ÿã˜ã€‚ var Promise = require('bluebird'); var AWS = require("aws-sdk"); AWS.config.update({region: 'ap-northeast-1'}); var cloudformation = Promise.pr
msykt 2015/08/10
lambda

aws

s3

rpm
ãƒªãƒ³ã‚¯
ã€ã‚¢ãƒƒãƒ—ãƒ‡ãƒ¼ãƒˆã€‘S3ã®ä¸€è²«æ€§ã«å¤‰æ›´ãŒã‚ã‚Šã¾ã™ | DevelopersIO
ã‚¦ã‚£ã‚¹ã‚ãƒ¼ã€ã‚·ã‚¬ãƒ¼ã€ãƒ‘ã‚¤ãƒ—ã‚’ã“ã‚ˆãªãæ„›ã™ã‚‹å¤§æ —ã§ã™ã€‚ æœ¬æ—¥ã®ã‚¢ãƒŠã‚¦ãƒ³ã‚¹(è¦ãƒã‚°ã‚¤ãƒ³)ã§ã€S3ã®ä¸€è²«æ€§ã«ä¸€éƒ¨å¤‰æ›´ãŒã‚ã‚‹ã“ã¨ãŒç™ºè¡¨ã•ã‚Œã¾ã—ãŸã€‚å¤‰æ›´ãŒã‚ã£ãŸã®ã¯"US Standard"ã¨ã„ã†èžããªã‚Œãªã„ãƒªãƒ¼ã‚¸ãƒ§ãƒ³ã¨ãªã‚Šã¾ã™ã€‚ ãƒªãƒ¼ã‚¸ãƒ§ãƒ³ï¼US Standardï¼Ÿ S3ã«è©³ã—ã„æ–¹ã¯ã”å˜çŸ¥ã‹ã¨æ€ã„ã¾ã™ãŒã€S3ã®ãƒªãƒ¼ã‚¸ãƒ§ãƒ³ã«ãƒãƒ¼ã‚¸ãƒ‹ã‚¢(us-east-1)ãŒå˜åœ¨ã—ã¾ã›ã‚“ã€‚æ›¿ã‚ã‚Šã«US Standardã¨ã„ã†ãƒªãƒ¼ã‚¸ãƒ§ãƒ³ãŒã‚ã‚Šã¾ã™ã€‚æ´å²çš„çµŒç·¯ã‚‚ã‚ã‚‹ã®ã ã¨æ€ã„ã¾ã™ãŒã€s3.amazon aws.comï¼ˆãƒãƒ¼ã‚¸ãƒ‹ã‚¢åŒ—éƒ¨ã¾ãŸã¯å¤ªå¹³æ´‹å²¸åŒ—è¥¿éƒ¨ï¼‰ã¨s3-external-1.amazon aws.comï¼ˆãƒãƒ¼ã‚¸ãƒ‹ã‚¢åŒ—éƒ¨ã®ã¿ï¼‰ã®2ç®‡æ‰€ã®ã‚¨ãƒ³ãƒ‰ãƒã‚¤ãƒ³ãƒˆãŒä½¿ç”¨ã§ãã¾ã™ã€‚ ä½•ãŒå¤‰ã‚ã£ãŸã®ï¼Ÿ US Standardã¯ä¸€è²«æ€§ãŒä»–ã®ãƒªãƒ¼ã‚¸ãƒ§ãƒ³ã¨ç•°ãªã£ã¦ã„ã¾ã—ãŸãŒã€ä»–ã®ãƒªãƒ¼ã‚¸ãƒ§ãƒ³ã¨åŒã˜æ•´åˆæ€§ãƒ¢ãƒ‡ãƒ«ãŒä½¿ç”¨ã§ãã‚‹æ§˜ã«ãªã‚Šã¾ã—ãŸã€‚å¤‰æ›´ç‚¹ã¯
msykt 2015/06/20
ã‚ˆã†ã‚„ãå…¨ãƒªãƒ¼ã‚¸ãƒ§ãƒ³åŒã˜ä»•æ§˜ã«ãªã£ãŸã‹ã€‚ã©ã†ã„ã†çµŒç·¯ã§US Standardã ã‘é•ã£ã¦ãŸã®ã‹è£è©±èžããŸã„

aws

s3
ãƒªãƒ³ã‚¯
Powering Big Data at Pinterest. Interview with Krishna Gade. | ODBMS Industry Watch
msykt 2015/04/26
S3ã«ãƒã‚°ã‚’æ›¸ãè¾¼ã‚€Secorã¨ã„ã†ãƒ„ãƒ¼ãƒ«ãŒæ°—ã«ãªã£ãŸã®ã¨ã€Sparkã¨MemSQLã®é€£æºãŒé¢ç™½ãã†

log

kafka

singer

s3

secor

spark

cascading

memsql
ãƒªãƒ³ã‚¯
[aws]s3ãƒã‚±ãƒƒãƒˆã«IPåˆ¶é™ã‚’ã‹ã‘ã‚‹
s3ã«ã¯basicèªè¨¼ã‚’è¨å®šã™ã‚‹ã“ã¨ã¯ã§ããªã„ã®ã§ã™ãŒã€IPåˆ¶é™ãªã‚‰è¨å®šã™ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ ãã®ã‚„ã‚Šæ–¹ã‚’èª¿ã¹ã¦ã¿ã¾ã—ãŸã€‚ ãƒã‚±ãƒƒãƒˆä½œæˆå¾Œã€ã€ŒPropertiesã€=>ã€ŒPermissionsã€=>ã€ŒEdit bucket policyã€ã‚’é¸æŠž ä»¥ä¸‹ã®ã‚ˆã†ãªãƒã‚±ãƒƒãƒˆãƒãƒªã‚·ãƒ¼ã‚’è¨å®šã—ã¾ã™ã€‚ Resourceã«ãƒã‚±ãƒƒãƒˆåã‚’å…¥åŠ›ã—ã€IpAddressã«è¨±å¯ã—ãŸã„IPã‚’å…¥åŠ›ã™ã‚Œã°IPåˆ¶é™ã‚’ã‹ã‘ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ ãŸã ã—ã€å€‹åˆ¥ã«publicè¨å®šã«ã—ãŸãƒ•ã‚¡ã‚¤ãƒ«ã«ã¯ãƒã‚±ãƒƒãƒˆãƒãƒªã‚·ãƒ¼ãŒé©ç”¨ã•ã‚Œãªã„ã®ã§æ³¨æ„ãŒå¿…è¦ã§ã™ã€‚ 2015/3/6è¿½è¨˜ï¼šã€Œ"Effect": "Allow"ã€ã§ã‚¢ã‚¯ã‚»ã‚¹è¨±å¯è¨å®šã‚’ã™ã‚‹ã¨ã†ã¾ãè¨å®šãŒé©ç”¨ã§ããªã‹ã£ãŸã®ã§ã€Œ"Effect": "Deny"ã€ã‚’ç”¨ã„ã‚‹ãƒ‘ã‚¿ãƒ¼ãƒ³ã«ä¿®æ£ã—ã¾ã—ãŸ { "Version": "2012-10-17", "Id": "S3Policy#{unique_id
msykt 2015/03/13
ãƒã‚±ãƒƒãƒˆãƒãƒªã‚·ãƒ¼ã§IPåˆ¶é™ã§ãã‚‹ã®ã‹ãƒ¼

s3
ãƒªãƒ³ã‚¯
S3ã‚’DBåˆ©ç”¨ ã‚·ãƒ§ãƒƒãƒ”ãƒ³ã‚°ã‚»ãƒ³ã‚¿ãƒ¼å‘ã‘ãƒã‚¤ãƒ³ãƒˆã‚·ã‚¹ãƒ†ãƒ æ¦‚è¦
2015å¹´2æœˆ7æ—¥ å¤§é˜ªã§é–‹å‚¬ã•ã‚ŒãŸJAWS-UG KANSAI ç‰¹åˆ¥ç·¨ã§ã—ã‚ƒã¹ã£ãŸã¨ãã®ã‚¹ãƒ©ã‚¤ãƒ‰ã§ã™ã€‚Read less
msykt 2015/02/22
ã‚·ã‚¹ãƒ†ãƒ ç‰¹æ€§ã«åˆã‚ã›ã¦é©åˆ‡ãªã‚¢ãƒ—ãƒãƒ¼ãƒã‚’å–ã£ã¦ã‚‹ã¨æ€ã†ã€‚ç´ æ™´ã‚‰ã—ã„

s3
ãƒªãƒ³ã‚¯
conma.me
This domain may be for sale!
msykt 2014/10/02
ãŠã‰/â€œä»Šã ã¨ã€å…ˆæ—¥ç‹¬è‡ªèªè¨¼ãƒ—ãƒãƒã‚¤ãƒ€ã«ã‚‚å¯¾å¿œã—ãŸAmazon Cognitoã‚’ä½¿ã†ã“ã¨ã§ç°¡å˜ã«å®Ÿè£…å‡ºæ¥ã¾ã™ã€‚Congnitoã¯èªè¨¼ã ã‘ã§ãªãã€ãƒ‡ãƒã‚¤ã‚¹é–“ã§ã®ãƒ‡ãƒ¼ã‚¿åŒæœŸãªã©ã‚‚å‚™ãˆã¦ã„ã‚‹ãŸã‚æ˜¯éžè©¦ã—ã¦ã¿ã¦ã¯ã„ã‹ãŒã§ã—ã‚‡ã†ã‹â€

aws

s3
ãƒªãƒ³ã‚¯
GitHub - Netflix/s3mper: s3mper - Consistent Listing for S3
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
msykt 2014/01/17
â€œS3mper is a library that provides an additional layer of consistency checking on top of Amazon's S3 index through use of a consistent, secondary index.â€

s3
ãƒªãƒ³ã‚¯
Netflix: s3mperã‚’åˆ©ç”¨ã—ã¦ã‚¯ãƒ©ã‚¦ãƒ‰ã§ã®ãƒ‡ãƒ¼ã‚¿æ•´åˆæ€§ã‚’ã¨ã‚‹ - ãƒ¯ã‚¶ãƒŽãƒ | wazanova
http://tech blog.netflix.com/2014/01/s3mper-consistency-in-cloud.html1 comment | 0 pointsNetflixã®Daniel WeeksãŒåŒç¤¾ã®ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ãƒ–ãƒã‚°ã§ã€ã‚¯ãƒ©ã‚¦ãƒ‰ã§ãƒ‡ãƒ¼ã‚¿æ•´åˆæ€§ã‚’ã¨ã‚‹ãŸã‚ã®ãƒ©ã‚¤ãƒ–ãƒ©ãƒªs3mperã‚’ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã¨ã—ã¦æä¾›ã—ãŸã“ã¨ã‚’å ±å‘Šã—ã¦ã„ã¾ã™ã€‚ å‰å›žã®ãƒã‚¹ãƒˆ "Hadoop Platform as a Service in the Cloud"ã§ã¯ã€Netflixã®Hadoopãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ ãŒã©ã®ã‚ˆã†ã«AWSã®S3ã‚’æ´»ç”¨ã—ã¦ã„ã‚‹ã‹ã‚’ç´¹ä»‹ã—ãŸã€‚Netflixã¯S3ã‚’å…¨ã¦ã®ãƒ‡ãƒ¼ã‚¿ã‚¦ã‚§ã‚¢ãƒã‚¦ã‚¹ã®SOT (Source of Truth) ã¨è¦‹ãªã—ã¦ã„ã‚‹ã€‚ã“ã®ã‚µãƒ¼ãƒ“ã‚¹ã«ã¯ã€99.999999999%ã®ãƒ‡ãƒ¼ã‚¿æ°¸ç¶šæ€§ã€99.99%ã®ç¨¼åƒçŽ‡ã€åŠ¹çŽ‡ã‚ˆãå®¹é‡åˆ¶é™ã®ãªã„ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã€ãƒãƒ¼ã‚¸ãƒ§ãƒ‹ãƒ³ã‚°ï¼ˆãƒ‡ãƒ¼
msykt 2014/01/17
â€œaspectsã‚’åˆ©ç”¨ã—ã¦s3mperã‚’å®Ÿè£…ã—ã€Hadoopã®ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã®ã‚¤ãƒ³ã‚¿ãƒ¼ãƒ•ã‚§ãƒ¼ã‚¹ã®ãƒ¡ã‚½ãƒƒãƒ‰ã‚’åˆ©ç”¨ã—ã€DynamoDBã‚’äºŒæ¬¡çš„ãªã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ã¨ã—ã¦ä½¿ã„ã€ãƒ•ã‚¡ã‚¤ãƒ«ã®ãƒ¡ã‚¿ãƒ‡ãƒ¼ã‚¿ã‚’ãƒˆãƒ©ãƒƒã‚¯ã—ã¾ã™ã€‚â€

s3
ãƒªãƒ³ã‚¯
S3ã¨Fluentdã‚’ç”¨ã„ãŸåŠ¹çŽ‡çš„ãªãƒã‚°ç®¡ç† | SmartNewsé–‹ç™ºè€…ãƒ–ãƒã‚°
ã‚´ã‚¯ãƒã®å¤§å¹³ã§ã™ã€‚ ç§ã«ã¨ã£ã¦ä¸€ç•ªå¤§äº‹ã§æ›¿ãˆã®åŠ¹ã‹ãªã„ãƒŸãƒ¥ãƒ¼ã‚¸ã‚·ãƒ£ãƒ³ã¯ã•ã ã¾ã•ã—ã•ã‚“ã§ã™ãŒã€ç§ã«ã¨ã£ã¦ã‚¯ãƒ©ã‚¦ãƒ‰ã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒ†ã‚£ãƒ³ã‚°ã®ã‚µãƒ¼ãƒ“ã‚¹ã®ä¸ã§ä¸€ç•ªå¤§äº‹ã§æ›¿ãˆãŒåŠ¹ã‹ãªã„ã‚µãƒ¼ãƒ“ã‚¹ã¯S3ã§ã™ã€‚ å¤šç¨®å¤šæ§˜ãªAPIã‚’ç”¨ã„ã¦æŸ”è»Ÿã«ãƒ•ã‚¡ã‚¤ãƒ«ã®æ“ä½œãŒå‡ºæ¥ã‚‹äº‹ã‚„ã€â€99.999999999ï¼…â€ã¨è¬³ã‚ã‚Œã¦ã„ã‚‹é«˜ã„è€éšœå®³æ€§ã€S3ã«ã‚ã‚‹ãƒ‡ãƒ¼ã‚¿ã‚’Elastic MapReduceã‚„Redshiftãªã©ã‚’ç”¨ã„ã¦æ‰‹è»½ã«ãƒ‡ãƒ¼ã‚¿è§£æžã‚’è¡Œãˆã‚‹åŸºç›¤ãŒæä¾›ã•ã‚Œã¦ã„ã‚‹ã“ã¨ãªã©ã€ã‚ã¾ã‚Šã«ä¾¿åˆ©ã™ãŽã¦S3ã®ä»£æ›¿ã¨ãªã‚‹ã‚µãƒ¼ãƒ“ã‚¹ã‚’æŽ¢ã—ã ã™ã®ãŒé›£ã—ã„çŠ¶æ…‹ã§ã™ã€‚ ã‚‚ã¡ã‚ã‚“å¤šãã®AWSãƒ¦ãƒ¼ã‚¶ãƒ¼ãŒåŒã˜ã‚ˆã†ã«S3ã®ä¾¿åˆ©ã•ã‚’äº«å—ã—ã¦ã„ã‚‹ã¨æ€ã„ã¾ã™ã—ã€ã‚¤ãƒ³ã‚¿ãƒ¼ãƒãƒƒãƒˆä¸Šã§ã‚‚å¤šãã®ãƒ–ãƒã‚°ç‰ã§ãã®é…åŠ›ãŒèªžã‚‰ã‚Œã¦ã„ã¾ã™ã€‚ãã®ä¸ã§æœ¬è¨˜äº‹ã¯æ—¢ã«å˜åœ¨ã™ã‚‹è¨˜äº‹ã¨ä¼¼ãŸã‚ˆã†ãªå†…å®¹ã‚’æ›¸ã„ã¦ã—ã¾ã†ã‹ã‚‚ã—ã‚Œã¾ã›ã‚“ãŒã€å¼Šç¤¾ãªã‚Šã®S3ã®ä½¿ã„æ–¹ã«ã¤ã„ã¦ã”ç´¹ä»‹ã—ãŸã„ã¨æ€ã„ã¾ã™ã€‚ ãªãŠ
msykt 2013/09/05
fluentd

s3
ãƒªãƒ³ã‚¯
Managing your storage lifecycle - Amazon Simple Storage Service
To manage your objects so that they're stored cost effectively throughout their lifecycle, create an Amazon S3 Lifecycle configuration. An Amazon S3 Lifecycle configuration is a set of rules that define actions that Amazon S3 applies to a group of objects. There are two types of actions: Transition actions â€“ These actions define when objects transition to another storage class. For example, you mi
msykt 2013/08/16
aws

s3
ãƒªãƒ³ã‚¯
Logã‚’s3ã¨redshiftã«æ ¼ç´ã™ã‚‹ä»•çµ„ã¿
1. Logã‚’S3ã¨ Hive Redshi/ ã« æ ¼ç´ã™ã‚‹ä»•çµ„ã¿ 2013å¹´5æœˆ22æ—¥ æ ªå¼ä¼šç¤¾ã‚†ã‚ã¿ æ£®ä¸‹ å¥ mokemokechicken@twi;er 1 2. ä½œã‚‹ãã£ã‹ã‘ ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ãƒã‚°ã‚’MySQLã«ä¿å˜ã—ã¦ã„ã‚‹ ï¼ˆèª¿æŸ»ç›®çš„ï¼‰ MySQLã ã¨ã‚¹ã‚±ãƒ¼ãƒ«ã—ãªã„ S3ã‚„Hadoop(Hive)ä¸Šã«ä¿å˜ã—ã‚ˆã† ï¼ˆã‚¹ã‚±ãƒ¼ãƒ«ã—ãã†ï¼‰ 2 ï¼‘ï¼ï¼ã€œï¼’ï¼ï¼Write/sec ãã‚‰ã„ã§ã‚ãƒ„ã‚¤
msykt 2013/05/23
s3

fluentd

redshift
ãƒªãƒ³ã‚¯
Announcement: ELB stickiness updates to support Feb 2020 Chromium CORs changes
msykt 2013/05/05
emr

s3
ãƒªãƒ³ã‚¯
Ruby aws-sdk - timeout error
msykt 2013/04/27
aws

s3

ruby
ãƒªãƒ³ã‚¯
Django Cloud Browser â€” Cloud Browser 0.5.4 documentation
Django Cloud BrowserÂ¶ Info A Django application browser for cloud (S3, Cloud Files) datastores. Author Ryan Roemer (http://github.com/ryan-roemer) Build Version Cloud Browser is a simple web-based object browser for cloud-based blob datastores. Just add as an application to a Django project, add some settings, and youâ€™ll be able to browse cloud containers and implied subdirectories, as well as vie
msykt 2013/03/23
s3

django
ãƒªãƒ³ã‚¯
Hadoopã®ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã¨ã—ã¦S3ã‚’åˆ©ç”¨ã™ã‚‹
EC2ã§Hadoopã‚’ä½¿ã†å ´åˆã€ã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹ã‚’åœæ¢ã™ã‚‹ã¨ãƒ‡ãƒ¼ã‚¿ãŒãªããªã‚‹ï¼ˆEBSã‚’ä½¿ã†ã¨ã„ã†æ‰‹ã‚‚ã‚ã‚Šã¾ã™ãŒï¼‰é–¢ä¿‚ã§ãƒ‡ãƒ¼ã‚¿ã¯S3ã«ç½®ãå ´åˆãŒå¤šã„ã¨æ€ã„ã¾ã™ã€‚ ãã®å ´åˆã€S3ã‹ã‚‰EC2ä¸Šã®HDFSã«èªã¿è¾¼ã‚“ã§ã‹ã‚‰å‡¦ç†ã‚’ã—ã¦æœ€å¾Œã«S3ã«æ›¸ãå‡ºã™ã¨ã„ã†ã“ã¨ã‚‚è€ƒãˆã‚‰ã‚Œã¾ã™ãŒã€ã‚„ã¯ã‚ŠS3ä¸Šã®ãƒ•ã‚¡ã‚¤ãƒ«ã‚’ç›´æŽ¥Hadoopã®ã‚¸ãƒ§ãƒ–å…¥å‡ºåŠ›ã¨ã—ã¦ä½¿ã†ã®ãŒåŠ¹çŽ‡ãŒè‰¯ã„ã€‚ï¼ˆæ›´ã«è¨€ã†ã¨ã€è¤‡æ•°ã®ã‚¸ãƒ§ãƒ–ã‚’å®Ÿè¡Œã™ã‚‹æ™‚ã®ä¸é–“ã®ãƒ‡ãƒ¼ã‚¿ã‚’ä¿å˜ã™ã‚‹ã®ã«ã¯HDFSã‚’ä½¿ã£ãŸæ–¹ãŒé€Ÿã„ï¼‰ ãã‚‚ãã‚‚Hadoopã¯ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã®éƒ¨åˆ†ã‚’æŠ½è±¡åŒ–ã—ã¦ã„ã¦ã€HDFSã‚‚ãã®å®Ÿè£…ã®ä¸€ã¤ã«éŽãŽã¾ã›ã‚“ã€‚ AmazonS3 â€“ Hadoop Wikiã«ã‚‚ã‚ã‚Šã¾ã™ãŒã€S3ã‚’ä½¿ã†ãŸã‚ã®å®Ÿè£…ã«ã¯2ç¨®é¡žã‚ã‚‹ã®ã§æ³¨æ„ã€‚ S3 Native FileSystem (URI scheme: s3n) æ™®é€šã«S3ä¸Šã®ãƒ•ã‚¡ã‚¤ãƒ«ã‚’èªã¿æ›¸ãã™ã‚‹ãƒã‚¤ãƒ†ã‚£ãƒ–ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã€‚H
msykt 2013/02/24
hadoop

s3
ãƒªãƒ³ã‚¯
Welcome to Ceph â€” Ceph Documentation
Welcome to CephÂ¶ Ceph uniquely delivers object, block, and file storage in one unified system. Ceph Object Store RESTful Interface S3- and Swift-compliant APIs S3-style subdomains Unified S3/Swift namespace User management Usage tracking Striped objects Cloud solution integration Multi-site deployment Multi-site replication Ceph Block Device Thin-provisioned Images up to 16 exabytes Configurable s
msykt 2013/01/26
s3

java
ãƒªãƒ³ã‚¯
1 2 æ¬¡ã®ãƒšãƒ¼ã‚¸