GitHub - awslabs/amazon-emr-cli: A command-line interface for packaging, deploying, and running your EMR Serverless Spark jobs
So we're all working on data pipelines every day, but wouldn't be nice to just hit a button and have our code automatically deployed to staging or test accounts? I thought so, too, thats why I created the EMR CLI (emr) that can help you package and deploy your EMR jobs so you don't have to. The EMR CLI supports a wide variety of configuration options to adapt to your data pipeline, not the other w
AWS CLIã§EMRクラスタを立ã¡ä¸Šã’Sparkã®ã‚¢ãƒ—リケーションを実行ã™ã‚‹ - sambaiz-net
Amazon EMRã¯EC2ã‚„EKS上ã«Sparkã‚„Hive, Prestoã®ã‚¯ãƒ©ã‚¹ã‚¿ã‚’構築ã™ã‚‹ã‚µãƒ¼ãƒ“ス。 Sparkã®ãƒžãƒãƒ¼ã‚¸ãƒ‰ã‚µãƒ¼ãƒ“スã§ã‚ã‚‹Glueã¨æ¯”ã¹ã‚‹ã¨ã€Glueã¯ã‚µãƒ¼ãƒãƒ¼ãƒ¬ã‚¹ã§æ‰‹è»½ã«Sparkã«ã‚ˆã‚‹ETL処ç†ã‚’ã‚’è¡Œãˆã‚‹ã®ã«å¯¾ã—ã¦ã€EMRã¯ã‚¹ãƒãƒƒãƒˆã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹ãªã©ã‚’利用ã—ãŸã‚³ã‚¹ãƒˆãƒ‘フォーマンスã®è‰¯ã•ã¨è©³ç´°ãªãƒãƒ¥ãƒ¼ãƒ‹ãƒ³ã‚°ã‚’è¡Œã†ã“ã¨ãŒã§ãã‚‹ã¨ã„ã†ç‰¹é•·ãŒã‚ã‚‹ãŒã€EMR ServerlessãŒãƒªãƒªãƒ¼ã‚¹ã•ã‚ŒãŸã®ã§ãã®å·®ã¯å°‘ã—縮ã¾ã£ã¦ã„るよã†ã«æ„Ÿã˜ã‚‹ã€‚Glueã«ã¯ã‚¹ã‚ーマを指定ã™ã‚‹å¿…è¦ãŒãªã„DynamicFrameã‚„å‰ã®ç¶šãã‹ã‚‰å®Ÿè¡Œã§ãã‚‹Bookmarkã¨ã„ã£ãŸä¾¿åˆ©ãªæ©Ÿèƒ½ã‚‚ã‚ã‚‹ãŒã€é‡ã„処ç†ã‚’ç«‹ã¦ç¶šã‘ã«å®Ÿè¡Œã™ã‚‹ã¨ã‚³ã‚¹ãƒˆãŒåµ©ã‚“ã ã‚ŠDPUãªã©ã®ã‚¯ã‚©ãƒ¼ã‚¿ã«å¼•ã£ã‹ã‹ã‚‹ã“ã¨ã‚‚ã‚ã‚‹ã®ã§é©å®œä½¿ã„分ã‘ã¦ã„ããŸã„。 AWS Glueã®Jobã®Bookmarkを有効ã«ã—ã¦å‰å›žã®ç¶šãã‹ã‚‰å‡¦ç†ã‚’行ㆠ- sambai
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
処ç†ã‚’実行ä¸ã§ã™
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
GitHub - mauropelucchi/aws-emr-docker-integration: AWS EMR Docker integration
Using Docker Images on EMR
{{#tags}}- {{label}}
{{/tags}}