Spark and YARN - Qiita
Sparkã¨YARNã«ã¤ã„ã¦æ›¸ãã¾ã™ã€‚テーマ的ã«ã‚¤ãƒ³ãƒ•ãƒ©ã‚¹ãƒˆãƒ©ã‚¯ãƒãƒ£ã«ã¤ã„ã¦ã®è©±ãŒå¤šããªã‚‹ã¨æ€ã„ã¾ã™ã€‚ Sparkã¨Hadoopã®é–¢ä¿‚性 Sparkã¯Hadoopクラスタã¸ã®ä¾å˜ã¯ã—ã¦ã„ãªã„。(ãŸã ã—ã€ã‚„ã‚„ã“ã—ã„ã®ã ãŒHDFSã‚„YARNã®ã‚¯ãƒ©ã‚¤ã‚¢ãƒ³ãƒˆãƒ©ã‚¤ãƒ–ラリã¸ã®ä¾å˜ã¯ã‚ã‚‹)ãªã®ã§Hadoopãªã—ã§ã‚‚å‹•ã‹ã™ã“ã¨ãŒã§ãる。ã—ã‹ã—ãã‚Œã§ã‚‚Hadoopã¨ä¸€ç·’ã«å‹•ä½œã•ã›ã‚‹ã“ã¨ãŒå¤šã„ã®ã¯ä»¥ä¸‹ã®ç†ç”±ã«ã‚ˆã‚‹ã€‚ クラスタマãƒãƒ¼ã‚¸ãƒ£ã¨ã—ã¦ã®YARN Sparkã¯ã‚¢ãƒ—リケーション（厳密ã«ã¯Sparkアプリケーション）ã”ã¨ã«ä¸‹è¨˜ã®ã‚ˆã†ãªã‚¯ãƒ©ã‚¹ã‚¿ãŒæ§‹ç¯‰ã•ã‚Œã‚‹ã€‚Driver Programã¨å‘¼ã°ã‚Œã‚‹ã€SparkContextオブジェクトをæŒã¡ã€ã‚¢ãƒ—リケーションコードã®ä¸»è¦éƒ¨åˆ†ã‚’実行ã™ã‚‹ã‚¢ãƒ—リケーションã®ãƒžã‚¹ã‚¿ã‚³ãƒ³ãƒãƒ¼ãƒãƒ³ãƒˆã¨ã€RDDã«å¯¾ã™ã‚‹ã‚ªãƒšãƒ¬ãƒ¼ã‚·ãƒ§ãƒ³ã‚’実行ã™ã‚‹Executor群。ãã—ã¦ã€Driver Progr
PKI certificates and requirements
KubeCon + CloudNativeCon 2024Join us for three days of incredible opportunities to collaborate, learn and share with the cloud native community. Buy your ticket now! 12 - 15 November | Salt Lake City PKI certificates and requirementsKubernetes requires PKI certificates for authentication over TLS. If you install Kubernetes with kubeadm, the certificates that your cluster requires are automatically
よãã‚ã‚‹è³ªå• - Amazon EKS | AWS
Amazon Elastic Kubernetes Service (Amazon EKS) ã¨ã¯ä½•ã§ã™ã‹? Amazon EKS ã¯ã€AWS ã§ã® Kubernetes ã®å®Ÿè¡Œã‚’容易ã«ã™ã‚‹ãƒžãƒãƒ¼ã‚¸ãƒ‰ã‚µãƒ¼ãƒ“スã§ã‚ã‚Šã€ç‹¬è‡ªã® Kubernetes コントãƒãƒ¼ãƒ«ãƒ—レーンやワーカーノードをインストールã—ã¦æ“作ã™ã‚‹å¿…è¦ã¯ã‚ã‚Šã¾ã›ã‚“。 Kubernetes ã¯ã‚ªãƒ¼ãƒ—ンソースã®ã‚³ãƒ³ãƒ†ãƒŠã‚ªãƒ¼ã‚±ã‚¹ãƒˆãƒ¬ãƒ¼ã‚·ãƒ§ãƒ³ã‚·ã‚¹ãƒ†ãƒ ã§ã‚ã‚Šã€ã‚³ãƒ³ãƒ†ãƒŠåŒ–ã•ã‚ŒãŸã‚¢ãƒ—リケーションを大è¦æ¨¡ã«ãƒ‡ãƒ—ãƒã‚¤ãŠã‚ˆã³ç®¡ç†ã§ãるよã†ã«ã—ã¾ã™ã€‚Kubernetes ã¯ã€ã‚³ãƒ³ãƒ†ãƒŠã‚’è«–ç†çš„ãªã‚°ãƒ«ãƒ¼ãƒ—ã«åˆ†ã‘ã¦ã‚’管ç†ã€ç™ºè¦‹ã§ãるよã†ã«ã—ã¦ã‹ã‚‰ã€Amazon Elastic Compute Cloud (Amazon EC2) インスタンスã®ã‚¯ãƒ©ã‚¹ã‚¿ãƒ¼ä¸Šã§èµ·å‹•ã—ã¾ã™ã€‚Kubernetes ã‚’ãŠä½¿ã„ã«ãªã‚‹ã¨ã€ã‚ªãƒ³ãƒ—レミスã€ã‚¯ãƒ©ã‚¦ãƒ‰ã§åŒã˜ãƒ„ールセットを用ã„ã¦ã€
minikube start
minikube is local Kubernetes, focusing on making it easy to learn and develop for Kubernetes. All you need is Docker (or similarly compatible) container or a Virtual Machine environment, and Kubernetes is a single command away: minikube start What you’ll need 2 CPUs or more 2GB of free memory 20GB of free disk space Internet connection Container or virtual machine manager, such as: Docker, QEMU, H
Share Process Namespace between Containers in a Pod
KubeCon + CloudNativeCon 2024Join us for three days of incredible opportunities to collaborate, learn and share with the cloud native community. Buy your ticket now! 12 - 15 November | Salt Lake City Share Process Namespace between Containers in a PodThis page shows how to configure process namespace sharing for a pod. When process namespace sharing is enabled, processes in a container are visible
Set up an Extension API Server
KubeCon + CloudNativeCon 2024Join us for three days of incredible opportunities to collaborate, learn and share with the cloud native community. Buy your ticket now! 12 - 15 November | Salt Lake City Set up an Extension API ServerSetting up an extension API server to work with the aggregation layer allows the Kubernetes apiserver to be extended with additional APIs, which are not part of the core
Namespaces
KubeCon + CloudNativeCon 2024Join us for three days of incredible opportunities to collaborate, learn and share with the cloud native community. Buy your ticket now! 12 - 15 November | Salt Lake City NamespacesIn Kubernetes, namespaces provide a mechanism for isolating groups of resources within a single cluster. Names of resources need to be unique within a namespace, but not across namespaces. N
Glossary
This glossary is intended to be a comprehensive, standardized list of Kubernetes terminology. It includes technical terms that are specific to Kubernetes, as well as more general terms that provide useful context.
Dynamic Admission Control
KubeCon + CloudNativeCon 2024Join us for three days of incredible opportunities to collaborate, learn and share with the cloud native community. Buy your ticket now! 12 - 15 November | Salt Lake City Dynamic Admission ControlIn addition to compiled-in admission plugins, admission plugins can be developed as extensions and run as webhooks configured at runtime. This page describes how to build, con
kops/docs/horizontal_pod_autoscaling.md at master · kubernetes/kops
With Horizontal Pod Autoscaling, Kubernetes automatically scales the number of pods in a replication controller, deployment, or replica set based on observed CPU utilization (or, with alpha support, on some other, application-provided metrics). The HorizontalPodAutscaler autoscaling/v2 stable API moved to GA in 1.23. The previous stable version, which only includes support for CPU autoscaling, can
料金 - Amazon EKS | AWS
Amazon Elastic Kubernetes Service (Amazon EKS) を利用ã™ã‚‹ã¨ã€ã‚¯ãƒ©ã‚¦ãƒ‰ã¾ãŸã¯ã‚ªãƒ³ãƒ—レミス㧠Kubernetes アプリケーションを柔軟ã«é–‹å§‹ã€å®Ÿè¡Œã€ãŠã‚ˆã³ã‚¹ã‚±ãƒ¼ãƒ«ã§ãã¾ã™ã€‚Amazon EKS ã¯ã€ãƒ‘ッãƒé©ç”¨ã€ãƒŽãƒ¼ãƒ‰ãƒ—ãƒãƒ“ジョニングã€æ›´æ–°ãªã©ã®ä¸»è¦ãªã‚¿ã‚¹ã‚¯ã‚’自動化ã—ãªãŒã‚‰ã€é«˜å¯ç”¨æ€§ã§å®‰å…¨ãªã‚¯ãƒ©ã‚¹ã‚¿ãƒ¼ã‚’æä¾›ã—ã¾ã™ã€‚ 作æˆã—㟠Amazon EKS クラスターã”ã¨ã«ã€1 時間ã‚ãŸã‚Š 0.10 USD ã®æ–™é‡‘ãŒç™ºç”Ÿã—ã¾ã™ã€‚Kubernetes ã®åå‰ç©ºé–“㨠IAM ã‚»ã‚ュリティãƒãƒªã‚·ãƒ¼ã‚’活用ã™ã‚‹ã¨ã€EKS クラスター 1 ã¤ã§è¤‡æ•°ã®ã‚¢ãƒ—リケーションを実行ã™ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚Amazon Elastic Compute Cloud (Amazon EC2) ã¾ãŸã¯ AWS Fargate を使用ã—㦠AWS 㧠EKS を実行ã—ã€AWS Outpos
AWS re:Invent 2017ã§ç™ºè¡¨ã•ã‚ŒãŸã‚³ãƒ³ãƒ†ãƒŠé–¢é€£ã‚µãƒ¼ãƒ“ス(EKSã€Fargate)ã®ã¾ã¨ã‚ã¨è‡ªç¤¾ã‚µãƒ¼ãƒ“スã¸ã®æ´»ç”¨ – PSYENCE.MEDIA
AWS re:Invent 2017ã§ç™ºè¡¨ã•ã‚ŒãŸã‚³ãƒ³ãƒ†ãƒŠé–¢é€£ã‚µãƒ¼ãƒ“ス(EKSã€Fargate)ã®ã¾ã¨ã‚ã¨è‡ªç¤¾ã‚µãƒ¼ãƒ“スã¸ã®æ´»ç”¨ 大島 雅人 2017.12.01 113 18323924 ã“ã®è¨˜äº‹ã¯ RECRUIT MARKETING PARTNERS Advent Calendar 2017 ã®æŠ•ç¨¿è¨˜äº‹ã§ã™ã€‚ ã“ã‚“ã«ã¡ã¯ã€é–‹ç™ºæ”¯æ´Gã§ã‚¤ãƒ³ãƒ•ãƒ©é‹ç”¨ã‚’ã—ã¦ã„る大島ã§ã™ã€‚アドベントカレンダー 2017 ã®è¨˜å¿µã™ã¹ã1日目㯠AWS re:Invent in ラスベガス ã‹ã‚‰ãŠå±Šã‘ã—ã¾ã™ã€‚ AWS re:Inventã¯Amazon Web Service, Inc.ãŒæ¯Žå¹´é–‹å‚¬ã—ã¦ã„る技術ã¨ãƒžãƒ¼ã‚±ãƒ†ã‚£ãƒ³ã‚°ã®ã®ã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ã§ã€ä»Šå›žã¯6回目ã®é–‹å‚¬ã¨ãªã‚Šã¾ã™ã€‚æ–°ã—ã„サービスや機能ã®ã‚¢ãƒƒãƒ—デートãŒåŸºèª¿è¬›æ¼”ã§ç™ºè¡¨ã•ã‚ŒãŸã‚Šã€1000以上ã®ã‚»ãƒƒã‚·ãƒ§ãƒ³ãŒè¡Œã‚ã‚ŒãŸã‚Šã™ã‚‹ãªã©ãŠæ¥½ã—ã¿ãŒç››ã‚Šæ²¢å±±ãª5日間ã®ã‚¤ãƒ™ãƒ³ãƒˆã§ã™ã€‚
Kubelet authentication/authorization
KubeCon + CloudNativeCon North America 2021 Los Angeles, California + Virtual. 5 days of incredible opportunites to collaborate, learn + share with the entire community! October 11 - 15, 2021. Overview A kubelet's HTTPS endpoint exposes APIs which give access to data of varying sensitivity, and allow you to perform operations with varying levels of power on the node and within containers. This doc
Container Runtimes
Note: Dockershim has been removed from the Kubernetes project as of release 1.24. Read the Dockershim Removal FAQ for further details. You need to install a container runtime into each node in the cluster so that Pods can run there. This page outlines what is involved and describes related tasks for setting up nodes. Kubernetes 1.31 requires that you use a runtime that conforms with the Container
kubectl Usage Conventions
KubeCon + CloudNativeCon 2024Join us for three days of incredible opportunities to collaborate, learn and share with the cloud native community. Buy your ticket now! 12 - 15 November | Salt Lake City kubectl Usage ConventionsRecommended usage conventions for kubectl. Using kubectl in Reusable ScriptsFor a stable output in a script: Request one of the machine-oriented output forms, such as -o name,
Reserve Compute Resources for System Daemons
KubeCon + CloudNativeCon 2024Join us for three days of incredible opportunities to collaborate, learn and share with the cloud native community. Buy your ticket now! 12 - 15 November | Salt Lake City Reserve Compute Resources for System DaemonsKubernetes nodes can be scheduled to Capacity. Pods can consume all the available capacity on a node by default. This is an issue because nodes typically ru
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
Manage Amazon EMR clusters - Amazon EMR
Plan, configure and launch Amazon EMR clusters - Amazon EMR
Google Kubernetes Engine (GKE)
GKE cluster architecture  | Google Kubernetes Engine (GKE)  | Google Cloud
{{#tags}}- {{label}}
{{/tags}}