SlideShare a Scribd company logo

Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6

•
•
AbemaTV, Inc.
AbemaTV, Inc.

This document discusses transcoding video files using Kubernetes jobs. It describes creating different job pods for each transcoding task, such as 1080p, 720p, etc. It shows configuring the jobs with specifications for parallelism, completions, and restart policies. The document also discusses using the Kubernetes client API to watch for job status changes in order to manage the transcoding workflow.

1 of 43


! "

 

Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6
! ! 

1080p! 720p! 480p" 240p"360p" 180p" HLS MPEG2-TS 1080p! 720p! 480p" 240p"360p" 180p" MPEG-DASH Fragmented MP4 # $ Thumbnail % DRM packaging
Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6
Worker 1 Worker 2 Worker 3 Queue ∠
Task1Task2Task3 Worker Pod1 Worker Pod2 Worker Pod3 Node (GCE VM Instance) Enqueue task ID Store task details Pull tasks Transcode Request Kubernetes Task1 Task2 program: XXX type: master status: inqueue program: YYY type: mbr status: inqueue ∠
Task1Task2Task3 Worker Pod1 Worker Pod2 Worker Pod3 Node (GCE VM Instance) Enqueue task ID Store task details Pull tasks Transcode Request Kubernetes Task1 Task2 program: XXX type: master status: inqueue program: YYY type: mbr status: inqueue ∠
Worker Pod 1 Worker Pod 2 Worker Pod 3 vCPUx5, Mem 8GB vCPUx5, Mem 8GB vCPUx5, Mem 8GB メタデータ生成 必要リソース小 240pトランスコード 必要リソース中 1080pトランスコード 必要リソース大 Kubernetes Node (vCPUx16, Mem 25GB)
Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6
Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6
Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6
kind: Pod spec: containers: - name: transcoder image: transcoder resources: requests: memory: “500Mi" cpu: "1" limits: memory: “8Gi" cpu: "5"
Kubernetes Worker Pod 1 use: vCPUx1, Mem “500Mi” Worker Pod 2 use: vCPUx5, Mem “8Gi” Node (vCPUx16, Mem 25GB) … Kubernetes Worker Pod 2 use: vCPUx1, Mem “500Mi” Worker Pod 1 use: vCPUx5, Mem “8Gi” Node (vCPUx16, Mem 25GB) …
Kubernetes Worker Pod 1 use: vCPUx8, Mem 10GB Worker Pod 2 use: vCPUx4, Mem 8GB Worker Pod 3 use: vCPUx4, Mem 8GB Worker Pod 4 use: vCPUx8, Mem 10GB Node (vCPUx16, Mem 32GB) Node Evict
Kubernetes Worker Pod 1 use: vCPUx8, Mem 10GB Worker Pod 2 use: vCPUx4, Mem 8GB Worker Pod 3 use: vCPUx4, Mem 8GB Worker Pod 4 use: vCPUx8, Mem 10GB Node (vCPUx16, Mem 32GB) Node Evict
Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6
High-perf Workers vCPUx4 Mem 8GB. Low-perf Workers vCPUx1 Mem 2GB. Kubernetes ex) metadata, 180p, 240p ex) 720p, 1080p Trancode ∠ ∠Transcode Request


Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6
MBR Queue ∠ Transcode Queue ∠ DRM Queue ∠ Metadata Generator vCPUx1 Mem 2GB. MBR Transcoder vCPUx4 Mem 8GB. DRM Packager vCPUx2 Mem 4GB. Kubernetes


Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6
Job Pod
Task1 Task2 program: XXX type: master status: complete program: YYY type: mbr status: running Create JobStore task details Kubernetes Transcode 1080p vCPUx8, Mem 10GB DRM packaging vCPUx1, Mem 1GB Transcode 240p vCPUx2, Mem 2GB Transcode 480p vCPUx4, Mem 4GB Transcode Manager
Task1 Task2 program: XXX type: master status: complete program: YYY type: mbr status: running Create JobStore task details Kubernetes Transcode 1080p vCPUx8, Mem 10GB DRM packaging vCPUx1, Mem 1GB Transcode 240p vCPUx2, Mem 2GB Transcode 480p vCPUx4, Mem 4GB Transcode Manager
∠
Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6
! Video Metadata " Thumbnail ! MPEG2-TS MBR ! FMP4 MBR # DRM packaging 1080p 1080p 720p 720p 480p 480p 360p 360p 240p 240p 180p 180p MPEG2-TS MBR FMP4 MBR
! Video Metadata " Thumbnail ! MPEG2-TS MBR ! FMP4 MBR # DRM packaging 1080p 1080p 720p 720p 480p 480p 360p 360p 240p 240p 180p 180p MPEG2-TS MBR FMP4 MBR
kind: Job spec: template: spec: containers: - name: transcoder image: abema-transcoder command: [“transcoder”, …] restartPolicy: OnFailure backOffLimit: 3
! Video Metadata " Thumbnail ! MPEG2-TS MBR ! FMP4 MBR # DRM packaging 1080p 1080p 720p 720p 480p 480p 360p 360p 240p 240p 180p 180p MPEG2-TS MBR FMP4 MBR
kind: Job spec: template: spec: containers: - name: transcoder image: abema-transcoder command: [“transcoder”, …] parallelism: 3 completions: 3
Job MBR Manager Job 1080p Job 720p Job 480p Job 360p Job 240p Job 180p
import ( “k8s.io/client-go/kubernetes" "k8s.io/client-go/pkg/watch" ) wi, err := client.BatchV1().Jobs(..).Watch(..) defer wi.Stop() for { r, _ := <- wi.ResultChan() switch r.Type { case watch.Modified: // ... case watch.Deleted: // ... case watch.Error: // … } (k8s.io/client-go/kubernetes)
 Clientset.BatchV1().Jobs().Watch()
https://kubernetes.io/docs/concepts/workloads/controllers/ jobs-run-to-completion/#job-patterns A single Job to create a pod which then creates other pods, acting as a sort of custom controller for those pods. This allows the most flexibility, but may be somewhat complicated to get started with and offers less integration with Kubernetes. $ %
https://kubernetes.io/docs/concepts/workloads/controllers/ jobs-run-to-completion/#job-patterns A single Job to create a pod which then creates other pods, acting as a sort of custom controller for those pods. This allows the most flexibility, but may be somewhat complicated to get started with and offers less integration with Kubernetes. $ % 

Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6
Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6


THANK YOU

More Related Content

Kubernetes Jobによるバッチシステムのリソース最適化 / AbemaTV DevCon 2018 TrackB Session B6