[B! spark] kma83ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

kma83 id:kma83

sparkã«é–¢ã™ã‚‹kma83ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (30)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Google Kubernetes Engine ã§ã® Dataproc Â |Â Dataproc Documentation Â |Â Google Cloud
ãƒ•ã‚£ãƒ¼ãƒ‰ãƒãƒƒã‚¯ã‚’é€ä¿¡ Google Kubernetes Engine ã§ã® Dataproc ã‚³ãƒ¬ã‚¯ã‚·ãƒ§ãƒ³ã§ã‚³ãƒ³ãƒ†ãƒ³ãƒ„ã‚’æ•´ç† å¿…è¦ã«å¿œã˜ã¦ã€ã‚³ãƒ³ãƒ†ãƒ³ãƒ„ã®ä¿å˜ã¨åˆ†é¡žã‚’è¡Œã„ã¾ã™ã€‚ ãƒ•ã‚£ãƒ¼ãƒ‰ãƒãƒƒã‚¯ã‚’é€ä¿¡ ç‰¹ã«è¨˜è¼‰ã®ãªã„é™ã‚Šã€ã“ã®ãƒšãƒ¼ã‚¸ã®ã‚³ãƒ³ãƒ†ãƒ³ãƒ„ã¯ã‚¯ãƒªã‚¨ã‚¤ãƒ†ã‚£ãƒ–ãƒ»ã‚³ãƒ¢ãƒ³ã‚ºã®è¡¨ç¤º 4.0 ãƒ©ã‚¤ã‚»ãƒ³ã‚¹ã«ã‚ˆã‚Šä½¿ç”¨è¨±è«¾ã•ã‚Œã¾ã™ã€‚ã‚³ãƒ¼ãƒ‰ã‚µãƒ³ãƒ—ãƒ«ã¯ Apache 2.0 ãƒ©ã‚¤ã‚»ãƒ³ã‚¹ã«ã‚ˆã‚Šä½¿ç”¨è¨±è«¾ã•ã‚Œã¾ã™ã€‚è©³ã—ãã¯ã€Google Developers ã‚µã‚¤ãƒˆã®ãƒãƒªã‚·ãƒ¼ã‚’ã”è¦§ãã ã•ã„ã€‚Java ã¯ Oracle ãŠã‚ˆã³é–¢é€£ä¼šç¤¾ã®ç™»éŒ²å•†æ¨™ã§ã™ã€‚ æœ€çµ‚æ›´æ–°æ—¥ 2023-08-14 UTCã€‚ ï¿½ï¿½cï¿½ V `7 ï¿½ V ï¿½Xï¿½ V ï¿½ZÙ® V
kma83 2018/06/08
spark

kubernetes

google
ãƒªãƒ³ã‚¯
Submitting User Applications with spark-submit | Amazon Web Services
AWS Big Data Blog Submitting User Applications with spark-submit Francisco Oliveira is a consultant with AWS Professional Services Customers starting their big data journey often ask for guidelines on how to submit user applications to Spark running on Amazon EMR. For example, customers ask for guidelines on how to size memory and compute resources available to their applications and the best reso
kma83 2018/05/24
emr

spark
ãƒªãƒ³ã‚¯
EMRã§ã®Sparkã‚¢ãƒ—ãƒªã®ãƒˆãƒ©ãƒ–ãƒ«ã‚·ãƒ¥ãƒ¼ãƒˆ - Qiita
Deleted articles cannot be recovered. Draft of this article would be also deleted. Are you sure you want to delete this article? æ¦‚è¦ EMRï¼ˆâ‰’YARNï¼‰ä¸Šã§å‹•ã‹ã™Sparkã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã‚’é‹ç”¨ã—ã¦ã„ã¦ã€ã‚¨ãƒ©ãƒ¼ãŒèµ·ããŸæ™‚ã«ã©ã®ã‚ˆã†ã«ãƒã‚°ã‚’è¿½ã£ã¦ã„ãã‹ã‚’è¨˜ã—ã¾ã™ã€‚ ç’°å¢ƒ Hadoop: 2.7 Spark: 1.6 EMR: 4.6 ã‚¨ãƒ©ãƒ¼ã®æ¤œçŸ¥ ãƒªãƒˆãƒ©ã‚¤ã‚‚æˆåŠŸã—ãªã‹ã£ãŸå ´åˆã€EMRã®ã‚³ãƒ³ã‚½ãƒ¼ãƒ«ã§å¤±æ•—ãŒè¡¨ç¤ºã•ã‚Œã‚‹ã¨æ€ã„ã¾ã™ã€‚ ï¼ˆã¡ãªã¿ã«çµŒé¨“ä¸Šã€å®Ÿè¡Œæ™‚é–“ãŒ30sä»¥å†…ã§ã‚¨ãƒ©ãƒ¼ã«ãªã£ãŸã‚‰ã ã„ãŸã„stepã®è¨˜è¼‰ãƒŸã‚¹ã€60sç¨‹åº¦ã§çµ‚ã‚ã£ã¦ãŸã‚‰ã‚¢ãƒ—ãƒªã®ãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ãƒŸã‚¹ãªã©ã«ã‚ˆã‚‹åˆæœŸåŒ–ã®å¤±æ•—ãªã©ã§ã¯ã¨æ€ã„ã¾ã™ã€‚ï¼‰ Sparkã‚¢ãƒ—ãƒªã®å®Ÿè¡Œã«ãŠã„ã¦ã¯ã€ã„ãã¤ã‹ã‚¨ãƒ©ãƒ¼ã¨ãªã‚Šã†ã‚‹ã¨ã“ã‚
kma83 2018/05/24
emr

spark
ãƒªãƒ³ã‚¯
ãƒãƒ¥ãƒ¼ãƒˆãƒªã‚¢ãƒ«å…¬é–‹ : Apache Spark ã‚’ Kubernetes Engine ã§ãƒã‚¤ãƒ†ã‚£ãƒ–ã«å®Ÿè¡Œ | Google Cloud å…¬å¼ãƒ–ãƒã‚°
ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã®ã‚¯ãƒ©ã‚¹ã‚¿ ã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒ†ã‚£ãƒ³ã‚° ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã§ã‚ã‚‹ Apache Spark ã¯ã€ç‰¹ã«é‡‘èžã‚„ãƒ¡ãƒ‡ã‚£ã‚¢ã€ãƒ˜ãƒ«ã‚¹ã‚±ã‚¢ã€å°å£²ã‚Šã¨ã„ã£ãŸæ¥ç¨®ã«ãŠã„ã¦ã€å¤§è¦æ¨¡ãªãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚„æ©Ÿæ¢°å¦ç¿’ã®ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ ã¨ã—ã¦äººæ°—ã®é«˜ã„ãƒ—ãƒãƒ€ã‚¯ãƒˆã§ã™ã€‚ ã“ã® Apache Spark ã‚’ Kubernetes ã¨ãƒã‚¤ãƒ†ã‚£ãƒ–ã«é€£æºã•ã›ã‚‹ã¹ãã€Google Cloud ã¯ 1 å¹´é–“ã«ã‚ãŸã£ã¦ãã®å–ã‚Šçµ„ã¿ã‚’ãƒªãƒ¼ãƒ‰ã—ã¦ãã¾ã—ãŸã€‚2016 å¹´ 12 æœˆã«å°è¦æ¨¡ãªã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ ã‚¤ãƒ‹ã‚·ã‚¢ãƒãƒ–ã¨ã—ã¦ã‚¹ã‚¿ãƒ¼ãƒˆã—ãŸã“ã®ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã¯æ‹¡å¤§ã‚’ç¶šã‘ã€ä¸¡è€…ã®é€£æºã‚’ç¶æŒã—ã‚µãƒãƒ¼ãƒˆã™ã‚‹æ´»ç™ºãªã‚³ãƒŸãƒ¥ãƒ‹ãƒ†ã‚£ã‚’è‚²ã‚“ã§ã„ã¾ã™ã€‚ Apache Spark ã®ãƒãƒ¼ã‚¸ãƒ§ãƒ³ 2.3 ã«ã¯ãƒã‚¤ãƒ†ã‚£ãƒ–ã® Kubernetes ã‚µãƒãƒ¼ãƒˆãŒå«ã¾ã‚Œã¦ãŠã‚Šã€ãƒ¦ãƒ¼ã‚¶ãƒ¼ã¯ Kubernetes ã®åå‰ç©ºé–“ã¨ã‚¯ã‚©ãƒ¼ã‚¿ã«ã‚ˆã‚‹ãƒžãƒ«ãƒãƒ†ãƒŠãƒ³ã‚·ãƒ¼ã‚„å…±æœ‰ã«åŠ ãˆã€ãƒ—ãƒ©ã‚¬ãƒ–ãƒ«ãªèªå¯
kma83 2018/05/20
spark

kubernetes

google
ãƒªãƒ³ã‚¯
Running Spark on Kubernetes - Spark 2.3.0 Documentation
Running Spark on Kubernetes Prerequisites How it works Submitting Applications to Kubernetes Docker Images Cluster Mode Dependency Management Using Remote Dependencies Secret Management Introspection and Debugging Accessing Logs Accessing Driver UI Debugging Kubernetes Features Namespaces RBAC Client Mode Future Work Configuration Spark Properties Spark can run on clusters managed by Kubernetes. T
kma83 2018/05/20
spark

kubernetes
ãƒªãƒ³ã‚¯
Apache Spark on Kubernetesã®æ¤œè¨¼ - Qiita
#Apache Spark on Kubernetes ã¨ã¯ Apache Spark(ä»¥ä¸‹ã€Spark)ã¯ã€v2.3.0ã‚ˆã‚ŠKubernetesã‚’Nativeã‚µãƒãƒ¼ãƒˆã—ã¾ã—ãŸã€‚ã“ã‚Œã¾ã§ã€ç®¡ç†è€…ã«ã‚ˆã‚Šäº‹å‰ã«ç”¨æ„ã•ã‚ŒãŸSpark/Hadoopã¨ã€ãã‚Œã«å‰²ã‚Šå½“ã¦ã‚‰ã‚ŒãŸã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒˆãƒªã‚½ãƒ¼ã‚¹ã‚’ä½¿ã£ã¦ä¸¦åˆ—è¨ˆç®—ã‚’å®Ÿè¡Œã—ã¦ã„ã¾ã—ãŸã€‚ä»Šå›žã®Nativeã‚µãƒãƒ¼ãƒˆã§ã¯ã€Kubernetesã‚’ç®¡ç†ã™ã‚‹ç®¡ç†è€…ãŒSparkç”¨ã®Pod(ã‚³ãƒ³ãƒ†ãƒŠ)ã‚’ã‚ã‚‰ã‹ã˜ã‚æº–å‚™ã™ã‚‹å¿…è¦ã¯ã‚ã‚Šã¾ã›ã‚“ã€‚Kubernetesç’°å¢ƒã•ãˆã‚ã‚Œã°ã€Sparkç”¨ã«ç”¨æ„ã•ã‚ŒãŸã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒˆãƒªã‚½ãƒ¼ã‚¹ã‚’æ°—ã«ã›ãšã«å®Ÿè¡Œã™ã‚‹ã“ã¨ãŒå¯èƒ½ã¨ãªã‚Šã¾ã™ã€‚ å¤§ã¾ã‹ãªå®Ÿè¡Œã®æµã‚Œã¨ã—ã¦ã¯ã€Sparkã®ãƒ¦ãƒ¼ã‚¶ãŒSparkã®CLI(spark-submitã‚³ãƒžãƒ³ãƒ‰)ã‚’å®Ÿè¡Œã™ã‚‹ã¨ã€Kubernetesã®API Serverã‚’é€šã˜ã€Schedulerã«ã¦Sparkã‚’å®Ÿè¡Œã™ã‚‹ã‚³ãƒ³ãƒ”ãƒ¥
kma83 2018/05/19
spark

kubernetes
ãƒªãƒ³ã‚¯
Event time processing in Apache Spark and Apache Flink
kma83 2018/05/14
stream

apacheflink

spark
ãƒªãƒ³ã‚¯
SparkInternalsã§çŸ¥ã‚‹ã€Sparkã®å†…éƒ¨æ§‹é€ æ¦‚è¦ï¼ˆæ¦‚è¦/Logical Planï¼‰ - Qiita
ã¯ã˜ã‚ã« ã“ã‚“ã«ã¡ã¯ã€‚ ã“ã®è¨˜äº‹ã¯Apache Spark Advent Calendar 2015ã®19æ—¥ç›®ã®è¨˜äº‹ã§ã™ã€‚ Sparkã‚’ä½¿ã†ã«ã‚ãŸã£ã¦å›°ã‚‹ç‚¹ Sparkã‚’ä½¿ã£ã¦ã„ã‚‹çš†ã•ã‚“ãªã‚‰å®Ÿæ„Ÿã—ã¤ã¤ã‚ã‚‹ã¨æ€ã„ã¾ã™ãŒã€Sparkã«ã¯å›°ã£ãŸç‚¹ã‚‚ã‚ã‚Šã¾ã™ã€‚ ã¨ã‚Šã‚ãˆãšã‚µãƒ³ãƒ—ãƒ«ã‚’å‹•ã‹ã™ã ã‘ã€ã§ã‚ã‚Œã°ãã‚Œã»ã©è‹¦åŠ´ã™ã‚‹ã“ã¨ã¯ç„¡ã„ã®ã§ã™ãŒã€å®Ÿéš›ã«å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã‚’ç”¨ã„ã¦ä½¿ç”¨ã—å‡ºã™ã¨ã€ä¸‹è¨˜ã®ã‚ˆã†ãªå•é¡ŒãŒé »ç™ºã—ã¾ã™ã€‚ é©åˆ‡ãªãƒãƒ¥ãƒ¼ãƒ‹ãƒ³ã‚°ãŒæ–½ã•ã‚Œã¦ã„ãªã„ã¨ã‚¹ãƒ”ãƒ¼ãƒ‰ãŒå‡ºãªã„ ä¸€éƒ¨ã®ã‚³ãƒ³ãƒãƒ¼ãƒãƒ³ãƒˆã¯ãã‚‚ãã‚‚å‹•ã‹ãªã„ï¼ˆç‰¹ã«MLlibï¼‰ å•é¡ŒãŒç™ºç”Ÿã—ãŸæ™‚ã€ã©ã“ã§ç™ºç”Ÿã—ãŸã‹ãŒã±ã£ã¨ã¯ã‚ã‹ã‚Šã«ãã„ ãã®ãŸã‚ã€ã‚ã‚‹ç¨‹åº¦å†…éƒ¨æ§‹é€ ã‹ã€ãƒãƒ¥ãƒ¼ãƒ‹ãƒ³ã‚°ã®å‹˜æ‰€ã‚’ãŠã•ãˆã¦ãŠã‹ãªã„ã¨è‹¦åŠ´ã™ã‚‹ã“ã¨ã«ãªã‚Šã¾ã™ã€‚ å„ªã‚Œã¦ã¯ã„ã‚‹ã‚“ã§ã™ãŒã€ãã®åˆ†ãƒ”ãƒ¼ã‚ãƒ¼ãªãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã€ã¨ã„ã†ã®ãŒMapReduceã¨æ¯”ã¹ãŸç§ã®å°è±¡ã§ã™ã€‚ ãã‚“ãªã‚ã‘ã§ã€Sparkã®å†…
kma83 2018/04/27
spark
ãƒªãƒ³ã‚¯
Introduction Â· The Internals of Apache Spark
$ docker run -u $UID --privileged -v `pwd`:/antora --rm -t antora/antora antora-playbook.yml // alternatively and recommended $ docker run --entrypoint ash --privileged -v `pwd`:/antora --rm -it antora/antora // Inside the container /antora # antora version 2.2.0 /antora # antora local-antora-playbook.yml // On your local computer (outside the container) $ open .out/local/index.html
kma83 2018/04/17
spark

gitbook
ãƒªãƒ³ã‚¯
ASF JIRA
kma83 2018/04/14
spark

rdd
ãƒªãƒ³ã‚¯
Why Spark RDD is invariant in T?
kma83 2018/04/14
spark

rdd
ãƒªãƒ³ã‚¯
JIRA
kma83 2018/04/14
spark

rdd
ãƒªãƒ³ã‚¯
Spark ã®ãƒ¡ã‚¿ã‚¹ãƒˆã‚¢ã¨ã—ã¦ AWS Glue ãƒ‡ãƒ¼ã‚¿ã‚«ã‚¿ãƒã‚°ã‚’ä½¿ç”¨ã™ã‚‹ SQL - Amazon EMR
ç¿»è¨³ã¯æ©Ÿæ¢°ç¿»è¨³ã«ã‚ˆã‚Šæä¾›ã•ã‚Œã¦ã„ã¾ã™ã€‚æä¾›ã•ã‚ŒãŸç¿»è¨³å†…å®¹ã¨è‹±èªžç‰ˆã®é–“ã§é½Ÿé½¬ã€ä¸ä¸€è‡´ã¾ãŸã¯çŸ›ç›¾ãŒã‚ã‚‹å ´åˆã€è‹±èªžç‰ˆãŒå„ªå…ˆã—ã¾ã™ã€‚ Spark ã®ãƒ¡ã‚¿ã‚¹ãƒˆã‚¢ã¨ã—ã¦ AWS Glue ãƒ‡ãƒ¼ã‚¿ã‚«ã‚¿ãƒã‚°ã‚’ä½¿ç”¨ã™ã‚‹ SQL Amazon EMRãƒªãƒªãƒ¼ã‚¹ 5.8.0 ä»¥é™ã‚’ä½¿ç”¨ã—ã¦ã€ AWS Glue Data Catalog ã‚’ãƒ¡ã‚¿ã‚¹ãƒˆã‚¢ã¨ã—ã¦ä½¿ç”¨ã™ã‚‹SQLã‚ˆã†ã« Spark ã‚’è¨å®šã§ãã¾ã™ã€‚æ°¸ç¶šçš„ãªãƒ¡ã‚¿ã‚¹ãƒˆã‚¢ã€ã¾ãŸã¯ã•ã¾ã–ã¾ãªã‚¯ãƒ©ã‚¹ã‚¿ãƒ¼ã€ã‚µãƒ¼ãƒ“ã‚¹ã€ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã€ã‚ã‚‹ã„ã¯ AWS ã‚¢ã‚«ã‚¦ãƒ³ãƒˆã§å…±æœ‰ã•ã‚Œã‚‹ãƒ¡ã‚¿ã‚¹ãƒˆã‚¢ãŒå¿…è¦ã§ã‚ã‚‹å ´åˆã¯ã€ã“ã®è¨å®šã‚’ãŠå‹§ã‚ã—ã¾ã™ã€‚ AWS Glue ã¯ã€ãƒ‡ãƒ¼ã‚¿ã®åˆ†é¡žã€ã‚¯ãƒªãƒ¼ãƒ‹ãƒ³ã‚°ã€å¼·åŒ–ã€ã•ã¾ã–ã¾ãªãƒ‡ãƒ¼ã‚¿ã‚¹ãƒˆã‚¢é–“ã®ä¿¡é ¼æ€§ã®é«˜ã„ç§»å‹•ã‚’ç°¡å˜ã‹ã¤è²»ç”¨å¯¾åŠ¹æžœã®é«˜ã„æ–¹æ³•ã§è¡Œãˆã‚‹ãƒ•ãƒ«ãƒžãƒãƒ¼ã‚¸ãƒ‰æŠ½å‡ºã€å¤‰æ›ã€ãƒãƒ¼ãƒ‰ (ETL) ã‚µãƒ¼ãƒ“ã‚¹ã§ã™ã€‚ AWS Glue Data Catalog ã¯ã€
kma83 2018/03/13
aws

emr

awsglue

spark
ãƒªãƒ³ã‚¯
Devsumi 2016 b_4 Kafkaã¨Sparkã‚’çµ„ã¿åˆã‚ã›ãŸãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ åˆ†æžåŸºç›¤ã®æ§‹ç¯‰
Kafkaã¨Sparkã‚’çµ„ã¿åˆã‚ã›ãŸãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ åˆ†æžåŸºç›¤ã‚’ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ã®è¦–ç‚¹ã§è¦‹ã¦ã„ãã¾ã™ã€‚Read less
kma83 2017/12/12
apachekafka

spark
ãƒªãƒ³ã‚¯
Spark2.0ã‚’EMRã§è©¦ã™ - Qiita
æ¦‚è¦ ä¸€é€±é–“å‰ãã‚‰ã„ã«Spark2.0ãŒå‡ºã¾ã—ãŸã€‚ãã—ã¦ã€ä»Šæ—¥ç¢ºèªã—ãŸã‚‰EMR5.0ãŒæ—¢ã«Spark2.0å¯¾å¿œã•ã‚Œã¦ã„ã¾ã—ãŸã€‚ã•ã™ãŒAWSã•ã‚“ã§ã™ï¼ ã¨ã„ã†ã“ã¨ã§ã€Spark2.0ã§ã‚¤ãƒžãƒ‰ã‚ã®Sparkå®Ÿè¡Œã‚’ç°¡å˜ã«ãƒ¡ãƒ¢ã—ã¦ãŠãã¾ã™ã€‚ ã‚´ãƒ¼ãƒ« Spark2.0ãŒå‹•ã Scala2.11ãƒ“ãƒ«ãƒ‰ã—ãŸjarãŒå‹•ã Java8ã§å‹•ã YARNåˆ†æ•£ç’°å¢ƒä¸Šã§å‹•ã EMRè¨å®š Software Configurationã«ã¦ä»¥ä¸‹ã®ã‚ˆã†ã«è¨å®š emr-5.0.0 hadoop2.7.2 Spark2.0.0 configurationã«ä»¥ä¸‹ã®jsonã‚’è¿½åŠ Java8ã®è¨å®šã¨ã€sparkã«æœ€é©åŒ–ã™ã‚‹è¨å®šã‚’è¡Œã£ã¦ã„ã¾ã™ã€‚ [ { "classification": "spark", "properties": { "maximizeResourceAllocation": "true" } }, { "Cla
kma83 2017/10/18
emr

spark
ãƒªãƒ³ã‚¯
Spark MLlibã§ã¯ã˜ã‚ã‚‹ã‚¹ã‚±ãƒ¼ãƒ©ãƒ–ãƒ«ãªæ©Ÿæ¢°å¦ç¿’
1. 1Copyright Â© 2014 NTT DATA Corporation NTTãƒ‡ãƒ¼ã‚¿ åŸºç›¤ã‚·ã‚¹ãƒ†ãƒ äº‹æ¥æœ¬éƒ¨ OSSãƒ—ãƒãƒ•ã‚§ãƒƒã‚·ãƒ§ãƒŠãƒ«ã‚µãƒ¼ãƒ“ã‚¹ çŒ¿ç”° æµ©è¼” 2014å¹´12æœˆ17æ—¥ JJUG ãƒŠã‚¤ãƒˆãƒ»ã‚»ãƒŸãƒŠãƒ¼ Spark/MLlibã§ã¯ã˜ã‚ã‚‹ã‚¹ã‚±ãƒ¼ãƒ©ãƒ–ãƒ«ãªæ©Ÿæ¢°å¦ç¿’ 2. 2Copyright Â© 2014 NTT DATA Corporation è‡ªå·±ç´¹ä»‹ ï® æ‰€å±ž/æ°å ï¬ NTTãƒ‡ãƒ¼ã‚¿ åŸºç›¤ã‚·ã‚¹ãƒ†ãƒ äº‹æ¥æœ¬éƒ¨ OSSãƒ—ãƒãƒ•ã‚§ãƒƒã‚·ãƒ§ãƒŠãƒ«ã‚µãƒ¼ãƒ“ã‚¹ ï¬ çŒ¿ç”° æµ©è¼”ï¼ˆã•ã‚‹ãŸ ã“ã†ã™ã‘ï¼‰ ï® ä½•ã‚’ã—ã¦ã„ã‚‹äºº? ï¬ OSSã‚’ä½¿ã£ãŸR&Dã‚„ã‚·ã‚¹ãƒ†ãƒ é–‹ç™ºã€ãƒ†ã‚¯ãƒ‹ã‚«ãƒ«ã‚µãƒãƒ¼ãƒˆã«æºã‚ã£ã¦ã„ã¾ã™ ï¬ 6å¹´ã»ã©å¤§è¦æ¨¡åˆ†æ•£å‡¦ç†åŸºç›¤ã€ŒHadoopã€é–¢é€£ã®R&Dã‚„ã‚·ã‚¹ãƒ†ãƒ é–‹ç™ºã«é–¢ã‚ã£ã¦ãã¾ ã—ãŸ ï¬ è¿‘å¹´ã¯Hadoopã‹ã‚‰æ´¾ç”Ÿã—ã¦ã€ã‚¤ãƒ³ãƒ¡ãƒ¢ãƒªåˆ†æ•£å‡¦ç†åŸºç›¤ã€ŒSparkã€ã«ã‚‚æºã‚ã£ã¦ã„ã¾ã™ ï® è‘—ä½œç‰©(å…±è‘—) ï¬
kma83 2017/02/24
ã‚ãƒ£ãƒƒã‚·ãƒ¥ã®ä»•çµ„ã¿ã¨ã‹

Spark

mllib
ãƒªãƒ³ã‚¯
Gmane -- Mail To News And Back Again
kma83 2016/01/26
spark
ãƒªãƒ³ã‚¯
Running Spark on YARN - Spark 1.4.1 Documentation
kma83 2016/01/26
spark

yarn
ãƒªãƒ³ã‚¯
EMC XtremSF is a server based PCIe Flash hardware
Please be advised our License Portal will be undergoing maintenance between March 15 10:30pm PST - March 16th 9:00am PST during which time users may experience intermittent performance issues. We apologize for the inconvenience. Please be advised that the Broadcom ERP system will be undergoing maintenance between March 28 7pm PST - Apr 1 7pm PST which will impact all new customer accounts created
kma83 2016/01/26
yarn

hadoop

spark
ãƒªãƒ³ã‚¯
How do I handle a task not serializable exception?
kma83 2015/11/10
scala

spark
ãƒªãƒ³ã‚¯
1 2 æ¬¡ã®ãƒšãƒ¼ã‚¸