[B! Spark] skozawaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

skozawa id:skozawa

Sparkã«é–¢ã™ã‚‹skozawaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (12)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

æ ªå¼ä¼šç¤¾ALBERTï¼ˆãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰ã‚¨ãƒ³ã‚¸ãƒ³ï¼‰
ãƒ‡ãƒ¼ã‚¿åˆ†æžã‹ã‚‰å°Žãå‡ºã•ã‚ŒãŸã‚¤ãƒ³ã‚µã‚¤ãƒˆç„¡ã—ã«AIï¼ˆäººå·¥çŸ¥èƒ½ï¼‰ã®æ´»ç”¨ã¯å§‹ã¾ã‚Šã¾ã›ã‚“ã€‚ç§ãŸã¡ã¯ã€å„æ¥ç•ŒçŸ¥è˜ã¨ãƒ‡ãƒ¼ã‚¿ãƒ»ã‚¢ãƒŠãƒªãƒ†ã‚£ã‚¯ã‚¹æŠ€è¡“ã‚’é§†ä½¿ã—ãƒ‡ãƒ¼ã‚¿ãƒ‰ãƒªãƒ–ãƒ³çµŒå–¶ã‚’å¼·åŠ›ã«æ”¯æ´ã—ã¾ã™ã€‚ ãƒ‡ãƒ¼ã‚¿ã€ã‚¢ãƒŠãƒªãƒ†ã‚£ã‚¯ã‚¹ã€AIã¯ä¼æ¥ã«ã¨ã£ã¦ç«¶åˆä»–ç¤¾ã¨ã®å·®åˆ¥åŒ–ã‚’å›³ã‚‹ã‹ã¤ã¦ãªã„ã»ã©å¤§ããªè¦å› ã«ãªã£ã¦ã„ã¾ã™ã€‚ä»Šæ—¥ã®çµŒå–¶å¹¹éƒ¨ãŒåŠ¹çŽ‡ã‚’å‘ä¸Šã—ãªãŒã‚‰æ–°ãŸãªåŽç›Šæºã‚’é–‹æ‹“ã—ã€æ–°ã—ã„ãƒ“ã‚¸ãƒã‚¹ãƒ¢ãƒ‡ãƒ«ã‚’ã‚¿ã‚¤ãƒ ãƒªãƒ¼ã«æ§‹ç¯‰ã™ã‚‹æ–¹æ³•ã‚’æ¨¡ç´¢ã™ã‚‹ä¸ã€ä¾¡å€¤ã‚’ç”Ÿã¿å‡ºã—æˆé•·ã‚’ç¶šã‘ã‚‹ä¼æ¥ã«ã¯ã€Œãƒ‡ãƒ¼ã‚¿æ´»ç”¨ã€ã¨ã„ã†å…±é€šé …ãŒã‚ã‚Šã¾ã™ã€‚ç§ãŸã¡ã¯ã€ç„¡æ•°ã®ãƒ‡ãƒ¼ã‚¿ã‹ã‚‰ä¼æ¥ã«ã¨ã£ã¦æœ¬å½“ã«å¿…è¦ãªãƒ‡ãƒ¼ã‚¿ã‚’æ´»ç”¨ã™ã‚‹ãŸã‚ã®æ–¹æ³•ã‚’çŸ¥ã£ã¦ã„ã¾ã™ã€‚ å°†æ¥ã‚’è¦‹æ®ãˆãŸã‚ªãƒšãƒ¬ãƒ¼ã‚·ãƒ§ãƒ³ä½“åˆ¶ã‚’å‚™ãˆã¦ã„ã‚‹ä¼æ¥ã®åŠæ•°ä»¥ä¸Šï¼ˆ52ï¼…ï¼‰ã¯ã€ã™ã§ã«ãƒ‡ãƒ¼ã‚¿ã¨ã‚¢ãƒŠãƒªãƒ†ã‚£ã‚¯ã‚¹ã‚’å¤§è¦æ¨¡ã«æ´»ç”¨ã—ã¦ã„ã¾ã™ã€‚ãƒ‡ãƒ¼ã‚¿ã¨AIã«é–¢ã™ã‚‹å–ã‚Šçµ„ã¿ã‚’ãƒ“ã‚¸ãƒã‚¹æˆ¦ç•¥ã«æ²¿ã£ã¦å®Ÿæ–½ã™ã‚‹ã“ã¨ã§æŠ•è³‡åˆ©ç›ŠçŽ‡ã‚’è¿…é€Ÿã«æœ€å¤§åŒ–ã—ã€æœ€çµ‚çš„ã«ã¯AIã‚’ãƒ“
skozawa 2016/06/17
Spark

apache
ãƒªãƒ³ã‚¯
åºƒå‘Šé…ä¿¡ç¾å ´ã¦ã‚™ä½¿ã†Sparkæ©Ÿæ¢°å¦ç¿’
8. ãƒ‡ãƒ¼ã‚¿æ§‹é€ 8 â€¢ id: ad identifier â€¢ click: 0/1 for non-click/click â€¢ hour: YYMMDDHH â€¢ C1 --Â anonymized categorical variable â€¢ banner_pos â€¢ site_id â€¢ site_domain â€¢ site_category â€¢ app_id â€¢ app_domain â€¢ app_category â€¢ device_id â€¢ device_ip â€¢ device_model â€¢ device_type â€¢ device_conn_type â€¢ C14-C21 -- anonymized categorical variables
skozawa 2015/08/02
Spark

åºƒå‘Š
ãƒªãƒ³ã‚¯
Apache Spark ã«ã‚ˆã‚‹æŽ¨è–¦ã‚·ã‚¹ãƒ†ãƒ æ¡ˆä»¶ä¾‹
2015-03-21 #TokyoWebmining 44th ã®ç™ºè¡¨è³‡æ–™ã§ã™ã€‚
skozawa 2015/03/21
Spark

recommend
ãƒªãƒ³ã‚¯
DMM.com - DMMã®ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿åˆ†æžã®ã”ç´¹ä»‹ã€€ï½žSparkã«ã‚ˆã‚‹ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰ï½ž
ç”Ÿtweet ã‚„ã£ã¦ã¿ã¦å‡ºæ¥ãŸäº‹ãƒ»ãƒ€ãƒ¡ã ã£ãŸäº‹ 1 è§£æžçµæžœ ãƒ©ã‚ªã‚¦ã¨ãƒˆã‚ã‚‚6å€‹æ¶ˆã—ã§3.75å€ã£ã¦è€ƒãˆã‚‹ã¨ æ‚ªããªã„ ãƒ©ã‚ªã‚¦,ãƒˆã‚,æ¶ˆã—,ããªã„ => ãŸã¶ã‚“ã“ã‚Œã®ã“ã¨ ã‚„ã£ã¦ã¿ã¦å‡ºæ¥ãŸäº‹ãƒ»ãƒ€ãƒ¡ã ã£ãŸäº‹ 2 æ‰‹ä½œæ¥ã§æ•™å¸«ãƒ‡ãƒ¼ã‚¿ä½œæˆ => æ–‡è„ˆã‚’å…¨ç„¶è€ƒæ…®ã—ã¦ãã‚Œãªã„ â‡’ãã‚Œã§ã‚‚ã‚´ãƒŸãŒå¤šå°‘ä¹—ã£ã¦ã—ã¾ã† â‡’æ•™å¸«ãƒ‡ãƒ¼ã‚¿ã®ä½œæˆæ–¹æ³• Spark ãƒãƒžã‚Šã©ã“ã‚ åŠ¹çŽ‡ãŒæ‚ªã„ä¸Šã€é€”ä¸ã§å¿ƒãŒæŠ˜ã‚Œãã†ã ã£ãŸã€‚ Map#filterKeys and Map#mapValues not serializable Spark ãƒãƒžã‚Šã©ã“ã‚ Scala 2.11
skozawa 2015/02/25
dmm

spark
ãƒªãƒ³ã‚¯
Cloudera Blog
We are thrilled to announce the general availability of the Cloudera AI Inference service, powered by NVIDIA NIM microservices, part of the NVIDIA AI Enterprise platform, to accelerate generative AI deployments for enterprises. This service supports a range of optimized AI models, enabling seamless and scala ble AI inference. Background The generative AI landscape is evolving [â€¦] Read blog post
skozawa 2014/10/05
Spark

machine learning
ãƒªãƒ³ã‚¯
Real-time Recommendations using Spark Comcast Labs
Unified governance for all data, analytics and AI assets
skozawa 2014/09/07
Spark

BigData
ãƒªãƒ³ã‚¯
æ©Ÿæ¢°å¦ç¿’ã«é©ã—ãŸå¤§è¦æ¨¡åˆ†æ•£è¨ˆç®—ç’°å¢ƒApache Spark - è‡ªç„¶è¨€èªžå‡¦ç† on Mac
å¤§è¦æ¨¡åˆ†æ•£è¨ˆç®—ç’°å¢ƒã®Apache Sparkã¯ã€Hadoopã®MacReduceã«æ¯”ã¹ã¦ãƒ¡ãƒ¢ãƒªå†…ã§åŠ¹çŽ‡çš„ã«å‡¦ç†ã‚’è¡Œã†ã“ã¨ãŒç‰¹é•·ã§ã€æ©Ÿæ¢°å¦ç¿’ã€ã‚¹ãƒˆãƒªãƒ¼ãƒ å‡¦ç†ã€ã‚°ãƒ©ãƒ•è§£æžã€SQL ãƒ‡ãƒ¼ã‚¿åˆ†æžãªã©ã®æ©Ÿèƒ½ãƒ©ã‚¤ãƒ–ãƒ©ãƒªãŒã‚ã‚Šã¾ã™ã€‚Sparkè‡ªä½“ã¯Scalaè¨€èªžã§å®Ÿè£…ã•ã‚Œã¦ã„ã¾ã™ãŒã€Scalaã®ä»–ã«Pythonç”¨ã®APIã‚„å¯¾è©±çš„ãªã‚·ã‚§ãƒ«ã‚‚ç”¨æ„ã•ã‚Œã¦ã„ã¦ã€ã©ã¡ã‚‰ã®è¨€èªžã§ã‚‚ãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°ã‚„å‹•ä½œç¢ºèªãŒå¯èƒ½ã§ã™ã€‚ å‹•ä½œç¢ºèªã¯ç°¡å˜ã«ã§ãã¦ã€Java6ä»¥ä¸ŠãŒã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã•ã‚Œã¦ã„ã‚Œã°ã€Downloadsãƒšãƒ¼ã‚¸ã‹ã‚‰é©å½“ãªãƒ“ãƒ«ãƒ‰æ¸ˆã¿ãƒ‘ãƒƒã‚±ãƒ¼ã‚¸ã‚’ãƒ€ã‚¦ãƒ³ãƒãƒ¼ãƒ‰ã—ã¦è§£å‡ã™ã‚‹ã¨ã€Scalaã¾ãŸã¯Pythonã®å¯¾è©±çš„ã‚·ã‚§ãƒ«ãŒèµ·å‹•ã§ãã¾ã™ï¼š $ curl -O http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-hadoop1.tgz $ tar zxf spark-1.0.2-b
skozawa 2014/08/15
Spark
ãƒªãƒ³ã‚¯
Scala ã§ã¯ã˜ã‚ã‚‹ Spark / MLlib ã®å˜ç´”ãƒ™ã‚¤ã‚ºåˆ†é¡žå™¨ - ALBERT Engineering Blog
ã¯ã˜ã‚ã« ä»¥å‰ã®ã‚¨ãƒ³ãƒˆãƒªã«ã¦ Spark / MLlib ã® K-means ã‚’å–ã‚Šä¸Šã’ã¾ã—ãŸãŒã€ä»Šå›žã¯åŒã˜ã MLlib ã«ã¦æä¾›ã•ã‚Œã¦ã„ã‚‹æ©Ÿæ¢°å¦ç¿’ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã®ä¸€ã¤ã€ å˜ç´”ãƒ™ã‚¤ã‚ºåˆ†é¡žå™¨ (Naive Bayes, ãƒŠã‚¤ãƒ¼ãƒ–ãƒ™ã‚¤ã‚º) ã‚’ä½¿ã£ã¦ã¿ã¾ã—ã‚‡ã†ï¼†K-åˆ†å‰²äº¤å·®æ¤œè¨¼ã‚’ã—ã¦ã¿ã¾ã—ã‚‡ã†ã€ ã¨ã„ã†ã‚¨ãƒ³ãƒˆãƒªã§ã™ã€‚ å˜ç´”ãƒ™ã‚¤ã‚ºåˆ†é¡žå™¨ãã®ã‚‚ã®ã«ã¤ã„ã¦ã¯èªè€…ã®çš†æ§˜ã¯ã”å˜çŸ¥ã€ã¨ã„ã†å‰æã§ã“ã®å¾Œã®è©±ã‚’é€²ã‚ã¦ã—ã¾ã„ã¾ã™ã®ã§ã€ ã€Œã¡ã‚‡ã£ã¨è‰¯ãã‚ã‹ã‚‰ãªã„ã‚ˆå¾…ã£ã¦ãã‚Œï¼ã€ã¨ã„ã†æ–¹ã¯ å˜ç´”ãƒ™ã‚¤ã‚ºåˆ†é¡žå™¨ â€“ Wikipedia ç¬¬3å›žã€€ãƒ™ã‚¤ã‚¸ã‚¢ãƒ³ãƒ•ã‚£ãƒ«ã‚¿ã‚’å®Ÿè£…ã—ã¦ã¿ã‚ˆã†ï¼šæ©Ÿæ¢°å¦ç¿’ ã¯ã˜ã‚ã‚ˆã†ï½œgihyo.jp â€¦ æŠ€è¡“è©•è«–ç¤¾ ç¬¬8å›žã€€è‡ªç”±å›žç”å¼ã‚¢ãƒ³ã‚±ãƒ¼ãƒˆã§é¡§å®¢ã®å£°ã‚’èžããŸã‚ã®è€ƒãˆæ–¹ï¼šMahoutã§ä½“æ„Ÿã™ã‚‹æ©Ÿæ¢°å¦ç¿’ã®å®Ÿè·µï½œgihyo.jp â€¦ æŠ€è¡“è©•è«–ç¤¾ ãªã©ã®ãƒšãƒ¼ã‚¸ã‚’ä¸€èªã™ã‚‹ã“ã¨ã‚’ãŠã™ã™ã‚ã—ã¾ã™ã€‚ å˜ç´”ãƒ™ã‚¤ã‚ºåˆ†é¡ž
skozawa 2014/08/05
Scala

Spark

MLlib
ãƒªãƒ³ã‚¯
Private Presentation
Private content!This content has been marked as private by the uploader.
skozawa 2014/07/09
æ©Ÿæ¢°å¦ç¿’

Spark
ãƒªãƒ³ã‚¯
Apache Sparkâ„¢ - Unified Engine for large-scale data analytics
Apache Sparkâ„¢ is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters.
skozawa 2014/05/09
Spark

Hadoop

apache
ãƒªãƒ³ã‚¯
Spot Instance + Spark + MLlibã§å®Ÿç¾ã™ã‚‹ç°¡å˜ä½Žã‚³ã‚¹ãƒˆæ©Ÿæ¢°å¦ç¿’
Spot Instance + Spark + MLlibï¿½ã§å®Ÿç¾ã™ã‚‹ç°¡å˜ä½Žã‚³ã‚¹ãƒˆï¿½é«˜é€Ÿæ©Ÿæ¢°å¦ç¿’ AWS Casual Talks#2 LT 2014/4/18 Read less
skozawa 2014/04/23
Spark

æ©Ÿæ¢°å¦ç¿’

aws
ãƒªãƒ³ã‚¯
Spark shark
6. ãŠã¾ãˆã ã‚Œã‚ˆï¼Ÿ Tsuyoshi Ozawa @oza_x86 OSS developer Apache Hadoop ã®é–‹ç™ºã‚’ã—ã¦ã¾ã™ github : oza â†ã®22ç« ã‚’æ›¸ã„ãŸã‚ˆï¼ Tuesday, October 22, 13
skozawa 2013/11/05
Spark

shark
ãƒªãƒ³ã‚¯
1