[B! BigData] kuchitamaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

kuchitama id:kuchitama

BigDataã«é–¢ã™ã‚‹kuchitamaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (8)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

OSSã®ãƒ‡ãƒ¼ã‚¿å¯è¦–åŒ–ãƒ„ãƒ¼ãƒ«ã€ŒMetabaseã€ãŒè¶…ä½¿ã„ã‚„ã™ã„ - Qiita
Metabaseã€ã¾ã˜ã§ã‚¤ã‚±ã¦ã‚‹ã€‚ 1æ—¥ã§ã€Re:Dashã‹ã‚‰ä¹—ã‚Šæ›ãˆã¾ã—ãŸã€‚ Metabaseã¨ã¯ OSSã®ãƒ‡ãƒ¼ã‚¿å¯è¦–åŒ–ãƒ„ãƒ¼ãƒ«ã€‚Re:Dashã¨ã‹ã¨åŒã˜é¡žã€‚ AWSã¨ã‹ã«ä¹—ã›ã¦ã€èª°ã‚‚ãŒè¦‹ã‚Œã‚‹ãƒ€ãƒƒã‚·ãƒ¥ãƒœãƒ¼ãƒ‰ã‚’ä½œã£ãŸã‚Šã™ã‚‹æ™‚ã«ä½¿ã†ã¨ã€ä¿ºã‹ã£ã‘ãƒ¼ã£ã¦ãªã‚Šã¾ã™ã€‚ ã‚¹ã‚¯ãƒªãƒ¼ãƒ³ã‚·ãƒ§ãƒƒãƒˆ å®Ÿéš›ã«ç¤¾å†…ã§é‹ç”¨ã—ã¦ã„ã‚‹æ§˜åã‚’ãƒ¢ã‚¶ã‚¤ã‚¯ä»˜ãã§ãŠè¦‹ã›ã—ã¾ã™ã€‚ æ¯Žæ—¥ã¿ã¦ã„ã‚‹ãƒ€ãƒƒã‚·ãƒ¥ãƒœãƒ¼ãƒ‰ã€‚ã“ã‚Œã®4å€ãã‚‰ã„ã®æ•°å€¤è¦‹ã¦ã‚‹ã€‚ ãƒ€ãƒƒã‚·ãƒ¥ãƒœãƒ¼ãƒ‰ã®ç·¨é›†ã€‚ãƒžã‚¦ã‚¹ã§ç°¡å˜ã«ä½ç½®ã‚„å¤§ãã•ã®å¤‰æ›´ãŒã§ãã‚‹ã€‚ ã‚¯ã‚¨ãƒªãƒ“ãƒ«ãƒ€ãƒ¼çš„ãªã‚„ã¤ãŒã‚ã‚Šã€ã‚°ãƒ«ãƒ¼ãƒ”ãƒ³ã‚°ã¨ã‹è¶…çµ¶ä¾¿åˆ©ã€‚SQLæ›¸ã‘ãªã„äººã§ã‚‚ã€å˜ä¸€ãƒ†ãƒ¼ãƒ–ãƒ«ã¨ã‹ã ã£ãŸã‚‰ä½™è£•ã‹ã¨ã€‚ ã‚°ãƒ©ãƒ•åŒ–ã‚‚ç°¡å˜ã€‚ SQLã§ã®ã‚¯ã‚¨ãƒªç·¨é›†ç”»é¢ã€‚ã¡ã‚ƒã‚“ã¨è£œå®Œã—ã¦ãã‚Œã‚‹ã€‚ Metabaseã®è‰¯ã„æ‰€ è¦‹ãŸç›®ãŒæ ¼å¥½ã„ã„ã€‚ ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ãŒç°¡å˜ã€‚ openjdk-8-jdkå…¥ã‚Œã¦ã€jarãƒ•ã‚¡ã‚¤ãƒ«ç½®ãã ã‘ã€‚ è±Šå¯Œãªãƒ‡ãƒ¼ã‚¿ã‚½ãƒ¼ã‚¹ã«å¯¾å¿œ
kuchitama 2018/01/04
ãŠãŠã€Clojureè£½ã¨ãªï¼è©¦ã—ã¦ã¿ã‚ˆã†

clojure

oss

ãƒ‡ãƒ¼ã‚¿

DataAnalytics

BigData
ãƒªãƒ³ã‚¯
Rettyã®åˆ†æžåŸºç›¤ã«ãŠã‘ã‚‹AWSæ´»ç”¨è¡“
#xtechjaws #xtechjaws01 2017/10/20 X-Tech JAWS ã®è³‡æ–™
kuchitama 2018/01/04
ã‚„ã£ã±ã€è¦ä»¶å®šç¾©å¤§äº‹ã ã‚ˆãªãã€‚ã§ã‚‚ã€è¦ä»¶å¼•ãå‡ºã™ã®ãŒé›£ã—ã„ã‚“ã ã‚ˆâ€¦ãƒ‡ãƒ¼ã‚¿åˆ†æžã—ã¦ä½•ã‚„ã‚ŠãŸã„ã¨ã‹ãªã‹ã£ãŸã‚Šã€å˜ã«ä»Šã‚„ã£ã¦ã‚‹æ—¥é ƒã®é›†è¨ˆã‚’ç°¡å˜ã«ã—ãŸã„ãã‚‰ã„ã—ã‹ãƒ“ã‚¸ãƒ§ãƒ³ãŒãªã„å ´åˆãŒå¤šã„â€¦

BigData

DataAnalytics
ãƒªãƒ³ã‚¯
How to create Treasure Data #dotsbigdata
This document provides an overview of Treasure Data's big data analytics platform. It discusses how Treasure Data ingests and processes large amounts of schema-less data from various sources in real-time and at scale. It also describes how Treasure Data stores and indexes the data for fast querying using SQL interfaces while maintaining schema flexibility.Read less
kuchitama 2015/08/02
treasure data

fluentd

bigdata
ãƒªãƒ³ã‚¯
GCP Next: GoogleãŒæãã€MapReduceã‚’è¶…ãˆãŸãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã®ä¸–ç•Œ
GCP Next 2015ã®ã‚»ãƒƒã‚·ãƒ§ãƒ³ã€ŒGoogleãŒæãã€MapReduceã‚’è¶…ãˆãŸãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ã®ä¸–ç•Œã€ã®ã‚¹ãƒ©ã‚¤ãƒ‰ã§ã™ã€‚
kuchitama 2015/06/19
Google

bigdata

GCP

mapreduce

bigquery

BigData
ãƒªãƒ³ã‚¯
Apache Drill 1.0ãŒãƒªãƒªãƒ¼ã‚¹ã•ã‚ŒãŸã®ã§Amazon S3ã¨é€£æºã—ã¦ã¿ãŸ | DevelopersIO
ã¯ã˜ã‚ã« Apache Drill 1.0ãŒãƒªãƒªãƒ¼ã‚¹ã•ã‚Œã¾ã—ãŸï¼ Drill 1.0 Released Apache Drillã¯ã€NoSQLå‘ã‘ã®SQLã‚¯ã‚¨ãƒªã‚¨ãƒ³ã‚¸ãƒ³ã§ã€ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ä¸Šã®JSON/CSV/Parquetãªã©ã®ãƒ•ã‚¡ã‚¤ãƒ«ã€Hiveã‚½ãƒ¼ã‚¹ã€HBaseã€Mongo DBãªã©ã«ç›´æŽ¥SQLã‚¯ã‚¨ãƒªã‚’æŠ•ã’ã‚‹ã“ã¨ãŒå‡ºæ¥ã¾ã™ã€‚ ä½•ã¯ã¨ã‚‚ã‚ã‚Œã€è§¦ã£ã¦ã¿ãªã„ã¨ã©ã‚“ãªãƒ¢ãƒŽãªã®ã‹ã‚ã‹ã‚Šã¾ã›ã‚“ã€‚æ—©é€Ÿè©¦ã—ã¦ã¿ã¾ã—ãŸï¼ ã‚„ã£ã¦ã¿ãŸ è©¦ã—ãŸç’°å¢ƒã¯EC2(on Amazon Linux 2015.03)ã§ã™ã€‚ ã‚»ãƒƒãƒˆã‚¢ãƒƒãƒ— Drillã®å‹•ä½œã«ã¯JDK 7ãŒå¿…è¦ã«ãªã‚Šã¾ã™ã®ã§ã€ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã•ã‚Œã¦ã„ã‚‹ã“ã¨ã‚’ç¢ºèªã—ã¾ã™ã€‚ $ java -version java version "1.7.0_79" OpenJDK Runtime Environment (amzn-2.5.5.1.59.amzn1-x86_64
kuchitama 2015/05/22
Apache

json

csv

ãƒ„ãƒ¼ãƒ«

Drill

aws

BigData
ãƒªãƒ³ã‚¯
Sparkã®è³‡æ–™ - Scalaæ—¥è¨˜
Sparkã®ç´ æ™´ã‚‰ã—ã„è³‡æ–™ãŒå…¬é–‹ã•ã‚Œã¾ã—ãŸã€‚ã“ã‚Œã‚’è¦‹ã‚Œã°ã™ãã«MapReduceã§PMIãŒè¨ˆç®—ã§ãã‚‹ã‚ˆã†ã«ãªã‚Šã¾ã™ã€‚ Apache Spark ãƒãƒ¥ãƒ¼ãƒˆãƒªã‚¢ãƒ« from K Yamaguchi www.slideshare.net
kuchitama 2015/05/08
spark

scala

apacheSpark

BigData

DataAnalytics
ãƒªãƒ³ã‚¯
Apache Spark ãƒãƒ¥ãƒ¼ãƒˆãƒªã‚¢ãƒ«
4. ç´ æœ´ãªå®Ÿè£… 3 frequency = defaultdict(int) for line in opened_file: for word in some_splitter(line): frequency[word] += 1 for word in frequency: some_output(word, frequency[word]) ãƒ•ã‚¡ã‚¤ãƒ« frequency ãƒ¡ãƒ¢ãƒªã« è¾žæ›¸/ãƒãƒƒã‚·ãƒ¥/ é€£æƒ³é…åˆ—/Mapã‚’ æŒã¤é »åº¦(Pythonic) frequency = collections.Counter( word for line in opened_file for word in some_splitter(line)) for word, count in frequency.iterit ems(): some_output(word, count)
kuchitama 2015/05/02
ã‚ã¨ã§èªã‚€

OSS

Apache

apacheSpark

spark

BigData
ãƒªãƒ³ã‚¯
ç¬¬1å›žã€€ITç®¡ç†è€…ã‚‚çŸ¥ã£ã¦ãŠããŸã„Googleã‚¢ãƒŠãƒªãƒ†ã‚£ã‚¯ã‚¹ã®æ¦‚è¦
ç¬¬1å›žã€€ITç®¡ç†è€…ã‚‚çŸ¥ã£ã¦ãŠããŸã„Googleã‚¢ãƒŠãƒªãƒ†ã‚£ã‚¯ã‚¹ã®æ¦‚è¦ï¼šGoogleã‚¢ãƒŠãƒªãƒ†ã‚£ã‚¯ã‚¹å…¥é–€ï¼ˆ1/2 ãƒšãƒ¼ã‚¸ï¼‰ ä»Šã‚„Webãƒšãƒ¼ã‚¸ã‚‚ä¼æ¥ã®ã€Œé¡”ã€ã®ä¸€ã¤ã¨ãªã£ã¦ã„ã‚‹ã€‚ãã®ã€Œé¡”ã€ãŒååˆ†ã«æ´»ç”¨ã•ã‚Œã¦ã„ã‚‹ã®ã‹ã€ç¢ºèªã™ã‚‹ã«ã¯ã‚¢ã‚¯ã‚»ã‚¹åˆ†æžãŒåŠ¹æžœçš„ã ã€‚ã—ã‹ã—ã€å…·ä½“çš„ã«ã¯ã©ã†ã™ã‚Œã°ã‚ˆã„ã®ã‹ã€‚æœ¬é€£è¼‰ã§ã¯ã€åˆ†æžãƒ„ãƒ¼ãƒ«ã¨ã—ã¦åºƒãåˆ©ç”¨ã•ã‚Œã¦ã„ã‚‹Googleã‚¢ãƒŠãƒªãƒ†ã‚£ã‚¯ã‚¹ã®æ¦‚è¦ã¨ã€ä»£è¡¨çš„ãªã‚¢ã‚¯ã‚»ã‚¹åˆ†æžã®æ–¹æ³•ã«ã¤ã„ã¦è§£èª¬ã—ã¦ã„ãã€‚ é€£è¼‰ç›®æ¬¡ ãƒ“ã‚¸ãƒã‚¹ã«ãŠã‘ã‚‹Webã®é‡è¦æ€§ã¨ãƒ‡ãƒ¼ã‚¿åˆ†æž ä»Šã‚„ã»ã¨ã‚“ã©ã®ä¼æ¥ãŒä¼æ¥æƒ…å ±ãªã©ã‚’Webã‚µã‚¤ãƒˆã§æä¾›ã—ã¦ãŠã‚Šã€å¤šãã®äººãŒãã®ä¼æ¥ã«ã¤ã„ã¦èª¿ã¹ã‚‹éš›ã«å‚è€ƒã«ã—ã¦ã„ã‚‹ã€‚ã¤ã¾ã‚Šã€Webã‚µã‚¤ãƒˆãŒä¼æ¥ã®ä¸€ã¤ã®ã€Œé¡”ã€ã¨ãªã£ã¦ã„ã‚‹ã‚ã‘ã ã€‚ã‚ªãƒ³ãƒ©ã‚¤ãƒ³ã‚·ãƒ§ãƒƒãƒ”ãƒ³ã‚°ã‚µã‚¤ãƒˆï¼ˆECã‚µã‚¤ãƒˆï¼‰ã‚„Webã‚³ãƒ³ãƒ†ãƒ³ãƒ„ã‚µã‚¤ãƒˆã§ã¯ä»¥å‰ã‹ã‚‰ã‚¢ã‚¯ã‚»ã‚¹ãƒã‚°ã‚’è§£æžã™ã‚‹ã“ã¨ã§ã€Webã‚µã‚¤ãƒˆã®ä½¿ã„å‹æ‰‹ã‚’å‘ä¸Šã•ã›ã¦ã€æ¥ç¸¾ã‚’ä¼¸ã°ã™åŠªåŠ›
kuchitama 2015/04/23
ã‚ã¨ã§èªã‚€

Google

Web

ã‚¢ã‚¯ã‚»ã‚¹è§£æž

BigData
ãƒªãƒ³ã‚¯
1