[B! Flink] nobusueã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

nobusue id:nobusue

Flinkã«é–¢ã™ã‚‹nobusueã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (17)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Apache Flink + Kafka + Zeppelin é€£æºã§ã‚¢ãƒ‰ãƒ›ãƒƒã‚¯ãªã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°åˆ†æž
å‰å›žã€Apache Flinkã¨ã¯ã§Flinkã®æ¦‚è¦ã«è§¦ã‚ŒãŸã®ã§ã™ãŒã€ä»Šå›žã¯å®Ÿè·µçš„ã«kafkaã¨é€£æºã•ã›ã€Apache Zeppelinã§ã‚¢ãƒ‰ãƒ›ãƒƒã‚¯ã«åˆ†æžã—ã¦ã¿ã¾ã™ã€‚ Zeppelinã‚’ä½¿ã†ã®ã¯ã€å˜ç´”ã«ã€jarä½œã£ã¦å®Ÿè¡Œã™ã‚‹ã®ãŒé¢å€’ã ã‹ã‚‰ã§ã™(^_-)-â˜† æ¦‚è¦ fluentdã§ãƒ‡ãƒ¼ã‚¿ã‚’åŽé›†ã—ã€Kafkaã«ãƒ‡ãƒ¼ã‚¿ã‚’é€ã‚Šã¾ã™ã€‚ Zeppelinä¸Šã®notebookã‹ã‚‰flinkã®é›†è¨ˆã‚¸ãƒ§ãƒ–ã‚’å®Ÿè¡Œã—ã¾ã™ã€‚ flinkã‚¸ãƒ§ãƒ–ã¯Kafkaã«é›†ç´„ã•ã‚ŒãŸã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ»ãƒ‡ãƒ¼ã‚¿ã‚’é›†è¨ˆã—ã¾ã™ã€‚ ã¨ã„ã†ã‚ã‘ã§fluentd, Kafkaã‚’ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã—ã¦ã„ãã¾ã™ã€‚ â€» ã“ã®è¨˜äº‹ã§ã¯CentOS 6.7 Finalã‚’ä½¿ã„ã¾ã™ã€‚ã¾ãŸã‚µãƒ¼ãƒä¸Šã§ã¯rootã§ä½œæ¥ã—ã¦ã„ã¾ã™ã€‚ â€» ã¾ãŸã€jdkã¯openJDKã®1.8ã‚’ä½¿ç”¨ã—ã¦ã„ã¾ã™ã€‚ fluentd, Kafkaã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ« ã“ã®è¨˜äº‹ã§ã¯fluentdã¨Kafkaã‚’
nobusue 2016/09/07
Kafka

Flink

Zeppelin
ãƒªãƒ³ã‚¯
Apache BeamÂ®
Introducing Apache BeamThe Unified Apache Beam ModelThe easiest way to do batch and streaming data processing. Write once, run anywhere data processing for mission-critical production workloads. Link to GitHub Repo Introducing Apache BeamThe Unified Apache Beam ModelThe easiest way to do batch and streaming data processing. Write once, run anywhere data processing for mission-critical production w
nobusue 2016/05/26
streaming

Beam

Flink

Spark
ãƒªãƒ³ã‚¯
JVMä¸Šã§ã®ã‚¹ãƒˆãƒªãƒ¼ãƒ å‡¦ç†ã‚¨ãƒ³ã‚¸ãƒ³ã®å¤‰é·
1. JVMä¸Šã® ã‚¹ãƒˆãƒªãƒ¼ãƒ å‡¦ç†ã‚¨ãƒ³ã‚¸ãƒ³ã®å¤‰é· 2016/05/21 JJUG CCC 2016 Spring æœ¨æ‘å®—å¤ªéƒŽï¼ˆ@kimutanskï¼‰ https://www.flickr.com/photos/mattridings/9138233663 2. è‡ªå·±ç´¹ä»‹ â€¢ æœ¨æ‘ å®—å¤ªéƒŽï¼ˆSotaro Kimuraï¼‰ â€¢ ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿ç•Œéšˆã«ç”Ÿæ¯ã™ã‚‹ä½•ã§ã‚‚å±‹ â€¢ ãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰ã‹ã‚‰ãƒ•ãƒãƒ³ãƒˆã‚¨ãƒ³ãƒ‰ã€æŠ€è¡“æ¤œè¨¼ã‹ã‚‰é‹ç”¨ã€ ãƒ‰ã‚ãƒ¥ãƒ¡ãƒ³ãƒˆæ›¸ãã¾ã§è‰²ã€… â€¢ Scala Matsurié‹å–¶ã‚¹ã‚¿ãƒƒãƒ• â€¢ å…ƒStormä½¿ã„ â€¢ Stormã®ãŸã‚ã ã‘ã«Clojureã‚’å‹‰å¼·ã—ã¦ã¾ã—ãŸã‚ˆãƒ»ãƒ»ãƒ» â€¢ ç¾Spark Streamingä½¿ã„ â€¢ ã“ã®ç™ºè¡¨ã§ã¯ãã‚Œè‡ªä½“ã«ã¯ã‚ã¾ã‚Šè§¦ã‚Œã¾ã›ã‚“ã€‚ â€¢ Twitterä»– : @kimutansk 1 4. 3 1. ã‚¹ãƒˆãƒªãƒ¼ãƒ å‡¦ç†ã¨ã¯ï¼Ÿ ãƒãƒƒãƒå‡¦ç† å¯¾è©±åž‹ã‚¯ã‚¨ãƒª ã‚¹ãƒˆãƒªãƒ¼ãƒ å‡¦ç† å®Ÿè¡Œã‚¿
nobusue 2016/05/25
streaming

Spark

Flink
ãƒªãƒ³ã‚¯
Introducing Complex Event Processing (CEP) with Apache Flink
April 6, 2016 - With the ubiquity of sensor networks and smart devices continuously collecting more and more data, we face the challenge to analyze an ever growing stream of data in near real-time. Being able to react quickly to changing trends or to deliver up to date business intelligence can be a decisive factor for a companyâ€™s success or failure. A key probl em in real time processing is the de
nobusue 2016/04/07
Flink

CEP
ãƒªãƒ³ã‚¯
æœ€æ–°ã®ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹æŠ€è¡“ã®æ–¹å‘æ€§ã§æ€ã†ã“ã¨
13. Transf ormation Meaning map(func) Return a new DStream by passing each element of the source DStream through a function func. flatMap(func) Similar to map, but each input it em can be mapped to 0 or more output it ems. filter(func) Return a new DStream by selecting only the records of the source DStream on which func returns true. repartition(numPartitions) Changes the level of parallelism in thi
nobusue 2016/04/06
streaming

Spark

Flink

Kafka
ãƒªãƒ³ã‚¯
Announcing Apache Flink 1.0.0
nobusue 2016/03/10
CEPã‚µãƒãƒ¼ãƒˆãŒæ°—ã«ãªã‚‹

Flink
ãƒªãƒ³ã‚¯
Benchmarking Streaming Computation Engines at Yahoo!
Benchmarking Streaming Computation Engines at Yahoo! (Yahoo Storm Team in alphabetical order) Sanket Chintapalli, Derek Dagit, Bobby Evans, Reza Farivar, Tom Graves, Mark Holderbaugh, Zhuo Liu, Kyle Nusbaum, Kishorkumar Patil, Boyang Jerry Peng and Paul Poulosky. DISCLAIMER: Dec 17th 2015 data-artisans has pointed out to us that we accidentally left on some debugging in the flink benchmark. So the
nobusue 2016/03/03
storm

Spark

Flink
ãƒªãƒ³ã‚¯
Yahoo! Benchmarks Apache Flink, Spark and Storm
InfoQ Software Architects' Newsletter A monthly overview of things you need to know as an architect or aspiring architect. View an example
nobusue 2016/03/03
Storm2300ãƒŽãƒ¼ãƒ‰ã£ã¦ã™ã’ã‡ãª

storm

Spark

Flink
ãƒªãƒ³ã‚¯
GitHub - yahoo/streaming-benchmarks: Benchmarks for Low Latency (Streaming) solutions including Apache Storm, Apache Spark, Apache Flink, ...
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
nobusue 2016/03/03
storm

Spark

Flink
ãƒªãƒ³ã‚¯
Bitbucket
nobusue 2016/01/22
Flink

R
ãƒªãƒ³ã‚¯
flink/flink-contrib/docker-flink at master Â· apache/flink Â· GitHub
nobusue 2016/01/15
Flink

Docker
ãƒªãƒ³ã‚¯
A Comparative Performance Evaluation of Apache Flink
I compare Apache Flink to Apache Spark, Apache Tez, and MapReduce in Apache Hadoop in terms of performance. I run experiments using two benchmarks, Terasort and Hashjoin.Read less
nobusue 2015/10/27
Flink
ãƒªãƒ³ã‚¯
æ³¨ç›®ã™ã¹ã5ã¤ã®ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿é–¢é€£ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆ
Andrew BrustÂ ï¼ˆSpecial to ZDNET.comï¼‰Â ç¿»è¨³æ ¡æ£ï¼šÂ çŸ³æ©‹å•“ä¸€éƒŽ 2015-09-25 06:30 èªè€…ã®ä¸ã«ã¯ã€Apache Software Foundationã®ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆãƒªã‚¹ãƒˆã‚’è¦‹ã¦ã€æ°—ãŒé ããªã£ãŸäººã‚‚ã„ã‚‹ã®ã§ã¯ãªã„ã ã‚ã†ã‹ã€‚ã“ã®ãƒªã‚¹ãƒˆã«ã¯ãƒˆãƒƒãƒ—ãƒ¬ãƒ™ãƒ«ã®é‡è¦ãªãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã‹ã‚‰ã€ç«‹ã¡ä¸ŠãŒã£ãŸã°ã‹ã‚Šã®ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã¾ã§ã€æ•°å¤šãã®ã‚‚ã®ãŒä¸¦ã‚“ã§ã„ã‚‹ãŒã€ã™ã¹ã¦ã‚’æŠŠæ¡ã™ã‚‹ã«ã¯æ•°ãŒå¤šã™ãŽã‚‹ã€‚ãƒªã‚¹ãƒˆã‚’ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿é–¢é€£ã®ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã«é™ã£ã¦ã•ãˆã€ãã®æ•°ã¯ã‹ãªã‚Šå¤šã„ã€‚ãã®ä¸Šã€Apache Software Foundationã®å‚˜ä¸‹ã«ã¯ãªã„ãŒã€æŠŠæ¡ã—ã¦ãŠãã¹ããƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã‚‚ã‚ã‚‹ã€‚ é€”æ–¹ã«æš®ã‚Œã‚‹èªè€…ã®ãŸã‚ã«ã€ã“ã®è¨˜äº‹ã§ã¯ã€æŠŠæ¡ã—ã¦ãŠãä¾¡å€¤ã®ã‚ã‚‹5ã¤ã®ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã‚’ç´¹ä»‹ã™ã‚‹ã€‚ Apache Flink ã€ŒFlinkã€ã¯ã€è€éšœå®³æ€§ã®ã‚ã‚‹ã€Œexactly onceã€å“è³ªã®ãƒ‡ãƒ¼ã‚¿ã‚¹
nobusue 2015/09/26
Spark

flink

mahout
ãƒªãƒ³ã‚¯
Flink Streaming
Flink Streaming is the real-time data processing framework of Apache Flink. Flink streaming provides high level functional apis in Scala and Java backed by a high performance true-streaming runtime.Read less
nobusue 2015/09/26
Windowå‡¦ç†ã‚’ã‚«ã‚¹ã‚¿ãƒžã‚¤ã‚ºã§ãã‚‹ã®ã¯å¬‰ã—ã„

flink
ãƒªãƒ³ã‚¯
Introducing Flink Streaming
February 9, 2015 - This post is the first of a series of blog posts on Flink Streaming, the recent addition to Apache Flink that makes it possible to analyze continuous data sources in addition to static files. Flink Streaming uses the pipelined Flink engine to process data streams in real time and offers a new API including definition of flexible windows. In this post, we go through an example th
nobusue 2015/09/26
Data Driven Windowã¯ä½¿ãˆã‚‹ã‹ã‚‚

flink
ãƒªãƒ³ã‚¯
GitHub - apache/flink: Apache Flink
A streaming-first runtime that supports both batch processing and data streaming programs Elegant and fluent APIs in Java A runtime that supports very high throughput and low event latency at the same time Support for event time and out-of-order processing in the DataStream API, based on the Dataflow Model Flexible windowing (time, count, sessions, custom triggers) across different time semantics
nobusue 2015/09/26
flink
ãƒªãƒ³ã‚¯
Apache Flink: API, runtime, and project roadmap
The document provides an overview of Apache Flink, an open source stream processing framework. It discusses Flink's programming model using DataSets and transf ormations, real-time stream processing capabilities, windowing functions, iterative processing, and visualization tools. It also provides details on Flink's runtime architecture, including its use of pipelined and staged execution, optimizat
nobusue 2015/09/26
Flink

streaming
ãƒªãƒ³ã‚¯
1