[B! kafka] teddy-gã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

teddy-g id:teddy-g

kafkaã«é–¢ã™ã‚‹teddy-gã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (5)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Apache Kafka & Google Cloud Pub/Sub ä¸»è¦æ©Ÿèƒ½ã®æ¯”è¼ƒ
åŸ·ç†è€…: ç§‹è‘‰åŽŸãƒ©ãƒœ Â sitotkfm&rshtykh ã¯ã˜ã‚ã« ç§‹è‘‰åŽŸãƒ©ãƒœã§ã¯å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿å‡¦ç†åŸºç›¤ã®ç ”ç©¶ãƒ»é–‹ç™ºãƒ»é‹ç”¨ã‚’è¡Œã£ã¦ãŠã‚Šã€ãƒ‡ãƒ¼ã‚¿ã®è»¢é€ã‚„è§£æžã‚·ã‚¹ãƒ†ãƒ ã®Hubã¨ã—ã¦Pubsubã‚·ã‚¹ãƒ†ãƒ ã®é‡è¦æ€§ãŒé«˜ã¾ã£ã¦ã„ã‚‹ã€‚ Pubsubãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ãƒ¢ãƒ‡ãƒ«ã¯ã€éžåŒæœŸãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ãƒ‘ãƒ©ãƒ€ã‚¤ãƒ ã®ä¸€ç¨®ã§ã‚ã‚Šã€ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã®é€ä¿¡è€…ï¼ˆå‡ºç‰ˆå´ï¼‰ãŒç‰¹å®šã®å—ä¿¡è€…ï¼ˆè³¼èªå´ï¼‰ã‚’æƒ³å®šã›ãšã«ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã‚’é€ã‚‹ã‚ˆã†ãƒ—ãƒã‚°ãƒ©ãƒ ã•ã‚ŒãŸã‚‚ã®ã§ã‚ã‚‹ï¼ˆwikipediaï¼‰ã€‚Pubsubãƒ¢ãƒ‡ãƒ«ã‚’ç”¨ã„ã‚‹ã“ã¨ã§ã‚·ã‚¹ãƒ†ãƒ ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ã«ãŠã„ã¦ã‚³ãƒ³ãƒãƒ¼ãƒãƒ³ãƒˆåŒå£«ã‚’ç–Žçµåˆã«ã—ã€é«˜ã‚¹ã‚±ãƒ¼ãƒ©ãƒ“ãƒªãƒ†ã‚£ã‚’å®Ÿç¾ã™ã‚‹ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ãƒ‘ã‚¿ãƒ¼ãƒ³ã§ã‚ã‚‹ã€‚ã“ã®ãƒ¢ãƒ‡ãƒ«ã«åŸºã¥ã„ãŸã‚½ãƒªãƒ¥ãƒ¼ã‚·ãƒ§ãƒ³ã¯æ—¢ã«è¤‡æ•°å˜åœ¨ã™ã‚‹ãŒã€ä»Šå›žã¯ç‰¹ã«æ³¨ç›®ã•ã‚Œã¦ã„ã‚‹Apache Kafkaã‚„Google Cloud Pub/Subã‚’ç´¹ä»‹ã—ã€ã‚·ã‚¹ãƒ†ãƒ é¢ã§é‡è¦ã¨æ€ã‚ã‚Œã‚‹æ©Ÿèƒ½ã‚’æ¯”è¼ƒã™ã‚‹ã€‚ Kafkaã¯ã€
teddy-g 2018/02/18
Apache Kafkaã¯Pubsub

apache

kafka
ãƒªãƒ³ã‚¯
åˆ†æ•£ã‚¹ãƒˆãƒªãƒ¼ãƒ å‡¦ç†ã‚¨ãƒ³ã‚¸ãƒ³ã‚ã‚Œã“ã‚Œ - Qiita
Deleted articles cannot be recovered. Draft of this article would be also deleted. Are you sure you want to delete this article? #åˆ†æ•£ã‚¹ãƒˆãƒªãƒ¼ãƒ å‡¦ç†ã‚¨ãƒ³ã‚¸ãƒ³ã®ç¾¤é›„å‰²æ‹ ã®æ™‚ä»£ ã‚¹ãƒˆãƒªãƒ¼ãƒ å‡¦ç†ã‚’å®Ÿç¾ã™ã‚‹åˆ†æ•£ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ ãŒã€å¤§åˆ†å¢—ãˆã¾ã—ãŸã€‚ ä½•ãŒè‰¯ã„ã‹ï¼æ‚ªã„ã‹ã¯ã€ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ ã«æ±‚ã‚ã‚‹å†…å®¹ã‚„ã€é–¢é€£ã™ã‚‹ã‚¨ã‚³ã‚·ã‚¹ãƒ†ãƒ ã«ã‚‚å½±éŸ¿ã™ã‚‹ãŸã‚ã€ä¸€æ¦‚ã«ã¯è¨€ãˆãªã„ã§ã™ãŒã€Apacheã§æä¾›ã•ã‚Œã¦ã„ã‚‹OSSã¨ã—ã¦ã€æƒ…å ±ã‚’ã¾ã¨ã‚ãŸã‚‚ã®ãŒã‚ã£ãŸã®ã§ã€ãƒã‚¤ãƒ³ãƒˆã‚’ã¾ã¨ã‚ãŸã„ã¨æ€ã„ã¾ã™ã€‚ AN OVERVIEW OF APACHE STREAMING TECHNO LOGIES https://database line.wordpress.com/2016/03/12/an-overview-
teddy-g 2016/11/14
è‹±èªžç‰ˆã®è¨˜äº‹ã®æ¦‚è¦ã¨ç†è€…ã®å€‹äººçš„è¦‹è§£ãŒæ›¸ã„ã¦ã‚ã‚‹ã€‚æŠ€è¡“è©³ç´°ã¯è‹±èªžç‰ˆè¦‹ãŸæ–¹ãŒè‰¯ã„ã€‚

apache

streaming

realtime

nifi

kafka

spark

storm

flink
ãƒªãƒ³ã‚¯
An Overview of Apache Streaming Technologies | Databaseline
There are many techno logies for streaming data: simple event processors, stream processors, and complex event processors. Even within the open-source community there is a bewildering amount of options with sometimes few major differences that are not well documented or easy to find. Thatâ€™s why Iâ€™ve decided to create an overview of Apache streaming techno logies, including Flume , NiFi , Apex ,Â Spa
teddy-g 2016/11/14
Apacheã®Streamingç³»ã®ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆ/æŠ€è¡“æ¯”è¼ƒä¸€è¦§ã¨èª¬æ˜Žã€‚ã“ã‚Œã«Fluentdã‚’ä½µã›ã¦è¦šãˆã¦ãŠã‘ã°Streamç³»ã¯å¤§ä½“ç¶²ç¾…ã§ãã‚‹ã‚“ã˜ã‚ƒãªã„ã‹ãªã€‚

apache

streaming

realtime

nifi

kafka

spark

storm

flink
ãƒªãƒ³ã‚¯
Apache Kafka ã«ã¤ã„ã¦ | ã‚®ãƒ£ãƒƒãƒ—ãƒ
Kafka ã¨ ZooKeeper ZooKeeper ã®è©³ç´° ZooKeeper ã¯ã€åˆ†æ•£ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã®ãŸã‚ã®ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã®åˆ†æ•£ã‚³ãƒ¼ãƒ‡ã‚£ãƒãƒ¼ã‚·ãƒ§ãƒ³ã‚µãƒ¼ãƒ“ã‚¹ã§ã™ã€‚ZooKeeper ã¯ Java ã§å‹•ä½œã—ã€Java ã¨ C ã®ãŸã‚ã®ãƒã‚¤ãƒ³ãƒ‡ã‚£ãƒ³ã‚°ãŒç”¨æ„ã•ã‚Œã¦ã„ã¾ã™ã€‚ ã‚¯ãƒ©ã‚¹ã‚¿ (ãƒžãƒ«ãƒã‚µãƒ¼ãƒãƒ¼) ã‚»ãƒƒãƒˆã‚¢ãƒƒãƒ— ZooKeeper ã¯å¥‡æ•°å˜ä½ã§ã‚¢ãƒ³ã‚µãƒ³ãƒ–ãƒ«ã¨å‘¼ã¶ã‚¯ãƒ©ã‚¹ã‚¿ã‚’çµ„ã¿ã¾ã™ã€‚ã‚¢ãƒ³ã‚µãƒ³ãƒ–ãƒ«ã®éŽåŠæ•°ãŒå‹•ä½œã—ã¦ã„ã‚‹å¿…è¦ãŒã‚ã‚‹ã®ã§ã€å¥‡æ•°å°ã®ãƒžã‚·ãƒ³ã‚’ä½¿ã†ã®ãŒãƒ™ã‚¹ãƒˆã§ã™ã€‚ãŸã¨ãˆã° 4 å°ã®ãƒžã‚·ãƒ³ã‚’ä½¿ã†å ´åˆã€ZooKeeper ãŒå¯¾å¿œã§ãã‚‹ã®ã¯ãƒžã‚·ãƒ³ 1 å°ã®éšœå®³ã¾ã§ã§ã™ã€‚2 å°ã®ãƒžã‚·ãƒ³ã«éšœå®³ãŒç™ºç”Ÿã™ã‚‹ã¨ã€æ®‹ã‚Šã® 2 å°ã§ã¯éŽåŠæ•°ã«ãªã‚‰ãªã„ãŸã‚ã§ã™ã€‚5 å°ã®ãƒžã‚·ãƒ³ã‚’ä½¿ãˆã°ã€ZooKeeper ã¯ãƒžã‚·ãƒ³ 2 å°ã¾ã§ã®éšœå®³ã«å¯¾å¿œã§ãã¾ã™ã€‚ ãã®ä¸ã® 1 å°ãŒãƒªãƒ¼ãƒ€ãƒ¼é¸æŒ™ã‚’é€šã—ã¦å…¨ã¦ã®æ›¸ãè¾¼ã¿ã‚’èª¿åœ
teddy-g 2016/11/13
ã“ã£ã¡ã¯Kafkaã®Brokerç®¡ç†ã§ä½¿ã‚ã‚Œã¦ã„ã‚‹Zookeeperã®ã“ã¨ã«ã‚‚è§¦ã‚Œã¦ã‚ã‚‹ã€‚

apache

kafka

distributed

architecture

hadoop

realtime

streaming
ãƒªãƒ³ã‚¯
Apache Kafkaã«å…¥é–€ã—ãŸ
Apache kafka æœ€è¿‘ä»•äº‹ã§Apache Kafkaã®å°Žå…¥ã‚’é€²ã‚ã¦ã„ã‚‹ï¼ŽKafkaã¨ã¯ä½•ã‹? ã©ã“ã§ä½¿ã‚ã‚Œã¦ã„ã‚‹ã®ã‹? ã©ã®ã‚ˆã†ãªç†ç”±ã§ä½œã‚‰ã‚ŒãŸã®ã‹? ã©ã®ã‚ˆã†ã«å‹•ä½œã™ã‚‹ã®ã‹ï¼ˆç‰¹ã«ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã®èªã¿å‡ºã—ã«ã¤ã„ã¦ï¼‰? ã‚’ç°¡å˜ã«ã¾ã¨ã‚ã¦ãŠãï¼ˆãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ã¯ã¾ã ã¾ã å‹‰å¼·ä¸ãªã®ã§ãŠã‹ã—ãªã¨ã“ã‚ãŒã‚ã‚Œã°ãƒ„ãƒƒã‚³ãƒŸã‚’ã„ãŸã ã‘ã‚Œã°å¹¸ã„ã§ã™ï¼‰ï¼Ž ãƒãƒ¼ã‚¸ãƒ§ãƒ³ã¯ 0.8.2 ã‚’å¯¾è±¡ã«æ›¸ã„ã¦ã„ã‚‹ï¼Ž Apache Kafkaã¨ã¯? 2011å¹´ã«LinkedInã‹ã‚‰å…¬é–‹ã•ã‚ŒãŸã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã®åˆ†æ•£ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ã‚·ã‚¹ãƒ†ãƒ ã§ã‚ã‚‹ï¼ŽKafkaã¯ã‚¦ã‚§ãƒ–ã‚µãƒ¼ãƒ“ã‚¹ãªã©ã‹ã‚‰ç™ºã›ã‚‰ã‚Œã‚‹å¤§å®¹é‡ã®ãƒ‡ãƒ¼ã‚¿ï¼ˆe.g., ãƒã‚°ã‚„ã‚¤ãƒ™ãƒ³ãƒˆï¼‰ã‚’é«˜ã‚¹ãƒ«ãƒ¼ãƒ—ãƒƒãƒˆ/ä½Žãƒ¬ã‚¤ãƒ†ãƒ³ã‚·ã«åŽé›†/é…ä¿¡ã™ã‚‹ã“ã¨ã‚’ç›®çš„ã«é–‹ç™ºã•ã‚Œã¦ã„ã‚‹ï¼Žå…¬å¼ã®ãƒˆãƒƒãƒ—ãƒšãƒ¼ã‚¸ã«æŽ²è¼‰ã•ã‚Œã¦ã„ã‚‹ã‚»ãƒ¼ãƒ«ã‚¹ãƒã‚¤ãƒ³ãƒˆã¯ä»¥ä¸‹ã®4ã¤ï¼Ž Fast ã¨ã«ã‹ãå¤§é‡ã®ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã‚’æ‰±ã†ã“ã¨ãŒã§ãã‚‹ Scal
teddy-g 2016/11/13
çµå±€ã€ã‚°ã‚°ã£ã¦ãƒˆãƒƒãƒ—ã«ãã‚‹ã“ã®è¨˜äº‹ãŒKafkaã®èª¬æ˜Žã§ä¸€ç•ªåˆ†ã‹ã‚Šã‚„ã™ã‹ã£ãŸã€‚å®Ÿè£…ã—ãªã„èº«ãªã‚‰ã€é«˜é€Ÿãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°å‡¦ç†ç”¨ã®Pub/Subåž‹ã‚·ã‚¹ãƒ†ãƒ ã¨è¦šãˆã¨ããã‚‰ã„ã§ã„ã„ã‹ã—ã‚‰ã€‚

apache

kafka

distributed

architecture

hadoop

realtime

streaming
ãƒªãƒ³ã‚¯
1