ストリーム処ç†ã‚’支ãˆã‚‹ã‚ューイングシステムã®é¸ã³æ–¹
This document discusses messaging queues and platforms. It begins with an introduction to messaging queues and their core components. It then provides a table comparing 8 popular open source messaging platforms: Apache Kafka, ActiveMQ, RabbitMQ, NATS, NSQ, Redis, ZeroMQ, and Nanomsg. The document discusses using Apache Kafka for streaming and integration with Google Pub/Sub, Dataflow, and BigQuery
Apache Kafkaã«å…¥é–€ã—ãŸ
Apache kafka 最近仕事ã§Apache Kafkaã®å°Žå…¥ã‚’進ã‚ã¦ã„る．Kafkaã¨ã¯ä½•ã‹? ã©ã“ã§ä½¿ã‚ã‚Œã¦ã„ã‚‹ã®ã‹? ã©ã®ã‚ˆã†ãªç†ç”±ã§ä½œã‚‰ã‚ŒãŸã®ã‹? ã©ã®ã‚ˆã†ã«å‹•ä½œã™ã‚‹ã®ã‹ï¼ˆç‰¹ã«ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã®èªã¿å‡ºã—ã«ã¤ã„ã¦ï¼‰? ã‚’ç°¡å˜ã«ã¾ã¨ã‚ã¦ãŠã（メッセージングã¯ã¾ã ã¾ã 勉強ä¸ãªã®ã§ãŠã‹ã—ãªã¨ã“ã‚ãŒã‚ã‚Œã°ãƒ„ッコミをã„ãŸã ã‘ã‚Œã°å¹¸ã„ã§ã™ï¼‰ï¼Ž ãƒãƒ¼ã‚¸ãƒ§ãƒ³ã¯ 0.8.2 を対象ã«æ›¸ã„ã¦ã„る． Apache Kafkaã¨ã¯? 2011å¹´ã«LinkedInã‹ã‚‰å…¬é–‹ã•ã‚ŒãŸã‚ªãƒ¼ãƒ—ンソースã®åˆ†æ•£ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ã‚·ã‚¹ãƒ†ãƒ ã§ã‚る．Kafkaã¯ã‚¦ã‚§ãƒ–サービスãªã©ã‹ã‚‰ç™ºã›ã‚‰ã‚Œã‚‹å¤§å®¹é‡ã®ãƒ‡ãƒ¼ã‚¿ï¼ˆe.g., ãƒã‚°ã‚„イベント）を高スループット/低レイテンシã«åŽé›†/é…ä¿¡ã™ã‚‹ã“ã¨ã‚’目的ã«é–‹ç™ºã•ã‚Œã¦ã„る．公å¼ã®ãƒˆãƒƒãƒ—ページã«æŽ²è¼‰ã•ã‚Œã¦ã„るセールスãƒã‚¤ãƒ³ãƒˆã¯ä»¥ä¸‹ã®4ã¤ï¼Ž Fast ã¨ã«ã‹ã大é‡ã®ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã‚’扱ã†ã“ã¨ãŒã§ãã‚‹ Scal
マイクãƒã‚µãƒ¼ãƒ“スã¨ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ã®ãªãœ [概è¦ç·¨] - 赤帽エンジニアブãƒã‚°
レッドãƒãƒƒãƒˆã§ã‚¤ãƒ³ãƒ†ã‚°ãƒ¬ãƒ¼ã‚·ãƒ§ãƒ³ã®ãŸã‚ã®ãƒŸãƒ‰ãƒ«ã‚¦ã‚§ã‚¢ã®ãƒ†ã‚¯ãƒ‹ã‚«ãƒ«ã‚µãƒãƒ¼ãƒˆã‚’担当ã—ã¦ã„る山下ã§ã™ã€‚ 最近ã¯ãƒžã‚¤ã‚¯ãƒã‚µãƒ¼ãƒ“スã§ã‚·ã‚¹ãƒ†ãƒ を開発ã—ã¦ã„ã‚‹ã¨ã„ã†è©±ã‚‚よãèžãよã†ã«ãªã£ã¦ãã¾ã—ãŸã€‚ã§ã¯ãã“ã§ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ã€ãã—ã¦Kafkaを使ã£ã¦ã¾ã™ã§ã—ょã†ã‹ï¼Ÿãƒžã‚¤ã‚¯ãƒã‚µãƒ¼ãƒ“スã§ã¯ä½•æ•…ã‹RESTã°ã‹ã‚ŠãŒä¸–ã®ä¸ã«æ³¨ç›®ã•ã‚Œã¦ã—ã¾ã†ã“ã¨ã‚‚多ã„ãŸã‚ã«ã€ä»Šå›žã¯ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°æŽ¨ã—ã®å†…容ã«ã—ã¦ã„ã¾ã™ã€‚ マイクãƒã‚µãƒ¼ãƒ“スã§ã¯ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ã‚’用ã„ãŸã‚³ãƒžãƒ³ãƒ‰ã‚„イベントã“ãä¸å¿ƒã§ã‚ã£ã¦ä¸å¯æ¬ ã§ã™ã€‚マイクãƒã‚µãƒ¼ãƒ“スã®ä¸ã§ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ã¯ã©ã®ã‚ˆã†ã«åˆ©ç”¨ã•ã‚Œã€ãã—ã¦ãªãœå¿…è¦ãªã®ã§ã—ょã†ã‹ã€‚今回ã¯ã€Œãƒžã‚¤ã‚¯ãƒã‚µãƒ¼ãƒ“スã¨ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ã®ãªãœ [概è¦ç·¨]ã€ã¨é¡Œã—ã¦ãれを概観ã—ã¦ã„ãã¾ã™ã€‚ Kafkaã®ç°¡å˜ãªãŠã•ã‚‰ã„ ã©ã“ã§ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ã¯åˆ©ç”¨ã•ã‚Œã‚‹ã®ã‹ï¼Ÿ RESTã¯ãŠæ‰‹è»½ãªè§£æ±ºç–？ ãªãœãƒžã‚¤ã‚¯ãƒã‚µãƒ¼ãƒ“スã«ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°(Kafka)ãŒå¿…
![マイクãƒã‚µãƒ¼ãƒ“スã¨ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ã®ãªãœ [概è¦ç·¨] - 赤帽エンジニアブãƒã‚°](https://cdn-ak-scissors.b.st-hatena.com/image/square/97795bc6ad2035868537aa276b25751b65d1c2b4/height=288;version=1;width=512/https%3A%2F%2Fcdn.image.st-hatena.com%2Fimage%2Fscale%2F25e249850cd064a8463f9be827f3f9180f55b897%2Fbackend%3Dimagemagick%3Bheight%3D1300%3Bversion%3D1%3Bwidth%3D1300%2Fhttps%253A%252F%252Fcdn-ak.f.st-hatena.com%252Fimages%252Ffotolife%252Ft%252Ftyamashi-oss%252F20190724%252F20190724112428.jpg)
分散型システム徹底入門 – Part 2. | POSTD
Cassandra å…ˆã»ã©è§¦ã‚ŒãŸCassandraã¯åˆ†æ•£åž‹ã®NoSQLデータベースã§ã€CAP定ç†ã®Aã¨P（å¯ç”¨æ€§ã¨åˆ†æ–è€æ€§ï¼‰ã®ç‰¹æ€§ã‚’基準ã«æœ€çµ‚çš„ãªä¸€è²«æ€§ãŒç¢ºä¿ã•ã‚Œã¦ã„ã¾ã™ã€‚ãŸã ã€ã“ã®ã‚ˆã†ã«è¨€ã£ã¦ã—ã¾ã†ã¨å°‘ã—誤解を招ãã‹ã‚‚ã—ã‚Œã¾ã›ã‚“。ã¨ã„ã†ã®ã‚‚ã€å®Ÿéš›ã®ã¨ã“ã‚Cassandraã®è¨å®šã¯éžå¸¸ã«æŸ”軟性ãŒé«˜ãã€å¯ç”¨æ€§ã‚’çŠ ç‰²ã«ã—ã¦å¼·ã„一貫性をæä¾›ã™ã‚‹ã“ã¨ã‚‚ã§ãã‚‹ã‹ã‚‰ã§ã™ã€‚ã§ã™ãŒã€ãã†ã—ãŸä½¿ç”¨ã‚±ãƒ¼ã‚¹ã¯ä¸€èˆ¬çš„ã§ã¯ã‚ã‚Šã¾ã›ã‚“。 Cassandraã§ã¯ã€ コンシステントãƒãƒƒã‚·ãƒ¥æ³• を使ã£ã¦ã€æ¸¡ãã†ã¨ã™ã‚‹ãƒ‡ãƒ¼ã‚¿ã‚’クラスタã®ã©ã®ãƒŽãƒ¼ãƒ‰ãŒç®¡ç†ã™ã‚‹ã®ã‹ã‚’決ã‚ã¦ã„ã¾ã™ã€‚ãã—ã¦ãã®éš›ã¯ã€ãƒ‡ãƒ¼ã‚¿ã‚’複製ã™ã‚‹ãƒŽãƒ¼ãƒ‰æ•°ã‚’示㙠レプリケーションファクタ ã‚’è¨å®šã—ã¾ã™ã€‚ 注釈： レプリケーションファクタï¼3 挿入（ã‚ーã€å€¤ï¼‰ Cassandraã®ãƒŽãƒ¼ãƒ‰ï¼ˆã‚³ãƒ¼ãƒ‡ã‚£ãƒãƒ¼ã‚¿ï¼‰ Cassandraã®ãƒŽãƒ¼ãƒ‰ ãƒãƒƒã‚·ãƒ¥ï¼ˆã‚ー）ï¼2 ノード#2
Kappa Architecture - Where Every Thing Is A Stream
kappa-architecture.com Repository dedicated to Kappa Architecture. I collect and publish articles, tutorials, talks, projects and examples related to Kappa Architecture. What is Kappa Architecture? Kappa Architecture is a software architecture pattern. Rather than using a relational DB like SQL or a key-value store like Cassandra, the canonical data store in a Kappa Architecture system is an appen
25K request/secã‚’ã•ã°ã„ãŸã€ŒLINEã®ãŠå¹´çŽ‰ã€ã®ã‚¢ãƒ¼ã‚テクãƒãƒ£ã®è£å´ - LINE ENGINEERING
ã¿ãªã•ã‚“ã“ã‚“ã«ã¡ã¯ã€LINEã®ã‚µãƒ¼ãƒãƒ¼ã‚µã‚¤ãƒ‰ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã®é•·è°·éƒ¨ã§ã™ã€‚普段ã¯ã€æœ€è¿‘ã§ã„ã†ã¨LINE Login ã‚„ LINE Customer Connect ãªã©ã®é–‹ç™ºã‚’担当ã—ã¦ã„ã¾ã™ã€‚ 2018å¹´ã®å¹´å§‹ã« LINEã®ãŠå¹´çŽ‰ ã¨ã„ã†ã‚¤ãƒ™ãƒ³ãƒˆã‚’実施ã—ã€ãã®é–‹ç™ºã‚’担当ã—ã¾ã—ãŸã€‚今回ã®è¨˜äº‹ã§ã¯ã€LINEã®ãŠå¹´çŽ‰ã®ã‚¢ãƒ¼ã‚テクãƒãƒ£ã®ç´¹ä»‹ã‚„ã€å½“日実際ã«ç™ºç”Ÿã—ãŸå•é¡Œ(サービスéŽè² è·èµ·å› ã®kafka consumeré…延)ãªã©ã®æŒ¯ã‚Šè¿”ã‚Šã«ã¤ã„ã¦æ›¸ã“ã†ã¨æ€ã„ã¾ã™ã€‚ LINEã®ãŠå¹´çŽ‰ã¨ã¯ ãŠå¹´çŽ‰ã‚¤ãƒ™ãƒ³ãƒˆæœŸé–“ä¸ã«ã€ãŠå¹´çŽ‰ã¨LINEスタンプをセットã§ã€ŒãŠå¹´çŽ‰ã¤ãスタンプã€ã¨ã—ã¦è²©å£²ã—ã¾ã—ãŸã€‚対象スタンプを購入ã—ãŸãƒ¦ãƒ¼ã‚¶ãƒ¼ã•ã‚“ã¯ã€ã‚¹ã‚¿ãƒ³ãƒ—購入数 x 10個 ã®ãŠå¹´çŽ‰ãŒä»˜ä¸Žã•ã‚Œã¾ã™ã€‚ ã“ã†ã„ã£ãŸãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãŒå—ä¿¡ã•ã‚Œã¾ã™ ユーザーã¯è‡ªåˆ†ãŒã‚‚ã£ã¦ã„ã‚‹ãŠå¹´çŽ‰ã‚’å‹ã ã¡ã«ç›´æŽ¥é€ã£ãŸã‚Šã‚°ãƒ«ãƒ¼ãƒ—ã«é€ä¿¡ã™ã‚‹ã“ã¨ãŒã§ãã€ãŠå¹´çŽ‰ã‚’å—ã‘å–
Apache Kafkaを使ã£ãŸã‚¢ãƒ—リè¨è¨ˆã§åçœã—ã¦ã„る件をæ£ç›´ãƒ™ãƒ¼ã‚¹ã§è©±ã™
Apache Kafka: Producer, Broker and Consumer2017å¹´ã¯ç”Ÿã¾ã‚Œã¦å§‹ã‚ã¦Apache Kafkaã‚’æœ¬æ ¼çš„ã«æ¥å‹™åˆ©ç”¨ï¼ˆPoCã§ã¯ãªã本番é‹ç”¨ï¼‰ã—ãŸå¹´ã§ã—ãŸã€‚Apache Kafkaçš„ãªãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°ãƒŸãƒ‰ãƒ«ã‚¦ã‚§ã‚¢ãã®ã‚‚ã®ã®ã¯ã€ç¤¾å†…çš„ãªäº‹æƒ…ã§ã‚ˆã使ã£ã¦ã„ãŸã®ã§ãã®ä½¿ã„å‹æ‰‹ã«å¯¾ã—ã¦å›°æƒ‘ã¯ã»ã¨ã‚“ã©ç„¡ã‹ã£ãŸã§ã™ã—ã€ãƒŸãƒ‰ãƒ«ã‚¦ã‚§ã‚¢ã¨ã—ã¦éžå¸¸ã«å®‰å®šã—ã¦ã„ã‚‹ãŸã‚ã€Kafkaクラスタãã®ã‚‚ã®ã§ã®ä¸å…·åˆã‚‰ã—ã„ä¸å…·åˆãŒç™ºç”Ÿã—ãŸã“ã¨ã¯ä¸€åº¦ã‚‚ã‚ã‚Šã¾ã›ã‚“ã§ã—ãŸã€‚ ã—ã‹ã—ã€Kafkaã®ãƒˆãƒ”ックè¨è¨ˆãªã©ã«é–¢ã—ã¦ã®ãƒ™ã‚¹ãƒˆãƒ—ラクティスã¯äº‹ä¾‹ãƒ™ãƒ¼ã‚¹ã§ã‚‚ã‚ã¾ã‚Šè¦‹ã‹ã‘ãŸã“ã¨ãŒãªãã€ãƒãƒ¼ãƒ メンãƒãƒ¼ã¨æ‚©ã‚€ã“ã¨ã‚‚多ã‹ã£ãŸã§ã™ã€‚ã“ã®ã‚¹ãƒˆãƒ¼ãƒªãƒ¼ã§ã¯ã€ä¸»ã«Kafkaを利用ã—ãŸã‚¢ãƒ—リè¨è¨ˆã§è€ƒãˆãŸã“ã¨ã‚„失敗ã—ãŸã“ã¨ã‚’振り返りã¤ã¤å…±æœ‰ã—ã¾ã™ã€‚ãªãŠã€ãƒ‘ーティション数やå„種ãƒãƒƒãƒ•ã‚¡ã‚µã‚¤ã‚ºãªã©ã®ãƒãƒ¥ãƒ¼ãƒ‹ãƒ³ã‚°è¦ç´ ã¯ä»Šå›žå–
1
ランã‚ング
ランã‚ング
ランã‚ング
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
lambda-architecture.net
{{#tags}}- {{label}}
{{/tags}}