[B! Kafka] HHRã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

HHR id:HHR

Kafkaã«é–¢ã™ã‚‹HHRã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (30)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

æ¬¡ä¸–ä»£ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°/ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ Apache Pulsarç´¹ä»‹ ï½žApache Kafkaã«æ¯”ã¹ã¦ä½•ãŒé€²æ©ã—ãŸã®ã‹ï¼Ÿ - Qiita
Deleted articles cannot be recovered. Draft of this article would be also deleted. Are you sure you want to delete this article? ã¯ã˜ã‚ã« æœ¬ç¨¿ã¯ã€æ¬¡ã®ï¼“éƒ¨æ§‹æˆã‹ã‚‰ãªã‚Šã¾ã™ã€‚ Apache Pulsarã‚’æ‰±ã†å‰ã«ã€ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãƒ³ã‚°åŠã³ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ†ã‚¯ãƒŽãƒã‚¸ãƒ¼ã®æ¦‚è¦³ã‹ã‚‰ã¯ã˜ã‚ã¾ã™ã€‚ æ¬¡ã«ã€Apache Pulsarã®è§£èª¬ã«é€²ã¿ã¾ã™ã€‚ã¾ãšPulsarã®æ´å²ã‚„ç¾çŠ¶ã®æ¦‚è¦³ã«ã¤ã„ã¦è§¦ã‚ŒãŸå¾Œã€ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ãƒ¼ã¨æ©Ÿèƒ½ã®èª¬æ˜Žã‚’è¡Œã„ã¾ã™ã€‚ ã“ã“ã§ã¯ã€Apache Pulsarã®ç‰¹å¾´ã‚’ã‚ˆã‚Šç†è§£ã—ã‚„ã™ã„ã‚ˆã†ã«ã€å…ˆè¡Œã—ã¦ç™»å ´ã—ãŸæŠ€è¡“ã§ã‚ã‚‹ã€Apache Kafkaã¨ã®æ¯”è¼ƒã‚’ç”¨ã„ã¦èª¬æ˜Žã—ã¾ã™ã€‚ æœ€å¾Œã«ã€æœ¬ç¨¿ã§Pulsarã«èˆˆå‘³ã‚’ä¿ãŸã‚ŒãŸæ–¹ãŒã€å®Ÿéš›ã«Pulsarã‚’åˆ©ç”¨ã™ã‚‹ãŸã‚ã®æƒ…å ±ã‚’æä¾›ã—ã¾
HHR 2023/07/29
â€œå®Ÿã¯ã€ã“ã®Splunkã®å†…éƒ¨ã§ã¯ã€PulsarãŒåˆ©ç”¨ã•ã‚Œã¦ã„ã¾ã™ã€‚â€ãƒžï¼Ÿ

Pulsar

Kafka
ãƒªãƒ³ã‚¯
How Real-Time Stream Processing Safely Scales with ksqlDB
HHR 2020/11/20
Kafka
ãƒªãƒ³ã‚¯
Apache Kafka Producer Improvements: Sticky Partitioner
HHR 2020/10/20
v2.4~ producerã®æŒ™å‹•ã€‚At first glance, it might seem like setting linger.msã‹ã‚‰æã‚ã—ã„ã“ã¨ãŒæ›¸ã„ã¦ã‚ã‚‹ã€‚defaultã®linger.ms=0ã§ã‚‚å®Ÿéš›ã¯ãƒãƒƒãƒãƒ³ã‚°ã•ã‚Œã‚‹ã‚‰ã—ã„

Kafka
ãƒªãƒ³ã‚¯
What is the difference in Kafka between a Consumer Group Coordinator and a Consumer Group Leader?
HHR 2020/06/29
Coordinatorã¨Group Leaderã®å½¹å‰²ã€‚Coordinatorã¯Brokerå´ã§Group Leaderã¯Consumerå´ã‹ã‚‰é¸å®šã•ã‚Œã‚‹

Kafka
ãƒªãƒ³ã‚¯
Producer Performance Tuning for Apache Kafka
HHR 2020/05/26
è‰¯è¨˜äº‹ã€‚ã ãŒå¤ã„ã€‚

Kafka
ãƒªãƒ³ã‚¯
Apache Kafkaã®æ¦‚è¦ã¨ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ - Qiita
Deleted articles cannot be recovered. Draft of this article would be also deleted. Are you sure you want to delete this article? åˆç‰ˆ: 2018/9/28 è‘—è€…: ä¼Šè—¤ é›…åš, æ ªå¼ä¼šç¤¾æ—¥ç«‹è£½ä½œæ‰€ ã¯ã˜ã‚ã« ã“ã®æŠ•ç¨¿ã§ã¯ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹2017.Enterpriseã§ç™ºè¡¨ã—ãŸã€Œã‚ã–ã›ï¼Kafkaãƒžã‚¹ã‚¿ãƒ¼ã€€ï½žApache Kafkaã§æœ€é«˜ã®æ€§èƒ½ã‚’å‡ºã™ã«ã¯ï½žã€ã®æ¤œè¨¼æ™‚ã«èª¿æŸ»ã—ãŸå†…å®¹ã‚’ç´¹ä»‹ã—ã¾ã™ï¼ˆå…¨8å›žã®äºˆå®šï¼‰ã€‚æœ¬æŠ•ç¨¿ã®å†…å®¹ã¯2017å¹´6æœˆã«ãƒªãƒªãƒ¼ã‚¹ã•ã‚ŒãŸKafka 0.11.0 æ™‚ç‚¹ã®ã‚‚ã®ã§ã™ã€‚ ç¬¬1å›žç›®ã¨ãªã‚‹ä»Šå›žã¯ã€Apache Kafkaã®æ¦‚è¦ã¨ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ã«ã¤ã„ã¦ç´¹ä»‹ã—ã¾ã™ã€‚ æŠ•ç¨¿ä¸€è¦§ï¼š Apache Kafkaã®æ¦‚è¦ã¨ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ (æœ¬æŠ•ç¨¿)
HHR 2019/10/04
å…¨ï¼˜å›ž

Kafka
ãƒªãƒ³ã‚¯
Apache Kafkaã§ç™ºç”Ÿã—ãŸåŽŸå› ä¸æ˜Žã®ãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹ä½Žä¸‹ã¨ã€ãã‚Œã‚’è§£æ±ºã™ã‚‹ãŸã‚ã«ã‚„ã£ãŸã“ã¨
2019å¹´7æœˆ17æ—¥ã€kafka.apache.jpãŒä¸»å‚¬ã™ã‚‹ã‚¤ãƒ™ãƒ³ãƒˆã€ŒApache Kafka Meetup Japan #7ã€ãŒLINEæ ªå¼ä¼šç¤¾ã«ã¦é–‹å‚¬ã•ã‚Œã¾ã—ãŸã€‚åˆ†æ•£ã‚¹ãƒˆãƒªãƒ¼ãƒŸãƒ³ã‚°ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ ã€ŒApache Kafkaã€ã«é–¢ã™ã‚‹ãƒŠãƒ¬ãƒƒã‚¸ã‚„æœ€æ–°æƒ…å ±ã‚’å…±æœ‰ã™ã‚‹æœ¬ã‚¤ãƒ™ãƒ³ãƒˆã€‚ä»Šå›žã¯4äººã®ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ãŒã€è‡ªèº«ã‚„è‡ªç¤¾ã«ãŠã‘ã‚‹çŸ¥è¦‹ã‚’èªžã‚Šã¾ã—ãŸã€‚ãƒ—ãƒ¬ã‚¼ãƒ³ãƒ†ãƒ¼ã‚·ãƒ§ãƒ³ã€ŒKafka Broker performance degradation by mysterious JVM pauseã€ã«ç™»å£‡ã—ãŸã®ã¯ã€LINEæ ªå¼ä¼šç¤¾ã®æ²³æ‘å‹‡äººæ°ã€‚ã‚ã‚‹æ—¥Kafkaã«èµ·ã“ã£ãŸçªç„¶ã®ãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹ä½Žä¸‹ã¨ãã®åŽŸå› ã«ã¤ã„ã¦ã€è§£æ±ºã¾ã§ã®è»Œè·¡ã‚’èªžã‚Šã¾ã—ãŸã€‚è¬›æ¼”è³‡æ–™ã¯ã“ã¡ã‚‰ Apache Kafkaã®ãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹ä½Žä¸‹ã¨ãã®åŽŸå› æ²³æ‘å‹‡äººæ°ï¼šã‚ˆã‚ã—ããŠé¡˜ã„ã—ã¾ã™ã€‚æœ€åˆã«è‡ªå·±ç´¹ä»‹ã‚’ã—ã¾ã™ã€‚æ²³æ‘å‹‡äººã¨ã„ã„ã¾ã™ã€‚ LINEã§å…¨ç¤¾å‘ã‘ã®
HHR 2019/09/17
Kafka
ãƒªãƒ³ã‚¯
Kafka Broker performance degradation by mysterious JVM pause
"Kafka Broker performance degradation by mysterious JVM pause" Yuto Kawamura (LINE Corporation) Presentation at Apache Kafka Meetup Japan #7 @LINE (â€¦
HHR 2019/07/19
ã¨ã„ã†ã‹javaã®ã‚³ã‚¢ãªè©±

Kafka
ãƒªãƒ³ã‚¯
Apache Kafkaã£ã¦æœ¬å½“ã«å¤§ä¸ˆå¤«ï¼Ÿï½žæ•…éšœæ¤œè¨¼ã®ã‚ªãƒ¼ãƒãƒ¼ãƒ“ãƒ¥ãƒ¼ã¨èˆˆå‘³æ·±ã„æŒ™å‹•ã®ç´¹ä»‹ï½ž
4. Â© 2019 NTT DATA Corporation 4 Who am I? â€¢ Bio â€“ Engineering and researching about the distributed computing, open source software, and so on. â€“ Consulting about IT infrastructure for the data processing and data utilization â€“ Leading technical teams â€¢ Presentations / Publications â€“ Spark Summit, Strata Data Conference, Kafka Summit, DataWorks (Hadoop) Summit, Developer Summit, and so on. â€“ Shoe
HHR 2019/05/29
Kafka
ãƒªãƒ³ã‚¯
â€Œ
â€Œâ€Œ â€Œâ€Œâ€Œâ€Œâ€Œâ€Œ
HHR 2019/04/28
Kafka
ãƒªãƒ³ã‚¯
ï¼»é€Ÿå ±ï¼½Googleã€å¤§æ‰‹ã‚¯ãƒ©ã‚¦ãƒ‰ã«ä¸æº€ã‚’è¡¨æ˜Žã—ã¦ã„ãŸMongoDBã€Redisã‚‰OSSãƒ™ãƒ³ãƒ€ã¨æˆ¦ç•¥çš„ææºã€‚Google Cloudã«OSSãƒ™ãƒ³ãƒ€ã®ãƒžãƒãƒ¼ã‚¸ãƒ‰ã‚µãƒ¼ãƒ“ã‚¹ã‚’çµ±åˆã€‚Google Cloud Next '19
ï¼»é€Ÿå ±ï¼½Googleã€å¤§æ‰‹ã‚¯ãƒ©ã‚¦ãƒ‰ã«ä¸æº€ã‚’è¡¨æ˜Žã—ã¦ã„ãŸMongo DBã€Redisã‚‰OSSãƒ™ãƒ³ãƒ€ã¨æˆ¦ç•¥çš„ææºã€‚Google Cloudã«OSSãƒ™ãƒ³ãƒ€ã®ãƒžãƒãƒ¼ã‚¸ãƒ‰ã‚µãƒ¼ãƒ“ã‚¹ã‚’çµ±åˆã€‚Google Cloud Next '19 Googleã¯ã‚µãƒ³ãƒ•ãƒ©ãƒ³ã‚·ã‚¹ã‚³ã§ã‚¤ãƒ™ãƒ³ãƒˆã€ŒGoogle Cloud Next '19ã€ã‚’é–‹å‚¬ã€‚4æœˆ9æ—¥ï¼ˆæ—¥æœ¬æ™‚é–“4æœˆ10æ—¥æœªæ˜Žï¼‰ã«è¡Œã£ãŸã‚ªãƒ¼ãƒ—ãƒ‹ãƒ³ã‚°ã‚ãƒ¼ãƒŽãƒ¼ãƒˆã§ã€Mongo DBã€Redisã€Confluentã€Elasticã‚’ã¯ã˜ã‚ã¨ã™ã‚‹ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ãƒ™ãƒ³ãƒ€ã¨ã®æˆ¦ç•¥çš„ææºã‚’ç™ºè¡¨ã—ã¾ã—ãŸã€‚ ææºç›¸æ‰‹ã¯Confluentã€DataStaxã€Elasticã€InfluxDataã€Mongo DBã€Neo4jã€Redis Labsã®7ç¤¾ã€‚ ã“ã®ææºã®ä¸‹ã§ã€Googleã¯ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ãƒ™ãƒ³ãƒ€ãŒæä¾›ã™ã‚‹ãƒžãƒãƒ¼ã‚¸ãƒ‰ã‚µãƒ¼ãƒ“ã‚¹ã‚’Google Cloudã«çµ±åˆã€‚ãƒ¦ãƒ¼ã‚¶ãƒ¼ã¯G
HHR 2019/04/10
å‡ºã—æŠœã„ãŸæ„Ÿã€‚ä»–ã¯ã©ã†ã™ã‚‹ã®ã‹ãŒæ¥½ã—ã¿

redis

MongoDB

google

Kafka

elasticsearch
ãƒªãƒ³ã‚¯
25K request/secã‚’ã•ã°ã„ãŸã€ŒLINEã®ãŠå¹´çŽ‰ã€ã®ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ã®è£å´
LINEæ ªå¼ä¼šç¤¾ã¯ã€2023å¹´10æœˆ1æ—¥ã«LINEãƒ¤ãƒ•ãƒ¼æ ªå¼ä¼šç¤¾ã«ãªã‚Šã¾ã—ãŸã€‚LINEãƒ¤ãƒ•ãƒ¼æ ªå¼ä¼šç¤¾ã®æ–°ã—ã„ãƒ–ãƒã‚°ã¯ã“ã¡ã‚‰ã§ã™ã€‚ LINEãƒ¤ãƒ•ãƒ¼ Tech Blog ã¿ãªã•ã‚“ã“ã‚“ã«ã¡ã¯ã€LINEã®ã‚µãƒ¼ãƒãƒ¼ã‚µã‚¤ãƒ‰ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã®é•·è°·éƒ¨ã§ã™ã€‚æ™®æ®µã¯ã€æœ€è¿‘ã§ã„ã†ã¨LINE Login ã‚„ LINE Customer Connect ãªã©ã®é–‹ç™ºã‚’æ‹…å½“ã—ã¦ã„ã¾ã™ã€‚ 2018å¹´ã®å¹´å§‹ã« LINEã®ãŠå¹´çŽ‰ ã¨ã„ã†ã‚¤ãƒ™ãƒ³ãƒˆã‚’å®Ÿæ–½ã—ã€ãã®é–‹ç™ºã‚’æ‹…å½“ã—ã¾ã—ãŸã€‚ä»Šå›žã®è¨˜äº‹ã§ã¯ã€LINEã®ãŠå¹´çŽ‰ã®ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ã®ç´¹ä»‹ã‚„ã€å½“æ—¥å®Ÿéš›ã«ç™ºç”Ÿã—ãŸå•é¡Œ(ã‚µãƒ¼ãƒ“ã‚¹éŽè² è·èµ·å› ã®kafka consumeré…å»¶)ãªã©ã®æŒ¯ã‚Šè¿”ã‚Šã«ã¤ã„ã¦æ›¸ã“ã†ã¨æ€ã„ã¾ã™ã€‚ LINEã®ãŠå¹´çŽ‰ã¨ã¯ ãŠå¹´çŽ‰ã‚¤ãƒ™ãƒ³ãƒˆæœŸé–“ä¸ã«ã€ãŠå¹´çŽ‰ã¨LINEã‚¹ã‚¿ãƒ³ãƒ—ã‚’ã‚»ãƒƒãƒˆã§ã€ŒãŠå¹´çŽ‰ã¤ãã‚¹ã‚¿ãƒ³ãƒ—ã€ã¨ã—ã¦è²©å£²ã—ã¾ã—ãŸã€‚å¯¾è±¡ã‚¹ã‚¿ãƒ³ãƒ—ã‚’è³¼å…¥ã—ãŸãƒ¦ãƒ¼ã‚¶ãƒ¼ã•ã‚“ã¯ã€ã‚¹ã‚¿ãƒ³ãƒ—è³¼å…¥æ•° x 1
HHR 2019/01/02
LINE

Kafka

prometheus
ãƒªãƒ³ã‚¯
LINEã®å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ãƒ‘ã‚¤ãƒ—ãƒ©ã‚¤ãƒ³ã‚’æ”¯ãˆã‚‹ã€Apache Kafkaãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ ã®é‹ç”¨ã®è£å´ - ãƒã‚°ãƒŸãƒ¼Tech
LINEã®å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ãƒ‘ã‚¤ãƒ—ãƒ©ã‚¤ãƒ³ã‚’æ”¯ãˆã‚‹ã€Apache Kafkaãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼ãƒ ã®é‹ç”¨ã®è£å´ Multi-Tenancy Kafka cluster for LINE services with 250 billion daily messages 2018å¹´11æœˆ21æ—¥ã€LINEæ ªå¼ä¼šç¤¾ãŒä¸»å‚¬ã™ã‚‹ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢å‘ã‘æŠ€è¡“ã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ã€ŒLINE DEVELOPER DAY 2018ã€ãŒé–‹å‚¬ã•ã‚Œã¾ã—ãŸã€‚4åº¦ç›®ã®é–‹å‚¬ã¨ãªã‚‹ä»Šå›žã®ãƒ†ãƒ¼ãƒžã¯ã€ŒNext LINEã€ã€‚ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã‚¢ãƒ—ãƒªã ã‘ã§ãªãã€ã•ã¾ã–ã¾ãªã‚µãƒ¼ãƒ“ã‚¹ã®é–‹ç™ºã‚„æ–°ãŸãªæŠ€è¡“é ˜åŸŸã¸ã®æŠ•è³‡ã‚’è¡Œã£ã¦ã„ã‚‹LINEãŒç›®æŒ‡ã™ãƒ“ã‚¸ãƒ§ãƒ³ã«ã¤ã„ã¦ã€ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ãŸã¡ã®æŠ€è¡“çš„çŸ¥è¦‹ã‚„æŒ‘æˆ¦ã‚’é€šã—ã¦ç´¹ä»‹ã—ã¾ã™ã€‚ã‚»ãƒƒã‚·ãƒ§ãƒ³ã€ŒMulti-Tenancy Kafka cluster for LINE services with 250 billion daily messages
HHR 2018/12/06
Kafka

LINE
ãƒªãƒ³ã‚¯
Upgrading Apache Kafka Clients Just Got Easier
This is a very exciting time to be part of the Apache KafkaÂ® community! Every four months, a new Apache Kafka release brings additional features and improvements. Weâ€™re particularly excited about the new Streams and Connect features, and exactly-once support. But to get the new stuff, you need to upgrade, and that used to be harder than it is today. In this post, Iâ€™ll talk a little bit about the n
HHR 2018/12/02
Kafka
ãƒªãƒ³ã‚¯
Apache Kafka â€•å…¥é–€ã‹ã‚‰Trifectaã‚’ç”¨ã„ãŸå¯è¦–åŒ–ã¾ã§â€• - Qiita
éŽåŽ»ã«ã‚¤ãƒ³ã‚¿ãƒ¼ãƒ³å…ˆã§ Apache Kafka ã¨ã„ã† OSS ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã«è§¦ã‚Œã‚‹æ©Ÿä¼šãŒã‚ã‚Šã¾ã—ãŸã€‚ä»Šå›žã¯ãã®æ™‚ã«ç†è§£ã—ãŸã“ã¨ã‚’ã¾ã¨ã‚ã‚ˆã†ã¨æ€ã„ã¾ã™ã€‚ èªè€…å¯¾è±¡ Apache Kafka ã«è§¦ã‚ŒãŸã“ã¨ãŒãªã„æ–¹ãŒå¯¾è±¡ã§ã™ã€‚ä½•ã‚’ã™ã‚‹ã‚‚ã®ã§ã€ã©ã†ä½¿ã†ã®ã‹ã€ã¨ã„ã†ã“ã¨ã«é–¢ã—ã¦æ›¸ãã¾ã™ã€‚å†…éƒ¨ã§ã©ã®ã‚ˆã†ã«ãƒ‡ãƒ¼ã‚¿ãŒç®¡ç†ã•ã‚Œã¦ã„ã‚‹ã‹ã€ä»–ã® OSS ã¨ã®æ¯”è¼ƒãªã©ã«ã¤ã„ã¦ã¯è¨˜è¿°ã—ã¾ã›ã‚“ã€‚ã™ã§ã«æ¦‚è¦ã‚’çŸ¥ã£ã¦ã„ã¦ã€ãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹ã®ãƒãƒ¥ãƒ¼ãƒ‹ãƒ³ã‚°ã€ç´°ã‹ã„æŒ™å‹•ã®è©³ç´°ãªã©ã‚’å¿…è¦ã¨ã—ã¦ã„ã‚‹æ–¹ã«ã¯ä¸è¦ãªå†…å®¹ã‹ã‚‚ã—ã‚Œã¾ã›ã‚“ã€‚ãã®ã‚ˆã†ãªå ´åˆã¯ã€è„³å†…ã®ã‚ãƒ£ãƒƒã‚·ãƒ¥ã‚’åˆ©ç”¨ã—ãªãŒã‚‰é©å®œèªã¿é£›ã°ã—ã¦ã„ãŸã ãã“ã¨ã§ãƒ‘ãƒ•ã‚©ãƒ¼ãƒžãƒ³ã‚¹ã®ä½Žä¸‹ã‚’é¿ã‘ã¦ã„ãŸã ã‘ã‚Œã°ã¨æ€ã„ã¾ã™ã€‚ å†…å®¹ Kafka ã®ç´¹ä»‹ èµ·å‹•ãƒ»è¨å®šæ–¹æ³• ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã®é€å—ä¿¡ ( Python ã‚¯ãƒ©ã‚¤ã‚¢ãƒ³ãƒˆåˆ©ç”¨ ) trifecta ã‚’ç”¨ã„ã¦ UI ã‹ã‚‰ç¢ºèª (è©³ç´°ã«é–¢ã—ã¦ã¯çœç•¥ã‹ã€ã‚¤ãƒ³
HHR 2018/12/01
Kafka
ãƒªãƒ³ã‚¯
Multitenancy: Kafka clusters for everyone at LINE
Yuto Kawamura from LINE Corporation presented on their use of Apache Kafka clusters to provide multitenancy for different internal teams. They face challenges in ensuring isolation between client workloads and preventing abusive clients. Their solutions include request quotas to limit client resource usage, slow logs to identify slow requests, and changes to the broker code to pre-warm caches and
HHR 2018/11/10
quota

Kafka

LINE
ãƒªãƒ³ã‚¯
https://www.ospn.jp/osc2017.enterprise/pdf/OSC2017.enterprise_Hitachi_Kafka.pdf
HHR 2018/06/22
å„è¨å®šå€¤ã‚’çµµçš„ã«è¡¨ç¾

Kafka

performance
ãƒªãƒ³ã‚¯
How to Choose the Number of Topics/Partitions in a Kafka Cluster? | Confluent
Start learning Apache Kafka or Apache Flink with our free courses on Confluent Developer! Our course catalog has options for developers, architects, and operators at all learning levels. This is a common question asked by many Apache Kafka users. The goal of this post is to explain a few important determining factors and provide a few simple formulas for when you are self-managing your Kafka clust
HHR 2018/05/25
â€œMore Partitions May Increase Unavailabilityâ€ã«è‰²ã€…ã¨è©°ã¾ã£ã¦ã„ã‚‹ã€‚ãŸã ã—ã€v0.8.x

Kafka
ãƒªãƒ³ã‚¯
25K request/secã‚’ã•ã°ã„ãŸã€ŒLINEã®ãŠå¹´çŽ‰ã€ã®ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ã®è£å´ - LINE ENGINEERING
ã¿ãªã•ã‚“ã“ã‚“ã«ã¡ã¯ã€LINEã®ã‚µãƒ¼ãƒãƒ¼ã‚µã‚¤ãƒ‰ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã®é•·è°·éƒ¨ã§ã™ã€‚æ™®æ®µã¯ã€æœ€è¿‘ã§ã„ã†ã¨LINE Login ã‚„ LINE Customer Connect ãªã©ã®é–‹ç™ºã‚’æ‹…å½“ã—ã¦ã„ã¾ã™ã€‚ 2018å¹´ã®å¹´å§‹ã« LINEã®ãŠå¹´çŽ‰ ã¨ã„ã†ã‚¤ãƒ™ãƒ³ãƒˆã‚’å®Ÿæ–½ã—ã€ãã®é–‹ç™ºã‚’æ‹…å½“ã—ã¾ã—ãŸã€‚ä»Šå›žã®è¨˜äº‹ã§ã¯ã€LINEã®ãŠå¹´çŽ‰ã®ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ã®ç´¹ä»‹ã‚„ã€å½“æ—¥å®Ÿéš›ã«ç™ºç”Ÿã—ãŸå•é¡Œ(ã‚µãƒ¼ãƒ“ã‚¹éŽè² è·èµ·å› ã®kafka consumeré…å»¶)ãªã©ã®æŒ¯ã‚Šè¿”ã‚Šã«ã¤ã„ã¦æ›¸ã“ã†ã¨æ€ã„ã¾ã™ã€‚ LINEã®ãŠå¹´çŽ‰ã¨ã¯ ãŠå¹´çŽ‰ã‚¤ãƒ™ãƒ³ãƒˆæœŸé–“ä¸ã«ã€ãŠå¹´çŽ‰ã¨LINEã‚¹ã‚¿ãƒ³ãƒ—ã‚’ã‚»ãƒƒãƒˆã§ã€ŒãŠå¹´çŽ‰ã¤ãã‚¹ã‚¿ãƒ³ãƒ—ã€ã¨ã—ã¦è²©å£²ã—ã¾ã—ãŸã€‚å¯¾è±¡ã‚¹ã‚¿ãƒ³ãƒ—ã‚’è³¼å…¥ã—ãŸãƒ¦ãƒ¼ã‚¶ãƒ¼ã•ã‚“ã¯ã€ã‚¹ã‚¿ãƒ³ãƒ—è³¼å…¥æ•° x 10å€‹ ã®ãŠå¹´çŽ‰ãŒä»˜ä¸Žã•ã‚Œã¾ã™ã€‚ ã“ã†ã„ã£ãŸãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ãŒå—ä¿¡ã•ã‚Œã¾ã™ ãƒ¦ãƒ¼ã‚¶ãƒ¼ã¯è‡ªåˆ†ãŒã‚‚ã£ã¦ã„ã‚‹ãŠå¹´çŽ‰ã‚’å‹ã ã¡ã«ç›´æŽ¥é€ã£ãŸã‚Šã‚°ãƒ«ãƒ¼ãƒ—ã«é€ä¿¡ã™ã‚‹ã“ã¨ãŒã§ãã€ãŠå¹´çŽ‰ã‚’å—ã‘å–
HHR 2018/02/13
LINE

Kafka
ãƒªãƒ³ã‚¯
Monitoring Kafka w/ Prometheus
This document discusses monitoring Apache Kafka clusters and applications with Prometheus. It provides an overview of the architecture used, including deploying Prometheus servers, Kafka and HBase exporters, and a JSON exporter for YARN applications. Specific exporters are discussed for Kafka brokers using JMX, Kafka clients using the Prometheus Java library, and exposing application metrics via H
HHR 2018/02/05
Kafka

LINE

prometheus
ãƒªãƒ³ã‚¯
1 2 æ¬¡ã®ãƒšãƒ¼ã‚¸