yassã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ / 2013å¹´12æœˆ28æ—¥ - ã¯ã¦ãªãƒ

Amazon Kinesisã¨Apache Kafkaã®é¡žä¼¼ç‚¹ï¼ç›¸é•ç‚¹ã¾ã¨ã‚ - å¤¢ã¨ã‚¬ãƒ©ã‚¯ã‚¿ã®é›†ç©å ´

ã“ã‚“ã«ã¡ã¯ã€‚ Amazon Kinesisã«ã¤ã„ã¦èª¿ã¹ãŸã‚Šå®Ÿè£…ã—ã¦ã¿ãŸã‚Šã—ãŸãŸã‚ã€ ãƒ¢ãƒ‡ãƒ«ãŒã‚ˆãä¼¼ãŸApache Kafkaã¨ã®é¡žä¼¼ç‚¹ã‚„ç›¸é•ç‚¹ãŒæ°—ã«ãªã£ã¦ãã¾ã—ãŸã€‚ ã¨ã„ã†ã‚ã‘ã§ã€å®Ÿéš›æ¯”ã¹ã¦ã¿ãŸçµæžœã©ã†ã ã£ãŸã®ã‹ã‚’ã¾ã¨ã‚ã¦ã¿ã¾ã™ã€‚ 1.2ã¤ã®ãƒ—ãƒãƒ€ã‚¯ãƒˆã®é¡žä¼¼ç‚¹ Amazon Kinesisã¨Apache Kafkaã®å¤§ããªé¡žä¼¼ç‚¹ã¨ã—ã¦ã€ä»¥ä¸‹ãŒã‚ã‚Šã¾ã™ã€‚ 1.ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã‚’å–å¾—ã—ãŸã‚¿ã‚¤ãƒŸãƒ³ã‚°ã§å‰Šé™¤ã™ã‚‹ã®ã§ã¯ãªãã€ä¸€å®šæœŸé–“çµŒéŽå¾Œã«å‰Šé™¤ã™ã‚‹ãƒ¢ãƒ‡ãƒ«ã‚’å–ã£ã¦ã„ã‚‹ Kestrelã€RabbitMQã¨ã„ã£ãŸãƒ—ãƒãƒ€ã‚¯ãƒˆã¯ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã‚’å–å¾—ã•ã‚Œã€åˆ©ç”¨å´ã‹ã‚‰å¿œç”ãŒè¿”ã£ãŸæ®µéšŽã§å‰Šé™¤ã™ã‚‹ãƒ¢ãƒ‡ãƒ«ã‚’å–ã£ã¦ã„ã‚‹ã€‚ ã€Œãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã‚ãƒ¥ãƒ¼ã€ã¨è¨€ã„ã‚ã‚‰ã‚ã—ãŸå ´åˆã€å‰Šé™¤ã™ã‚‹ãƒ¢ãƒ‡ãƒ«ã®æ–¹ãŒã‚¤ãƒ¡ãƒ¼ã‚¸çš„ã«ã¯è¿‘ã„ï¼Ÿ Amazon Kinesisã€Apache Kafkaã¯ãƒ¡ãƒƒã‚»ãƒ¼ã‚¸ã®æä¾›ã‚’è¡Œã†ã®ã¿ã§å‰Šé™¤ã¯è¡Œã‚ãªã„ã€‚ä¸€å®šæ™‚é–“çµŒéŽå¾Œã«å‰Šé™¤ã—ã¦ã„ã‚‹

yass 2013/12/28

" IRecordProcessorã‚’ç¶™æ‰¿ã—ãŸã‚¢ãƒ—ãƒªã‚’ä½œæˆã™ã‚‹ã“ã¨ã§ã€DynamoDBä¸Šã«è‡ªå‹•çš„ã«ä¿å˜ã€‚/ IRecordProcessorã‚’åˆ©ç”¨ã™ã‚‹å ´åˆã¯1ã‚¹ãƒ¬ãƒƒãƒ‰ã§1ãƒ‘ãƒ¼ãƒ†ã‚£ã‚·ãƒ§ãƒ³åˆ†ã—ã‹ãƒ‡ãƒ¼ã‚¿ã‚’å–å¾—ã§ããªã„ã€‚"

ãƒªãƒ³ã‚¯

e-trees.Japan, inc. :: KC705ã§PCIeã®é€šä¿¡æ€§èƒ½ã‚’æ¸¬å®šã—ã¦ã¿ãŸ

PCIe gen1 x1ã®è»¢é€é€Ÿåº¦ã¯ç†è«–(è¦æ ¼)ä¸Šã§ã¯2.5GTpsï¼Œ0.5GBpsã§ã™ï¼Ž ã—ã‹ã—ï¼Œå®Ÿéš›ã«ã¯PCIeä¸Šã®ãƒ‡ãƒ¼ã‚¿ã‚’é§†å‹•ã™ã‚‹ã‚¨ãƒ³ã‚¸ãƒ³ã®å‡¦ç†æ™‚é–“ã‚„ï¼Œãƒ‡ãƒ¼ã‚¿ä¾›çµ¦æºã‹ã‚‰ã®èªã¿å‡ºã—ï¼Œæ›¸ãè¾¼ã¿å…ˆã§ã®æ›¸ãå‡ºã—æ™‚é–“ãªã©ï¼Œã„ã‚ã„ã‚ãªã‚ªãƒ¼ãƒãƒ˜ãƒƒãƒ‰ãŒå…¥ã£ã¦ãã¾ã™ï¼Ž ã§ï¼Œå®Ÿéš›ã©ã®ãã‚‰ã„ãªã®ã‚ˆï¼Ÿã¨ã„ã†ã®ã‚’ EDKã§PCIe+DDR3ã®ã‚¢ã‚¯ã‚»ãƒ©ãƒ¬ãƒ¼ã‚¿ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã‚’ä½œã‚‹(1) ã§ä½œã£ãŸã‚·ã‚¹ãƒ†ãƒ ã§è©•ä¾¡ã—ã¦ã¿ã¾ã—ãŸï¼Ž çµæžœ ã§ï¼Œã„ããªã‚Šã§ã™ãŒï¼Œçµæžœã§ã™ï¼Ž æ¸¬å®šã—ãŸã®ã¯ï¼Œ è»¢é€é€Ÿåº¦ã¯æ¬¡ã®4ãƒ‘ã‚¿ãƒ³ï¼Œ Gen1 x1 Gen2 x1 Gen1 x4 Gen2 x4 DMAè»¢é€å˜ä½ã¯æ¬¡ã®7ãƒ‘ã‚¿ãƒ³ã‚’è©¦ã—ã¾ã—ãŸ 1KB 4KB 16KB 64KB 256KB 1MB 4MB Gen2 x4ã®å ´åˆã¯ï¼Œ DMAè»¢é€å˜ä½ãŒ1KBã®ã¨ãã«Hostâ†’DDR3/FPGAã¨DDR3/FPGAâ†’Hostã®ã©ã¡ã‚‰ã‚‚ç´„58MBps DMA

yass 2013/12/28

" Gen2 x4ã®å ´åˆã¯ï¼ŒDMAè»¢é€å˜ä½ãŒ1KBã®ã¨ãã«Hostâ†’DDR3/FPGAã¨DDR3/FPGAâ†’Hostã®ã©ã¡ã‚‰ã‚‚ç´„58MBps / DMAè»¢é€å˜ä½ãŒ4MBã®ã¨ãã«Hostâ†’DDR3/FPGAãŒç´„850MBpsï¼ŒDDR3/FPGAâ†’HostãŒç´„1035MBps / PCIeã®ç†è«–çš„ãªè»¢é€ãƒ¬ãƒ¼ãƒˆã¯Gen2 x4ã®å ´åˆ2GBps "

ãƒªãƒ³ã‚¯

Re: PCI Express (PCIe) for Networkers - wataken44's blog

ã“ã®è¨˜äº‹ã¯æ¬¡ã®è¨˜äº‹ã¸ã®ãƒ¬ã‚¹ãƒãƒ³ã‚¹ã ã£ãŸã‚Šã—ã¾ã™ã€‚ PCI Express (PCIe) for Networkers : http://terrasense.wordpress.com/2011/01/08/pci-express-pcie-for-networkers/ 8b/10bã‚¨ãƒ³ã‚³ãƒ¼ãƒ‡ã‚£ãƒ³ã‚°ã¨ã€å„ãƒ¬ã‚¤ãƒ¤ã®ãƒ˜ãƒƒãƒ€ï¼ˆãƒ•ãƒ¬ãƒ¼ãƒŸãƒ³ã‚°ï¼‰ã«ã‚ˆã‚‹ã‚ªãƒ¼ãƒãƒ¼ãƒ˜ãƒƒãƒ‰ãŒä¸€ç•ªå¤§ãã„ã€‚ 8b/10bï¼šå®ŸåŠ¹å¸¯åŸŸï¼80% ãƒ˜ãƒƒãƒ€ï¼šãƒ‡ãƒ¼ã‚¿ã‚µã‚¤ã‚ºï¼64Bã®å ´åˆã€å®ŸåŠ¹å¸¯åŸŸï¼84%è¤‡æ•°ãƒ¬ãƒ¼ãƒ³ã®å ´åˆã€ãƒ•ãƒ¬ãƒ¼ãƒ ã‚µã‚¤ã‚ºã«ã‚ˆã£ã¦ã¯ãƒ‘ãƒ‡ã‚£ãƒ³ã‚°ã®ã‚ªãƒ¼ãƒãƒ¼ãƒ˜ãƒƒãƒ‰ãŒç™ºç”Ÿã™ã‚‹ã€‚ ãã®ä»–ã‚·ã‚°ãƒŠãƒªãƒ³ã‚°(TLP, DLLP)ã‚‚åŒã˜å¸¯åŸŸã‚’ä½¿ç”¨ã™ã‚‹ãŸã‚ã€å¤šå°‘ã®ã‚ªãƒ¼ãƒãƒ¼ãƒ˜ãƒƒãƒ‰ãŒç™ºç”Ÿã™ã‚‹ã€‚å¸‚å ´ã«å‡ºã¦ã„ã‚‹ã‚«ãƒ¼ãƒ‰ã¯åŸºæœ¬çš„ã« PCIe å´ã®å¸¯åŸŸ > ã‚¤ãƒ³ã‚¿ãƒ¼ãƒ•ã‚§ãƒ¼ã‚¹ã®å¸¯åŸŸã¨ãªã‚‹ã‚ˆã†ã«è¨è¨ˆã•ã‚Œã‚‹ã¯ãšã§ã™ã€‚ 10GbE 2port ã§ Gen2 x8 ã® N

yass 2013/12/28

" å¸‚å ´ã«å‡ºã¦ã„ã‚‹ã‚«ãƒ¼ãƒ‰ã¯åŸºæœ¬çš„ã« PCIe å´ã®å¸¯åŸŸ > ã‚¤ãƒ³ã‚¿ãƒ¼ãƒ•ã‚§ãƒ¼ã‚¹ã®å¸¯åŸŸã¨ãªã‚‹ã‚ˆã†ã«è¨è¨ˆã•ã‚Œã‚‹ã¯ãšã§ã™ã€‚10GbE 2port ã§ Gen2 x8 ã® NIC ã‚’ä¾‹ã«ã¨ã‚‹ã¨ / Network ã®å¸¯åŸŸ : 10Gbps * 2 = 20Gbps / PCIe ã®å¸¯åŸŸ: 5GT/s * 8lane * (8b/10b) = 32Gbps "

ãƒªãƒ³ã‚¯

LAN Ethernet Maximum Rates, Generation, Capturing & Monitoring

LAN Ethernet Maximum Rates, Generation, Capturing & Monitoring Introduction Designing and managing an IP network requires an in-depth understanding of both the network infrastructure and the performance of devices that are attached, including how packets are handled by each network device. Network computing engineers typically refer to the performance of network devices by using the speed of the i

yass 2013/12/28

ãƒªãƒ³ã‚¯

Network Tuning and Performance Guide @ Calomel.org

home rss search January 01, 2017 Network Tuning and Performance a simple guide to enhancing network speeds Many of today's desktop systems and servers come with on board gigabit network controllers. After some simple speeds tests you will soon find out that you are not be able to transfer data over the network much faster than you did with a 100MB link. There are many factors which affect network

yass 2013/12/28

" PCI Express is a newer technology which elevates bus bandwidth from hundreds of megabytes per second to many gigabytes per second. This allows a single machine to support multiple gigabit ports per interface card or even multiple 10 gigabit ports. / 161,112 pps) and 1.9333 gigabits per second "

ãƒªãƒ³ã‚¯

https://nuclearcat.com/mediawiki/index.php/Intel_Gigabit_Performance

yass 2013/12/28

"PCI-Express just great for 1Gbps / embedded in ICH8 Gigabit adapter 82566DC, have because of bug, or by design internal FIFO only 16K! It's means 8K for RX, and 8K for TX / Because of small FIFO buffer and probably PCI-Express latency, on high packet-rate (around 100Kpps) you will hit a wall. "

ãƒªãƒ³ã‚¯

100Gbpsã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ãƒ«ãƒ¼ã‚¿ã®å®Ÿç¾å¯èƒ½æ€§ã«é–¢ã™ã‚‹è«–æ–‡

1. Building a Single-Box 100 Gbps Software Router Sangjin Han, Keon Jang, KyoungSoo Park, Sue Moon KAIST In IEEE Workshop on Local and Metropolitan Area Networks, 2010 id:y_uuki / @y_uuk1 1 2. PacketShader: a GPU-accelerated Software Router Sangjin Han, Keon Jang, KyoungSoo Park and Sue Moon. In proceedings of ACM SIGCOMM 2010, Delhi, India. September 2010 SSLShader: Cheap SSL Acceleration with Co

yass 2013/12/28

1299cycle

ãƒªãƒ³ã‚¯

ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

ã‚¿ã‚°

2013å¹´12æœˆ28æ—¥ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (7ä»¶)

Amazon Kinesisã¨Apache Kafkaã®é¡žä¼¼ç‚¹ï¼ç›¸é•ç‚¹ã¾ã¨ã‚ - å¤¢ã¨ã‚¬ãƒ©ã‚¯ã‚¿ã®é›†ç©å ´

e-trees.Japan, inc. :: KC705ã§PCIeã®é€šä¿¡æ€§èƒ½ã‚’æ¸¬å®šã—ã¦ã¿ãŸ

Re: PCI Express (PCIe) for Networkers - wataken44's blog

LAN Ethernet Maximum Rates, Generation, Capturing & Monitoring

Network Tuning and Performance Guide @ Calomel.org

https://nuclearcat.com/mediawiki/index.php/Intel_Gigabit_Performance

100Gbpsã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ãƒ«ãƒ¼ã‚¿ã®å®Ÿç¾å¯èƒ½æ€§ã«é–¢ã™ã‚‹è«–æ–‡

ãŠçŸ¥ã‚‰ã›

ä»Šé€±ã®ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯æ•°ãƒ©ãƒ³ã‚ãƒ³ã‚°ï¼ˆ2025å¹´5æœˆç¬¬2é€±ï¼‰

ä»Šé€±ã®ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯æ•°ãƒ©ãƒ³ã‚ãƒ³ã‚°ï¼ˆ2025å¹´5æœˆç¬¬1é€±ï¼‰

æœˆé–“ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯æ•°ãƒ©ãƒ³ã‚ãƒ³ã‚°ï¼ˆ2025å¹´4æœˆï¼‰

å…¬å¼Twitter

ã‚ãƒ¼ãƒœãƒ¼ãƒ‰ã‚·ãƒ§ãƒ¼ãƒˆã‚«ãƒƒãƒˆä¸€è¦§

ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

å…¬å¼Twitter

ã¯ã¦ãªã®ã‚µãƒ¼ãƒ“ã‚¹

ã‚¿ã‚°

2013å¹´12æœˆ28æ—¥ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (7ä»¶)

ãŠçŸ¥ã‚‰ã›

ä»Šé€±ã®ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯æ•°ãƒ©ãƒ³ã‚­ãƒ³ã‚°ï¼ˆ2025å¹´5æœˆç¬¬2é€±ï¼‰

ä»Šé€±ã®ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯æ•°ãƒ©ãƒ³ã‚­ãƒ³ã‚°ï¼ˆ2025å¹´5æœˆç¬¬1é€±ï¼‰

æœˆé–“ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯æ•°ãƒ©ãƒ³ã‚­ãƒ³ã‚°ï¼ˆ2025å¹´4æœˆï¼‰

å…¬å¼Twitter

ã‚­ãƒ¼ãƒœãƒ¼ãƒ‰ã‚·ãƒ§ãƒ¼ãƒˆã‚«ãƒƒãƒˆä¸€è¦§

å…¬å¼Twitter

ã¯ã¦ãªã®ã‚µãƒ¼ãƒ“ã‚¹

2013å¹´12æœˆ28æ—¥ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (7ä»¶)

ãŠçŸ¥ã‚‰ã›

ä»Šé€±ã®ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯æ•°ãƒ©ãƒ³ã‚ãƒ³ã‚°ï¼ˆ2025å¹´5æœˆç¬¬2é€±ï¼‰

ä»Šé€±ã®ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯æ•°ãƒ©ãƒ³ã‚ãƒ³ã‚°ï¼ˆ2025å¹´5æœˆç¬¬1é€±ï¼‰

æœˆé–“ã¯ã¦ãªãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯æ•°ãƒ©ãƒ³ã‚ãƒ³ã‚°ï¼ˆ2025å¹´4æœˆï¼‰

å…¬å¼Twitter

ã‚ãƒ¼ãƒœãƒ¼ãƒ‰ã‚·ãƒ§ãƒ¼ãƒˆã‚«ãƒƒãƒˆä¸€è¦§

å…¬å¼Twitter

ã¯ã¦ãªã®ã‚µãƒ¼ãƒ“ã‚¹