Infrastructure Processing billions of events in real time at Twitter At Twitter, we process approximately 400 billion events in real time and generate petabyte (PB) scale data every day. There are various event sources we consume data from, and they are produced in different platforms and storage systems, such as Hadoop, Vertica, Manhattan distributed databases, Kafka, Twitter Eventbus, GCS, BigQu
ãªã³ã°ãããã¡ã®ã¤ã¡ã¼ã¸å³ 1. ãªã³ã°ãããã¡ã¨ã¯ä½ã æ©è½çã«ã¯First In First Out (FIFO)ã¨ãå¼ã°ãããã¥ã¼ã®ä¸ç¨®ã§ãããããªã³ã°ç¶ã«ãããã¡ãç½®ãã¦ããã®ä¸ã§Readã¨Writeã®ã¤ã³ããã¯ã¹ãã°ã«ã°ã«ã¨åãæ§é ãã¨ãäºã«ãã£ã¦å®¹éã«ä¸éãã§ãããã¨ã¨å¼ãæãã«é«éãªèªã¿æ¸ãé度ãå¾ããã®ã§ããããã¥ã¼ãåã«å®è£ ããã ããªãå±±ã»ã©æ¹æ³ããã£ã¦ç·å½¢ãªã¹ãã使ã£ã¦ããããã¹ã¿ãã¯ã2ã¤ä½¿ã£ã¦ãåççã«ã¯å¯è½ã ããã®ä¸ã§ããªã³ã°ãããã¡ãç¨ããæ¹æ³ã®å©ç¹ã¯ã²ã¨ãã«æ§è½ã®é«ãã§ããã¡ã¢ãªç¢ºä¿ãªã©ãè¡ããªããé°ã§ã·ã¹ãã ç³»ã®æ§ã ãªå ´æã§ä½¿ããã¦ããã ããã®å®è£ èªä½ã¯æ å ±ç³»ã®å¤§å¦çã®æ¼ç¿ã¬ãã«ã®é£åº¦ã§ãããå°ã奥ãæ·±ããã¾ããªã³ã°ãããã¡ã®ã¹ã¿ã³ãã¼ããªã¤ã³ã¿ãã§ã¼ã¹ã¨å®è£ ã¯ä»¥ä¸ã®ãããªãã®ã§ããã class RingBuffer { public: explicit
Microsoftã¯Windowsã®ããã©ã¼ãã³ã¹ã¨ä¿¡é ¼æ§ã®åä¸ãã¢ããªã®æé©åãçã¨ãè¨å®ãªã©ã®åãçµã¿ææãçºè¡¨ããããã¹ã¯ãããã¸ã®ã¢ã¯ã»ã¹æéã10ï¼ ç縮ããããã¢ããªã±ã¼ã·ã§ã³ã®èµ·åæéã50ï¼ ä»¥ä¸ç縮ããããã¦ããã
ä»åã¯ãSQLãæ¸ãä¸ã§ç¹ã«ããã©ã¼ãã³ã¹ã«å½±é¿ã®ããSQLã®å®è¡è¨ç»ã®èªã¿æ¹ã«ã¤ãã¦è§£èª¬ãã¾ããå®è¡è¨ç»ã¯ãã¼ã¿ãã¼ã¹è£½åã«ãã£ã¦ãã¾ãã¾ã«å·®ç°ãããã¾ãããããã§ã¯æ¯è¼çã©ã®ãã¼ã¿ãã¼ã¹è£½åã§ãå ±éããå 容ã«ã¤ãã¦è§£èª¬ãã¾ãã å®è¡è¨ç»ã¨ã¯è¨è¿°ããSQLãå®éã«ãã¼ã¿ãã¼ã¹ã®å é¨ã§ã©ã®ããã«å¦çããã¦çµæãè¿ããããã®å¦çæ¹æ³ãè¨è¿°ããæ å ±ã§ãã A5:SQL Mk-2ã§ã¯ãSQLã¨ãã£ã¿ã§å®è¡è¨ç»ãè¦ãã SQL ã®ä¸ã«ãã£ã¬ãããããç¶æ ã§ã¡ãã¥ã¼ãã [SQL(S)] â [SQLã®å®è¡è¨ç»(J)] ã¾ãã¯ãCtrl+E ã§è¡¨ç¤ºã§ãã¾ãã 表示ã®ä»æ¹ã¯ãã¼ã¿ãã¼ã¹è£½åãã¨ã«ç°ãªãã¾ãããå¤ãã®ãã¼ã¿ãã¼ã¹è£½åã§ã¯ããªã¼ç¶ã®æ å ±ã¨ãã¦è¡¨ç¾ããã¾ããï¼ãã®ãã A5:SQL Mk-2ã§ãããªã¼ãã¥ã¼ã§å®è¡è¨ç»ã表示ãã¾ããï¼ ããªã¼ã®ãªã¼ãï¼ç«¯ï¼ããå¦çãè¡ãããã«ã¼ãï¼æ ¹ï¼ã«åãã£
ã¦ã¼ã¶ã¼å士ã®ã¤ãªãããå ã«æç³»åã«140æåã®ã¡ãã»ã¼ã¸ã20åã»ã©è¡¨ç¤ºããââãTwitterã®ãµã¼ãã¹ã¯ãæåã«ãã¦ãã¾ãã¨å®ã«ã·ã³ãã«ã ããèå¾ã«ã¯é常ã«å¤§ããªæè¡çãã£ã¬ã³ã¸ã横ããã£ã¦ãããã¤ã¶ããæ°ã¯æé10å件ãçªç ´ãTwitterãæµããã¡ãã»ã¼ã¸æ°ã¯ç§é120ä¸ã«ãéããã¦ã¼ã¶ã¼å士ã®ã¤ãªããã表ãã½ã¼ã·ã£ã«ã»ã°ã©ãã§ããã¡ã¢ãªã«è¼ãéãè¶ ãã¦ãããéæ¹ããªãã¹ã±ã¼ã«ã®ãã¼ã¿ãã¤ãªãã§ããã«ãé¢ãããã0.1ç§ä»¥ä¸ã§Webãã¼ã¸ã®è¡¨ç¤ºãå®äºãããªããã°ãªããªãããã®ããã«åãã¼ã¿ã¹ãã¬ã¼ã¸ã¯1ï½5msç¨åº¦ã§å¿çããªããã°ãªããªãã Twitterã®ãªã¹ãæ©è½ã®å®è£ ã§ããã¸ã§ã¯ããªã¼ãã¼ãåãããã¨ãããNick Kallenæ°ãæ¥æ¥ãã2010å¹´4æ19æ¥ãã2æ¥éã®äºå®ã§éå¬ä¸ã®ãQCon Tokyo 2010ãã§åºèª¿è¬æ¼ãè¡ã£ãããData Architecture
人æ°ã®ãã¹ãã©ãã¥ã¼ã³3ããéãã¦ãææ°ã®éä¿¡æè¡ã®åºæ¬ãå¦ãã§ãã¾ããã¨ãããã®ç¹éãä»åã¯å®éã«ã¹ãã©ãã¥ã¼ã³3ã®ãã±ãããWiresharkã§ãã£ããã£ã¼ãã¦ãUDPã®éä¿¡ãNATè¶ãã®ä»çµã¿ãè¦ã¦ããã¾ãã ãªãè¨è¼å 容ã«ã¤ãã¦ã¯ãå®éã®éä¿¡ããã£ããã£ã¼ããçµæã«åºã¥ããçè ã®èå¯ãæ¨æ¸¬ã«ãããã®ã§ããã任天å ã®å ¬å¼è¦è§£ã§ã¯ãªããã¨ãè¨ãã¦ããã¾ãã 任天å ãéçºããæ±ç¨ã²ã¼ã ãã©ãããã©ã¼ã ãNPLNã ã¹ãã©ãã¥ã¼ã³3ã§ã¯ããNPLNãã¨ããã任天å ãã¹ã¯ã©ããããéçºããã任天å ãã©ãããã©ã¼ã åãæ±ç¨ã²ã¼ã ãµã¼ãã¼ã使ããå§ãã¾ãããå¾æ¥ã®ãã©ãããã©ã¼ã ã§ãããNEXãã¯ãã³ãã³ãã¼3DSï¼Wii Uããã®ä¸ä»£ã§ä½¿ããã¦ãã¦ãNPLNã¯ããã§ã®ãã¦ãã¦ãåºã«2021å¹´ãããGoogle Cloud Platformï¼GCPï¼ãã§ç¨¼åãã¦ããããã§ãããªã³ã©ã¤ã³å¯¾æ¦ã®ãã
ãã®è¨äºã¯ãSupershipã°ã«ã¼ã Advent Calendar 2021ã®7æ¥ç®ã®è¨äºã«ãªãã¾ãã å æ¥ãsshã使ç¨ãããã¡ã¤ã«è»¢éãåç·é度ã¨æ¯ã¹ã¦ç°å¸¸ã«é ãã¨ããç¾è±¡ã«ééããã®ã§ããã®éã«è¡ã£ã調æ»ãåç¾ãã¤ã¤åå ã対çã«ã¤ãã¦æ¸ãã¦ã¿ããã¨æãã¾ãã è¦ç´ OpenSSHã¯ããã©ã«ãã§interactiveãªã»ãã·ã§ã³ã« af21 ãnon-interactiveãªã»ãã·ã§ã³ã« cs1 ãDSCPå¤ã¨ãã¦IPãããã«è¨å®ãã ãã¬ãã網ã¯IPãããã®DSCPå¤ã帯ååªå ãµã¼ãã¹ã§ä½¿ç¨ãã¦ãããå¥ç´ã«å¿ãã¦æå®ãããåªå 度以å¤ãè¨å®ããããã±ããã®è»¢éã¯ä¿è¨¼ãããªã ãã®ãããOpenSSHãããã©ã«ãè¨å®ã®ã¾ã¾ãã¬ãã網ã§ä½¿ãã¨éä¿¡ãã§ããªãã£ãããé度ä½ä¸ãªã©ã®æªå½±é¿ãåããå¯è½æ§ããã OpenSSHãDSCPå¤ãè¨å®ããªãããã«ããããã«ã¯ãIPQoS noneãè¨
å社å·è¡å½¹å¡ã®æ¢ ç°æäºæ¬é¨é·ï¼ããã¤ã¹ãã¼ããã¼å¶æ¥çµ±æ¬æ¬é¨ï¼ã¯ããããããªã¼ã®ä¸å ·åããOSç«ã¡ä¸ããã®é ããã¡ã¢ãªã®å¯¿å½ã«ããã¯ã©ãã·ã¥ãªã©ã®ãã©ãã«ã4å¹´ç®ä»¥éã«é »çºãããã¨åæã ããã«ãæ¥æ¬ã®ä¸å°ä¼æ¥ã®PCè²·ãæ¿ããµã¤ã¯ã«ã¯5.4å¹´ã¨ãç±³å½ã®4.5å¹´ãã°ãã¼ãã«ã®4.3å¹´ãããé·ãå¾åãããã¨ããã é·æéåè½ã¡ã®PCãå©ç¨ãããã¨ã«ãããã¡ãªããã«ã¤ãã¦ã調æ»ä¼ç¤¾ã®ç±³Tech Aisleãè¡ã£ãPCå©ç¨ã®ã³ã¹ãæ¯è¼èª¿æ»ãæ¢ ç°æ¬é¨é·ãç´¹ä»ã å©ç¨æéã4å¹´æªæºã®PC1å°ã«ãããã¡ã³ããã³ã¹ã³ã¹ãã¯322ãã«ã§ãçç£æ§ã³ã¹ãï¼èµ·åã«ãããæéãªã©ãæ¬æ¥çç£ãå¯è½ã ã£ãæéã失ãã³ã¹ãï¼ã¯1056ãã«ãè¨1379ãã«ï¼ç´15ä¸åï¼ã§ããã®ã«å¯¾ãã4年以ä¸ã®PC1å°ã«ãããããããã®ã³ã¹ãã¯497ãã«ã2574ãã«ã§è¨3071ãã«ï¼ç´35ä¸åï¼ã¨ããçµæã ã£ãã ãä¸å°ã§æã使ããã¦ã
ã»ãã·ã§ã³IDãã¢ã¯ã»ã¹ãã¼ã¯ã³ãã¯ãã¾ãæ¥åä¸ã§ä½¿ãä¸æã®èå¥åãªã©ãããããªã¨ããã§ä¸æã®IDãçæããªãããããªãã±ã¼ã¹ãåå¨ãã¾ãã ããã§ä¸éã§ä½¿ããã¦ããIDã®çææ¹æ³ã«ã¤ãã¦èª¿ã¹ã¦ã¿ã¾ããã é¸æåºæº IDçæã«ãããè¦æ±ã¨ãã¦ã以ä¸ã®è¦³ç¹ãä¸ãããããã¨æãã¾ãã çæã®é度 大éã«ãã¼ã¿ãçæéã§å¦çãããããã«IDãä»ä¸ããå ´åãIDçæãã®ãã®ãããã«ããã¯ã¨ãªããã¨ãããã¾ãã æ¨æ¸¬å°é£æ§ IDãæ©å¯æ å ±ã¨çµã³ã¤ããå ´åãIDãæ¹ããããã¦ããæ©å¯ãã¼ã¿ãè¦ããªãããã«ã§ãã¦ããå¿ è¦ãããã¾ãã é åºæ§ æ¡çªããé ã«ãã¼ã¿ãã½ã¼ãããå¿ è¦ãããå ´åã¯ãIDãã½ã¼ããã¼ã¨ãã¦ä½¿ããªãã¨ããã¾ããã ããããã«ã¤ãã¦åçææ段ãè©ä¾¡ãã¾ãã IDçæã®æ段 ãã¼ã¿ãã¼ã¹ã®æ¡çªãã¼ãã« æ¡çªç¨ã®ãã¼ãã«ãä½ããããã§çªå·ãUPDATEããªããåå¾ãã¦ããããããã§ããå¤ã
æåã«ãã£ã¦ãããããã¯è² ãæãã¿ã ã SPAã¨PWAã®ç¾ç¶ èªåã¯æ¥æ¬ã§Reactã®åæã¨ã´ã¡ã³ã¸ã§ãªã¹ãã¿ãããªãã¨ããã£ã¦ãã¦ãSPAã®éãã®ã³ã³ãã³ããããä½ã£ã¦ãããããããåãããã³ãã¨ã³ããç©äºãããããããã¦ãéããã¼ã¸ãéç£ãã¦ã¦ã§ããå£åããã¦ããããªããï¼ãã¿ãããªæ¹å¤ããåæãã§ããåããããªãã§åã«ãããããããªããã©ãReact = SPA ã¿ãããªã¤ã¡ã¼ã¸ã§ã¹ã±ã¼ãã´ã¼ãã«ããã¦ããã ãããããã¯ã¾ãããã èªåã®ä»äºã§SPAæè¡ã使ãã¨ããã¯ãã¡ããã¨å¿ è¦æ§ããããçç±ã説æã§ããããã ããã¯ãè¿å¹´ã®è¤éå/ééåã«ã¤ãã¦æãã¨ããã¯ããã®ã§ãéã«æ¯ã£ã¦ AMP/PWA ã¨ããé¸æè¢ãæã£ã¦ããããã¦ãæ£ç´è¨ãã¨ä¾é ¼ãããR&Dã®ä»äºã§ããã£ããã ãã©ãä¸éãè¦ããããªãã ãã©ãä»ã®ã¨ããä»äºã§ä½¿ãã¿ã¤ãã³ã°ããªãã PWAæè¡ãä»äºã§ä½¿ããªãã£ãçç±ã¨ãã¦
ãçéããã¦ç¬ãããéããã¦éã«ä¸å®ã«ãªãã¬ãã«ãââãdev.toãã¨ããç±³å½ã®Webãµã¤ãã®è¡¨ç¤ºé度ãç°å¸¸ã«éãã¨ããããä¸ã§è©±é¡ã ããªãéãã®ãããµã¤ãåµè¨è ã®ãã³ã»ãã«ãã¼ã³ããããµã¤ãå ã®è¨äºã§èª¬æãã¦ããã 2016å¹´ã«ãªã¼ãã³ããdev.toã¯ãããã°ã©ãã¼ãæ å ±ã交æããããè°è«ãããã§ããã³ãã¥ããã£ã¼ãµã¤ããããããã¼ã¸ã«ã¯ã¦ã¼ã¶ã¼ãæ稿ããããã°è¨äºã®ã¿ã¤ãã«ãã³ã¡ã³ãããSNSã®ã¿ã¤ã ã©ã¤ã³ã®ããã«ä¸¦ãã§ããããã®ããããã¼ã¸ã®è¡¨ç¤ºã«ãããæéãåè¨äºãã¯ãªãã¯ããã¨ãã®ãã¼ã¸é·ç§»ãâä¸ç¬âãªã®ã ã ãªãéãã®ãããã«ãã¼ã³ããã17å¹´2æã«æ稿ããããã°è¨äºã«ããã°ãç±³Fastlyãæä¾ããCDNï¼Content Delivery Networkï¼ãæ´»ç¨ãã¦ãããCDNã¯ä¸çä¸ã«ãã£ãã·ã¥ãµã¼ããåæ£é ç½®ããã¦ã¼ã¶ã¼ãã¨ã«æãï¼ãããã¯ã¼ã¯çã«ï¼è¿ããµã¼ãã«ãã£ã
Netflixã®ããã©ã¼ãã³ã¹ã¨ã³ã¸ãã¢ã§ããçè ããã®ãtopã³ãã³ããªã©ã§è¡¨ç¤ºãããCPU使ç¨ç(%CPU)ã¯ããã¾ãæ¬å½ã®ä½¿ç¨çã表ãã¦ãããããã¥ã¼ãã³ã°ãªã©ã®ããã®ææ¨ã¨ãã¦ä½¿ããªãã¨ããææããªããããªã£ã¦ãã¾ã£ãã®ããä½ãè¦ãã°æ¬å½ã®CPU使ç¨çããããã®ãããããããã解説ããè¨äºã ç§ãã¡ã¿ããªãCPU使ç¨çã¨ãã¦ä½¿ã£ã¦ããææ¨ã¯é常ã«èª¤è§£ãæããã®ã§ããã®ç¶æ³ã¯æ¯å¹´æªåãã¦ãã¾ããCPU使ç¨çã¨ã¯ä½ã§ããããï¼ããã»ããµã¼ãã©ã®ãããå¿ãããï¼éãã¾ããCPU使ç¨çã表ãã¦ããã®ã¯ããã§ã¯ããã¾ãããç§ã話ãã¦ããã®ã¯ããã¡ãã¡ã§ããããã人ãã¡ã«ãããããç£è¦è£½åã§ããããã¯top(1)ã§ã使ããã¦ããã"%CPU"ã¨ããææ¨ã®ãã¨ã§ãã ããªãã®èãã¦ããã§ãããCPU使ç¨ç90% : å®é : "stalled"(訳注 : 以ä¸ã¹ãã¼ã«ã¨è¨ã)ã¨ã¯ãããã»ããµã¼ã
ãªãªã¼ã¹ãé害æ å ±ãªã©ã®ãµã¼ãã¹ã®ãç¥ãã
ææ°ã®äººæ°ã¨ã³ããªã¼ã®é ä¿¡
å¦çãå®è¡ä¸ã§ã
j次ã®ããã¯ãã¼ã¯
kåã®ããã¯ãã¼ã¯
lãã¨ã§èªã
eã³ã¡ã³ãä¸è¦§ãéã
oãã¼ã¸ãéã
{{#tags}}- {{label}}
{{/tags}}