AWS Summit Tokyo 2016 Developer Conference (2016/06/03)
Norikra ã¨ã¯ Norikra ã¨ã¯ãªã¢ã«ã¿ã¤ã ã¤ãã³ãã¹ããªã¼ã ã«å¯¾ã㦠SQL ã©ã¤ã¯ãªè¨èªã§å¦çã§ãã cool ãªãããã¯ãã§ãã ä¾ãã°ãNginx ã®ã¢ã¯ã»ã¹ãã°ã Norikra ã«æµãè¾¼ã¿ãnåãããã®ã¢ã¯ã»ã¹æ°ãã¬ã¹ãã³ã¹ã¿ã¤ã ããªã¢ã«ã¿ã¤ã ã«éè¨ããã¨ãã£ãäºãå¯è½ã§ãã ãã¡ãã Nginx ã ãã§ã¯ãªãããèªèº«ãæ¸ãããã¢ããªãåºåãããã°ãæµãè¾¼ãã§éè¨ã§ãã¾ãã æ´ã« Fluentd ãçµã¿åããã㨠GrowthForecast ã Mackerel ã¨ãã£ããã¼ã«ã«éè¨çµæã渡ãã¦å¯è¦åãããªã©ã¨ãã£ããã¨ã容æãªã®ã§ãéå ±å¤éè¨ãã·ã¹ãã éç¨ç¶æ³ã®å¯è¦åã«æã£ã¦ããã§ãã Fluentd 㨠Norikra ãæ´»ç¨ãã¦å¯è¦åããä¾ fluent-plugin-norikra ã¨å¯è¦åãã¼ã«(GrowthForecastç)ãçµã¿åãããã¨ããã«å¯è¦å
Googleããªã¼ãã³ã½ã¼ã¹ã¨ãã¦å ¬éããKubernetesã¯ãã³ã³ããåä»®æ³åã½ããã¦ã§ã¢ã®Dockerã管çãããã¼ã«ã§ããéçºããã¸ã§ã¯ãã«ã¯DockerãRedHatãIBMãVMwareããã¤ã¯ãã½ãããªã©å¤æ°ã®ä¼æ¥ãåå ã表æãã¦ãã¾ãã Kubernetesã¯ãè¤æ°ã®Dockerã³ã³ããã«ã¾ã¨ãã¦ã¢ããªã±ã¼ã·ã§ã³ããããã¤ããè¨å®ãè¡ãã稼åç¶æ³ãç£è¦ã管çãããµã¼ãã¹ã¸ã®ãã©ãã£ãã¯ãã«ã¼ãã£ã³ã°ãããªã©ãã¯ã©ã¹ã¿ã¨ãã¦Dockerãéç¨ããããã®å¤ãã®æ©è½ãåãã¦ãã¾ãã ãã®Kubernetesã§ä½¿ãããæ¨æºã®ãã°åéãã¼ã«ã¨ãã¦ããªã¼ãã³ã½ã¼ã¹ã®fluentdãæ¡ç¨ããããã¨ãæããã«ãªãã¾ãããä¸è¨ã¯ãããä¼ããGoogleä½è¤æ°ã®ãã¤ã¼ãã fluentdãKubernetesã®æ¨æºãã°ã³ã¬ã¯ã¿ã«æ¡ç¨ããããï¼ï¼ï¼ https://t.co/V8VDM4IE7e
ã¾ãã両è ã¯ããªãæ§è³ªã®ç°ãªããããã¯ããªã®ã§ã以ä¸ã®æ¯è¼ã¯çéãã çéãã§ãããã¨ãåæã«ãã¹ããªã¼ã ãã¼ã¿å¦çãã©ãããã©ã¼ã ã¨ãã¦ã®ä¸¡è ãæ¯ã¹ã¦ã¿ãã åºæ¬æ å ± fluentd http://fluentd.org/ ä»ãã¨ããããã°ã³ã¬ã¯ã¿ã¼/ã¤ãã³ãã¢ã°ãªã²ã¼ã¿ã¼ãRubyã§å®è£ ããã¦ããã軽éé«éã RPCåºç¤ã§ã¯ãªãããã®ä¸ã®ã¬ã¤ã¤ã¼ã«ä½ç½®ãããããã¯ãã Storm http://storm-project.net/ åæ£RPCåºç¤ãã¹ããªã¼ã ãã¼ã¿çMapReduce風ãã¬ã¼ã ã¯ã¼ã¯ãJava+Clojureã§å®è£ ããã¦ããã æ¦è¦ã«ã¤ãã¦ã¯ãä¸è¨ã®ã¹ã©ã¤ããã¨ã¦ãããããããã£ãã Twitterã®ãªã¢ã«ã¿ã¤ã åæ£å¦çã·ã¹ãã ãStormãå ¥é ã¹ããªã¼ã ãã¼ã¿å¦çã§ä½ãããã®ãã«ã¤ã㦠ã¹ããªã¼ã ãã¼ã¿å¦çã®ãã¼ãºã«ã¤ãã¦ãèªåãç解ãã¦ããç¯å²ã§ã®ç°¡åãªèª¬æã å ¸
2013-01-08 å»å¹´æå¾ã®èã®æ¥ã§ãã12/29ã«ããã°åéãã¼ã« fluentd ã使ã£ã¦ãgroongaãã¼ã¿ãã¼ã¹ã®ã¬ããªã±ã¼ã·ã§ã³ãè¡ãã©ã¤ãã©ãªfluent-plugin-groonga 1.0.1ããªãªã¼ã¹ãã¾ããã ããããblogroongaã§ã¯ä»ã¾ã§ãã®fluent-plugin-groongaã«ã¤ãã¦èª¬æãã¦ãã¾ããã§ããã ããã§ããã®è¨äºã§ã¯fluent-plugin-groongaã«ã¤ãã¦èª¬æãã¾ãã fluent-plugin-groongaã¨ã¯ fluent-plugin-groongaã¯fluentdã®ãã©ã°ã¤ã³éã§ãRubyã§å®è£ ããã¦ãã¾ãã é常ãfluentdã¯ãã°ã®åéã«ä½¿ãã¾ãããfluent-plugin-groongaã§ã¯groongaã®ã¯ã¨ãªãè¤æ°ã®groongaãµã¼ãã¼ã«è»¢éããããã«ä½¿ã£ã¦ãã¾ããã¤ã¾ããfluentdã®ã¡ãã»ã¼ã¸
fluent-plugin-rewriteã¨ããfluentdã®ãã©ã°ã¤ã³ãä½æããã以ä¸ããã®ãã©ã°ã¤ã³ã®è§£æ±ºããåé¡ã«ã¤ãã¦è¿°ã¹ãã https://github.com/kentaro/fluent-plugin-rewrite https://rubygems.org/gems/fluent-plugin-rewrite åé¡ ãããµã¼ãã¹ã®ã¬ã¹ãã³ã¹ã¿ã¤ã æ¹åããã¦ãã¦ãã¾ãã¯ç¶æ³ã®å¯è¦åã®ããã«ãfluentdãç¨ãããã¨ã«ããããã®éããã¨ãã° ããããã¼ã¸ ã¦ã¼ã¶ãã¼ã¸ æ¸ç±æ¤ç´¢ãã¼ã¸ æ¸ç±è©³ç´°ãã¼ã¸ ... ãã®ä» ã¨ãã£ãã°ã«ã¼ãã«ããã¦ãã¬ã¹ãã³ã¹ã¿ã¤ã ã®å種統è¨ãåããããã¾ãããã°ãå ¨é¨éè¨ããã°ããã¨ãããã®ã§ã¯ãªããé¤å¤ãã¹ãmessageãè¤æ°ç¨®é¡ããã®ã§ãæè»ã«ãã£ã«ã¿ã«ã¼ã«ãè¨å®ãããã æ¢åã®ãã©ã°ã¤ã³ã ã¨out_exec_filterã使ããã¨ã«ãã£
ããããã¾ã§Rack::CommonLogger::Fluentãéç¨ãã¯ããã¾ããã Mongoã«è¨é²ãã¦ãã«ããã«ãã¨ããè²ã ãªæ¹æ³ããããããã®ã§ãããã¨ãããããã¡ã¤ã«ã«è¨é²ãã¦ããã¦ãç¨éã¯ãã¨ã§èãããã¨ããæãã§ä½¿ã£ã¦ãã¾ãã ã¨ãããã¨ã§ããã°ãã¡ã¤ã«ãè²ã ã¿ã¦ã¿ã¦ã¢ã¯ã»ã¹ãã©ããªæããªã®ããã調ã¹ããããã¨ãã«ä¾¿å©ãããªflgrepã¨ãããã¼ã«ãä½ã£ã¦ã¿ã¾ããã(æ¢ã«ã©ããã«ãããããªæ°ããã¤ã¤ãRack::CommonLogger::Fluentã®v0.4.0ããå梱ããã¦ãã¾ãã Rack::CommonLogger::Fluentã«å ¥ã£ã¦ã¯ãã¾ãããfluentdã®ãã°å½¢å¼ã§ããã°ä½¿ãããã§ã¯ãªããã¨æãã¾ãã $ flgrep Usage: flgrep [options] -c, --column=s column name to target -v, --va
RRDãªã©ã«ã¡ããªã¯ã¹ãæ¸ãè¾¼ãã§ã°ã©ããçæãã¦ããå ´åãæ¨æºçãªãµã¼ãã ã¨CPUãHDDãããã«ããã¯ã«ãªã£ã¦ãNagiosãµã¼ã1å°ãããæã¦ãã¯ã©ã¤ã¢ã³ãã¯ã300å°ã700å°ããããé度ã¨ãã£ãã¨ããã§ãããã æ°ä¸å°ã®ãµã¼ãã管çããæ§ãªç°å¢ã§ã¯ãNagiosãµã¼ãåä½ã§æ å ±ãåæããã¦ãã¾ãã®ã§ãé¢é£ããã·ã¹ãã ï¼ç¹ã«ä»é¨ç½²ã管çãã¦ããæ§ãªï¼ã®ç¶æ³ãææ¡ãã¥ãããã¨ãããããã¾ãã å ¨ãµã¼ãã®ç¶æ³ã横æãã¦æ¤ç´¢ããªã¹ãã£ã³ã°ãã§ããã¨ãé害æã®å¯¾å¿æéãç縮ã§ããããçµ±è¨æ å ±ã®åå¾ãã§ããã¨ã¡ããªã¯ã¹drivenãªéç¨ï¼éçºããããããªãããããã¯ãã®è³ªãåä¸ããã ããã¨ãããã¨ã§fluentdã§ãã£ã¦ã¿ã¾ãããã¨ã¯è¨ã£ã¦ãã¾ã å§ã¾ã£ãã°ãããªã®ã§ãããä¸ã®å³ã®æ§ãªæ§æã§ãNagiosä¸ã®ã¤ãã³ããã°ãfluentdãtailããå¿ è¦ãªã¤ãã³ããã°ãfluentd serv
1. Fluentdã¨Redisã§ã¯ãããã«ã¸ã¥ã¢ã«ã§ ãªã¢ã«ã¿ã¤ã ãªã¬ã³ã¡ã³ãã¼ã·ã§ã³ GMO Media, Inc. Business Promotion Office System Architect Hitoshi Asai 2. WORK ScalaJP backend system data reporting aws FAVORITE PLUGINS TWITTER input: tail Hitoshi ASAI buffer: memory @hito_asa ããã°ã©ãã§ããã output: exec machida, tokyo, jp #fluentdcasual
ã¿ããªå¤§å¥½ãfluentdã¯ããã¸ã便å©ã§ããããã°ã®åéï¼éç´ã ããããããã¨ããã¨ãã«ã¡ãã£ã¨ãªã¼ãã¼ã¹ããã¯æ°å³ã®ã¨ãããããã¾ããç¹ã« in_tail ã¯ãã°ã®èªã¿è¾¼ã¿ã¨åæã« parse ãããä»çµã¿ã«ãªã£ã¦ãããã¾ããã°ãæ¸ãããå ´æãªãparseã®ã«ã¼ã«ãããã£ã¦ãã§ãããã¨ããã¨ããã¯åççãªãã®ã§ãããã®ã§ãããloadavgãé«ããµã¼ãã§ãããããã¨ãããã®ã¯æ£ç´ã«ãã£ã¦ãªããªãå³ããã§ãã ããããããã§ä»¥åã« scribeline ã¨ããã¨ã¼ã¸ã§ã³ããã¼ã«ãä½ã£ãã®ã§ããã fluentd 以éå¾ã使ãç¶ãã¦ããã®ã§ããããã°ããã£ããéç´ããã¨ããã® fluentd ãCPU使ç¨ççã«ãã£ã±ããã£ã±ãã«ãªã£ã¦å³ãããã®ãããã¾ããããscribe(Thrift)ãããªãã¦MessagePackã«ããã°åããããã°ããããã¨ãããã¨ãæéçºè ãè¨ã£ã¦ãããããªæ°ãã
åã«èªåã§æ¸ãã fluentdã®ããã®ãã©ã°ã¤ã³ãã¤ãããæ¸ãæé - tagomorisã®ã¡ã¢ç½®ãå ´ ã¯ããã¸ãéå®ãã¦ããã®ã ããæ¸ãããããå¾ã«ãªã£ã¦å®ã¯ç¾å¨ãã§ã« bundle gem ã³ãã³ãã使ããããããè¯ãããã ã¨ãããã¨ãããã£ã¦ãã¾ã£ããã°ãã°ããã¦ã¦ç§»è¡ãã¦ãªãã£ãã ã§ãã¾ãã²ã¨ã¤ãã©ã°ã¤ã³ãæ¸ããã¨ã«ããã®ã§ã¤ãã§ã« bundle ã使ã£ãæé ããã£ããã¾ã¨ãã¦ããã以ä¸ã®ã¨ã³ããªãããã¸ãåèã«ããã¦ããã£ãã T-POINTãåå¾ããã¹ã¯ãªãããGistãã移å, Bundlerã使ã£ãGemä½æã¡ã¢ (èªåç¨) - ãã ã®ã«ã£ã(2012-02-18) æºåã¨ãã£ã¬ã¯ããªããªã¼ã®ä½æ bundler ã¯å¿ è¦ãªã®ã§ããªã«ã¯ãªãã¨ãå ¥ãã¦ãããã gem install bundler ããã¦ãã©ã°ã¤ã³ç¨ãã£ã¬ã¯ããªããªã¼ãä½æãããä»å㯠DataCount
Fluentd meetup in Japanãªãã¤ãã³ãããããã©ããã¹ããªãï¼ ã¨ãããèªãããã£ãã®ã§ããããããåããã¦åå ãã1ã»ãã·ã§ã³ããã¹ã£ã¦ããã ã¾ã ä¸ã«åºã¦å年足ããã®ããã«ã¦ã§ã¢ã®ã¤ãã³ããªã®ã«éããéã£ãã120人*1ãã¾ãã¾ãåæ¥éã²ãããé«æ¿åº¦ãªæéã ã£ãã話ãã¦ã¿ãã¨ãã¿ããªfluentdããã©ã¼ã«ã¹ãã¦ããããã«ãã£ã±ãåé¡æèããã£ã¦ãã¦ããããã£ã±ãããã¯åºãã¹ããã¦åºãã®ã ãªãã¨ããå°è±¡ã ã£ãã ãã¨ããtogetterã®ã¾ã¨ããã¼ã¸ãè¦ããã©ç°æ§ã«é·ããã©ãã ãçæ³ã ã£ãããããããéå¬æéä¸ãTwitterã®æ¥æ¬ã®ãã¬ã³ãã« #fluentd ãåºã¦ãããªã ä¼å ´ãã»ãã¨ã«ãã°ããããéå¶ãUstreamãç¡ç·LAN解æ¾ãé»æºã®ç¢ºä¿ãã飲ã¿ç©æä¾ã¾ã§æ¥µãã¦è¯ãç¶æ ã ã£ãã主å¬ãéå¶ååã®æ¹ã ããã³ãã¥ã¼ãã£ã¼ã¢ã¼ããã¯ãæ§ãã»ãã¨ãã«ãããã¨ãã
Fluentd meetup in Japan. I talked about "Dive into Fluent plugin". Some contents are outdated. See this slide: http://www.slideshare.net/repeatedly/dive-into-fluentd-plugin-v012Read less
çããããã°æ¸ãã¦ã¾ããï¼ï¼(æ¨æ¶) Fluentd meetup in Japan ãéå¬éè¿ãæè¿å¤§å¤ç±ãã¤ãã³ããã°åéã·ã¹ãã Fluentd ãªããã§ãããPerl ãããã°ãåºåãã Fluent::Logger ã¨ããã¢ã¸ã¥ã¼ã«ã CPAN ã«ãªãªã¼ã¹ããã®ã§ãç¥ãããã¾ãã (æåã®ç㯠id:hirose31 ãããæ¸ããã¦ãããã«ååã® id:shin1rosei ã¨æãå ãããã®ã§ã) ã¤ã³ã¹ãã¼ã«ã¯ cpanm ãªã©ã§ã©ããã使ãæ¹ã¯ POD ã«ãããããã«ç°¡åã§ãã use Fluent::Logger; my $logger = Fluent::Logger->new( host => "127.0.0.1", port => 24224 ); $logger->post( "myapp.info" => { foo => "bar" } ); ä¸è¨ã®ä¾ã§éä¿¡ã
ãªãªã¼ã¹ãé害æ å ±ãªã©ã®ãµã¼ãã¹ã®ãç¥ãã
ææ°ã®äººæ°ã¨ã³ããªã¼ã®é ä¿¡
å¦çãå®è¡ä¸ã§ã
j次ã®ããã¯ãã¼ã¯
kåã®ããã¯ãã¼ã¯
lãã¨ã§èªã
eã³ã¡ã³ãä¸è¦§ãéã
oãã¼ã¸ãéã
{{#tags}}- {{label}}
{{/tags}}