You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
誰ãä½ã®ããã«ä½¿ãã®ãããããããªããã©ï¼æ°ããfluentd pluginæ¸ãã¾ããï¼ fluntdã«æµãã¦ããæ¥æ¬èªã®ãã¼ã¿ãmecabã«é£ããã¦å½¢æ ç´ è§£æçµæãããä¸åº¦fluentdã«æµããã©ã°ã¤ã³ã§ãï¼ install Gemfileã«ä»¥ä¸ãæ¸ãã¦ï¼bundle installããã°ããã§ã gem 'fluent-plugin-mecab', git: 'git://github.com/katsyoshi/fluent-plugin-mecab.git' 使ãæ¹ã¯ç°¡åã§ãï¼ä»¥ä¸ã®ããã«è¨å®ãã¾ã <source> type forward </source> <match mikutter.**> type copy <store> type stdout </store> <store> type mecab key message tag mecab.meacab </st
graphite ã«ã¡ããªã¯ã¹ããã¹ããã fluent-plugin ãæ¸ãã¾ãã å ã« github ã§å ¬éããã¦ãã fluent-plugin-graphite ãããã¾ããããã¤ãããæ¸ã㦠gem release ãããã¾ãã https://github.com/studio3104/fluent-plugin-graphite http://rubygems.org/gems/fluent-plugin-graphite ãªãã¤ãããæ¸ããã®ã 以ä¸ã®ãããªç®æã«æ¸å¿µããããä¿®æ£ã ã¨çµå±ã¾ãã£ã¨æ¸ãç´ãã®ã¨å¤ãããªãã¨æãã¤ãããæ¸ãã¦ãã¾ãã¾ãã å è¡ãã©ã°ã¤ã³ã¯ã Fluent::BufferedOutput ãç¶æ¿ããå é¨ã§ãµã³ããªã³ã°ãã«ã¦ã³ããªã©ã®è¨ç®ããã¦ããããããããã®ã¯ä»ã®ãã©ã°ã¤ã³ã«ä»»ãã¦ãæ¥ãå¤ããã®ã¾ã¾æãã¦ãããã°ããã®ã§ã¯ãªããã¨æã£ã ã¬ã³ã¼ã
ES + kibanaã§ãµã¼ãã¢ãã¿ãªã³ã°ããã£ã¦ã¿ãã®ã§ãããESã®CPUè² è·ãããªãé«ãã¦ããªã¢ã«ã¿ã¤ã ã«ã¢ãã¿ãªã³ã°ã§ããªãç¶æ³ã ã£ãã®ã§ãgraphite + grafanaã«ãã¦ã¿ããã¡ãªã¿ã«ãESã®ãµã¼ãã®CPUè² è·ã¯ãããªæãã Grafanaã¯Graphiteç¨ã®Dash boardãä½ããã¼ã«ãæè¿ãinfluxDBã«ã対å¿ãã¦ãã¦ãªããªãéå¿çã Grafana - Graphite Dashboard kibanaãforkããã ããã£ã¦ãç»é¢ã¯ãã£ãããã¾ã ä¿®æ£ãããããã®ããã¡ãã»ã¼ã¸ã«kibanaã£ã¦æåãã§ã¦ãããã¨ããã ã»ããã¢ãã ããããã®ã»ããã¢ããã®ã¡ã¢ ç£è¦ãµã¼ã ã¾ããç£è¦ãµã¼ãã«Graphiteã¨Grafanaãããããç°å¢ã¯CentOS6 CentOS6.x - CentOSã«RPMã§Graphite+Diamondãã¤ã³ã¹ãã¼ã«ãã -
# cat /etc/nginx/log_format.conf log_format ltsv 'time:$time_iso8601' '\tremote_addr:$remote_addr' '\trequest_method:$request_method' '\trequest_length:$request_length' '\trequest_uri:$request_uri' '\thttps:$https' '\turi:$uri' '\tquery_string:$query_string' '\tstatus:$status' '\tbytes_sent:$bytes_sent' '\tbody_bytes_sent:$body_bytes_sent' '\treferer:$http_referer' '\tuseragent:$http_user_agent' '
Norikraã¨ã¯ Norikraã¨ã¯ãªã¢ã«ã¿ã¤ã éè¨ãããã¯ãã§ããã¤ãã³ãã¹ããªã¼ã ã«å¯¾ãã¦SQLã©ã¤ã¯ãªè¨èªã§å¦çãæ¸ããã¨ãåºæ¥ã¾ãã ä¾ãã°ãApacheã®ã¢ã¯ã»ã¹ãã°ãNorikraã«æµãè¾¼ã¿ã1åãããã®ã¢ã¯ã»ã¹æ°ãã¬ã¹ãã³ã¹ã¿ã¤ã ã®æ大å¤ããªã¢ã«ã¿ã¤ã ã«éè¨ãããã¨ãåºæ¥ã¾ãã Norikraã®å©ç¨ä¾ã¯ä½è ã§ããtagomorisããã®ããã°ã§ç´¹ä»ãããã¾ãã ä»åã¯ãNorikraã使ã£ã¦DoSæ»æããããã¯ããä»çµã¿ãä½ã£ã¦ã¿ã¾ããã DoSæ»æãããã¯ã®ä»çµã¿ ã¢ã¯ã»ã¹å ã¯Apacheã®ã¢ã¯ã»ã¹ãã°ããåå¾ãããã°ã®åã渡ãã«ã¯Fluentdãå©ç¨ãã¾ããã ãããã¯ã®æé ã¯ä»¥ä¸ã®ããã«ãªãã¾ãã ã¢ã¯ã»ã¹ãã°ãFluentdã®in_tailãã©ã°ã¤ã³ã§åå¾ã Fluentdã®out_norikraãã©ã°ã¤ã³ã§ãã¢ã¯ã»ã¹ãã°ãNorikraã«æµãè¾¼ã¿ã Norikra
æ®æ®µã¯ãµã¼ãã®ã¡ããªã¯ã¹å¯è¦åã®ããã«cloudforecastã使ã£ã¦ãã¾ãããææ¡ä»¶ç¨ã«æ°ç§åä½ã§æ°åå°ã®ãµã¼ãã®ã¡ããªã¯ã¹ã表示ãããã®ã§ãè¨äºã¿ã¤ãã«ã®ãããªæ§æãä½ã£ã¦ã¿ãã dstatã§ã¨ã£ãå種å¤ã®ä»ã«ãnginxã¨memcachedã®æ å ±ãåããã¦è¡¨ç¤ºããã¦ãã¾ãã ã»ããã¢ãã ããããã®ã»ããã¢ããã®ã¡ã¢ ç£è¦ãµã¼ã ã¾ããç£è¦ãµã¼ãã«Elasticsearchã¨kibanaãããããç°å¢ã¯CentOS6 $ sudo yum install java-1.7.0-openjdk $ sudo rpm -Uvh https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.x.x.noarch.rpm Elasticsearchã¯ç¹ã«è¨å®ãªãèµ·å $ sudo service
ansibleã®ãã°ãfluentdã«æµã¶ Chef ã®ãã°ã Fluentd ã«æµã ã¨ããè¨äºãèªã¿ã¾ãããAnsibleã«ããã®ããã®æ©è½ãããã«ãé¢ããããã ã¾ãæ å ±ããªãã®ã§ã¾ã¨ãã¦ã¿ã¾ããã callback plugin¶ Ansibleã«ã¯moduleã¨ã¯å¥ã«pluginã¨ããæ©æ§ãããã¾ããä¾ãã° "{{lookup('file', '/etc/foo.txt') }}" ã¨ããããã«ä½¿ãlookup pluginããã ã¾ããä»ã«ãconnectionãvarsã®pluginãããã¾ããããã®ãã¡ã®ä¸ã¤ã«ã callback pluginãããã¾ãã callback pluginã¯ãã®åã®éãããããããªã¤ãã³ããçºçããæã«èªåçã« å¼ã³åºãããcallbackãç»é²ããããã®pluginã§ããã¤ãã³ãã®ä¾ã¨ãã¦ã¯ã playbookã®éå§æ taskã®éå§æ
ç®ç 家åºå ã®é»åãã¼ã¿ãéãããï¼ã§ããã°éãããã¼ã¿ãå©ç¨ãã¦æ¶è²»é»åãæ¸ããããï¼ BeagelBoardã¨ã¯ï¼ ARMï¼OMAP3ï¼ãè¼ã£ãã¯ã³ãã¼ãPCã§ãï¼ã¡ã¢ãªã512MBã¨å²ã¨ãªããï¼ï¼ï¼ã§ãï¼ td-agent for BeagleBoard-xM ããã¯å»å¹´ã®Kernel/VM Advent Calendarã大æ±æ¸Rubyä¼è°04ãè¦ã¦ä¸ããï¼ ã©ãããæµãã§éãã¦ãã®ï¼ Arduinoã«ä»ããã»ã³ãµã¼ãZigBeeï¼Xbeeï¼ã§åéãã·ã³ï¼BeagleBoardï¼ã«æãã¾ãï¼åéãã·ã³ã«æãããããï¼fluent-plugin-serialportãå©ç¨ãã¦ãã¼ã¿ãéãã¾ãï¼éãããããã¼ã¿ã¯ãµã¼ãã«VPNãéãã¦fluentdã§éä¿¡ããã¾ãï¼ ãã¼ã¿ã¯ãµã¼ãå´ã«ã¨ããããmongodb使ã£ã¦è²¯ãã¦ãã¾ãï¼ãã®ãã¡GrowthForecastãªã©å©ç¨ãã¦å¯è¦åããã
Twilioï¼ãã¥ã¤ãªãªï¼ã¨ãã話é¡ã®é»è©±APIãµã¼ãã¹ããåãã§ããããã ãããFluentdã«çµã¿è¾¼ããã©ã°ã¤ã³ãfluent-plugin-twilioããä½æã»å ¬éãã¾ããã Boundioã¨ããé»è©±APIãµã¼ãã¹ãå»æ¢ã¨ãªããè¡ãå ´ã失ã£ã¦ããæ¹ã«ãæå ±ã§ãã fluent-plugin-twilio https://github.com/y-ken/fluent-plugin-twilio Fluentd Output plugin to make a call with twilio. Twilioã¨ã¯ Web APIãå©ç¨ããé»è©±ãæããããåãããã¨ã®åºæ¥ããµã¼ãã¹ã§ãã TwiMLã¨ããXMLè¨èªãç¨ãã大å¤èªç±åº¦ã®é«ãåä½ãå®ç¾©ãããã¨ãåºæ¥ã¾ãã æååã¨ãã¦æ¸¡ããæ¼¢åãå«ãæ¥æ¬èªãããé³å£°åæããã¦èªã¿ä¸ããæ©è½ãåãã£ã¦ãã¾ãã Twilioã®æéï¼2013å¹´
fluentd ã®ãã©ã°ã¤ã³ã®ãã¹ãã§ãä»ã®outputãå©ç¨ããã¿ã¤ãã®ãã©ã°ã¤ã³ã¯ã©ããã¹ãããã®ããªãã¨æã£ã¦ããã¹ã¦ã¿ãã@tagomoris ããã® out_forest ã¯ã¡ããã¨ãã¹ããåå¨ããã®ã§ãããèªãäºã«ã https://github.com/tagomoris/fluent-plugin-forest/blob/master/test/plugin/test_out_forest.rb#L416 ã¾ã fluentd ã®ãã©ã°ã¤ã³ã®ãã¹ãã®æ¸ãæ¹ããç解ãã¦ããªãã®ã§ç解ããªããã°ãã¨ å ¬å¼ããã¥ã¡ã³ã ãèªãã¨: Please see Fluentdâs source code for details. ãã£ãã¯ã⦠ã¾ãåæºåã« Fluent::Test.setup ãå¼ã°ãªãã¨ãããªãã£ã½ãããããengine.rb ã«æ¸ãã¦ããæçºã§grepããªãã¨è¦ã¤ã
ä¸ã¤ã®ãµã¼ãã¹ã§ãè² è·åæ£ã®ããã«è¤æ°ã®Webãµã¼ãï¼nginxï¼ãéç¨ãã¦ããå ´åãã¢ã¯ã»ã¹ãã°ãªã©ããã©ãã©ã«æ£ãã°ã£ã¦ãã¦ããã°ãããªã«ã調ã¹ããã¨æã£ãããçµ±è¨ãã¨ã£ããããã®ãé¢åã ã£ãããã¾ãã ããã§ãããæè¿ã¯ãã£ã¦ããfluentdãå°å ¥ãã¦ã¿ã¾ããããããéãç°¡åã ã¤ã¡ã¼ã¸ã¯ãããªæãã fluentdã®ã¤ã³ã¹ãã¼ã« ãã°ãéä¿¡ããå´ã¨ãåä¿¡ããå´ã両æ¹ã«fluentdãå ¥ã£ã¦ããå¿ è¦ãããã¾ãã ä»åã¯AWSã®ãã£ã¹ããªã ã£ãã®ã§ãyumçµç±ã§ã¤ã³ã¹ãã¼ã«ãå ¬å¼ãµã¤ãã«è¨è¼ããã¦ããä¸è¨ã³ãã³ãä¸çºã§å®äºã§ããä»ã«ãapt-get / gem / brewã§ãç°¡åã«å ¥ãããã¾ãã curl -L http://toolbelt.treasure-data.com/sh/install-redhat.sh | sh Installing Fluentd Using r
ã´ã¯ãã®å¤§å¹³ã§ãããããããã¾ã§ãã Redisã¯é«éã§ããã¤ãã¼ã¿ã®æ°¸ç¶åããè¤æ°ã®ãã¼ã¿åã«ããã¹ãã¢(list,set,sorted setçï¼ã対å¿ãã¦ãããæ©è½çãè±å¯ã¨ãããã¨ããæç¨è ã®å¤ãKVSå®è£ ã®ä¸ã¤ã ã¨æãã¾ãã ç¹ã«ç§ã®ãããªã¢ããªã±ã¼ã·ã§ã³ã¨ã³ã¸ãã¢ã®äººéã«ã¨ã£ã¦ã¯ãã¼ã¿åã®ããªã¨ã¼ã·ã§ã³ã®è±å¯ãã便å©ããæããé¨åã§ããã¨ãã°listãç¨ãã¦ã¿ã¤ã ã©ã¤ã³çãªæ å ±ãå±¥æ´æ å ±ã®ç®¡çãsorted setãç¨ãã¦ã©ã³ãã³ã°æ å ±ã®ç®¡çããªã©ã®ããã«ã¢ããªã±ã¼ã·ã§ã³ã®éè¦ã®å¤ãã«Redisã対å¿ãããã¨ãã§ãã¾ãã ãããã®æ å ±ãç»é²ããéã®ããã¼ã¨ãã¦ã¯èªä½ã®ã¢ããªã±ã¼ã·ã§ã³ããç´æ¥ãã¨ããã±ã¼ã¹ãå¤ãã¨æãã¾ããããã£ããFluentdã®ãããªä¾¿å©ãªlog collectorå®è£ ãããã®ã§ãFluentdã¨Redisãçµã¿åãããäºã§ã«ã¸ã¥ã¢ã«ã«æ å ±ã®èç©ãè¡ããã
miyagawaããã®Podcast Rebuild: 19ã§Kibanaã®è©±ããã£ã¦ã¡ãã£ã¨çãä¸ããå§ãã¦ãã®ã§ãç°¡åã«åä½ã試ãããµã³ãã«ã¢ããªã»ãããä½ã£ã¦ã¿ã¾ããã https://github.com/y310/kibana-trial git cloneãã¦READMEã«æ¸ãã¦ããæé ãå®è¡ãã¦ããã¨å¤§ä½åãã¨æãã¾ãã railsããfluentdã«ãã°ãéãé¨åã¯ããããªæãã§rack middlewareã使ã£ã¦éãã¾ãã # application_controller.rb class ApplicationController < ActionController::Base around_filter :collect_metrics def collect_metrics yield # ensureã使ãã®ã¯ä¾å¤æã®ãã°ãææãããã ensure # co
ããã«ã¡ã¯ã@jedipunkz ã§ãã èªååã®æµããæ¤è¨ããä¸ã§ãã°è§£æãå¿ãã¦ã¯ãããªãã¨ãããã¨ã§ ElasticSearch ã使ããããªãã¨ã¼ãããèãã¦ã㦠Logstash ã¨ã Kibana ã¨ããããã¼ã¯ã¼ããç® ã«æ¢ã¾ãããã«ãªã£ã¦ãã¾ããã ElasticSaerch 㯠API ã§æ å ±ãæ¤ç´¢åºæ¥ããããã®ã§èªååã«ãã£ã¦ãããããã¯ã¨ ã³ãã« Logstash ã使ã£ã¦â¦ ã¨æã£ã¦ãã®ã§ããæè¿ããèãããã«ãªã£ã fluentd ãããããçé¢ç®ã«ä½¿ãããï¼ã¨ãããã¨ã§ãä»å㯠Kibana + ElasticSearch + fluentd ã®çµã¿åããã§ãã°è§£æã·ã¹ãã ãçµãæ¹æ³ãã¡ã¢ãã¦ããã¾ãã åèã«ããã¦é ãã URL http://memocra.blogspot.jp/2013/04/kibanakibanaelasticsearchfluent
ããä½çªç ãã ï¼ã£ã¦ãããæ¸ããã¦ã¾ãããã³ãªãããæ¸ãã¦ã¿ã¾ããã Elasticsearchï¼Kibanaã®ç°å¢ãä½ã£ã¦ãã¿ã ã¿ã ããã®ãã°çæãã¼ã«ããApacheã®ããã¼ãã°ãæµãããã§å ¥ãã¦ã¿ã¾ããã åèURL memorycraftããã®ããã° Kibana Elasticsearch fluentd apache-loggen ã¤ã³ã¹ãã¼ã«ã¨èµ·å ä»åã¯CentOSã¸ã®ã¤ã³ã¹ãã¼ã«ã§ãã åºæ¬çã«ã¯memorycraftããã®ããã°ã®æµãã®ã¾ã¾ã§ãã elasticserchã®ã¤ã³ã¹ãã¼ã«ã¨èµ·å ãã¦ã³ãã¼ããã¦ãèµ·åããã ãã ã試ãã¨ãããã¨ã§ã-fãªãã·ã§ã³ã«ã¦ã³ã³ã½ã¼ã«ã«ãã°åºåã curl -OL https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.1.
以åç´¹ä»ããè¨äºãsplunkã£ã¦ãªãããï¼ï¼splunk enterpriseã使ã£ã¦ã¿ãï¼ãã§ã¯ splunk enterpriseã 使ã£ã¦ã¿ã¾ããã ä»åããã°è§£æãã©ãããã¼ã ã§ãããKibanaã使ã£ã¦ã¿ã¾ãã Kibanaã¯æ¤ç´¢ãªã©ã«Elasticsearchãå©ç¨ãã¾ãã ã¾ãKibanaã¯ãã¼ã¿ã®åéã«Logstashã®å©ç¨ãæ¨å¥¨ãã¦ãã¾ãã ãããã以ä¸ã®ãããªãããã¯ãã§ãã âLogstash ãã°ãã¼ã¿ãåéãã解æãã¦ä¿åãã¾ãã ãã®çµã¿åããã§ä½¿ç¨ããå ´åãä¿åå ã¯elasticsearchã«ãªãã¾ãã âElasticsearch ãªã¢ã«ã¿ã¤ã ãã¼ã¿ã®å ¨ææ¤ç´¢ãçµ±è¨ãªã©ãRestfulã¤ã³ã¿ã¼ãã§ã¼ã¹ã§æä¾ãã¾ãã âKibana ãã¼ã¿ã®æ å ±ãæç»ããæ¤ç´¢ãããããªã«ãã¦ã³ã§æ å ±ããã©ããGUIã¢ããªã±ã¼ã·ã§ã³ã§ãã ãã®3ã¤ãçµã¿åããã¦ä½¿ç¨ããã¨ä¾¿å©
ã©ã³ãã³ã°
ã©ã³ãã³ã°
ã©ã³ãã³ã°
ãªãªã¼ã¹ãé害æ å ±ãªã©ã®ãµã¼ãã¹ã®ãç¥ãã
ææ°ã®äººæ°ã¨ã³ããªã¼ã®é ä¿¡
å¦çãå®è¡ä¸ã§ã
j次ã®ããã¯ãã¼ã¯
kåã®ããã¯ãã¼ã¯
lãã¨ã§èªã
eã³ã¡ã³ãä¸è¦§ãéã
oãã¼ã¸ãéã
{{#tags}}- {{label}}
{{/tags}}