鐃緒申鐃緒申鐃醇く鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申發削申帖鐃�

鐃緒申蠅随申鐃緒申鐃緒申函鐃�
鐃緒申鐃緒申鐃緒申鐃縮なワ申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申fluentd鐃緒申鐃縮わ申鐃銃￥申elasticsearch鐃祝わ申鐃曙た鐃緒申鐃緒申
鐃緒申鐃緒申鐃銃最緒申的鐃祝わ申Kibana鐃叔みわ申鐃緒申鐃緒申諭鐃�


鐃緒申襪鰹申函鐃�
鐃緒申鐃緒申鐃出とデ￥申鐃緒申鐃塾なわ申鐃緒申
鐃緒申Web鐃緒申鐃緒申鐃出」￥申屮鐃緒申鐃緒申鐃緒申鵐機鐃緒申弌廖鐃緒申elasticsearch+kibana鐃塾ワ申鐃緒申鐃出￥申
鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃術ワ申鐃緒申鐃出では逸申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃夙ってわ申鐃緒申鐃殉わ申鐃緒申

鐃緒申Web鐃緒申鐃緒申鐃出のワ申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申LTSV鐃緒申鐃緒申鐃祝わ申鐃暑。
鐃緒申鐃緒申鐃緒申鐃緒申鐃藷サ￥申鐃出では￥申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申存鐃緒申鐃銃￥申鐃緒申鐃緒申的鐃祝はわ申鐃祝ょご鐃祝わ申彁鐃緒申箸鐃緒申發件申鐃緒申鐃�
鐃緒申鐃緒申鐃緒申鐃緒申鐃藷サ￥申鐃出わ申鐃緒申fluentd鐃緒申鐃縮わ申鐃緒申elasticsearch鐃祝ワ申鐃緒申檗鐃緒申鐃�
鐃緒申elasticsearch+kibana鐃緒申鐃緒申鐃出は種申鐃緒申妨鐃緒申鐃�

鐃緒申鐃緒申戮襪鰹申函鐃緒申錣�鐃緒申覆鐃緒申鐃緒申函鐃�
鐃緒申fluentd鐃緒申鐃緒申鐃緒申侫鐃緒申鐃緒申鐃塾緒申鐃緒申
鐃緒申elasticsearch鐃塾ワ申鐃緒申鐃緒申鐃殉テワ申廛譟種申箸僚鐃緒申鐃緒申鐃�
鐃緒申鐃緒申鐃緒申鐃緒申鐃所ス鐃夙ワ申鵐阿鐃淑�鐃緒申鐃緒申鐃書きわ申鐃緒申討曚鐃緒申鐃緒申覆鐃緒申鐃�
鐃緒申鐃緒申index鐃熟￥申logtash-YYYYMMDD鐃竣とわ申鐃緒申鐃緒申鐃緒申type鐃熟「ワ申鐃緒申鐃俊ワ申名_access_log鐃竣とわ申鐃緒申鐃緒申鐃緒申
鐃緒申鐃緒申1鐃縦のワ申鐃緒申妊奪鐃緒申鐃緒申砲鐃緒申鐃暑こ鐃夙で￥申鐃緒申鐃緒申佞僚鼎鐃宿緒申鐃緒申箸鐃緒申鐃緒申任鐃緒申鐃塾わ申鐃緒申鐃藷そわ申鐃緒申鐃緒申鐃緒申鐃淑と￥申鐃緒申鐃緒申鐃竣リケ鐃緒申鐃緒申鐃緒申鐃熟のワ申鐃緒申鐃緒申鼎佑討澆鐃緒申箸茲わ申鐃粛￥申

鐃緒申襪鰹申箸鐃緒申鐃�1鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申泙鐃緒申鐃�LTSV鐃緒申鐃緒申鐃祝わ申鐃緒申鐃緒申鐃緒申

LTSV鐃塾件申鐃緒申鐃熟わ申鐃緒申粉鐃緒申鐃緒申砲鐃緒申鐃緒申鐃�
LogFormat "domain:%V\thost:%h\tserver:%A\tident:%l\tuser:%u\ttime:%{%d/%b/%Y:%H:%M:%S %z}t\tmethod:%m\tpath:%U%q\tprotocol:%H\tstatus:%>s\tsize:%b\treferer:%{Referer}i\tagent:%{User-Agent}i\tresponse_time:%D" combined


鐃緒申鐃緒申鐃緒申鐃�Y-ken鐃緒申鐃緒申離屮鐃緒申鐃緒申鮖温佑砲鐃緒申鐃緒申討鐃緒申鐃緒申鐃緒申鐃緒申泙鐃緒申鐃緒申鐃�
Apache鐃緒申鐃緒申鐃緒申LTSV鐃緒申鐃緒申鐃祝わ申鐃緒申櫃裡鐃緒申弔鐃緒申鐃夙わ申鐃緒申鐃緒申仆鐃祝￥申鐃�Apache鐃緒申Fluentd鐃緒申LTSV鐃緒申鐃所サ鐃緒申廛鐃�

鐃緒申鐃緒申鐃緒申鐃緒申鐃塾￥申鐃緒申暴颪い討鐃緒申泙鐃緒申鐃塾わ申鐃緒申鐃緒申鐃初し鐃緒申鐃叔わ申鐃緒申鐃緒申鐃緒申鐃銃わ申鐃縦わ申鐃緒申鐃緒申辰討鐃緒申泙鐃緒申鐃�

httpd.conf鐃塾逸申鐃緒申抜鐃処。
status.html鐃塾ヘルス鐃緒申鐃緒申鐃獣ワ申鐃術のワ申鐃緒申鐃熟出なわ申鐃緒申鐃緒申鬚靴討鐃緒申鐃塾わ申
LogFormat鐃祝誌申鐃所し鐃緒申鐃夙わ申鐃緒申暴鐃緒申呂鐃緒申鐃処う鐃祝と￥申


SetEnvIf Request_URI "status.html" nolog

LogFormat "domain:%V\thost:%h\tserver:%A\tident:%l\tuser:%u\ttime:%{%Y/%m/%d %H:%M:%S %z}t\tmethod:%m\tpath:%U%q\tprotocol:%H\tstatus:%>s\tsize:%b\treferer:%{Referer}i\tagent:%{User-Agent}i\tresponse_time:%D" combined_lts

CustomLog  /var/www/sss/redirect/logs/apache_access.log  combined_lts env=!nolog


鐃緒申鐃緒申Cookie鐃熟はふわ申鐃緒申討鐃緒申泙鐃緒申鵝７鐃縮￥申鐃淑駕申鐃塾よう鐃淑ワ申鐃緒申鐃叔緒申鐃熟わ申鐃緒申鐃粛緒申蝓�
domain

host

server

ident 

user

time

method

path

protocol

status

size

referer

agent

response_time


鐃緒申襪鰹申箸鐃緒申鐃�2鐃緒申Web鐃緒申鐃緒申鐃出に￥申fluent-agent-lite鐃竣をインスト￥申鐃緒申鐃緒申鐃緒申

fluent-agent-lite
#fluentd 鐃術ワ申鐃緒申鐃緒申鐃緒申鐃緒申鐃術のワ申鐃緒申鐃緒申鐃緒申鐃緒申鐃� fluent-agent-lite 鐃書いわ申

鐃緒申鐃藷スト￥申鐃緒申鐃緒申飽焚鐃緒申鬚い鐃淑わ申鐃夙￥申鐃緒申鐃緒申幣鐃順が鐃緒申鐃緒申泙鐃緒申鐃緒申鐃�
<
yum -y install perl-ExtUtils-MakeMaker



vim /etc/fluent-agent-lite.conf
apache.access.hoge01 鐃夙わ申鐃緒申鐃緒申鐃駿ワ申肇鐃緒申鐃緒申鐃緒申覆鐃緒申鐃銃わ申鐃緒申鐃熟わ申鐃叔わ申鐃粛￥申

# fluentd tag prefix of all LOGS

TAG_PREFIX="apache"



LOGS=$(cat <<"EOF"

access.hoge01 /var/www/sss/redirect/logs/apache_access.log 

EOF

)



# SERVERNAME[:PORTNUM]

# port number is optional (default: 24224)

PRIMARY_SERVER="192.168.x.x:24224"


余鐃縮でわ申鐃緒申鐃緒申fluent-agent-lite鐃緒申fluent-agent鐃塾居申動鐃塾削申鐃祝つわ申鐃銃細わ申鐃緒申鐃緒申鐃述わ申鐃緒申鐃緒申泙鐃緒申鐃緒申鐃�
http://blog.livedoor.jp/sonots/archives/24617519.html

鐃緒申襪鰹申箸鐃緒申鐃�3鐃緒申鐃緒申鐃緒申鐃緒申鐃藷サ￥申鐃出に￥申fluentd鐃竣とプラグ鐃緒申鐃藷インスト￥申鐃暑、鐃緒申鐃緒申

fluentd鐃緒申鐃藷スト￥申鐃暑し鐃殉わ申鐃緒申

鐃竣ラグ鐃緒申鐃藷インスト￥申鐃暑し鐃殉わ申鐃緒申
 /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-elasticsearch

 /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-parser

 /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-file-alternative


鐃緒申鐃緒申侫鐃緒申鐃緒申鐃緒申鐃緒申鐃銃でわ申鐃緒申
鐃叔初、鐃緒申鐃塾まわ申elasticsearch鐃祝デ￥申鐃緒申鐃緒申流鐃緒申鐃夙￥申message鐃緒申鐃緒申鐃緒申鐃緒申砲呂鐃緒申辰鐃緒申泙鐃殉ワ申鐃緒申妊奪鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃銃わ申鐃殉って￥申鐃緒申鐃緒申鐃緒申鐃緒申鐃夙わ申鐃竣ってわ申鐃緒申鐃塾でわ申鐃緒申鐃緒申
鐃緒申鐃緒申鐃緒申蠅刻申鐃緒申鵑�￥申鐃緒申鐃宿バワ申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃殉わ申鐃緒申鐃緒申



鐃緒申鐃緒申埜鐃塾�鐃処く鐃緒申鐃緒申鐃緒申鐃塾わ申鐃緒申鐃緒申鐃緒申鐃緒申鐃夙思わ申鐃縦つ以駕申鐃叔でわ申鐃殉わ申鐃緒申鐃緒申



<source>

鐃緒申type forward

鐃緒申port 24200

</source>

<match apache.access.**>

鐃緒申type copy

鐃緒申<store>

鐃緒申鐃緒申type file_alternative

鐃緒申鐃緒申path /tmp/httpd/access.*.log

鐃緒申鐃緒申time_slice_format %Y%m%d

鐃緒申鐃緒申compress gzip

鐃緒申鐃緒申output_include_time false

鐃緒申鐃緒申output_include_tag false

鐃緒申鐃緒申output_data_type attr:message

鐃緒申鐃緒申add_newline true

鐃緒申</store>

鐃緒申<store>

鐃緒申鐃緒申type parser

鐃緒申鐃緒申add_prefix parsed

鐃緒申鐃緒申format ltsv

鐃緒申鐃緒申time_format %d/%b/%Y:%H:%M:%S %z

鐃緒申鐃緒申key_name message

鐃緒申</store>

</match>

<match parsed.apache.access.**>

鐃緒申type copy

鐃緒申<store>

鐃緒申鐃緒申type elasticsearch

鐃緒申鐃緒申index_name adminpack

鐃緒申鐃緒申type_name redirect_apache

鐃緒申鐃緒申include_tag_key true

鐃緒申鐃緒申tag_key @log_name

鐃緒申鐃緒申host 192.168.xx.xx

鐃緒申鐃緒申port 9200

鐃緒申鐃緒申logstash_format true

鐃緒申鐃緒申flush_interval 3s

鐃緒申</store>

</match>



鐃遵記鐃塾よう鐃緒申鐃緒申鐃緒申鐃�elasticsearch鐃緒申流鐃緒申鐃緒申鐃緒申函鐃緒申焚鐃緒申里茲�鐃淑ワ申鐃緒申妊奪鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃殉わ申鐃緒申



Sense鐃叔醐申鐃緒申



xxx.xxx.xxx.xxxx:9200/logstash-2013.12.05/redirect_apache

GET _search

{

   "query": {

      "match_all": {}

   }

}




{

   "took": 1,

   "timed_out": false,

   "_shards": {

      "total": 5,

      "successful": 5,

      "failed": 0

   },

   "hits": {

      "total": 1016,

      "max_score": 1,

      "hits": [

         {

            "_index": "logstash-2013.12.05",鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申妊奪鐃緒申鐃�

            "_type": "redirect_apache",鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申type鐃緒申鐃緒申鐃緒申

            "_id": "MB-NARl7Tpm4pXqEvbBTtA",

            "_score": 1,

            "_source": {

               "domain": "hogehoge.co.jp",

               "host": "xxx.xxx.0.176",

               "server": "xxx.xxx.11.175",

               "ident": "-",

               "user": "-",

               "method": "GET",

               "path": "/favicon.ico",

               "protocol": "HTTP/1.1",

               "status": "301",

               "size": "328",

               "referer": "-",

               "agent": "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Win64; x64; Trident/6.0)",

               "response_time": "142",

               "time": "2013/12/05 09:53:28 +0900",

               "@log_name": "parsed.apache.access.hoge",鐃緒申鐃緒申tag_key鐃緒申鐃所し鐃緒申名鐃緒申

               "@timestamp": "2013-12-05T09:53:28+09:00"

            }

         },


鐃緒申襪鰹申箸鐃緒申鐃�4鐃緒申elasticsearch+kibana鐃薯インスト￥申鐃暑、鐃緒申鐃緒申

鐃緒申elaticsearch鐃塾ワ申鐃藷スト￥申鐃緒申
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.6.tar.gz

tar -xf elasticsearch.tar.gz
rm elasticsearch.tar.gz
mv elasticsearch-* elasticsearch
sudo mv elasticsearch /usr/local/share


鐃緒申鐃叔￥申鐃緒申鷁修鐃緒申撞鐃銃逸申鐃緒申鐃� 
curl -L http://github.com/elasticsearch/elasticsearch-servicewrapper/tarball/master | tar -xz



mv *servicewrapper*/service /usr/local/share/elasticsearch/bin/



rm -Rf *servicewrapper*



sudo /usr/local/share/elasticsearch/bin/service/elasticse



鐃緒申kibana鐃塾ワ申鐃藷スト￥申鐃緒申
tar展鐃緒申鐃緒申鐃銃ドワ申鐃緒申鐃緒申肇襦種申箸鐃緒申屬鐃緒申鐃緒申鐃緒申鐃�
http://www.elasticsearch.org/overview/kibana/installation/

wget https://download.elasticsearch.org/kibana/kibana/kibana-3.0.0milestone4.tar.gz


鐃処う鐃巡く鐃緒申鐃緒申鐃緒申鐃緒申
http://xxx.xxx.xxx.xxx/kibana/index.html#/dashboard/file/logstash.json


鐃緒申鐃藷スト￥申鐃緒申亮鐃緒申覆鵑�はわ申鐃緒申鐃緒申鐃緒申任討鐃緒申鐃塾でわ申鐃緒申鐃緒申鮖温佑砲鐃緒申討鐃緒申鐃緒申鐃緒申鐃緒申鐃�
鐃緒申鐃夙￥申鐃緒申鐃緒申離鐃緒申鐃緒申个鐃緒申鐃�Firewall鐃夙わ申iptables鐃夙わ申鐃緒申鐃緒申鐃所が鐃緒申鐃緒申里如鐃�
鐃緒申認鐃緒申鐃緒申必鐃竣なポ￥申鐃夙なんかは誌申鐃緒申鐃祝わ申鐃緒申鐃銃わ申鐃緒申鐃殉わ申鐃順う鐃緒申

鐃殉わ申鐃叔わ申鐃銃なわ申鐃緒申鐃緒申
鐃緒申鐃盾い鐃獣わ申鐃所、鐃緒申侫鐃緒申蕁�鐃醇ー鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申箸鐃淑�鐃緒申鐃緒申鐃書きわ申鐃緒申討鐃緒申襦�orz
鐃緒申鐃緒申鐃緒申蓮鐃�Mapping鐃薯いわ申襪鰹申箸如鐃緒申鐃処し鐃殉わ申鐃緒申鐃緒申
elasticsearch鐃緒申鐃塾わ申里鐃緒申鐃緒申拔鐃緒申覆里如鐃�sense鐃叔ワ申鐃緒申鐃所が鐃緒申鐃緒申鐃銃わ申鐃淑わ申鐃緒申orz
鐃夙わ申鐃緒申鐃緒申鐃夙で￥申鐃緒申鐃縮までわ申鐃緒申鐃塾でわ申鐃緒申鐃緒申elasticsearch鐃緒申鐃緒申鐃緒申鐃緒申鐃駿駈申鐃緒申鐃銃わ申鐃殉わ申鐃緒申


鐃殉わ申鐃殉わ申鐃緒申蠅随申鐃緒申鐃緒申鐃�
鐃緒申鐃祝わ申鐃殉わ申砲鐃緒申泙辰鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃�LTSV鐃緒申鐃緒申鐃祝なわ申鐃緒申鐃銃￥申fluentd鐃緒申elasticsearch鐃祝ワ申鐃緒申檗鐃緒申箸鐃緒申鐃緒申鐃緒申鐃�
鐃緒申鐃塾常申隋�鐃緒申鐃緒申妊奪鐃緒申鐃緒申鬚修鐃緒申鐃緒申悩鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃宿￥申鐃緒申鐃緒申鐃緒申fluentd鐃叔誌申鐃緒申任鐃緒申鐃緒申蠅刻申鐃塾わ申鐃淑￥申
鐃緒申鐃緒申鐃出￥申鐃緒申鐃緒申鐃緒申鐃淑ｏ申鐃緒申侫鐃緒申鐃緒申襪�鐃緒申鐃緒申里如鐃緒申鐃緒申腓件申鐃�LTSV鐃祝わ申鐃銃￥申fluentd鐃緒申流鐃緒申鐃夙わ申鐃祝￥申鐃緒申鐃緒申蓮鐃�2012/09/08鐃竣のワ申鐃緒申妊奪鐃緒申鐃緒申鐃緒申茵�鐃夙わ申鐃緒申鐃獣わ申流鐃緒申鐃緒申鐃淳わ申鐃緒申鐃緒申

鐃緒申鐃夙￥申鐃緒申鐃術わ申鐃緒申鐃緒申鐃緒申鐃緒申未鐃緒申匹里鐃緒申蕕わ申埜続鐃緒申佑砲覆鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申(kibana鐃緒申鐃重わ申鐃夙わ申鐃粛￥申
鐃緒申辰討澆覆鐃緒申箸錣�鐃緒申覆鐃緒申鐃緒申福鐃�


鐃夙わ申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申鐃緒申里悩鐃緒申鐃緒申単鐃緒申鐃緒申鐃縮わ申鐃緒申鐃緒申鐃緒申鐃叔わ申鐃緒申鐃緒申
elasticsearch鐃夙わ申鐃緒申鐃緒申妊奪鐃緒申鐃緒申旅鐃渋わ申砲弔鐃緒申鐃縦器申戮泙鐃緒申鐃�
平鐃峻わ申鐃緒申fluentd鐃緒申鐃緒申鐃緒申鐃夙わ申鐃竣わ申鐃縦つ￥申鐃緒申鐃緒申鐃緒申務鐃緒申鐃宿わ申鐃縦つなので￥申
鐃緒申鐃緒申鐃殉でわ申鐃緒申里某鐃緒申鐃緒申屬虜鐃緒申擇鐃緒申鐃瞬わ申鐃竣わ申鐃殉わ申鐃緒申鐃緒申orz

頭鐃祝ワ申鐃緒申謄襪わ申譴随申鐃緒申鐃�

鐃緒申呂蝓�鐃緒申鐃緒申鐃宿ワ申鐃緒申鐃緒申箸離鐃春ワ申鐃緒申鵐垢鐃緒申弔鼎鐃緒申匹犂駈申鐃緒申任鐃緒申鐃緒申諭鐃緒申鐃�
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index.html