请问多台服务器的nginx日志如何进行实时分析?
网友回复
http { include mime.types; default_type application/octet-stream; log_format json '{"@timestamp":"$time_iso8601",' '"host":"$server_addr",' '"clientip":"$remote_addr",' '"request":"$request",' '"size":$body_bytes_sent,' '"responsetime":$request_time,' '"upstreamtime":"$upstream_response_time",' '"upstreamhost":"$upstream_addr",' '"http_host":"$host",' '"url":"$uri",' '"referer":"$http_referer",' '"agent":"$http_user_agent",' '"status":"$status"}'; access_log /data/nginx/logs/access_json.log json; server_names_hash_bucket_size 128; client_header_buffer_size 32K; large_client_header_buffers 4 32k;Logstash安装要求JRE 7版本及以上,否则部分工具无法使用。 点击下载Logstash 2.4.1,下载地址 https://www.elastic.co/cn/downloads/past-releases/logstash-2-4-1 通过如下命令解压即可使用: $ tar -xzvf logstash-2.4.1.tar.gz $ cd logstash-2.4.1
配置logstash配置文件配置logstash配置文件:
vim logstash_nginxlog2print.confinput { file { path => ['/data/nginx/logs/access_json.log'] start_position => "beginning" codec => "json" tags => ['user'] type => "nginx" } } output { if [type] == "nginx" { redis { host => "172.17.0.90" port => "6379" key => "nginx" db => "10" data_type => "list" } } }
logstash -f logstash_nginxlog2print.conf
直接将nginx日志输出到redis的list队列中了,当然还可以输出到阿里云的datahub的实时数据处理中心中进行可视化bi分析,这个output还支持插件自定义