回答

可以使用Logstash结合redis flink等存储来进行分析

先将nginx的日志输出改成json格式的

http {
include mime.types;
default_type application/octet-stream;
log_format json '{"@timestamp":"$time_iso8601",'
'"host":"$server_addr",'
'"clientip":"$remote_addr",'
'"request":"$request",'
'"size":$body_bytes_sent,'
'"responsetime":$request_time,'
'"upstreamtime":"$upstream_response_time",'
'"upstreamhost":"$upstream_addr",'
'"http_host":"$host",'
'"url":"$uri",'
'"referer":"$http_referer",'
'"agent":"$http_user_agent",'
'"status":"$status"}';
access_log /data/nginx/logs/access_json.log json;
server_names_hash_bucket_size 128;
client_header_buffer_size 32K;
large_client_header_buffers 4 32k;

Logstash安装要求JRE 7版本及以上，否则部分工具无法使用。
点击下载Logstash 2.4.1，下载地址 https://www.elastic.co/cn/downloads/past-releases/logstash-2-4-1

通过如下命令解压即可使用：

$ tar -xzvf logstash-2.4.1.tar.gz
$ cd logstash-2.4.1

配置logstash配置文件配置logstash配置文件：

vim logstash_nginxlog2print.conf 

input {
 file {
  path => ['/data/nginx/logs/access_json.log']
  start_position => "beginning"
  codec => "json"
  tags => ['user']
  type => "nginx"
 }
}
output {
  if [type] == "nginx" {
   redis {
    host => "172.17.0.90"
    port => "6379"
    key => "nginx"
    db => "10"
    data_type => "list"
   }
  }
}

logstash -f  logstash_nginxlog2print.conf 

直接将nginx日志输出到redis的list队列中了，当然还可以输出到阿里云的datahub的实时数据处理中心中进行可视化bi分析，这个output还支持插件自定义