分析机制：
自动化实现每日凌晨2点分析前一天的日志，并把结果记录用nginx来实现访问.如访问https://devops.xx.xxx/xyapi1/20240928，即访问的是20240928 业务日志分析情况。
分析日志内容：
nginx服务器下配置多个域名，对当前nginx所有的请求进行分析，比如统计请求数有多少/有效的请求/失败的请求/独立访客/日志大小/数据传输量等; 异常的url请求；根据点击量列出对应的IP及所对应的城市；有没有异常的refer请求域名接口等数据。
1.日志格式定义如下
{"remote_addr":"100.89.1.19","X-Forwarded-For":"220.163.193.10, 172.16.0.176","remote_user":"","time_local":"30/Sep/2024:14:15:00 +0800","method":"GET","uri":"/api/xy/toc/v1/queryContentByDocId","request_uri":"/api/xy/toc/v1/queryContentByDocId?companyId=xxxx&userId=xxxxx&appCode=FABU_YUNSHI&productId=bstv","server_protocol":"HTTP/1.1","request_body":"","status":"200","body_bytes_sent":"1325","http_referer":"https://mtydazzle.xxxx.xxxx/","user_agent":"Mozilla/5.0 (Linux; Android 12; GLA-AL00 Build/HUAWEIGLA-AL00; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/126.0.6478.188 Mobile Safari/537.36 XWEB/1260117 MMWEBSDK/20240501 MMWEBID/5902 MicroMessenger/8.0.50.2701(0x2800325B) WeChat/arm64 Weixin NetType/WIFI Language/zh_CN ABI/arm64","upstream_response_time":"","upstream_cache_status":"STALE"}2.goaccess 部署
goaccess安装：
yum install libmaxminddb-devel
tar -zxvf goaccess-1.9.2.tar.gz
cd goaccess-1.9.2/
./configure --prefix=/opt/goaccess --enable-utf8 --enable-geoip=mmdb
make -j 4
make install
城市地图下载：
cd /opt/goaccess/etc/goaccess
wget https://download.db-ip.com/free/dbip-city-lite-2024-06.mmdb.gz
gunzip dbip-city-lite-2024-06.mmdb.gz
日志处理配置及地图库加载：
cat /opt/goaccess/etc/goaccess/goaccess.conf
time-format %H:%M:%S
date-format %d/%b/%Y
log-format %^:%^,%^:"~h{, }",%^:%^,%^:"%d:%t %^",%^:"%m",%^:"%U",%^:%^,%^:"%H",%^:%^,%^:"%s",%^:"%b",%^:"%R",%^:"%u",%^:%^,%^:%^
geoip-database /opt/goaccess/etc/goaccess/dbip-city-lite-2024-06.mmdb3.定时任务执行
run_goaccess.sh
#!/bin/bash
export LANG="zh_CN.UTF-8"
yesterday=$(date -d "yesterday" +%Y-%m-%d)
yesterday_file_date=$(date -d "yesterday" +%Y%m%d)
/opt/goaccess/bin/goaccess /data/logs/nginx/xyinterface_$yesterday.log -p /opt/goaccess/etc/goaccess/goaccess.conf -o /datanfs/ftp/nginx/172.16.0.70/goaccess-$yesterday_file_date.html00 01 * * * /opt/goaccess/run_goaccess.sh >/dev/null 2>&14.nginx 访问页面配置
location ~ ^/xyapi1/(d+)$ {
root /datanfs/ftp/nginx/172.16.0.70;
try_files /goaccess-$1.html =404;
}