CentOS7 部署ElastiFlow网络流量分析平台
(图片可点击放大查看)
(图片可点击放大查看)
(图片可点击放大查看)
本文参考如下链接完成
https://docs.elastiflow.com/docs/install_linux
https://cloud.tencent.com/developer/article/1648854
https://blog.csdn.net/weixin_43838503/article/details/122432963
https://blog.51cto.com/coolsky/3190806
(图片可点击放大查看)
条件准备
-
1、host-sflow agent
https://github.com/sflow/host-sflow/releases/download/v2.0.25-3/hsflowd-centos7-2.0.25-3.x86_64.rpm
-
2、ELK的安装包
elasticsearch-7.17.2-x86_64.rpm
kibana-7.17.2-x86_64.rpm
logstash-7.17.2-x86_64.rpm
具体安装部署步骤如下
一、调整相关的内核参数并关闭防火墙
sed -i 's/enable/disabled/g' /etc/selinux/config
setenforce 0
hostnamectl set-hostname elastiflow
echo "vm.max_map_count=262144" | sudo tee /etc/sysctl.d/70-elasticsearch.conf > /dev/null
echo -e "net.core.netdev_max_backlog=4096nnet.core.rmem_default=262144nnet.core.rmem_max=67108864nnet.ipv4.udp_rmem_min=131072nnet.ipv4.udp_mem=2097152 4194304 8388608" | tee /etc/sysctl.d/60-net.conf > /dev/null
sysctl -w vm.max_map_count=262144 && sysctl -w net.core.netdev_max_backlog=4096 && sysctl -w net.core.rmem_default=262144 && sysctl -w net.core.rmem_max=67108864 && sysctl -w net.ipv4.udp_rmem_min=131072 && sysctl -w net.ipv4.udp_mem='2097152 4194304 8388608'
systemctl stop firewalld.service && systemctl disable firewalld.service
(图片可点击放大查看)
(图片可点击放大查看)
二、安装JDK环境并安装ELK
yum install java-openjdk-devel java-openjdk
(图片可点击放大查看)
(图片可点击放大查看)
rpm -ivh logstash-7.17.2-x86_64.rpm
rpm -ivh elasticsearch-7.17.2-x86_64.rpm
rpm -ivh kibana-7.17.2-x86_64.rpm
(图片可点击放大查看)
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl enable kibana.service
systemctl enable logstash.service
(图片可点击放大查看)
三、Elasticsearch和Kibana配置文件修改
vim /etc/elasticsearch/elasticsearch.yml
(图片可点击放大查看)
vim /etc/elasticsearch/jvm.options
(图片可点击放大查看)
vim /etc/kibana/kibana.yml
(图片可点击放大查看)
(图片可点击放大查看)
四、elastiflow安装包解压
https://codeload.github.com/robcowart/elastiflow
(图片可点击放大查看)
mv elastiflow-master elastiflow
cp -a elastiflow/logstash/elastiflow/. /etc/logstash/elastiflow/
cp -a elastiflow/logstash.service.d/. /etc/systemd/system/logstash.service.d/
(图片可点击放大查看)
vim /etc/logstash/pipelines.yml
修改添加成如下
#- pipeline.id: main
# path.config: "/etc/logstash/conf.d/*.conf"
- pipeline.id: elastiflow
path.config: "/etc/logstash/elastiflow/conf.d/*.conf"
pipeline.workers: 4
(图片可点击放大查看)
(图片可点击放大查看)
chown -R logstash:logstash /etc/logstash/elastiflow
chown -R logstash:logstash /etc/logstash/pipelines.yml vi /etc/logstash/jvm.options
vim /etc/logstash/startup.options
(图片可点击放大查看)
(图片可点击放大查看)
(图片可点击放大查看)
/usr/share/logstash/bin/logstash-plugin install logstash-codec-sflow
/usr/share/logstash/bin/logstash-plugin install logstash-codec-netflow
/usr/share/logstash/bin/logstash-plugin install logstash-input-udp
/usr/share/logstash/bin/logstash-plugin install logstash-input-tcp
/usr/share/logstash/bin/logstash-plugin install logstash-filter-dns
/usr/share/logstash/bin/logstash-plugin install logstash-filter-geoip
/usr/share/logstash/bin/logstash-plugin install logstash-filter-translate
(图片可点击放大查看)
(图片可点击放大查看)
/usr/share/logstash/bin/system-install
(图片可点击放大查看)
五、启动服务
systemctl restart elasticsearch.service
systemctl restart kibana.service
systemctl restart logstash.service
(图片可点击放大查看)
六、Kibana配置
导入elastiflow.kibana.7.8.x.ndjson到kibana
(图片可点击放大查看)
(图片可点击放大查看)
修改如下kibana中的配置
(图片可点击放大查看)
(图片可点击放大查看)
七、Linux服务器安装及配置hsflow
rpm -ivh hsflowd-centos7-2.0.25-3.x86_64.rpm vim /etc/hsflowd.conf
添加如下配置
sampling = 1
#截取包大小为256
headerBytes = 256
#设置收集器地址和端口
collector { ip=192.168.31.189 udpport=6343 }
#设置采样的网卡
pcap { dev = ens33 }
systemctl enable hsflowd
systemctl start hsflowd
systemctl status hsflowd
(图片可点击放大查看)
(图片可点击放大查看)
八、效果测试截图
(图片可点击放大查看)
(图片可点击放大查看)
(图片可点击放大查看)
(图片可点击放大查看)
(图片可点击放大查看)
九、Tips
1、交换机配置sflow暂未进行测试
后续有环境会可以进行测试
2、若logstash启动失败
需要/var/log/logstash/logstash-plain.log 和journalctl -xe -u logstash进行排错
3、上面的kibana与ES未配置安全认证 后续有时间再做调整