一、基于Kibana实现RBAC实战案例

1.1 基于nginx的反向代理控制访问kibana

说明：这种方法比较鸡肋，别人不通过域名也可以直接访问kibana

1.1.1 搭建nginx环境

1、在elk123节点上添加yum源

[root@elk123 ~]# cat > /etc/yum.repos.d/nginx.repo <<'EOF'
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true
[nginx-mainline]
name=nginx mainline repo
baseurl=http://nginx.org/packages/mainline/centos/$releasever/$basearch/
gpgcheck=1
enabled=0
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true
EOF

2、在elk123节点上安装nginx

[root@elk123 ~]# yum -y install nginx
[root@elk123 ~]# systemctl start nginx

3、在浏览器输入http://192.168.1.123进行访问测试

1.1.2 基于nginx的反向代理控制访问kibana

1、在elk123节点上编写nginx的配置文件

[root@elk123 ~]# cat > /etc/nginx/conf.d/kibana.conf <<'EOF'
server {
listen 80;
server_name kibana.oldboyedu.com;
location / {
proxy_pass http://192.168.1.123:5601$request_uri;
auth_basic "oldboyedu kibana web!";
auth_basic_user_file conf/htpasswd;
}
}
EOF

2、在elk123节点上创建账号文件

[root@elk123 ~]# mkdir -pv /etc/nginx/conf
[root@elk123 ~]# yum -y install httpd-tools
[root@elk123 ~]# htpasswd -c -b /etc/nginx/conf/htpasswd admin oldboyedu

3、在elk123节点上

[root@elk123 ~]# nginx -t
[root@elk123 ~]# systemctl reload nginx

4、添加hosts解析(C:\Windows\System32\drivers\etc\hosts)

192.168.1.123 kibana.oldboyedu.com

5、打开浏览器输入http://kibana.oldboyedu.com/后，输入账号admin，密码oldboyedu

1.2 配置ES集群TSL认证

1、在elk121节点上生成证书文件

[root@elk121 ~]# cd /es/softwares/es7/elasticsearch-7.17.5/
[root@elk121 elasticsearch-7.17.5]# ./bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass "" --days 3650

2、在elk121节点上为证书文件修改属主和属组

[root@elk121 ~]# cd /es/softwares/es7/elasticsearch-7.17.5/
[root@elk121 elasticsearch-7.17.5]# chown es:es config/elastic-certificates.p12

3、在elk121节点上同步证书文件到其他节点

[root@elk121 ~]# cd /es/softwares/es7/elasticsearch-7.17.5/
[root@elk121 elasticsearch-7.17.5]# data_rsync.sh `pwd`/config/elastic-certificates.p12

4、所有节点上修改ES集群的配置文件

(1) 在elk121节点上修改ES集群的配置文件

在文件末尾处添加如下内容

[root@elk121 elasticsearch-7.17.5]# vim /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml
…
…
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

配置文件内容修改完成后，如下：

[root@elk121 elasticsearch-7.17.5]# egrep -v "^$|^#" /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml
cluster.name: es
path.data: /es/data/es7
path.logs: /es/logs/es7
network.host: 192.168.1.121
discovery.seed_hosts: ["elk121","elk122","elk123"]
cluster.initial_master_nodes: ["elk123"]
reindex.remote.whitelist: "192.168.1.*:19200"
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

(2) 在elk122节点上修改ES集群的配置文件

在文件末尾处添加如下内容

[root@elk122 ~]# vim /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml…
…
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

配置文件内容修改完成后，如下：

[root@elk122 ~]# egrep -v "^$|^#" /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml
cluster.name: es
path.data: /es/data/es7
path.logs: /es/logs/es7
network.host: 192.168.1.122
discovery.seed_hosts: ["elk121","elk122","elk123"]
cluster.initial_master_nodes: ["elk123"]
reindex.remote.whitelist: "192.168.1.*:19200"
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

(3) 在elk123节点上修改ES集群的配置文件

在文件末尾处添加如下内容

[root@elk123 ~]# vim /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml
…
…
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

配置文件内容修改完成后，如下：

[root@elk123 ~]# egrep -v "^$|^#" /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml
cluster.name: es
path.data: /es/data/es7
path.logs: /es/logs/es7
network.host: 192.168.1.123
discovery.seed_hosts: ["elk121","elk122","elk123"]
cluster.initial_master_nodes: ["elk123"]
reindex.remote.whitelist: "192.168.1.*:19200"
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

5、所有节点重启ES集群

[root@elk121 ~]# systemctl restart es7
[root@elk122 ~]# systemctl restart es7
[root@elk123 ~]# systemctl restart es7

6、在elk121节点上生成随机密码

[root@elk121 ~]# cd /es/softwares/es7/elasticsearch-7.17.5/
[root@elk121 elasticsearch-7.17.5]# ./bin/elasticsearch-setup-passwords auto
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Future versions of Elasticsearch will require Java 11; your Java version from [/es/softwares/jdk1.8.0_291/jre] does not meet this requirement. Consider switching to a distribution of Elasticsearch with a bundled JDK. If you are already using a distribution with a bundled JDK, ensure the JAVA_HOME environment variable is not set.
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y
Changed password for user apm_system
PASSWORD apm_system = DLpFdnwyTE3VQ0zFRnLZ
Changed password for user kibana_system
PASSWORD kibana_system = 2qH0cXBJBIxeV08icISL
Changed password for user kibana
PASSWORD kibana = 2qH0cXBJBIxeV08icISL
Changed password for user logstash_system
PASSWORD logstash_system = c7JcP3myoyLR901tL22A
Changed password for user beats_system
PASSWORD beats_system = N71zGmfOXQpiymLdPxGI
Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = wkVCfI0WGPClMpWj6Y0Q
Changed password for user elastic
PASSWORD elastic = Qq4wqznwY5kt5z58I6aX

7、使用postman访问测试

填写GET请求192.168.1.121:9200/_cat/nodes，选择【Authorization】-【Basic Auth】后，输入账号apm_system，密码DLpFdnwyTE3VQ0zFRnLZ进行发送

1.3 kibana添加ES认证

1.3.1 配置kibana连接ES集群

1、在elk123节点上修改kibana的配置文件

复制第47行和第48行内容并进行修改

[root@elk123 ~]# vim /etc/kibana/kibana.yml
…
…
elasticsearch.username: "kibana_system"
elasticsearch.password: "2qH0cXBJBIxeV08icISL"

修改完成后，文件内容为

[root@elk123 ~]# egrep -v "^$|^#" /etc/kibana/kibana.yml
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://192.168.1.121:9200","http://192.168.1.122:9200","http://192.168.1.123:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "2qH0cXBJBIxeV08icISL"
i18n.locale: "zh-CN"

2、在elk123节点上重启kibana

[root@elk123 ~]# systemctl restart kibana

3、打开浏览器输入http://192.168.1.123:5601,输入账号elastic，密码：Qq4wqznwY5kt5z58I6aX

(1)点击【配置文件】

(2)进行密码更换，更换密码为123456

1.4 kibana的RBAC

1.4.1 kibana基于角色访问控制

1.4.1.1 创建开发者用户并赋权

1、登录kibana新建角色dev

(1)点击【角色】-【创建角色】

(2)定义角色名称(无法第二次修改)为dev, 设置索引权限为create_doc,read

(3) 添加Kibana权限

(4)点击【创建角色】

2、登录kibana新建用户

(1)点击【用户】-【创建用户】

(2)创建用户并绑定角色

3、重新登录kibana(使用dev01用户)

(1)查看主界面，观察到只能查看到【Discover】和【Dashboard】

(2)查看Dashboard,正常查看

(3)点击【Discover】，只能正常查看索引linux85-logstash-nginx-project-2024.01*

1.4.1.2 创建运维用户并赋权

1、登录kibana新建角色sre

(1)点击【角色】-【创建角色】

(2)定义角色名称(无法第二次修改)为sre, 设置索引权限为*

(3) 添加Kibana权限

(4)点击【创建角色】

2、登录kibana新建用户

(1)点击【用户】-【创建用户】

(2)创建用户并绑定角色

3、重新登录kibana(使用sre01用户)

(1)查看主界面，观察到拥有所有权限

(2)查看Dashboard,所有都正常查看

(3)点击【Discover】，所有索引都正常查看

1.5 logstash写入ES加密集群案例

1.5.1 配置logstash连接ES集群

1.5.1.1 elastic角色使用logstash组件写入数据到ES集群

1、在elk121节点上修改logstash配置文件

[root@elk121 ~]# vim /logstash/config/16-file-to-es_tsl.conf
input {
file {
# 指定本地文件的路径
path => "/tmp/linux85-file"
# 指定读取文件的起始位置，但前提是该文件之前未读取过或者未在".sincedb"文件中>记录。
start_position => "beginning"
}
}
output {
# stdout {}
elasticsearch {
hosts => ["http://192.168.1.121:9200","http://192.168.1.122:9200","http://192.168.1.123:9200"]
index => "linux85-logstash-file"
user => "elastic"
password => "123456"
}
}

2、在elk121节点上定义测试文件

[root@elk121 ~]# vim /tmp/linux85-file
AAAAAA
BBBBBB
CCCCCC
DDDDDD

3、在elk121节点上启动logstash实例

[root@elk121 ~]# logstash -rf /logstash/config/16-file-to-es_tsl.conf

4、登录kibana进行验证

5、在elk121节点上查看logstash采集文件的偏移量路径

[root@elk121 ~]# ls -la /es/softwares/logstash-7.17.5/data/plugins/inputs/file/.sincedb_d6b0037d6dc2e99190203c1515d33007
-rw-r--r-- 1 root root 56 1月 15 21:52 /es/softwares/logstash-7.17.5/data/plugins/inputs/file/.sincedb_d6b0037d6dc2e99190203c1515d33007

6、在elk121节点上指定偏移量

注意：偏移量文件如果没有，在启动会自动创建

(1) 在elk121节点上修改采集文件中的偏移量为09

[root@elk121 ~]# vim /es/softwares/logstash-7.17.5/data/plugins/inputs/file/.sincedb_d6b0037d6dc2e99190203c1515d33007
34796906 0 64768 09 1705326533.049527 /tmp/linux85-file

(2) 删除之前创建好的索引，在elk121节点上再次启动logstash实例

[root@elk121 ~]# logstash -rf /logstash/config/16-file-to-es_tsl.conf

7、打开kibana，使用开发者工具进行验证。观察到从第二行的第三个字母开始(第一行6个字母和一个空格为7个偏移量值，不包含第9个字符)

GET linux85-logstash-file/_search
{
"_source": ["message"]
}

1.5.1.2 自定义角色使用logstash组件写入数据到ES集群

1、登录kibana新建角色logstash

(1)点击【角色】-【创建角色】

(2)定义角色名称(无法第二次修改)为logstash, 指定特定索引并设置索引权限为create_doc,write,设置集群权限为all

(3)点击【创建角色】

2、登录kibana新建用户logstash01

(1)点击【用户】-【创建用户】

(2)创建用户并绑定角色

3、在postman添加POST请求http://192.168.1.121:9200/linux85-jiaoshi07-001/_doc并填写认证信息

{
    "name": "李文轩",
    "hobby": ["吃鸡","丝袜","rap"]
}

观察到成功创建文档

4、在postman添加GET请求http://192.168.1.121:9200/linux85-jiaoshi07-001/_search并填写认证信息,观察到因为权限无法查询

1.6 filebeat写入ES加密集群案例

1.6.1 配置filebeat连接ES集群

1、在elk123节点上修改filebeat配置文件

[root@elk123 ~]# cd /es/softwares/filebeat-7.17.5-linux-x86_64
[root@elk123 filebeat-7.17.5-linux-x86_64]# vim config/24-log-to-es_tls.yaml
filebeat.inputs:
- type: log
paths:
- /tmp/es/test.log
output.elasticsearch:
hosts: ["http://192.168.1.121:9200","http://192.168.1.122:9200","http://192.168.1.123:9200"]
username: "elastic"
password: "123456"
index: "oldboyedu-jiaoshi07-test"
setup.ilm.enabled: false
setup.template.name: "oldboyedu-jiaoshi07"
setup.template.pattern: "oldboyedu-jiaoshi07-*"
setup.template.overwrite: true
setup.template.settings:
index.number_of_shards: 3
index.number_of_replicas: 0

2、在elk123节点上定义测试文件

[root@elk123 filebeat-7.17.5-linux-x86_64]# vim /tmp/es/test.log
aaa
bbb
ccc

3、在elk123节点上启动filebeat实例

[root@elk123 filebeat-7.17.5-linux-x86_64]# filebeat -e -c config/24-log-to-es_tls.yaml

4、在kibana上验证结果

二、ElasticSearch技术各组件瓶颈分析及解决方案

1、垂直扩容:

(1)升级单节点的硬件设备。

(2)缺点:

  • 存在单点故障;

  • 容易达到民用设备的上限;

  • 价钱昂贵;

2、水平扩容:

(1)加相同配置的机器。

(2)缺点:

  • 高峰期外的时间服务器性能利用率较低!

3、消息队列:

(1)使用消息队列解耦,缓冲数据,数据削峰