一、基于租户和团队的资源限制¶
在一个 Kubernetes 集群中，可能会有不同的团队或者不同的租户共同使用，此时可以针对不同的租户和不同的团队进行资源限制。
1、创建两个Namespace模拟两个租户
[root@k8s-master01 ~]# k create ns c1
[root@k8s-master01 ~]# k create ns c2
2、配置租户1的资源限制
定义资源限制的yaml文件
[root@k8s-master01 ~]# vim c1-resourcequota.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
name: c1-quota
namespace: c1
spec:
hard:
requests.cpu: "2"
requests.memory: 4Gi
limits.cpu: "8"
limits.memory: 16Gi
pods: "50"
requests.storage: 40Gi
persistentvolumeclaims: "10"
services: "40"
count/replicasets.apps: 1k
应用yaml文件
[root@k8s-master01 ~]# kaf c1-resourcequota.yaml
3、配置租户2的资源限制
定义资源限制的yaml文件
[root@k8s-master01 ~]# vim c2-resourcequota.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
name: c2-quota
namespace: c2
spec:
hard:
requests.cpu: "4"
requests.memory: 8Gi
limits.cpu: "16"
limits.memory: 32Gi
pods: "50"
count/replicasets.apps: 1k
应用yaml文件
[root@k8s-master01 ~]# kaf c2-resourcequota.yaml
4、创建一个deployment进行测试
[root@k8s-master01 ~]# k create deploy test-quota --image=registry.cn-hangzhou.aliyuncs.com/abroad_images/redis:7.2.5 -n c1
创建完成后，查看创建情况，观察到创建后没有deployment资源
[root@k8s-master01 ~]# kgp -n c1
No resources found in c1 namespace.
因为deployment底层由rs进行管理，再查看rs资源创建情况,观察到rs资源创建成功
[root@k8s-master01 ~]# kg rs -n c1
NAME DESIRED CURRENT READY AGE
test-quota-779ff9bb4b 1 0 0 14s
查看RS资源日志信息,提示需要添加资源请求
[root@k8s-master01 ~]# k describe rs -n c1 test-quota-779ff9bb4b
...
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 4m32s replicaset-controller Error creating: pods "test-quota-779ff9bb4b-6t8s2" is forbidden: failed quota: c1-quota: must specify limits.cpu for: redis; limits.memory for: redis; requests.cpu for: redis; requests.memory for: redis
Warning FailedCreate 4m32s replicaset-controller Error creating: pods "test-quota-779ff9bb4b-9wfbf" is forbidden: failed quota: c1-quota: must specify limits.cpu for: redis; limits.memory for: redis; requests.cpu for: redis; requests.memory for: redis
Warning FailedCreate 4m32s replicaset-controller Error creating: pods "test-quota-779ff9bb4b-8h7fn" is forbidden: failed quota: c1-quota: must specify limits.cpu for: redis; limits.memory for: redis; requests.cpu for: redis; requests.memory for: redis
Warning FailedCreate 4m32s replicaset-controller Error creating: pods "test-quota-779ff9bb4b-94445" is forbidden: failed quota: c1-quota: must specify limits.cpu for: redis; limits.memory for: redis; requests.cpu for: redis; requests.memory for: redis
Warning FailedCreate 4m32s replicaset-controller Error creating: pods "test-quota-779ff9bb4b-w2dc4" is forbidden: failed quota: c1-quota: must specify limits.cpu for: redis; limits.memory for: redis; requests.cpu for: redis; requests.memory for: redis
Warning FailedCreate 4m31s replicaset-controller Error creating: pods "test-quota-779ff9bb4b-pws8w" is forbidden: failed quota: c1-quota: must specify limits.cpu for: redis; limits.memory for: redis; requests.cpu for: redis; requests.memory for: redis
Warning FailedCreate 4m31s replicaset-controller Error creating: pods "test-quota-779ff9bb4b-97rhg" is forbidden: failed quota: c1-quota: must specify limits.cpu for: redis; limits.memory for: redis; requests.cpu for: redis; requests.memory for: redis
Warning FailedCreate 4m31s replicaset-controller Error creating: pods "test-quota-779ff9bb4b-sg5kg" is forbidden: failed quota: c1-quota: must specify limits.cpu for: redis; limits.memory for: redis; requests.cpu for: redis; requests.memory for: redis
Warning FailedCreate 4m30s replicaset-controller Error creating: pods "test-quota-779ff9bb4b-st2kh" is forbidden: failed quota: c1-quota: must specify limits.cpu for: redis; limits.memory for: redis; requests.cpu for: redis; requests.memory for: redis
Warning FailedCreate 108s (x7 over 4m29s) replicaset-controller (combined from similar events): Error creating: pods "test-quota-779ff9bb4b-4z4zq" is forbidden: failed quota: c1-quota: must specify limits.cpu for: redis; limits.memory for: redis; requests.cpu for: redis; requests.memory for: redis
5、查看c1租户的resourceQuota,观察到count/replicasets.apps的数值已经变为1
[root@k8s-master01 ~]# kg resourceQuota -n c1
NAME AGE REQUEST LIMIT
c1-quota 10m count/replicasets.apps: 1/1k, persistentvolumeclaims: 0/10, pods: 0/50, requests.cpu: 0/2, requests.memory: 0/4Gi, requests.storage: 0/40Gi, services: 0/40 limits.cpu: 0/8, limits.memory: 0/16Gi
6、给deployment添加资源请求,这里演示资源请求超出资源限制
[root@k8s-master01 ~]# k edit deploy test-quota -n c1
# 在第40行内容下面添加资源请求
...
...
resources:
limits:
cpu: "4"
memory: 1Gi
requests:
cpu: "3"
memory: 10Mi
...
...
# 观察到没有pod生成
[root@k8s-master01 ~]# kgp -n c1
No resources found in c1 namespace.
# 查看rs,观察到资源请求资源超出了资源限制，命名空间中定义的 ResourceQuota（名为 c1-quota）对 requests.cpu 做了限制（limited: requests.cpu=2），但当前创建的 Pod 申请的 CPU 资源为 requests.cpu=3，超出配额允许范围，因此被集群拒绝。
[root@k8s-master01 ~]# kg rs -nc1
NAME DESIRED CURRENT READY AGE
test-quota-779ff9bb4b 1 0 0 16m
test-quota-c8c4c6d7b 1 0 0 3m54s
[root@k8s-master01 ~]# k describe rs test-quota-c8c4c6d7b -n c1
...
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 5m12s replicaset-controller Error creating: pods "test-quota-c8c4c6d7b-tzm5l" is forbidden: exceeded quota: c1-quota, requested: requests.cpu=3, used: requests.cpu=0, limited: requests.cpu=2
Warning FailedCreate 5m12s replicaset-controller Error creating: pods "test-quota-c8c4c6d7b-w9hhn" is forbidden: exceeded quota: c1-quota, requested: requests.cpu=3, used: requests.cpu=0, limited: requests.cpu=2
Warning FailedCreate 5m12s replicaset-controller Error creating: pods "test-quota-c8c4c6d7b-mcs5c" is forbidden: exceeded quota: c1-quota, requested: requests.cpu=3, used: requests.cpu=0, limited: requests.cpu=2
Warning FailedCreate 5m12s replicaset-controller Error creating: pods "test-quota-c8c4c6d7b-fwq9l" is forbidden: exceeded quota: c1-quota, requested: requests.cpu=3, used: requests.cpu=0, limited: requests.cpu=2
Warning FailedCreate 5m12s replicaset-controller Error creating: pods "test-quota-c8c4c6d7b-jg9mh" is forbidden: exceeded quota: c1-quota, requested: requests.cpu=3, used: requests.cpu=0, limited: requests.cpu=2
Warning FailedCreate 5m12s replicaset-controller Error creating: pods "test-quota-c8c4c6d7b-jhm68" is forbidden: exceeded quota: c1-quota, requested: requests.cpu=3, used: requests.cpu=0, limited: requests.cpu=2
Warning FailedCreate 5m12s replicaset-controller Error creating: pods "test-quota-c8c4c6d7b-jm8jz" is forbidden: exceeded quota: c1-quota, requested: requests.cpu=3, used: requests.cpu=0, limited: requests.cpu=2
Warning FailedCreate 5m12s replicaset-controller Error creating: pods "test-quota-c8c4c6d7b-qvs9m" is forbidden: exceeded quota: c1-quota, requested: requests.cpu=3, used: requests.cpu=0, limited: requests.cpu=2
Warning FailedCreate 5m11s replicaset-controller Error creating: pods "test-quota-c8c4c6d7b-czqvl" is forbidden: exceeded quota: c1-quota, requested: requests.cpu=3, used: requests.cpu=0, limited: requests.cpu=2
Warning FailedCreate 2m28s (x7 over 5m10s) replicaset-controller (combined from similar events): Error creating: pods "test-quota-c8c4c6d7b-vl5f4" is forbidden: exceeded quota: c1-quota, requested: requests.cpu=3, used: requests.cpu=0, limited: requests.cpu=2
7、给deployment修改资源请求
[root@k8s-master01 ~]# k edit deploy test-quota -n c1
# 主要修改requests的cpu数为1
...
...
resources:
limits:
cpu: "4"
memory: 1Gi
requests:
cpu: "1"
memory: 10Mi
...
...
8、重新查看pod创建情况,观察到当资源请求合理时，能正常启动
[root@k8s-master01 ~]# kgp -nc1
NAME READY STATUS RESTARTS AGE
test-quota-7f648c585b-bs7g6 1/1 Running 0 69s