一、先决条件¶
1.k8s集群要求:
- 最低版本支持Kubernetes v1.21或更高版本
- 至少五个节点，每个节点的内存不低于5G，CPU不低于2核
- 至少有三个存储节点，并且每个节点至少有一个裸盘
- K8s集群所有的节点时间必须一致
2.CPU架构:
- amd64
- x86_64
- arm64
3.分区或设备不能使用文件系统进行格式化
[root@k8s-master01 ~]# lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sdb
sr0 iso9660 CentOS 7 x86_64 2020-11-02-15-15-23-00
sda
├─sda2 LVM2_member lwte0g-uYRm-g35q-IUjr-H2Aa-NU7S-Wb3Ag8
│ ├─centos-swap swap 842f2feb-e334-45f6-be5b-2fddbcd72486
│ └─centos-root xfs 8739385b-6757-4209-94bf-ca84878fb961 /
└─sda1 xfs 33acd5a3-ae42-4564-9243-124f7c4baf81 /boot
二、检查环境¶
1.检查污点以及node状态，确保无污点以及node是ready状态
[root@k8s-master01 ~]# kubectl describe node | grep Taint
Taints: <none>
Taints: <none>
Taints: <none>
Taints: <none>
Taints: <none>
[root@k8s-master01 ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-master01 Ready <none> 103d v1.23.17
k8s-master02 Ready <none> 103d v1.23.17
k8s-master03 Ready <none> 103d v1.23.17
k8s-node01 Ready <none> 103d v1.23.17
k8s-node02 Ready <none> 103d v1.23.17
2.查看时间是否同步
$ date
若没有同步
$ ntpdate time2.aliyun.com
3.查看三个存储节点上的磁盘挂载情况,保证三个节点每个都有一个裸盘
[root@k8s-node02 ~]# lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sdb
sr0 iso9660 CentOS 7 x86_64 2020-11-02-15-15-23-00
sda
├─sda2 LVM2_member FwFkVh-KQZQ-jPVX-ZFhr-KWXw-apOr-qyTTNl
│ ├─centos-swap swap f2ecae66-0aa8-4cd7-808f-137643a10360
│ └─centos-root xfs 5c040742-70c2-43bf-8923-e11c7b439723 /
└─sda1 xfs 3f980dfc-fa56-4bfe-8f58-cd70fd4fc852 /boot
[root@k8s-node01 ~]# lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sdb
sr0 iso9660 CentOS 7 x86_64 2020-11-02-15-15-23-00
sda
├─sda2 LVM2_member InPkzD-1116-NcSy-VYZ6-UqAo-Xi3e-TMcXgS
│ ├─centos-swap swap 94b48f88-4f53-42fc-899d-df8f10d32e57
│ └─centos-root xfs eaefe8a0-c12d-47d5-8f73-f41e875d5102 /
└─sda1 xfs efc4bf10-7b41-41cb-9497-b4b0301615ef /boot
[root@k8s-master03 ~]# lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sdb
sr0 iso9660 CentOS 7 x86_64 2020-11-02-15-15-23-00
sda
├─sda2 LVM2_member 6U2rVH-r90V-vRzK-1VAx-FQsH-XuSg-eRcwLe
│ ├─centos-swap swap d198ed44-10f3-4359-b1b8-f715c39d414e
│ └─centos-root xfs ae303dc0-0f2c-4536-a7af-e6ec9baa622f /
└─sda1 xfs 2b57b5d1-7634-4b16-9c28-2de601accd76 /boot
三、开始部署¶
1.克隆Rook相关代码
$ git clone --single-branch --branch v1.11.8 https://github.com/rook/rook.git
2.修改operator.yaml，替换国外镜像为国内镜像
$ cd rook/deploy/examples
$ vim operator.yaml
ROOK_CSI_CEPH_IMAGE: "registry.cn-hangzhou.aliyuncs.com/abroad_images/cephcsi:v3.8.0"
ROOK_CSI_REGISTRAR_IMAGE: "registry.cn-hangzhou.aliyuncs.com/abroad_images/csi-node-driver-registrar:v2.7.0"
ROOK_CSI_RESIZER_IMAGE: "registry.cn-hangzhou.aliyuncs.com/abroad_images/csi-resizer:v1.7.0"
ROOK_CSI_PROVISIONER_IMAGE: "registry.cn-hangzhou.aliyuncs.com/abroad_images/csi-provisioner:v3.4.0"
ROOK_CSI_SNAPSHOTTER_IMAGE: "registry.cn-hangzhou.aliyuncs.com/abroad_images/csi-snapshotter:v6.2.1"
ROOK_CSI_ATTACHER_IMAGE: "registry.cn-hangzhou.aliyuncs.com/abroad_images/csi-attacher:v4.1.0"
替换为rook/ceph:v1.11.8为registry.cn-hangzhou.aliyuncs.com/abroad_images/ceph:v1.11.8
另外需要开启 自 动 发 现 容 器,将ROOK_ENABLE_DISCOVERY_DAEMON 改成 true
3.部署Rook集群
$ cd rook/deploy/examples
$ kubectl create -f crds.yaml -f common.yaml -f operator.yaml
4.验证部署成功,观察到Ready为1/1,状态为Running
$ kubectl -n rook-ceph get po -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
rook-ceph-operator-d68878968-n9spk 1/1 Running 0 13m 172.25.92.90 k8s-master02 <none> <none>
rook-discover-4jsq2 1/1 Running 0 13m 172.17.125.25 k8s-node01 <none> <none>
rook-discover-bns2w 1/1 Running 0 13m 172.25.92.91 k8s-master02 <none> <none>
rook-discover-fw6gf 1/1 Running 0 13m 172.27.14.223 k8s-node02 <none> <none>
rook-discover-kkl6h 1/1 Running 0 13m 172.18.195.23 k8s-master03 <none> <none>
rook-discover-sv6zm 1/1 Running 0 13m 172.25.244.225 k8s-master01 <none> <none>