来自AI助手的总结

介绍用 Argo CD 和 Rollout 实现蓝绿部署的实战流程

一、实战练习

登录http://gitlab.example.com/，账号为

root,密码为。登录成功后，参考前面内容创建名为Argocd Example Apps的项目，标识字符串为argocd-example-apps

创建一个 Rollout 资源和一个针对该资源的 Kubernetes Service 对象，这里的 Rollout 采用了蓝绿部署：

[root@master01 ~]# cd /root/17/argo-rollouts
[root@master01 argo-rollouts]# mkdir -p argorollout-examples/bluegreen
[root@master01 argo-rollouts]# cd argorollout-examples/bluegreen
[root@master01 bluegreen]# vim rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: newapp-rollout-0902
  namespace: demo
spec:
  replicas: 3
  strategy:
    blueGreen:     
      activeService: bluegreen-appv1
      previewService: bluegreen-appv2  
      autoPromotionEnabled: false
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: web
        image: registry.cn-hangzhou.aliyuncs.com/abroad_images/kubernetes-bootcamp:v1
        ports:
        - name: http
          containerPort: 8080
          protocol: TCP

web-app 应用的 appv1-svc：

[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples/bluegreen
[root@master01 bluegreen]# vim web-app-appv1.yaml
apiVersion: v1
kind: Service
metadata:
  name: bluegreen-appv1
  namespace: demo
spec:
  selector:
    app: web-app
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080

web-app 应用的 appv2-svc：

[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples/bluegreen
[root@master01 bluegreen]# vim web-app-appv2.yaml
apiVersion: v1
kind: Service
metadata:
  name: bluegreen-appv2
  namespace: demo
spec:
  selector:
    app: web-app
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080

把上述3个文件放入代码仓库 http://gitlab.example.com/demoteam/argocd-example-apps.git的argorollout-examples/bluegreen 目录下


[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples

[root@master01 argorollout-examples]# git init

# 添加远端仓库

[root@master01 argorollout-examples]# git remote add origin http://gitlab.example.com/demoteam/argocd-example-apps.git

# 验证查看

[root@master01 argorollout-examples]# git remote -v

origin  http://gitlab.example.com/demoteam/argocd-example-apps.git (fetch)

origin  http://gitlab.example.com/demoteam/argocd-example-apps.git (push)

# 添加到暂存区

[root@master01 argorollout-examples]# git add .

# 提交到本地仓库

[root@master01 argorollout-examples]# git commit -m "first for argocd-example-apps"

# 切换到main分支

[root@master01 argorollout-examples]# git branch -M main

# 上传到main分支

[root@master01 argorollout-examples]# git push -uf origin main

Username for 'http://gitlab.example.com': root

Password for 'http://root@gitlab.example.com': <gitlab-password>

添加gitlab仓库


# 先使用初始密码登录

[root@master01 17]# argocd login argocd.example.com

Username: admin

Password: <new-admin-password>

# 添加前需要保证私有gitlab仓库不为空

[root@master01 argo-rollouts]# argocd repo add http://gitlab.example.com/demoteam/argocd-example-apps.git  --username root --password <gitlab-password> --insecure-skip-server-verification 

# 验证查看

[root@master01 argorollout-examples]# argocd repo list | grep argocd-example-apps

git         http://gitlab.example.com/demoteam/argocd-example-apps.git  true      false  false  true   Successful

后续在 Argo CD 上使用 YAML 资源清单创建服务：

[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
[root@master01 argorollout-examples]# vim argorollout-blue-green.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: argorollout-blue-green
  namespace: argocd
spec:
  destination:
    name: ''
    namespace: demo
    server: 'https://kubernetes.default.svc'
  source:
    path: bluegreen
    repoURL: 'http://gitlab.example.com/demoteam/argocd-example-apps.git'
    targetRevision: main
  sources: []
  project: default
# 应用  
[root@master01 argorollout-examples]# kubectl apply -f argorollout-blue-green.yaml

最终目录文件结构：


[root@master01 argorollout-examples]# tree ../argorollout-examples/

../argorollout-examples/

├── argorollout-blue-green.yaml

└── bluegreen

    ├── rollout.yaml

    ├── web-app-appv1.yaml

    └── web-app-appv2.yaml

1 directory, 4 files

web页面手动实现同步

依次点击【SYNC】-【SYNCHEONIZE】

web 页面查看状态：

查看网络流向：

Argo Rollouts 查看服务：


# 查看服务

[root@master01 argorollout-examples]# kubectl argo rollouts list rollouts -ndemo

NAME                 STRATEGY   STATUS        STEP  SET-WEIGHT  READY  DESIRED  UP-TO-DATE  AVAILABLE

newapp-rollout-0902  BlueGreen  Healthy       -     -           3/3    3        3           3        

# 查看节点状态

[root@master01 argorollout-examples]# kubectl argo rollouts status newapp-rollout-0902 -ndemo

Healthy

获取滚动更新的信息：

[root@master01 argorollout-examples]# kubectl argo rollouts get rollout newapp-rollout-0902 -ndemo
Name:            newapp-rollout-0902
Namespace:       demo
Status:          ✔ Healthy
Strategy:        BlueGreen
Images:          registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable, active)
Replicas:
  Desired:       3
  Current:       3
  Updated:       3
  Ready:         3
  Available:     3
NAME                                            KIND        STATUS     AGE    INFO
⟳ newapp-rollout-0902                           Rollout     ✔ Healthy  2m33s  
└──# revision:1                                                               
   └──⧉ newapp-rollout-0902-746fbdc85           ReplicaSet  ✔ Healthy  2m33s  stable,active
      ├──□ newapp-rollout-0902-746fbdc85-h77np  Pod         ✔ Running  2m33s  ready:1/1
      ├──□ newapp-rollout-0902-746fbdc85-kvn98  Pod         ✔ Running  2m33s  ready:1/1
      └──□ newapp-rollout-0902-746fbdc85-md5j7  Pod         ✔ Running  2m33s  ready:1/1

测试验证：均为 v1 版本


# 查看服务

[root@master01 argorollout-examples]# kgs -ndemo | grep bluegreen

bluegreen-appv1   ClusterIP   <cluster-ip>    <none>        8080/TCP   10m

bluegreen-appv2   ClusterIP   <cluster-ip>     <none>        8080/TCP   10m

# 查看bluegreen-appv1对应的版本为v1

[root@master01 argorollout-examples]# curl <cluster-ip>:8080

Hello Kubernetes bootcamp! | Running on: newapp-rollout-0902-746fbdc85-kvn98 | v=1

# 查看bluegreen-appv2对应的版本为v1

[root@master01 argorollout-examples]# curl <cluster-ip>:8080

Hello Kubernetes bootcamp! | Running on: newapp-rollout-0902-746fbdc85-kvn98 | v=1

更新镜像：

1）更新 git 仓库配置文件，修改镜像版本从 v1 –> v2

2)通过命令行的方式更新：

更新镜像


#语法格式

kubectl argo rollouts set image ROLLOUT_NAME CONTAINER=IMAGE [flags]

# 实际替换镜像为v2

[root@master01 argorollout-examples]# kubectl argo rollouts set image newapp-rollout-0902 web=registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 -ndemo

动态观察服务切换信息

[root@master01 argorollout-examples]# kubectl argo rollouts get rollout newapp-rollout-0902 -ndemo --watch
Name:            newapp-rollout-0902
Namespace:       demo
Status:          ॥ Paused
Message:         BlueGreenPause
Strategy:        BlueGreen
Images:          registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable, active)
                 registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (preview)
Replicas:
  Desired:       3
  Current:       6
  Updated:       3
  Ready:         3
  Available:     3
NAME                                             KIND        STATUS     AGE  INFO
⟳ newapp-rollout-0902                            Rollout     ॥ Paused   12m  
├──# revision:2                                                              
│  └──⧉ newapp-rollout-0902-7466b89448           ReplicaSet  ✔ Healthy  41s  preview
│     ├──□ newapp-rollout-0902-7466b89448-75x4s  Pod         ✔ Running  41s  ready:1/1
│     ├──□ newapp-rollout-0902-7466b89448-8wh8l  Pod         ✔ Running  41s  ready:1/1
│     └──□ newapp-rollout-0902-7466b89448-nn84z  Pod         ✔ Running  41s  ready:1/1
└──# revision:1                                                              
   └──⧉ newapp-rollout-0902-746fbdc85            ReplicaSet  ✔ Healthy  12m  stable,active
      ├──□ newapp-rollout-0902-746fbdc85-h77np   Pod         ✔ Running  12m  ready:1/1
      ├──□ newapp-rollout-0902-746fbdc85-kvn98   Pod         ✔ Running  12m  ready:1/1
      └──□ newapp-rollout-0902-746fbdc85-md5j7   Pod         ✔ Running  12m  ready:1/1

手动执行版本切换


# 手动进行切换，等待大概10s后，v1版本的pod会被删除

[root@master01 argorollout-examples]# kubectl argo rollouts promote newapp-rollout-0902 -ndemo

# 实时查看，观察到目前只剩下v2版本的pod

[root@master01 argorollout-examples]# kubectl argo rollouts get rollout newapp-rollout-0902 -ndemo --watch

Name:            newapp-rollout-0902

Namespace:       demo

Status:          ✔ Healthy

Strategy:        BlueGreen

Images:          registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (stable, active)

Replicas:

  Desired:       3

  Current:       3

  Updated:       3

  Ready:         3

  Available:     3

NAME                                             KIND        STATUS        AGE    INFO

⟳ newapp-rollout-0902                            Rollout     ✔ Healthy     18m    

├──# revision:2                                                                   

│  └──⧉ newapp-rollout-0902-7466b89448           ReplicaSet  ✔ Healthy     6m55s  stable,active

│     ├──□ newapp-rollout-0902-7466b89448-75x4s  Pod         ✔ Running     6m55s  ready:1/1

│     ├──□ newapp-rollout-0902-7466b89448-8wh8l  Pod         ✔ Running     6m55s  ready:1/1

│     └──□ newapp-rollout-0902-7466b89448-nn84z  Pod         ✔ Running     6m55s  ready:1/1

└──# revision:1                                                                   

   └──⧉ newapp-rollout-0902-746fbdc85            ReplicaSet  • ScaledDown  18m

测试验证：均为 v2 版本


# 查看服务

[root@master01 argorollout-examples]# kgs -ndemo | grep blue

bluegreen-appv1   ClusterIP   <cluster-ip>    <none>        8080/TCP   19m

bluegreen-appv2   ClusterIP   <cluster-ip>     <none>        8080/TCP   19m

# 测试访问v1

[root@master01 argorollout-examples]# curl <cluster-ip>:8080

Hello Kubernetes bootcamp! | Running on: newapp-rollout-0902-7466b89448-nn84z | v=2

# 测试访问v2

[root@master01 argorollout-examples]# curl <cluster-ip>:8080

Hello Kubernetes bootcamp! | Running on: newapp-rollout-0902-7466b89448-nn84z | v=2

浏览器中输入http://argorollouts.example.com/，打开argorollouts Dashboard 界面

如何中断 Rollout ？

在生产过程中，如果新版本部署后测试有问题，如何在更新过程中手动中止 Rollout ？

首先，使用 set image 命令部署一个新的 V1 版本的容器，并等待 rollout 再次达到暂停的步骤。


[root@master01 ~]# kubectl argo rollouts set image newapp-rollout-0902 web=registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 -ndemo

查看验证：


[root@master01 ~]# kubectl argo rollouts get rollout newapp-rollout-0902 -ndemo --watch

中止更新，而不是将滚动切换到下一步，这样它就回到了 V2 版本，该插件同样提供了一个 abort 命令，可以在更新过程中的任何时候手动中止 Rollout 。


[root@master01 ~]# kubectl argo rollouts abort newapp-rollout-0902 -ndemo

当中止滚动时，它将扩大 ReplicaSet 的 stable 版本（在本例中是 V2 版本），并缩小任何其他版本。

尽管 ReplicaSet 的稳定版本可能正在运行，并且是健康的，但整个 Rollout 仍然被认为是退化的，因为期望的版本（ V1 版本）不是实际运行的版本。

[root@master01 ~]# kubectl argo rollouts get rollout newapp-rollout-0902 -ndemo
Name:            newapp-rollout-0902
Namespace:       demo
Status:          ✖ Degraded
Message:         RolloutAborted: Rollout aborted update to revision 3
Strategy:        BlueGreen
Images:          registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (preview)
                 registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (stable, active)
Replicas:
  Desired:       3
  Current:       6
  Updated:       3
  Ready:         3
  Available:     3
NAME                                             KIND        STATUS      AGE    INFO
⟳ newapp-rollout-0902                            Rollout     ✖ Degraded  32m    
├──# revision:3                                                                 
│  └──⧉ newapp-rollout-0902-746fbdc85            ReplicaSet  ✔ Healthy   32m    preview,delay:1s
│     ├──□ newapp-rollout-0902-746fbdc85-44hq9   Pod         ✔ Running   3m47s  ready:1/1
│     ├──□ newapp-rollout-0902-746fbdc85-sz5rv   Pod         ✔ Running   3m47s  ready:1/1
│     └──□ newapp-rollout-0902-746fbdc85-z7cj4   Pod         ✔ Running   3m47s  ready:1/1
└──# revision:2                                                                 
   └──⧉ newapp-rollout-0902-7466b89448           ReplicaSet  ✔ Healthy   21m    stable,active
      ├──□ newapp-rollout-0902-7466b89448-75x4s  Pod         ✔ Running   21m    ready:1/1
      ├──□ newapp-rollout-0902-7466b89448-8wh8l  Pod         ✔ Running   21m    ready:1/1
      └──□ newapp-rollout-0902-7466b89448-nn84z  Pod         ✔ Running   21m    ready:1/1

模拟测试访问,发现实际访问的版本为v2，但是用户实际想访问的是v1

[root@master01 ~]# curl <cluster-ip>:8080
Hello Kubernetes bootcamp! | Running on: newapp-rollout-0902-7466b89448-8wh8l | v=2
[root@master01 ~]# curl <cluster-ip>:8080
curl: (7) Failed connect to <cluster-ip>:8080; Connection refused

二、基于ArgoCD的金丝雀部署

2.1 传统金丝雀(灰度)流程

2.2 ArgoRollout 金丝雀(灰度)流程

实现原理：

金丝雀发布是一种部署新版本应用的方法，其中 新版本 先部署给一小部分用户，如果一切正常，则逐渐增加新版本的流量，直到完全替换旧版本。

2.3 基于 Replica Shifting(版本替换) 金丝雀发布

2.3 部署 Rollout

创建数据目录


[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples

[root@master01 argorollout-examples]# mkdir canary-replica

在数据目录中创建文件

[root@master01 argorollout-examples]# cd canary-replica/
[root@master01 canary-replica]# vim canary-rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: canary-rollouts-demo
  namespace: demo
spec:
  replicas: 3 # 定义3个副本
  strategy: # 定义升级策略
    canary: # 金丝雀发布
      steps: # 发布的节奏
        - setWeight: 20
        - pause: {} # 会一直暂停,需要手动 promote
        - setWeight: 40
        - pause: { duration: 10 } # 暂停10s
        - setWeight: 60
        - pause: { duration: 10 }
        - setWeight: 80
        - pause: { duration: 10 }
  revisionHistoryLimit: 2 # 下面部分其实是和 Deployment 兼容的
  selector:
    matchLabels:
      app: rollouts-demo
  template:
    metadata:
      labels:
        app: rollouts-demo
    spec:
      containers:
        - name: rollouts-demo
          image: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP

可以看到除了 apiVersion，kind 以及 strategy 之外，其他和 Deployment 无异。

strategy 字段定义的是发布策略，其中：

setWeight：设置流量的权重
pause：暂停，如果里面没有跟duration: 10则表示需要手动更新，如果跟了表示等待多长时间会自动更新。

2.3 定义 service

[root@master01 argorollout-examples]# cd canary-replica/
[root@master01 canary-replica]# vim canary-rollout-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: rollouts-demo
  namespace: demo
spec:
  ports:
  - port: 8080
    targetPort: http
    protocol: TCP
    name: http
  selector:
    app: rollouts-demo

2.3 创建 argorollout-canary-replica

[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
[root@master01 argorollout-examples]# vim argorollout-canary-replica.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: argorollout-canary-replica
  namespace: argocd
spec:
  destination:
    name: ''
    namespace: demo
    server: 'https://kubernetes.default.svc'
  source:
    path: canary-replica
    repoURL: 'http://gitlab.example.com/demoteam/argocd-example-apps.git'
    targetRevision: main
  sources: []
  project: default

把上述2个文件放入代码仓库 http://gitlab.example.com/demoteam/argocd-example-apps.git canary-replica 目录下，后续在 Argo CD 上使用 YAML 资源清单创建服务。


[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples

[root@master01 argorollout-examples]# git init

# 添加远端仓库

[root@master01 argorollout-examples]# git remote add origin http://gitlab.example.com/demoteam/argocd-example-apps.git

# 验证查看

[root@master01 argorollout-examples]# git remote -v

origin  http://gitlab.example.com/demoteam/argocd-example-apps.git (fetch)

origin  http://gitlab.example.com/demoteam/argocd-example-apps.git (push)

# 添加到暂存区

[root@master01 argorollout-examples]# git add .

# 提交到本地仓库

[root@master01 argorollout-examples]# git commit -m "first for argocd-example-apps"

# 切换到main分支

[root@master01 argorollout-examples]# git branch -M main

# 上传到main分支

[root@master01 argorollout-examples]# git push -uf origin main

Username for 'http://gitlab.example.com': root

Password for 'http://root@gitlab.example.com': <gitlab-password>

创建相关资源


[root@master01 argorollout-examples]# kubectl apply -f argorollout-canary-replica.yaml

2.3 手动 sync

点击【SYNC】-【SYNCHRONIZE】进行手动同步

底层命令观察：

# 查看回滚列表
[root@master01 ~]# kubectl argo rollouts list rollouts -ndemo
NAME                  STRATEGY   STATUS        STEP  SET-WEIGHT  READY  DESIRED  UP-TO-DATE  AVAILABLE
canary-rollouts-demo  Canary     Healthy       8/8   100         3/3    3        3           3        
newapp-rollout-0902   BlueGreen  Healthy       -     -           3/3    3        3           3       
# 获取滚动更新的信息
[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-demo -ndemo
Name:            canary-rollouts-demo
Namespace:       demo
Status:          ✔ Healthy
Strategy:        Canary
  Step:          8/8
  SetWeight:     100
  ActualWeight:  100
Images:          registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
Replicas:
  Desired:       3
  Current:       3
  Updated:       3
  Ready:         3
  Available:     3
NAME                                             KIND        STATUS     AGE  INFO
⟳ canary-rollouts-demo                           Rollout     ✔ Healthy  63s  
└──# revision:1                                                              
   └──⧉ canary-rollouts-demo-d7975f98c           ReplicaSet  ✔ Healthy  63s  stable
      ├──□ canary-rollouts-demo-d7975f98c-sdsb8  Pod         ✔ Running  62s  ready:1/1
      ├──□ canary-rollouts-demo-d7975f98c-vlxr5  Pod         ✔ Running  62s  ready:1/1
      └──□ canary-rollouts-demo-d7975f98c-wkwlg  Pod         ✔ Running  62s  ready:1/1

打开浏览器输入http://argorollouts.example.com/打开argorollout dashboard，命名空间选择demo

2.3 更新 Rollout

先打开一个新的 xshell 窗口，便于动态观察：

[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-demo -ndemo --watch
# 初始页面
Name:            canary-rollouts-demo
Namespace:       demo
Status:          ✔ Healthy
Strategy:        Canary
  Step:          8/8
  SetWeight:     100
  ActualWeight:  100
Images:          registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
Replicas:
  Desired:       3
  Current:       3
  Updated:       3
  Ready:         3
  Available:     3
NAME                                             KIND        STATUS     AGE   INFO
⟳ canary-rollouts-demo                           Rollout     ✔ Healthy  5m3s  
└──# revision:1                                                               
   └──⧉ canary-rollouts-demo-d7975f98c           ReplicaSet  ✔ Healthy  5m3s  stable
      ├──□ canary-rollouts-demo-d7975f98c-sdsb8  Pod         ✔ Running  5m2s  ready:1/1
      ├──□ canary-rollouts-demo-d7975f98c-vlxr5  Pod         ✔ Running  5m2s  ready:1/1
      └──□ canary-rollouts-demo-d7975f98c-wkwlg  Pod         ✔ Running  5m2s  ready:1/1

使用 set image 的命令更新：


[root@master01 ~]# kubectl argo rollouts set image canary-rollouts-demo rollouts-demo=registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 -ndemo

动态观察数据：

Name:            canary-rollouts-demo
Namespace:       demo
Status:          ॥ Paused #暂停，等待进一步promote
Message:         CanaryPauseStep
Strategy:        Canary
  Step:          1/8
  SetWeight:     20
  ActualWeight:  25
Images:          registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
                 registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (canary)
Replicas:
  Desired:       3
  Current:       4
  Updated:       1
  Ready:         4
  Available:     4
NAME                                              KIND        STATUS     AGE    INFO
⟳ canary-rollouts-demo                            Rollout     ॥ Paused   5m50s  
├──# revision:2                                                                 
│  └──⧉ canary-rollouts-demo-68bfb5ccbb           ReplicaSet  ✔ Healthy  15s    canary # 版本被标记为canary
│     └──□ canary-rollouts-demo-68bfb5ccbb-zhh74  Pod         ✔ Running  15s    ready:1/1
└──# revision:1                                                                 
   └──⧉ canary-rollouts-demo-d7975f98c            ReplicaSet  ✔ Healthy  5m50s  stable
      ├──□ canary-rollouts-demo-d7975f98c-sdsb8   Pod         ✔ Running  5m49s  ready:1/1
      ├──□ canary-rollouts-demo-d7975f98c-vlxr5   Pod         ✔ Running  5m49s  ready:1/1
      └──□ canary-rollouts-demo-d7975f98c-wkwlg   Pod         ✔ Running  5m49s  ready:1/1

当 demo rollout 到达第二步时，我们可以从插件中看到，Rollout 处于暂停状态，现在有 4 个副本中的1 个运行新版本的 pod，其余 3 个仍然运行旧版本，这相当于 setWeight: 20 步骤所定义的 20% 的金丝雀权重。

2.3 Promote Rollout

Rollout 现在处于暂停状态，要手动将 Rollout 切换到下一个步骤，运行插件的 promote 命令。


[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-demo -ndemo

然后，每隔 10s 进行 v2 版本的逐步替换，最后全部替换为 v2 版本：


[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-demo -ndemo

Name:            canary-rollouts-demo

Namespace:       demo

Status:          ✔ Healthy

Strategy:        Canary

  Step:          8/8

  SetWeight:     100

  ActualWeight:  100

Images:          registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (stable)

Replicas:

  Desired:       3

  Current:       3

  Updated:       3

  Ready:         3

  Available:     3

NAME                                              KIND        STATUS         AGE    INFO

⟳ canary-rollouts-demo                            Rollout     ✔ Healthy      9m21s  

├──# revision:2                                                                     

│  └──⧉ canary-rollouts-demo-68bfb5ccbb           ReplicaSet  ✔ Healthy      3m46s  stable

│     ├──□ canary-rollouts-demo-68bfb5ccbb-zhh74  Pod         ✔ Running      3m46s  ready:1/1

│     ├──□ canary-rollouts-demo-68bfb5ccbb-n956r  Pod         ✔ Running      33s    ready:1/1

│     └──□ canary-rollouts-demo-68bfb5ccbb-mb4fc  Pod         ✔ Running      22s    ready:1/1

└──# revision:1                                                                     

   └──⧉ canary-rollouts-demo-d7975f98c            ReplicaSet  • ScaledDown   9m21s  

      └──□ canary-rollouts-demo-d7975f98c-wkwlg   Pod         ◌ Terminating  9m20s  ready:1/1

也可以底层 svc 访问验证：


# 查看svc地址为<cluster-ip>

[root@master01 argorollout-examples]# kgs -ndemo | grep rollouts-demo

rollouts-demo     ClusterIP   <cluster-ip>   <none>        8080/TCP   2d4h

# 测试访问服务，目前版本变为v2

[root@master01 ~]# curl <cluster-ip>:8080

Hello Kubernetes bootcamp! | Running on: canary-rollouts-demo-68bfb5ccbb-mb4fc | v=2

2.4 基于 Traffic Shifting(流量接入) 金丝雀发布

官方数据：https://argoproj.github.io/argo-rollouts/features/traffic-management/nginx/

上面我们并没有接入外部流量，仅仅是在内部使用展示了金丝雀部署过程，下面我们接入外部流量进行测试。

Argo-Rollout 主要集成了 Ingress 和 ServiceMesh 两种流量控制方法。

主要 Ingress 支持：（主要使用 Nginx-Ingress）

2.4 创建 Rollout

Rollout 里用 canaryService 和 stableService 分别定义了该应用：

灰度的 Service Name (rollouts-traffic-canary)
当前版本的 Service Name (rollouts-traffic-stable)

[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
[root@master01 argorollout-examples]# mkdir canary-traffic
[root@master01 argorollout-examples]# cd canary-traffic
[root@master01 canary-traffic]# vim canary-rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: canary-rollouts-traffic-demo
  namespace: demo
spec:
  replicas: 3
  strategy:
    canary:
      canaryService: rollouts-traffic-canary
      stableService: rollouts-traffic-stable
      trafficRouting:
        nginx:
          stableIngress: rollouts-traffic-stable
      # 发布的节奏
      steps:
      - setWeight: 30
      # 会一直暂停
      - pause: {}
      - setWeight: 60
      - pause: {}
      - setWeight: 100
      - pause: {}
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app: rollouts-traffic-demo
  template:
    metadata:
      labels:
        app: rollouts-traffic-demo
    spec:
      containers:
      - name: rollouts-traffic-demo
        image: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1
        ports:
        - name: http
          containerPort: 8080
          protocol: TCP
        resources:
          requests:
            memory: 512Mi
            cpu: 512m

2.4 创建应用的 stable/canary SVC

Service rollouts-traffic-canary 和 rollouts-traffic-stable，二者内容一样。selector 中暂时没有填上pod-template-hash。

Argo-Rollout Controller 会根据实际的 ReplicaSet hash 来修改该值:

[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples/canary-traffic
[root@master01 canary-traffic]# vim rollouts-traffic-stable-canary.yaml
apiVersion: v1
kind: Service
metadata:
  name: rollouts-traffic-canary
  namespace: demo
spec:
  ports:
  - port: 8080
    targetPort: http
    protocol: TCP
    name: http
  selector:
    app: rollouts-traffic-demo
    # This selector will be updated with the pod-template-hash of the canary ReplicaSet. e.g.:
    # rollouts-pod-template-hash: 7bf84f9696
---
apiVersion: v1
kind: Service
metadata:
  name: rollouts-traffic-stable
  namespace: demo
spec:
  ports:
  - port: 8080
    targetPort: http
    protocol: TCP
    name: http
  selector:
    app: rollouts-traffic-demo

2.4 创建应用的 ingress 路由

Ingress 则定义了规则，nginx 将 canary.example.com 域名的请求转发到当前版本的 Service(rollouts-traffic-stable)

[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples/canary-traffic
[root@master01 canary-traffic]# vim canary-traffic-ing.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rollouts-traffic-stable
  namespace: demo
spec:
  ingressClassName: nginx
  rules:
  - host: canary.example.com
    http:
      paths:
      - backend:
          service:
            name: rollouts-traffic-stable
            port:
              number: 8080
        path: /
        pathType: Prefix

2.4 创建 APP

[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
[root@master01 argorollout-examples]# vim argorollout-canary-traffic.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: argorollout-canary-traffic
  namespace: argocd
spec:
  destination:
    name: ''
    namespace: default
    server: 'https://kubernetes.default.svc'
  source:
    path: canary-traffic
    repoURL: 'http://gitlab.example.com/demoteam/argocd-example-apps.git'
    targetRevision: main
  sources: []
  project: default

把上述 3 个文件放入代码仓库 http://gitlab.example.com/demoteam/argocd-example-apps.git canary-traffic 目录下，后续在 Argo CD 上使用 YAML 资源清单创建服务。


[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples

# 初始化

[root@master01 argorollout-examples]# git init

# 添加远端仓库

[root@master01 argocd]# git remote add origin http://gitlab.example.com/demoteam/argocd-example-apps.git

# 验证查看

[root@master01 argorollout-examples]# git remote -v

origin  http://gitlab.example.com/demoteam/argocd-example-apps.git (fetch)

origin  http://gitlab.example.com/demoteam/argocd-example-apps.git (push)

# 添加到暂存区

[root@master01 argorollout-examples]# git add .

# 提交到本地仓库

[root@master01 argorollout-examples]# git commit -m "second for argorollout-examples"

# 切换到main分支

[root@master01 argorollout-examples]# git branch -M main

# 上传到main分支

[root@master01 argorollout-examples]# git push -uf origin main

Username for 'http://gitlab.example.com': root

Password for 'http://root@gitlab.example.com': <gitlab-password>

...

...

gitlab页面查看

应用创建


[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples

[root@master01 argorollout-examples]# kubectl apply -f argorollout-canary-traffic.yaml

WEB UI 手动同步：

点击【SYNC】-【SYNCHRONIZE】

traffic 流向：

2.4 Canary Ingress

Rollout Controller 会根据 ingress rollouts-traffic-stable 内容，自动创建一个 ingress 用于灰度的流量，名字为 –canary，

所以这里多了一个 ingress canary-rollouts-traffic-demo-rollouts-traffic-stable-canary，将流量导向Canary Service (rollouts-traffic-canary):


[root@master01 ~]# kubectl get ing -ndemo | grep canary

canary-rollouts-traffic-demo-rollouts-traffic-stable-canary   nginx   canary.example.com   <node-ip>   80      8m36s

rollouts-traffic-stable                                       nginx   canary.example.com   <node-ip>   80      8m39s

2.4 触发更新


[root@master01 ~]# kubectl argo rollouts set image canary-rollouts-traffic-demo rollouts-traffic-demo=registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 -ndemo

动态观察，第一步：

[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-traffic-demo -ndemo --watch
# 回显内容
Name:            canary-rollouts-traffic-demo
Namespace:       demo
Status:          ॥ Paused
Message:         CanaryPauseStep
Strategy:        Canary
  Step:          1/6
  SetWeight:     30  #权重比例
  ActualWeight:  30  #权重比例
Images:          registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
                 registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (canary)
Replicas:
  Desired:       3
  Current:       4
  Updated:       1
  Ready:         4
  Available:     4
NAME                                                      KIND        STATUS     AGE    INFO
⟳ canary-rollouts-traffic-demo                            Rollout     ॥ Paused   12m    
├──# revision:2                                                                         
│  └──⧉ canary-rollouts-traffic-demo-69f4dd6c8            ReplicaSet  ✔ Healthy  2m27s  canary
│     └──□ canary-rollouts-traffic-demo-69f4dd6c8-f5bqd   Pod         ✔ Running  2m27s  ready:1/1
└──# revision:1                                                                         
   └──⧉ canary-rollouts-traffic-demo-5c8c98d57f           ReplicaSet  ✔ Healthy  12m    stable
      ├──□ canary-rollouts-traffic-demo-5c8c98d57f-6fwcj  Pod         ✔ Running  12m    ready:1/1
      ├──□ canary-rollouts-traffic-demo-5c8c98d57f-jbqcv  Pod         ✔ Running  12m    ready:1/1
      └──□ canary-rollouts-traffic-demo-5c8c98d57f-mrqmx  Pod         ✔ Running  12m    ready:1/1

curl 命令测试：

# 目前v2的权重占30%
[root@master01 ~]# for i in {1..10}; do curl canary.example.com; done
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-6fwcj | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-jbqcv | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-mrqmx | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-6fwcj | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-6fwcj | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-f5bqd | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-jbqcv | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-mrqmx | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-jbqcv | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-jbqcv | v=1

因为配置文件中 pause: {} ，需要在命令行手动执行灰度发布，命令执行完成后就能达到 60% 灰度


[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-traffic-demo -ndemo

再次查看滚动更新状态：

[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-traffic-demo -ndemo --watch
# 回显
Name:            canary-rollouts-traffic-demo
Namespace:       demo
Status:          ॥ Paused
Message:         CanaryPauseStep
Strategy:        Canary
  Step:          3/6
  SetWeight:     60 #权重比例
  ActualWeight:  60 #权重比例
Images:          registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
                 registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (canary)
Replicas:
  Desired:       3
  Current:       5
  Updated:       2
  Ready:         5
  Available:     5

查看 canary ingress 资源：

[root@master01 ~]# kgi -ndemo canary-rollouts-traffic-demo-rollouts-traffic-stable-canary -oyaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "60"
  creationTimestamp: "2025-04-24T12:01:59Z"
  generation: 1
  name: canary-rollouts-traffic-demo-rollouts-traffic-stable-canary
  namespace: demo
  ownerReferences:
  - apiVersion: argoproj.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: Rollout
    name: canary-rollouts-traffic-demo
    uid: a40a95e0-617a-4f40-b7ed-79f07e17cefa
  resourceVersion: "2571984"
  uid: 465ee9e2-71d1-45cb-92ad-ad1f5e7ffc27
spec:
  ingressClassName: nginx
  rules:
  - host: canary.example.com
    http:
      paths:
      - backend:
          service:
            name: rollouts-traffic-canary
            port:
              number: 8080
        path: /
        pathType: Prefix
status:
  loadBalancer:
    ingress:
    - ip: <node-ip>

继续执行命令完成 100% 的灰度流程：


[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-traffic-demo -ndemo

再次查看滚动更新状态：

[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-traffic-demo -ndemo --watch
# 回显
Name:            canary-rollouts-traffic-demo
Namespace:       demo
Status:          ॥ Paused
Message:         CanaryPauseStep
Strategy:        Canary
  Step:          5/6
  SetWeight:     100
  ActualWeight:  100
Images:          registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
                 registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (canary)
Replicas:
  Desired:       3
  Current:       6
  Updated:       3
  Ready:         6
  Available:     6
NAME                                                      KIND        STATUS     AGE    INFO
⟳ canary-rollouts-traffic-demo                            Rollout     ॥ Paused   16m    
├──# revision:2                                                                         
│  └──⧉ canary-rollouts-traffic-demo-69f4dd6c8            ReplicaSet  ✔ Healthy  6m29s  canary
│     ├──□ canary-rollouts-traffic-demo-69f4dd6c8-f5bqd   Pod         ✔ Running  6m29s  ready:1/1
│     ├──□ canary-rollouts-traffic-demo-69f4dd6c8-cw95h   Pod         ✔ Running  3m11s  ready:1/1
│     └──□ canary-rollouts-traffic-demo-69f4dd6c8-npwvj   Pod         ✔ Running  62s    ready:1/1
└──# revision:1                                                                         
   └──⧉ canary-rollouts-traffic-demo-5c8c98d57f           ReplicaSet  ✔ Healthy  16m    stable
      ├──□ canary-rollouts-traffic-demo-5c8c98d57f-6fwcj  Pod         ✔ Running  16m    ready:1/1
      ├──□ canary-rollouts-traffic-demo-5c8c98d57f-jbqcv  Pod         ✔ Running  16m    ready:1/1
      └──□ canary-rollouts-traffic-demo-5c8c98d57f-mrqmx  Pod         ✔ Running  16m    ready:1/1

打开浏览器输入http://argorollouts.example.com/查看 argorollouts dashboard：

浏览器输入https://argocd.example.com/打开argocd dashboard

完成金丝雀发布及销毁 v1 版本：


[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-traffic-demo -ndemo

等待一小段时间，再次查看底层数据，观察到v1版本的pod删除，目前只剩下v2版本的pod

Name:            canary-rollouts-traffic-demo
Namespace:       demo
Status:          ✔ Healthy
Strategy:        Canary
  Step:          6/6
  SetWeight:     100
  ActualWeight:  100
Images:          registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (stable)
Replicas:
  Desired:       3
  Current:       3
  Updated:       3
  Ready:         3
  Available:     3
NAME                                                     KIND        STATUS        AGE    INFO
⟳ canary-rollouts-traffic-demo                           Rollout     ✔ Healthy     19m    
├──# revision:2                                                                           
│  └──⧉ canary-rollouts-traffic-demo-69f4dd6c8           ReplicaSet  ✔ Healthy     9m44s  stable
│     ├──□ canary-rollouts-traffic-demo-69f4dd6c8-f5bqd  Pod         ✔ Running     9m44s  ready:1/1
│     ├──□ canary-rollouts-traffic-demo-69f4dd6c8-cw95h  Pod         ✔ Running     6m26s  ready:1/1
│     └──□ canary-rollouts-traffic-demo-69f4dd6c8-npwvj  Pod         ✔ Running     4m17s  ready:1/1
└──# revision:1                                                                           
   └──⧉ canary-rollouts-traffic-demo-5c8c98d57f          ReplicaSet  • ScaledDown  19m

curl 数据验证：

# 因为版本切换为v2，所以版本全为v2
[root@master01 ~]# for i in {1..10}; do curl canary.example.com; done
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-npwvj | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-f5bqd | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-cw95h | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-npwvj | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-f5bqd | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-f5bqd | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-cw95h | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-cw95h | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-cw95h | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-npwvj | v=2

三、回退应用

3.1 命令行回退

有时候在应用上线过后，有些BUG并没有发现，这时候要回退怎么办呢？argo rollouts 有一个 undo 命令，可以进行回退。

比如我们要将版本回退到第一个版本，则执行一下命令：


[root@master01 ~]# kubectl argo rollouts undo canary-rollouts-demo -ndemo --to-revision=1

通过 watch 界面可以看到如下信息：


[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-demo -ndemo

再次 promote 执行更多的 canary 过程：


[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-demo -ndemo

最终数据：

Name: canary-rollouts-traffic-demo
Namespace: demo
Status: ॥ Paused
Message: CanaryPauseStep
Strategy: Canary
Step: 5/6
SetWeight: 100
ActualWeight: 100
Images:
  registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
  registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (canary)
Replicas:
  Desired: 3
  Current: 6
  Updated: 3
  Ready: 6
  Available: 6

3.2 argorollouts界面回退

浏览器中输入http://argorollouts.example.com/打开argorollouts界面后，点击【Rollback】

依次执行promote直至完全回退到上个版本


# 第一次回退，此时权重比例变为60%

[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-traffic-demo -ndemo

# 第二次回退，此时权重比例变为100%

[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-traffic-demo -ndemo

# 第三次回退，删除现有版本的pod，完全回滚到上一个版本

[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-traffic-demo -ndemo

# 等待一小段时间进行测试访问，目前全部为v1,回退成功

[root@master01 ~]# for i in {1..10}; do curl canary.example.com; done

Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-bxz2c | v=1

Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-95snl | v=1

Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-bxz2c | v=1

Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-95snl | v=1

Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-8f9q2 | v=1

Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-95snl | v=1

Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-8f9q2 | v=1

Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-8f9q2 | v=1

Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-8f9q2 | v=1

Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-8f9q2 | v=1

四、总结

Argo Rollout 可以认为是 Deployment 的扩展，增加了蓝绿发布和金丝雀发布策略配置，并且支持通过自动测试实现服务发布或者回滚。此外，Argo Rollout 还提供了更精细的流量管理和监控能力，开发者能够在部署过程中实施健康检查、A/B测试以及性能评估，从而确保新版本的稳定性和可靠性。

文章版权归作者所有，未经允许请勿转载。

THE END

ArgoCD
# ArgoCD # GitOps # 持续交付 # Argo Rollouts # 渐进式交付

Argo Rollouts 发布实战：蓝绿、金丝雀与回滚操作