官方文档：apache/skywalking-kubernetes: Apache SkyWalking Kubernetes Deployment Helm Chart (github.com)

一、认识skywalking组件¶

Skywalking OAP Server：Skywalking OAP Server 是 Skywalking 分析系统的核心组件之一。

主要负责接收和处理来自 Skywalking Agent 的数据，并将数据存储到后端存储中（如 Elasticsearch 或者 MySQL）。
OAP Server 还会对数据进行预处理、聚合和压缩，以提高数据处理和存储的效率。

Skywalking UI：Skywalking UI 是 Skywalking 分析系统的 Web UI。

用于展示应用程序的性能指标、调用链信息、拓扑图等分析结果。
用户可以通过 Skywalking UI 快速定位应用程序中的性能问题和瓶颈。

Skywalking ES Init：Skywalking ES Init 是用于初始化 Elasticsearch 的工具。

可以快速创建 Elasticsearch 索引和映射，以便存储和查询 Skywalking 分析系统的数据。

Elasticsearch：Elasticsearch 是一个开源的分布式搜索和分析引擎，用于存储 Skywalking 分析系统的数据。

Skywalking OAP Server 会将数据存储到 Elasticsearch 中，并通过 Elasticsearch 进行数据查询和分析。

二、部署配置¶

2.1 控制器方式部署¶

2.1.1 部署ES（集群版）¶

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: es
  namespace: devops
spec:
  serviceName: elasticsearch
  replicas: 3
  selector:
    matchLabels:
      app: elasticsearch
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      imagePullSecrets:
        - name: harborsecret
      initContainers:
      - name: increase-vm-max-map
        image: registry.cn-hangzhou.aliyuncs.com/abroad_images/busybox:1.28
        command: ["sysctl", "-w", "vm.max_map_count=262144"]
        securityContext:
          privileged: true
      - name: increase-fd-ulimit
        image: registry.cn-hangzhou.aliyuncs.com/abroad_images/busybox:1.28
        command: ["sh", "-c", "ulimit -n 65536"]
        securityContext:
          privileged: true
      containers:
      - name: elasticsearch
        image: registry.cn-hangzhou.aliyuncs.com/abroad_images/elasticsearch:7.6.2
        ports:
        - name: rest
          containerPort: 9200
        - name: inter
          containerPort: 9300
        resources:
          limits:
            cpu: 1000m
          requests:
            cpu: 1000m
        volumeMounts:
        - name: data
          mountPath: /usr/share/elasticsearch/data
        env:
        - name: cluster.name
          value: k8s-logs
        - name: node.name
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: cluster.initial_master_nodes
          value: "es-0,es-1,es-2"
        - name: discovery.zen.minimum_master_nodes
          value: "2"
        - name: discovery.seed_hosts
          value: "elasticsearch"
        - name: ES_JAVA_OPTS
          value: "-Xms512m -Xmx2048m"
        - name: network.host
          value: "0.0.0.0"
  volumeClaimTemplates:
  - metadata:
      name: data
      labels:
        app: elasticsearch
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: nfs-storage
      resources:
        requests:
          storage: 50Gi
---
kind: Service
apiVersion: v1
metadata:
  name: elasticsearch
  namespace: devops
  labels:
    app: elasticsearch
spec:
  selector:
    app: elasticsearch
  clusterIP: None
  ports:
    - port: 9200
      name: rest
    - port: 9300
      name: inter-node
---
kind: Service
apiVersion: v1
metadata:
  name: elasticsearch-client
  namespace: devops
  labels:
    app: elasticsearch
spec:
  selector:
    app: elasticsearch
  ports:
    - port: 9200
      name: rest
    - port: 9300
      name: inter-node

2.1.2 部署ES（单机版）¶

vim elasticsearch-alone.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: elasticsearch
  namespace: devops
spec:
  replicas: 1
  selector:
    matchLabels:
      app: elasticsearch
  serviceName: elasticsearch
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      containers:
      - env:
        - name: ES_JAVA_OPTS
          value: -Xms2048m -Xmx2048m
        - name: node.data
          value: "true"
        - name: node.master
          value: "true"
        - name: path.data
          value: /usr/share/elasticsearch/data
        # 自定义集群名
        - name: cluster.name
          value: es-cluster
        # 定义节点名，使用metadata.name名称
        - name: node.name
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
          # 初始化集群时，ES从中选出master节点
        - name: cluster.initial_master_nodes
          # 对应metadata.name名称加编号，编号从0开始
          value: "elasticsearch-0"
        - name: discovery.zen.minimum_master_nodes
          value: "1"
        # 发现节点的地址，discovery.seed_hosts的值应包括所有master候选节点
        # 如果discovery.seed_hosts的值是一个域名，且该域名解析到多个IP地址，那么es将处理其所有解析的IP地址。
        - name: discovery.seed_hosts
          value: "elasticsearch"
        name: elasticsearch
        image: registry.cn-hangzhou.aliyuncs.com/abroad_images/elasticsearch:7.17.4
        imagePullPolicy: IfNotPresent
        lifecycle:
          postStart:
            exec:
              command:
              - /bin/sh
              - -c
              - |
                sysctl -w vm.max_map_count=262144
                ulimit -l unlimited
                ulimit -n 65536
                chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/data
        ports:
        - containerPort: 9200
          name: 9200tcp2
          protocol: TCP
        - containerPort: 9300
          name: 9300tcp2
          protocol: TCP
        resources:
          limits:
            cpu: "2"
            memory: 4Gi
          requests:
            cpu: "1"
            memory: 2Gi
        # 设置挂载目录
        volumeMounts:
          - name: elasticsearch-data
            mountPath: /usr/share/elasticsearch/data
  volumeClaimTemplates:
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
     # 对应容器中volumeMounts.name
      name: elasticsearch-data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 50Gi
      storageClassName: nfs-storage
---
apiVersion: v1
kind: Service
metadata:
  name: elasticsearch
  namespace: devops
spec:
  clusterIP: None
  ports:
  - name: elasticsearch-in
    port: 9300
    protocol: TCP
    targetPort: 9300
  - name: elasticsearch-out
    port: 9200
    protocol: TCP
    targetPort: 9200
  selector:
    app: elasticsearch
  type: ClusterIP

2.1.3 部署RBAC¶

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: skywalking
  name: skywalking-oap
  namespace: devops
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: skywalking
  namespace: devops
  labels:
    app: skywalking
rules:
  - apiGroups: [""]
    resources: ["pods", "endpoints", "services", "nodes"]
    verbs: ["get", "watch", "list"]
  - apiGroups: ["extensions"]
    resources: ["deployments", "replicasets"]
    verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: skywalking
  namespace: devops
  labels:
    app: skywalking
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: skywalking
subjects:
  - kind: ServiceAccount
    name: skywalking-oap
    namespace: devops

2.1.4 数据初始化Job¶

apiVersion: batch/v1
kind: Job
metadata:
  name: "skywalking-es-init"
  namespace: devops
  labels:
    app: skywalking-job
spec:
  template:
    metadata:
      name: "skywalking-es-init"
      labels:
        app: skywalking-job
    spec:
      serviceAccountName: skywalking-oap
      restartPolicy: Never
      initContainers:
      - name: wait-for-elasticsearch
        image: registry.cn-hangzhou.aliyuncs.com/abroad_images/busybox:1.30
        imagePullPolicy: IfNotPresent
        command: ['sh', '-c', 'for i in $(seq 1 60); do nc -z -w3 elasticsearch 9200 && exit 0 || sleep 5; done; exit 1']
      containers:
      - name: oap
        image: registry.cn-hangzhou.aliyuncs.com/github_images1024/skywalking-oap-server:8.9.0
        imagePullPolicy: IfNotPresent
        env:
        - name: JAVA_OPTS
          value: "-Xmx2g -Xms2g -Dmode=init"
        - name: SW_STORAGE
          value: elasticsearch
        - name: SW_STORAGE_ES_CLUSTER_NODES
          value: "elasticsearch:9200"
        volumeMounts:
      volumes:

2.1.5 部署OAP¶

---
apiVersion: v1
kind: Service
metadata:
  name: oap-svc
  namespace: devops
  labels:
    app: oap
spec:
  type: ClusterIP
  ports:
  - port: 11800
    name: grpc
  - port: 12800
    name: rest
  selector:
    app: oap
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: oap
  name: oap
  namespace: devops
spec:
  replicas: 1
  selector:
    matchLabels:
      app: oap
  template:
    metadata:
      labels:
        app: oap
    spec:
      serviceAccountName: skywalking-oap
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            podAffinityTerm:
              topologyKey: kubernetes.io/hostname
              labelSelector:
                matchLabels:
                  app: "skywalking"
      initContainers:
      - name: wait-for-elasticsearch
        image: registry.cn-hangzhou.aliyuncs.com/abroad_images/busybox:1.30
        imagePullPolicy: IfNotPresent
        command: ['sh', '-c', 'for i in $(seq 1 60); do nc -z -w3 elasticsearch.devops.svc 9200 && exit 0 || sleep 5; done; exit 1']
      containers:
      - name: oap
        image: registry.cn-hangzhou.aliyuncs.com/github_images1024/skywalking-oap-server:8.9.0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          tcpSocket:
            port: 12800
          initialDelaySeconds: 15
          periodSeconds: 20
        readinessProbe:
          tcpSocket:
            port: 12800
          initialDelaySeconds: 15
          periodSeconds: 20
        ports:
        - containerPort: 11800
          name: grpc
        - containerPort: 12800
          name: rest
        env:
        - name: JAVA_OPTS
          value: "-Dmode=no-init -Xmx2g -Xms2g"
        - name: SW_CLUSTER
          value: kubernetes
        - name: SW_CLUSTER_K8S_NAMESPACE
          value: "default"
        - name: SW_CLUSTER_K8S_LABEL
          value: "app=skywalking,release=skywalking,component=oap"
        # 记录数据
        - name: SW_CORE_RECORD_DATA_TTL
          value: "2"
        # Metrics数据
        - name: SW_CORE_METRICS_DATA_TTL
          value: "2"
        - name: SKYWALKING_COLLECTOR_UID
          valueFrom:
            fieldRef:
              fieldPath: metadata.uid
        - name: SW_STORAGE
          value: elasticsearch
        - name: SW_STORAGE_ES_CLUSTER_NODES
          value: "elasticsearch.devops.svc:9200"

2.1.6 部署UI¶

---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: ui
  name: ui-svc
  namespace: devops
spec:
  type: ClusterIP
  ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
  selector:
    app: ui
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ui
  namespace: devops
  labels:
    app: ui
spec:
  replicas: 1
  selector:
    matchLabels:
        app: ui
  template:
    metadata:
      labels:
        app: ui
    spec:
      affinity:
      containers:
      - name: ui
        image: registry.cn-hangzhou.aliyuncs.com/github_images1024/skywalking-ui:8.9.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
          name: page
        env:
        - name: SW_OAP_ADDRESS
          value: http://oap-svc:12800  #根据oap的svc一致

2.1.7 部署Ingress¶

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: devops
  name: skywalking-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: skywalking.zhang-qing.com
    http:
      paths:
        - pathType: Prefix
          backend:
            service:
              name: ui-svc
              port:
                number: 8080
          path: /

SkyWalking 部署实战：控制器方式部署 OAP、UI 与 Elasticsearch