官方站点：Prometheus Alert是开源的运维告警中心消息转发系统

手册：README - PrometheusAlert (gitbook.io)

一、为什么要用PrometheusAlert？

1.1 背景

  • 通过configmap配置文件维护告警媒介辨析度低；
  • 部分快消息告警媒介需要额外部署webhook做转发；
  • alertmanager web UI 较简单，不足以展示更多的数据信息；

1.2 架构原理

Exports --> Prometheus --> Prometheus Rules --> Alertmanager --> webhook --> PrometheusAlert --> Routes --> 企微/钉钉/飞书...

Day07-容器云平台监控一体化-图14

1.3 PrometheusAlert优点

配置灵活：通过web UI界面配置操作，相较于configmap较灵活。

功能强大：支持在线修改告警模版，告警路由，告警记录，测试等。

  • 针对Prometheus增加了告警级别，并且支持按照不同级别发送消息到不同目标对象。
  • 简化Prometheus分组配置，支持按照具体消息发送到单个或多个接收方。
  • 增加Dashboard，提供配置测试，告警消息模版自定义，告警模版测试等功能

自定义强度高：通过路由匹配规则触发更多的告警媒介（企微，钉钉，飞书，公有云电话/短消息接口）。

  • 增加手机号码配置项和号码自动轮询配置，可固定发送给单一个人告警信息，也可以通过自动轮询的方式发送到多个人员且支持按照不同日期发送到不同人员。

开源免费：PrometheusAlert 是一款完全开源的软件，可以免费使用和定制，而且有广泛的社区支持。

二、prometheusAlert部署配置

2.1 在kubernetes中运行

#Kubernetes中运行可以直接执行以下命令行即可(注意默认的部署模版中未挂载模版数据库文件db/PrometheusAlertDB.db，为防止模版数据丢失，请自行增加挂载配置 )
$ kubectl apply -n monitor -f https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/example/kubernetes/PrometheusAlert-Deployment.yaml

#启动后可使用浏览器打开以下地址查看：http://[YOUR-PrometheusAlert-URL]:8080
#默认登录帐号和密码在app.conf中有配置

2.2 使用helm部署

#clone项目源代码
$ git clone https://github.com/feiyu563/PrometheusAlert.git
$ cd PrometheusAlert/example/helm

#如需修改配置文件,请更新config中的app.conf
#helm部署模版支持配置Ingress域名，可在values.yaml中进行配置
#配置修改完成后，通过以下命令启动即可(注意默认的部署模版中未挂载模版数据库文件db/PrometheusAlertDB.db，为防止模版数据丢失，请自行增加挂载配置 )

$ helm upgrade --install monitor prometheusalert -n monitor

#启动后可使用浏览器打开以下地址查看: http://[Ingress_url]:[Ingress_port]
#默认登录帐号和密码在app.conf中有配置

2.3 使用控制器文件部署(目前使用这种部署)

2.3.1 配置PrometheusAlert使用mysql作为后端数据存储

部署mysql到Kubernetes集群：

# 创建秘钥
kubectl create secret generic mysql-root-password --from-literal=password=mysqlpwd123 -n monitor

# 创建pvc
[root@master01 7]# vim mysql-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pvc
  namespace: monitor
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: nfs-storage

# 创建mysql控制器
[root@master01 7]# vim mysql-Deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: mysql
  name: mysql
  namespace: monitor
spec:
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/abroad_images/mysql:5.7.23
        name: mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-root-password
              key: password
          # 如果你不想使用secret对象保存mysql登录密码，可以直接使用下面的方式指定，简单粗暴未尝不可
          #value: "123456"
        ports:
        - containerPort: 3306
        volumeMounts:
        - name: mysqlvolume
          mountPath: /var/lib/mysql
      volumes:
      - name: mysqlvolume
        # 使用pvc
        persistentVolumeClaim:
          claimName: mysql-pvc
---
#定义mysql的Service
apiVersion: v1
kind: Service
metadata:
  labels:
    app: svc-mysql
  name: svc-mysql
  namespace: monitor
spec:
  selector:
    app: mysql
  type: ClusterIP
  ports:
  - port: 3306
    protocol: TCP
    targetPort: 3306

# 应用
[root@master01 7]# kaf  mysql-pvc.yaml
[root@master01 7]# kaf  mysql-Deployment.yaml

创建数据库

# 获取mysql相关pod
[root@master01 7]# kubectl get pod -n monitor|grep mysql
mysql-69fdbd9999-wfbjx               1/1     Running   0             34s

# 进入 mysql pod
kubectl exec -it mysql-69fdbd9999-wfbjx -n monitor /bin/bash
# 在mysql的pod容器内执行
# 登录mysql，密码为mysqlpwd123
mysql -uroot -p
# 创建数据库prometheusalert
CREATE DATABASE prometheusalert CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;

获取 prometheusalert.sql

[root@master01 7]# vim prometheusalert.sql
-- ----------------------------
-- Records of prometheus_alert_d_b
-- ----------------------------
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (1, 'dd', 'Prometheus', 'prometheus-dd', '{{ $var := .externalURL}}{{ range $k,$v:=.alerts }}
{{if eq $v.status "resolved"}}
## [Prometheus恢复信息]({{$v.generatorURL}})
#### [{{$v.labels.alertname}}]({{$var}})
###### 告警级别：{{$v.labels.level}}
###### 开始时间：{{TimeFormat $v.startsAt "2006-01-02 15:04:05"}}
###### 结束时间：{{TimeFormat $v.endsAt "2006-01-02 15:04:05"}}
###### 故障主机IP：{{$v.labels.instance}}
##### {{$v.annotations.description}}
![Prometheus](https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png)
{{else}}
## [Prometheus告警信息]({{$v.generatorURL}})
#### [{{$v.labels.alertname}}]({{$var}})
###### 告警级别：{{$v.labels.level}}
###### 开始时间：{{TimeFormat $v.startsAt "2006-01-02 15:04:05"}}
###### 结束时间：{{TimeFormat $v.endsAt "2006-01-02 15:04:05"}}
###### 故障主机IP：{{$v.labels.instance}}
##### {{$v.annotations.description}}
![Prometheus](https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png)
{{end}}
{{ end }}
{{ $urimsg:=""}}{{ range $key,$value:=.commonLabels }}{{$urimsg =  print $urimsg $key "%3D%22" $value "%22%2C" }}{{end}}[*** 点我屏蔽该告警]({{$var}}/#/silences/new?filter=%7B{{SplitString $urimsg 0 -3}}%7D)', '2022-05-26 10:00:05');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (2, 'wx', 'Prometheus', 'prometheus-wx', '{{ $var := .externalURL}}{{ range $k,$v:=.alerts }}{{if eq $v.status "resolved"}}[PROMETHEUS-恢复信息]({{$v.generatorURL}})
> **[{{$v.labels.alertname}}]({{$var}})**
> <font color="info">告警级别:</font> {{$v.labels.level}}
> <font color="info">开始时间:</font> {{$v.startsAt}}
> <font color="info">结束时间:</font> {{$v.endsAt}}
> <font color="info">故障主机IP:</font> {{$v.labels.instance}}
> <font color="info">**{{$v.annotations.description}}**</font>{{else}}[PROMETHEUS-告警信息]({{$v.generatorURL}})
> **[{{$v.labels.alertname}}]({{$var}})**
> <font color="warning">告警级别:</font> {{$v.labels.level}}
> <font color="warning">开始时间:</font> {{$v.startsAt}}
> <font color="warning">结束时间:</font> {{$v.endsAt}}
> <font color="warning">故障主机IP:</font> {{$v.labels.instance}}
> <font color="warning">**{{$v.annotations.description}}**</font>{{end}}{{ end }}
{{ $urimsg:=""}}{{ range $key,$value:=.commonLabels }}{{$urimsg =  print $urimsg $key "%3D%22" $value "%22%2C" }}{{end}}[*** 点我屏蔽该告警]({{$var}}/#/silences/new?filter=%7B{{SplitString $urimsg 0 -3}}%7D)', '2022-05-26 09:59:49');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (3, 'fs', 'Prometheus', 'prometheus-fs', '{{ $var := .externalURL}}{{ range $k,$v:=.alerts }}{{if eq $v.status "resolved"}}**[Prometheus恢复信息]({{$v.generatorURL}})**
*[{{$v.labels.alertname}}]({{$var}})*
告警级别：{{$v.labels.level}}
开始时间：{{TimeFormat $v.startsAt "2006-01-02 15:04:05"}}
结束时间：{{TimeFormat $v.endsAt "2006-01-02 15:04:05"}}
故障主机IP：{{$v.labels.instance}}
**{{$v.annotations.description}}**{{else}}**[Prometheus告警信息]({{$v.generatorURL}})**
*[{{$v.labels.alertname}}]({{$var}})*
告警级别：{{$v.labels.level}}
开始时间：{{TimeFormat $v.startsAt "2006-01-02 15:04:05"}}
结束时间：{{TimeFormat $v.endsAt "2006-01-02 15:04:05"}}
故障主机IP：{{$v.labels.instance}}
**{{$v.annotations.description}}**{{end}}{{ end }}
{{ $urimsg:=""}}{{ range $key,$value:=.commonLabels }}{{$urimsg =  print $urimsg $key "%3D%22" $value "%22%2C" }}{{end}}[*** 点我屏蔽该告警]({{$var}}/#/silences/new?filter=%7B{{SplitString $urimsg 0 -3}}%7D)', '2022-05-26 10:10:03');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (4, 'txdx', 'Prometheus', 'prometheus-dx', '{{ range $k,$v:=.alerts }}{{if eq $v.status "resolved"}}
[Prometheus恢复信息]
{{$v.labels.alertname}}
告警级别：{{$v.labels.level}}
故障主机IP：{{$v.labels.instance}}
{{$v.annotations.description}}
{{else}}
[Prometheus告警信息]
{{$v.labels.alertname}}
告警级别：{{$v.labels.level}}
故障主机IP：{{$v.labels.instance}}
{{$v.annotations.description}}
{{end}}
{{ end }}', '2021-09-24 08:44:36');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (5, 'txdh', 'Prometheus', 'prometheus-dh', '{{ range $k,$v:=.alerts }}{{if eq $v.status "resolved"}}恢复信息故障主机IP{{$v.labels.instance}}{{$v.annotations.description}}{{else}}告警信息故障主机IP{{$v.labels.instance}}{{$v.annotations.description}}{{end}}{{ end }}', '2020-07-29 09:39:56');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (6, 'email', 'Prometheus', 'prometheus-email', '{{ $var := .externalURL}}{{ range $k,$v:=.alerts }}
{{if eq $v.status "resolved"}}
<h1><a href ={{$v.generatorURL}}>Prometheus恢复信息</a></h1>
<h2><a href ={{$var}}>{{$v.labels.alertname}}</a></h2>
<h5>告警级别：{{$v.labels.level}}</h5>
<h5>开始时间：{{$v.startsAt}}</h5>
<h5>结束时间：{{$v.endsAt}}</h5>
<h5>故障主机IP：{{$v.labels.instance}}</h5>
<h3>{{$v.annotations.description}}</h3>
<img src=https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png />
{{else}}
<h1><a href ={{$v.generatorURL}}>Prometheus告警信息</a></h1>
<h2><a href ={{$var}}>{{$v.labels.alertname}}</a></h2>
<h5>告警级别：{{$v.labels.level}}</h5>
<h5>开始时间：{{$v.startsAt}}</h5>
<h5>结束时间：{{$v.endsAt}}</h5>
<h5>故障主机IP：{{$v.labels.instance}}</h5>
<h3>{{$v.annotations.description}}</h3>
<img src=https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png />
{{end}}
{{ end }}', '2020-09-30 09:46:05');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (7, 'dd', 'Graylog2', 'graylog2-dd', '## [Graylog2告警信息](http://graylog.org)
#### {{.check_result.result_description}}
{{ range $k,$v:=.check_result.matching_messages }}
###### 告警索引：{{$v.index}}
###### 开始时间：{{GetCSTtime $v.timestamp}}
###### 告警主机：{{$v.fields.gl2_remote_ip}}:{{$v.fields.gl2_remote_port}}
##### {{$v.message}}
{{end}}
![Prometheus](https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png)', '2020-07-29 09:45:33');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (8, 'wx', 'Graylog2', 'graylog2-wx', '[Graylog2告警信息](http://graylog.org)
>**{{.check_result.result_description}}**
{{ range $k,$v:=.check_result.matching_messages }}
>告警索引:{{$v.index}}
开始时间:{{GetCSTtime $v.timestamp}}
告警主机:{{$v.fields.gl2_remote_ip}}:{{$v.fields.gl2_remote_port}}
**{{$v.message}}**
{{end}}', '2020-07-29 09:49:58');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (9, 'fs', 'Graylog2', 'graylog2-fs', '**[Graylog2告警信息](http://graylog.org)**
*{{.check_result.result_description}}*
{{ range $k,$v:=.check_result.matching_messages }}
告警索引：{{$v.index}}
开始时间：{{GetCSTtime $v.timestamp}}
告警主机：{{$v.fields.gl2_remote_ip}}:{{$v.fields.gl2_remote_port}}
**{{$v.message}}**
{{end}}', '2020-09-30 10:08:04');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (10, 'txdx', 'Graylog2', 'graylog2-dx', '{{ range $k,$v:=.check_result.matching_messages }}
告警主机 {{$v.fields.gl2_remote_ip}}端口 {{$v.fields.gl2_remote_port}}告警消息 {{$v.message}}
{{end}}', '2020-07-29 09:55:27');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (20, 'txdh', 'Graylog2', 'graylog2-dh', '{{ range $k,$v:=.check_result.matching_messages }}
告警主机 {{$v.fields.gl2_remote_ip}}端口 {{$v.fields.gl2_remote_port}}告警消息 {{$v.message}}
{{end}}', '2020-07-29 09:56:14');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (27, 'email', 'Graylog2', 'graylog2-email', '<h1><a href =http://graylog.org>Graylog2告警信息</a></h1>
<h2>{{.check_result.result_description}}</h2>
{{ range $k,$v:=.check_result.matching_messages }}
<h5>告警索引：{{$v.index}}</h5>
<h5>开始时间：{{GetCSTtime $v.timestamp}}</h5>
<h5>告警主机：{{$v.fields.gl2_remote_ip}}:{{$v.fields.gl2_remote_port}}</h5>
<h3>{{$v.message}}</h3>
{{end}}
<img src=https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png />', '2020-07-29 09:59:08');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (28, 'dd', 'Graylog3', 'graylog3-dd', '## [Graylog3告警信息](.check_result.Event.Source)
#### {{.check_result.event_definition_description}}
{{ range $k,$v:=.backlog }}
###### 告警索引：{{$v.index}}
###### 开始时间：{{GetCSTtime $v.timestamp}}
###### 告警主机：{{$v.fields.gl2_remote_ip}}:{{$v.fields.gl_2_remote_port}}
##### {{$v.message}}
{{end}}
![Prometheus](https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png)', '2020-07-29 10:00:32');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (29, 'wx', 'Graylog3', 'graylog3-wx', '[Graylog3告警信息](.check_result.Event.Source)
>**{{.check_result.event_definition_description}}**
{{ range $k,$v:=.backlog }}
>告警索引:{{$v.index}}
开始时间:{{GetCSTtime $v.timestamp}}
告警主机:{{$v.fields.gl2_remote_ip}}:{{$v.fields.gl_2_remote_port}}
**{{$v.message}}**
{{end}}', '2020-07-29 10:03:29');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (30, 'fs', 'Graylog3', 'graylog3-fs', '**[Graylog3告警信息](.check_result.Event.Source)**
*{{.check_result.event_definition_description}}*
{{ range $k,$v:=.backlog }}
告警索引：{{$v.index}}
开始时间：{{GetCSTtime $v.timestamp}}
告警主机：{{$v.fields.gl2_remote_ip}}:{{$v.fields.gl_2_remote_port}}
**{{$v.message}}**
{{end}}', '2020-09-30 09:52:11');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (31, 'txdx', 'Graylog3', 'graylog3-dx', '{{ range $k,$v:=.backlog }}
告警主机 {{$v.fields.gl2_remote_ip}}端口 {{$v.fields.gl_2_remote_port}}告警消息 {{$v.message}}
{{end}}', '2020-07-29 10:50:06');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (32, 'txdh', 'Graylog3', 'graylog3-dh', '{{ range $k,$v:=.backlog }}
告警主机 {{$v.fields.gl2_remote_ip}}端口 {{$v.fields.gl_2_remote_port}}告警消息 {{$v.message}}
{{end}}', '2020-07-29 10:50:30');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (33, 'email', 'Graylog3', 'graylog3-email', '<h1><a href =.check_result.Event.Source>Graylog3告警信息</a></h1>
<h2>{{.check_result.event_definition_description}}</h2>
{{ range $k,$v:=.backlog }}
<h5>告警索引：{{$v.index}}</h5>
<h5>开始时间：{{GetCSTtime $v.timestamp}}</h5>
<h5>告警主机：{{$v.fields.gl2_remote_ip}}:{{$v.fields.gl_2_remote_port}}</h5>
<h3>{{$v.message}}</h3>
{{end}}
<img src=https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png />', '2020-07-29 10:53:20');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (34, 'dd', 'Grafana', 'grafana-dd', '{{if eq .state "ok"}}
## [Grafana恢复信息]({{.ruleUrl}})
#### {{.ruleName}}
###### 告警级别：严重
###### 开始时间：{{GetCSTtime ""}}
##### {{.message}}
{{else}}
## [Grafana告警信息]({{.ruleUrl}})
#### {{.ruleName}}
###### 告警级别：严重
###### 开始时间：{{GetCSTtime ""}}
##### {{.message}}
{{end}}
![Prometheus](https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png)', '2020-07-29 11:40:33');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (35, 'wx', 'Grafana', 'grafana-wx', '{{if eq .state "ok"}}
[Grafana恢复信息]({{.ruleUrl}})
>**{{.ruleName}}**
>告警级别:严重
开始时间:{{GetCSTtime ""}}
{{.message}}
{{else}}
[Grafana告警信息]({{.ruleUrl}})
>**{{.ruleName}}**
>告警级别:严重
开始时间:{{GetCSTtime ""}}
{{.message}}
{{end}}', '2020-07-29 11:41:07');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (36, 'fs', 'Grafana', 'grafana-fs', '{{if eq .state "ok"}}
**[Grafana恢复信息]({{.ruleUrl}})**
*{{.ruleName}}*
告警级别：严重
开始时间：{{GetCSTtime ""}}
**{{.message}}**
{{else}}
**[Grafana告警信息]({{.ruleUrl}})**
*{{.ruleName}}*
告警级别：严重
开始时间：{{GetCSTtime ""}}
**{{.message}}**
{{end}}', '2020-09-30 09:53:04');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (37, 'txdx', 'Grafana', 'grafana-dx', '{{if eq .state "ok"}}
Grafana恢复信息{{.message}}
{{else}}
Grafana告警信息{{.message}}
{{end}}', '2020-07-29 11:44:28');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (38, 'txdh', 'Grafana', 'grafana-dh', '{{if eq .state "ok"}}
Grafana恢复信息{{.message}}
{{else}}
Grafana告警信息{{.message}}
{{end}}', '2020-07-29 11:44:49');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (39, 'email', 'Grafana', 'grafana-email', '{{if eq .state "ok"}}
<h1><a href ={{.ruleUrl}}>Grafana恢复信息</a></h1>
<h2>{{.ruleName}}</h2>
<h5>告警级别：严重</h5>
<h5>开始时间：{{GetCSTtime ""}}</h5>
<h3>{{.message}}</h3>
{{else}}
<h1><a href ={{.ruleUrl}}>Grafana恢复信息</a></h1>
<h2>{{.ruleName}}</h2>
<h5>告警级别：严重</h5>
<h5>开始时间：{{GetCSTtime ""}}</h5>
<h3>{{.message}}</h3>
{{end}}
<img src=https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png />', '2020-07-29 11:52:25');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (40, 'dd', 'SonarQube', 'sonar-dd-example', '## [Sonar告警信息]({{.serverUrl}})
###### 检测状态：{{.status}}
###### 检测时间：{{.analysedAt}}
###### ---------------------------------
{{ range $k,$v:=.qualityGate.conditions}}
###### metric：{{$v.metric}}
###### errorThreshold：{{$v.errorThreshold}}
###### operator：{{$v.operator}}
###### status：{{$v.status}}
###### -----------------------------------
{{ end }}
![Prometheus](https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png)', '2020-07-29 11:53:35');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (41, 'dd', 'Jenkins', 'jenkins-dd-example', '## [Jenkins构建信息]({{.buildUrl}})
###### Jenkins地址：[{{.buildUrl}}]({{.buildUrl}})
###### 构建项目：{{.projectName}}
###### 构建事件：{{.event}}
###### 构建名称：{{.buildName}}
###### 构建时间：{{GetCSTtime ""}}
![Prometheus](https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png)', '2020-07-29 11:54:02');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (42, 'fs', 'Prometheus', 'prometheus-fsv2', '{{ $var := .externalURL}}{{ range $k,$v:=.alerts }}
{{if eq $v.status "resolved"}}
**[Prometheus恢复信息]({{$v.generatorURL}})**
*[{{$v.labels.alertname}}]({{$var}})*
告警级别：{{$v.labels.level}}
开始时间：{{$v.startsAt}}
结束时间：{{$v.endsAt}}
故障主机IP：{{$v.labels.instance}}
**{{$v.annotations.description}}**
{{else}}
**[Prometheus告警信息]({{$v.generatorURL}})**
*[{{$v.labels.alertname}}]({{$var}})*
告警级别：{{$v.labels.level}}
开始时间：{{$v.startsAt}}
结束时间：{{$v.endsAt}}
故障主机IP：{{$v.labels.instance}}
**{{$v.annotations.description}}**
{{end}}
{{ end }}', '2020-11-20 08:05:04');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (43, 'workwechat', 'Prometheus', 'prometheus-wechatapp', '{{ $var := .externalURL}}{{ range $k,$v:=.alerts }}
{{if eq $v.status "resolved"}}
[Prometheus恢复信息]($v.generatorURL}})
>**[{{$v.labels.alertname}}]({{$var}})**
>告警级别: {{$v.labels.level}}
开始时间: {{$v.startsAt}}
结束时间: {{$v.endsAt}}
故障主机IP: {{$v.labels.instance}}
**{{$v.annotations.description}}**
{{else}}
[Prometheus告警信息]($v.generatorURL}})
>**[{{$v.labels.alertname}}]({{$var}})**
>告警级别: {{$v.labels.level}}
开始时间: {{$v.startsAt}}
结束时间: {{$v.endsAt}}
故障主机IP: {{$v.labels.instance}}
**{{$v.annotations.description}}**
{{end}}
{{ end }}', '2021-01-18 08:13:55');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (44, 'rl', 'Prometheus', 'prometheus-ruliu', '{{ $var := .externalURL}}{{ range $k,$v:=.alerts }}
{{if eq $v.status "resolved"}}
## [Prometheus恢复信息]({{$v.generatorURL}})
#### [{{$v.labels.alertname}}]({{$var}})
###### 告警级别：{{$v.labels.level}}
###### 开始时间：{{$v.startsAt}}
###### 结束时间：{{$v.endsAt}}
###### 故障主机IP：{{$v.labels.instance}}
##### {{$v.annotations.description}}
![Prometheus](https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png)
{{else}}
## [Prometheus告警信息]({{$v.generatorURL}})
#### [{{$v.labels.alertname}}]({{$var}})
###### 告警级别：{{$v.labels.level}}
###### 开始时间：{{$v.startsAt}}
###### 结束时间：{{$v.endsAt}}
###### 故障主机IP：{{$v.labels.instance}}
##### {{$v.annotations.description}}
![Prometheus](https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png)
{{end}}
{{ end }}', '2021-02-02 07:45:30');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (45, 'webhook', 'WebHook', 'prometheus-webhook', '{
{{ $var := index .alerts 0}}
{{if eq $var.status "resolved"}}
"title": "Prometheus恢复信息",
"prometheus-url": "{{$var.generatorURL}}",
"alert-name": "{{$var.labels.alertname}}",
"alertmanager-url": "{{.externalURL}}",
"alert-level": "{{$var.labels.level}}",
"alert-start-time": "{{$var.startsAt}}",
"alert-end-time": "{{$var.endsAt}}",
"alert-instance": "{{$var.labels.instance}}",
"message": "{{$var.annotations.description}}"
{{else}}
"title": "Prometheus告警信息",
"prometheus-url": "{{$var.generatorURL}}",
"alert-name": "{{$var.labels.alertname}}",
"alertmanager-url": "{{.externalURL}}",
"alert-level": "{{$var.labels.level}}",
"alert-start-time": "{{$var.startsAt}}",
"alert-end-time": "{{$var.endsAt}}",
"alert-instance": "{{$var.labels.instance}}",
"message": "{{$var.annotations.description}}"
{{end}}
}', '2021-04-29 08:04:12');
INSERT INTO prometheus_alert_d_b (id, tpltype, tpluse, tplname, tpl, created) VALUES (46, 'wx', 'ALiYun', 'aliyun', 'ALiYun {{.AlertState}}信息
>**{{.AlertName}}**
>告警级别: {{.TriggerLevel}}
开始时间: {{GetTime .Timestamp}}
故障主机: {{.InstanceName}}
------------详细信息--------------
metricName: {{.MetricName}}
expression: {{.Expression}}
signature: {{.Signature}}
metricProject: {{.MetricProject}}
userId: {{.UserId}}
namespace: {{.Namespace}}
preTriggerLevel: {{.PreTriggerLevel}}
ruleId: {{.RuleId}}
dimensions: {{.Dimensions}}
**当前值：{{.CurValue}}**', '2021-07-14 06:57:31');

导入sql

利用Navicat或命令行将db目录中的 prometheusalert.sql 导入数据库prometheusalert：

# 复制sql文件到mysql容器的/var/lib/mysql
[root@master01 7]# kubectl cp prometheusalert.sql mysql-69fdbd9999-wfbjx:/var/lib/mysql
# 导入SQL语句
kubectl exec -n monitor mysql-69fdbd9999-wfbjx -- \
bash -c "mysql -uroot -pmysqlpwd123 prometheusalert < /var/lib/mysql/prometheusalert.sql"

# 验证
[root@master01 7]# kubectl exec -it mysql-69fdbd9999-wfbjx -n monitor -- /bin/bash
root@mysql-69fdbd9999-wfbjx:/# mysql -uroot -pmysqlpwd123 -e "SHOW TABLES FROM prometheusalert"
mysql: [Warning] Using a password on the command line interface can be insecure.
+---------------------------+
| Tables_in_prometheusalert |
+---------------------------+
| alert_record              |
| alert_router              |
| prometheus_alert_d_b      |
+---------------------------+