一、部署模式和方法

注意:

  • 从minio.RELEASE.2025-05-24T17-08-30Z 版本开始界面变化较大,功能很大的缺失
  • 建议使用前一个功能完整的版本: minio.RELEASE.2025-04-22T22-12-26Z

部署模式

  • 单机单硬盘
  • 单机多硬盘
  • 多机多硬盘

部署方法

  • 包安装

  • 二进制安装

  • Docker 容器化安装

  • 基于 Kubernetes 部署

二、单机部署

2.1 单机部署说明

https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-miniosingle-node-single-drive.html
https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-miniosingle-node-multi-drive.html
https://min.io/docs/minio/linux/index.html
https://www.minio.org.cn/docs/minio/linux/index.html

minio server的standalone模式,即单机模式,所有管理的磁盘都在一个主机上。

该启动模式一般仅用于实验环境、测试环境、开发环境的验证和学习使用。

在standalone模式下,还可以分为non-erasure code mode和erasure code mode(纠删码)。

1、non-erasure code mode

当minio server 运行时只传入一个本地磁盘参数。即为 non-erasure code mode

在此启动模式下,对于每一份对象数据, minio直接存储这份数据,不会建立副本,也不会启用纠删码机制。

因此,这种模式无论是服务实例还是磁盘都是”单点",无任何高可用保障,磁盘损坏就意味着数据丢失。

2、erasure code mode

此模式需要为minio server实例传入多个本地磁盘参数。

一旦遇到多于一 个磁盘的参数, minio server会自动启用erasure code mode.

erasure code对磁盘的个数是有要求的,至少4个磁盘, 如不满足要求,实例启动将失败。

erasure code启用后,要求传给minio server的endpoint(standalone模式下,即本地磁盘上的目录)至少为4个。

2.2 包安装

2.2.1 Debian/Ubuntu包安装

https://www.minio.org.cn/docs/minio/linux/index.html
https://dl.min.io/server/minio/release/

image

范例: Debian/Ubuntu包安装

#安装指定版本,建议使用minio.RELEASE.2025-04-22T22-12-26Z学习,注意:minio-05-24T17-08-30Z 版本以后界面功能功能缺失
[root@ubuntu2204 ~]#MINIO_VERION=20250422221226.0.0
[root@ubuntu2204 ~]#MINIO_VERION=20241013133411.0.0
[root@ubuntu2204 ~]#MINIO_VERION=20230829230735.0.0
[root@ubuntu2204 ~]#wget https://dl.min.io/server/minio/release/linux-amd64/archive/minio_${MINIO_VERION}_amd64.deb -O minio.deb
[root@ubuntu2204 ~]#dpkg -i minio.deb
[root@ubuntu2204 ~]#dpkg -L minio 
/lib
/lib/systemd
/lib/systemd/system
/lib/systemd/system/minio.service
/usr
/usr/local
/usr/local/bin
/usr/local/bin/minio

[root@ubuntu2204 ~]#which minio 
/usr/local/bin/minio

[root@ubuntu2204 ~]#minio -v
minio version RELEASE.2023-08-29T23-07-35Z (commit-id=07b1281046c8934c47184d1b56c78995ef960f7d)
Runtime: go1.19.12 linux/amd64
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Copyright: 2015-2023 MinIO, Inc.
#新版service文件
[root@ubuntu2404 ~]#grep -Ev "#|^$" /lib/systemd/system/minio.service
[Unit]
Description=MinIO
Documentation=https://docs.min.io
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/minio
[Service]
Type=notify
WorkingDirectory=/usr/local
User=minio-user
Group=minio-user
ProtectProc=invisible
EnvironmentFile=-/etc/default/minio
ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES
Restart=always
LimitNOFILE=1048576
MemoryAccounting=no
TasksMax=infinity
TimeoutSec=infinity
OOMScoreAdjust=-1000
SendSIGKILL=no
[Install]
WantedBy=multi-user.target
#缺失环境变量定义,导致无法启动
[root@ubuntu2404 ~]#ls /etc/default/minio
ls: cannot access '/etc/default/minio': No such file or directory
[root@ubuntu2404 ~]#systemctl enable --now minio.service 
Created symlink /etc/systemd/system/multi-user.target.wants/minio.service  /usr/lib/systemd/system/minio.service.
Job for minio.service failed because the control process exited with error code.
See "systemctl status minio.service" and "journalctl -xeu minio.service" for details.
#查看service文件
[root@ubuntu2204 ~]#grep -Ev "#|^$" /lib/systemd/system/minio.service
[Unit]
Description=MinIO
Documentation=https://docs.min.io
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/minio
[Service]
Type=notify
WorkingDirectory=/usr/local
User=minio-user
Group=minio-user
ProtectProc=invisible
EnvironmentFile=-/etc/default/minio
ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /etc/default/minio\"; exit 1; fi"
ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES
Restart=always
LimitNOFILE=1048576
TasksMax=infinity
TimeoutStopSec=infinity
SendSIGKILL=no
[Install]
WantedBy=multi-user.target
#需要手动创建用户
#方法1:
[root@ubuntu2204 ~]#useradd -M -r -s /sbin/nologin minio-user
#方法2:
[root@ubuntu2204 ~]#groupadd -r minio-user
[root@ubuntu2204 ~]#useradd -M -r -s /sbin/nologin -g minio-user minio-user
#准备多个数据目录,minio数据目录不允许和根文件系统在一起,注意:新版允许
[root@ubuntu2204 ~]#mkdir -p /data
[root@ubuntu2204 ~]#lvcreate -n minio -L 10G ubuntu-vg
[root@ubuntu2204 ~]#mkfs.ext4 /dev/ubuntu-vg/minio
[root@ubuntu2204 ~]#echo /dev/ubuntu-vg/minio /data ext4 defaults 0 0 >> /etc/fstab
[root@ubuntu2204 ~]#mount -a
[root@ubuntu2204 ~]#mkdir -p /data/minio{1..4} 
[root@ubuntu2204 ~]#chown -R minio-user.minio-user /data/
[root@ubuntu2204 ~]#ll -d /data/minio{1..4} 
drwxr-xr-x 3 minio-user minio-user  4096 Oct 17 15:33 /data/minio1/
drwxr-xr-x 3 minio-user minio-user  4096 Oct 17 15:33 /data/minio2/
drwxr-xr-x 3 minio-user minio-user  4096 Oct 17 15:33 /data/minio3/
drwxr-xr-x 3 minio-user minio-user  4096 Oct 17 15:33 /data/minio4/
[root@ubuntu2204 ~]#cat > /etc/default/minio <<EOF
MINIO_ROOT_USER=admin                #默认minioadmin
MINIO_ROOT_PASSWORD=12345678         #默认minioadmin,密码不能低于8位
MINIO_VOLUMES="/data/minio{1...4}"   #必选项
MINIO_OPTS='--console-address :9001' #默认使用随机端口,可以访问9000进行跳转到此随机端口
#MINIO_CONSOLE_ADDRESS=":9001" #功能同上
MINIO_PROMETHEUS_AUTH_TYPE="public"  #支持promethues监控:curl -s http://127.:9000/minio/v2/metrics/cluster
EOF
[root@ubuntu2204 ~]#systemctl enable --now minio.service 
Created symlink /etc/systemd/system/multi-user.target.wants/minio.service  /lib/systemd/system/minio.service.
[root@ubuntu2204 ~]#systemctl status minio.service  minio.service - MinIO
     Loaded: loaded (/lib/systemd/system/minio.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-10-17 15:42:50 CST; 5s ago
       Docs: https://docs.min.io
    Process: 1871 ExecStartPre=/bin/bash -c if [ -z "${MINIO_VOLUMES}" ]; then echo "Variable MINIO_VOLUMES not set i>
   Main PID: 1873 (minio)
     Tasks: 7
     Memory: 87.7M
     CPU: 362ms
     CGroup: /system.slice/minio.service
             └─1873 /usr/local/bin/minio server --console-address :9001 /data/minio{1...4}
Oct 17 15:42:49 ubuntu2204 systemd[1]: Starting MinIO...
Oct 17 15:42:50 ubuntu2204 systemd[1]: Started MinIO.
Oct 17 15:42:50 ubuntu2204 minio[1873]: MinIO Object Storage Server
Oct 17 15:42:50 ubuntu2204 minio[1873]: Copyright: 2015-2023 MinIO, Inc.
Oct 17 15:42:50 ubuntu2204 minio[1873]: License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Oct 17 15:42:50 ubuntu2204 minio[1873]: Version: RELEASE.2023-08-29T23-07-35Z (go1.19.12 linux/amd64)
Oct 17 15:42:50 ubuntu2204 minio[1873]: Status:         4 Online, 0 Offline.
Oct 17 15:42:50 ubuntu2204 minio[1873]: S3-API: http://10.0.0.100:9000 http://127.0.0.1:9000
Oct 17 15:42:50 ubuntu2204 minio[1873]: Console: http://10.0.0.100:9001 http://127.0.0.1:9001
Oct 17 15:42:50 ubuntu2204 minio[1873]: Documentation: https://min.io/docs/minio/linux/index.html
[root@ubuntu2204 ~]#ss -ntlp|grep minio
LISTEN 0      4096       127.0.0.1:9000      0.0.0.0:*   users:(("minio",pid=1873,fd=10))       
LISTEN 0      4096             *:9000        *:*   users:(("minio",pid=1873,fd=11))       
LISTEN 0      4096           [::1]:9000       [::]:*   users:(("minio",pid=1873,fd=12))       
LISTEN 0      4096             *:9001        *:*   users:(("minio",pid=1873,fd=7))

#登录 http://10.0.0.100:9001/用户名密码 admin/12345678

image

2.2.2 红帽系统包安装

https://min.io/download? license=enterprise&platform=linux&subscription=enterprise-plus

image

dnf install https://dl.min.io/enterprise/minio/release/linux-amd64/minio-20240731080646.0.0-1.x86_64.rpm MINIO_ROOT_USER=admin MINIO_ROOT_PASSWORD=password minio server /mnt/data -- console-address ":9001"

范例: 红帽系统包安装

wget https://dl.min.io/server/minio/release/linux-amd64/archive/minio-20230829230735.0.0.x86_64.rpm -o minio.rpm
sudo df install minio.rpm
mkdir ~/minio
minio server ~/minio --console-address :9090

#默认用户和密码
#RootUser: minioadmin
#RootPass: minioadmin

2.3 二进制部署

2.3.1 二进制部署单机单磁盘模式

注意:建议minio.RELEASE.2025-04-22T22-12-26Z安装, 从minio-05-24T17-08-30Z 版本后界面变大较大功能缩水

https://dl.min.io/server/minio/release/
https://min.io/download?
license=enterprise&platform=linux&subscription=enterprise-plus

image-20260406110740987

image-20260406110747098

image

image

wget https://dl.min.io/server/minio/release/linux-amd64/minio
nmod +x minio
MINIO_ROOT_USER=admin MINIO_ROOT_PASSWORD=password ./minio server /mnt/data --console-address ':9001'

范例: 二进制部署单机单磁盘

#如果启动报错:原因是下载了enterpise版本
FATAL Unable to validate license, terminating process...: license: no license found
#国内镜像下载推荐使用
[root@ubuntu2204 ~]#wget https://dl.minio.org.cn/server/minio/release/linux-amd64/minio
[root@ubuntu2204 ~]#install minio /usr/local/bin/
#准备数据目录,建议此目录为LVM
[root@ubuntu2204 ~]#mkdir -p /data/minio
#启动服务命令选项说明
#示例: minio server /data/minio --console-address :9090
server #表示start object storage server
--console-address #指定管理后面端口,管理后台登录用户和密码,默认为minioadmin/minioadmin.
也可以通过还境变量MINIO_ROOT_USER(至少3个字符)和MINIO_ROOT_PASSWORD(至少8个字符)进行指定
#手动执行启动
[root@ubuntu2204 ~]#minio server /data/minio --console-address :9090
#输出信息如下
Formatting 1st pool, 1 set(s), 1 drives per set.
WARNING: Host local has more than 0 drives of set. A host failure will result in data becoming unavailable.
WARNING: Detected default credentials 'minioadmin:minioadmin', we recommend that you change these values with 'MINIO_ROOT_USER' and 'MINIO_ROOT_PASSWORD'
environment variables
MinIO Object Storage Server
Copyright: 2015-2023 MinIO, Inc.
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Version: RELEASE.2023-08-29T23-07-35Z (go1.19.12 linux/amd64)
Status:         1 Online, 0 Offline.
S3-API: http://10.0.0.101:9000 http://10.10.53.81:9000 http://127.0.0.1:9000
RootUser: minioadmin
RootPass: minioadmin
Console: http://10.0.0.101:9090 http://10.10.53.81:9090 http://127.0.0.1:9090
RootUser: minioadmin
RootPass: minioadmin
Command-line: https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart
   $ mc alias set myminio http://10.0.0.101:9000 minioadmin minioadmin
Documentation: https://min.io/docs/minio/linux/index.html
Warning: The standard parity is set to 0. This can lead to data loss.
[root@ubuntu2404 ~]#useradd -r -s /sbin/nologin minio
[root@ubuntu2404 ~]#chown minio: /data/minio/
#创建service启动
[root@ubuntu2404 ~]#cat /lib/systemd/system/minio.service
[Unit]
Description=Minio
After=systemd-networkd.service systemd-resolved.service
Documentation=https://min.io
[Service]
Type=simple
Environment=MINIO_ROOT_USER=admin MINIO_ROOT_PASSWORD=12345678
ExecStart=/usr/local/bin/minio server /data/minio  --console-address ":9001"
Restart=on-failure
User=minio
Group=minio
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
OOMScoreAdjust=-1000
[Install]
WantedBy=multi-user.target
[root@ubuntu2404 ~]#systemctl daemon-reload
[root@ubuntu2404 ~]#systemctl enable --now minio.service

2.3.2 Shell 脚本实现单机单磁盘模式

范例: 一键安装Shell 脚本实现单机单磁盘模式

[root@ubuntu2204 ~]#cat install_minio_single_node.sh 
#!/bin/bash

#支持在线和离线安装
#管理后台用户和密码,注意:用户名至少3个字符,密码至少8个字符,如果不指定下面环境变量默认用户名/密码为minioadmin/minioadmin
MINIO_ROOT_USER=admin
MINIO_ROOT_PASSWORD=12345678
#管理后台监听端口
MINIO_PORT=9999
#数据目录
MINIO_DATA=/data/minio
#文件名和下载URL
MINIO_FILE=minio
MINIO_URL="https://dl.minio.org.cn/server/minio/release/linux-amd64/${MINIO_FILE}"
#MINIO_URL="https://dl.min.io/server/minio/release/linux-amd64/${MINIO_FILE}"
HOST_IP=`hostname -I|awk '{print $1}'`
COLOR_SUCCESS="echo -e \\033[1;32m"
COLOR_FAILURE="echo -e \\033[1;31m"
END="\033[m"
color () {
    RES_COL=60
    MOVE_TO_COL="echo -en \\033[${RES_COL}G"
    SETCOLOR_SUCCESS="echo -en \\033[1;32m"
    SETCOLOR_FAILURE="echo -en \\033[1;31m"
    SETCOLOR_WARNING="echo -en \\033[1;33m"
    SETCOLOR_NORMAL="echo -en \E[0m"
    echo -n "$1" && $MOVE_TO_COL
    echo -n "["
    if [ $2 = "success" -o $2 = "0" ] ;then
        ${SETCOLOR_SUCCESS}
        echo -n $" OK "    
    elif [ $2 = "failure" -o $2 = "1" ] ;then 
        ${SETCOLOR_FAILURE}
        echo -n $"FAILED"
    else
        ${SETCOLOR_WARNING}
        echo -n $"WARNING"
    fi
    ${SETCOLOR_NORMAL}
    echo -n "]"
    echo
}
install_minio(){
    if [ ! -e ${MINIO_FILE} ] ;then
        wget ${MINIO_URL} || ${COLOR_FAILURE} "下载失败!" ${END}
    fi
    install ${MINIO_FILE} /usr/local/bin/minio
    useradd -s /sbin/nologin  -r minio
    mkdir -p ${MINIO_DATA}
    chown -R minio:minio  ${MINIO_DATA}
    cat > /lib/systemd/system/minio.service <<EOF
[Unit]
Description=Minio
After=systemd-networkd.service systemd-resolved.service
Documentation=https://min.io
[Service]
Type=simple
Environment=MINIO_ROOT_USER=${MINIO_ROOT_USER}
Environment=MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD}
ExecStart=/usr/local/bin/minio server ${MINIO_DATA} --console-address ":${MINIO_PORT}"
Restart=on-failure
User=minio
Group=minio
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
OOMScoreAdjust=-1000
[Install]
WantedBy=multi-user.target
EOF
    systemctl daemon-reload 
    systemctl enable --now minio.service &>/dev/null
    systemctl is-active minio.service &>/dev/null
    if [ $?  -eq 0 ];then  
        echo
        color "MinIO 安装完成!" 0
        echo "------------------------------------------------------------------"
        echo -e "请访问链接: \E[32;1mhttp://${HOST_IP}:${MINIO_PORT}/\E[0m"
        echo -e "用户和密码: \E[32;1m${MINIO_ROOT_USER}/${MINIO_ROOT_PASSWORD}\E[0m"
    else
        color "MinIO 安装失败!" 1
        exit
    fi
}
install_minio

2.4 容器化部署

https://min.io/docs/minio/container/index.html

范例: 容器化部署单机单磁盘

docker pull registry.cn-beijing.aliyuncs.com/wangxiaochun/minio:RELEASE.2024-11-07T00-52-20Z
docker pull minio/minio
#可选
mkdir -p /data/minio
#docker部署
docker run -d \
   -p 9000:9000 \
   -p 9090:9090 \
   --name minio \
   -v /data/minio:/data \
   -e "MINIO_ROOT_USER=admin" \
   -e "MINIO_ROOT_PASSWORD=12345678" \
   registry.cn-beijing.aliyuncs.com/wangxiaochun/minio:RELEASE.2025-04-22T22-12-26Z server /data --console-address ":9090"
   #registry.cn-beijing.aliyuncs.com/wangxiaochun/minio:RELEASE.2024-11-07T00-52-20Z server /data --console-address ":9090"
#minio/minio server /data --console-address ":9090"

#podman 部署  
podman run -p 9000:9000 -p 9001:9001 minio/minio server /data/minio --console-address ":9001"

范例: Docker 部署单机多磁盘启用纠删除码功能

[root@ubuntu2204 ~]#docker run -d \
  -p 9000:9000 \
  -p 9999:9999 \
  --name minio \
  -e "MINIO_ROOT_USER=admin" \
  -e "MINIO_ROOT_PASSWORD=12345678" \
  -v /minio/data1:/data1 \
  -v /minio/data2:/data2 \
  -v /minio/data3:/data3 \
  -v /minio/data4:/data4 \
  -v /minio/data5:/data5 \
  -v /minio/data6:/data6 \
  -v /minio/data7:/data7 \
  -v /minio/data8:/data8 \
 registry.cn-beijing.aliyuncs.com/wangxiaochun/minio:RELEASE.2025-04-22T22-12-26Z server /data{1...8} --console-address ":9999"
  #registry.cn-beijing.aliyuncs.com/wangxiaochun/minio:RELEASE.2024-11-07T00-52-20Z server /data{1...8} --console-address ":9999"
  #minio/minio server /data{1...8} --console-address ":9999"

[root@ubuntu2204 ~]#docker logs minio
Automatically configured API requests per node based on available memory on the system: 17
Status:         8 Online, 0 Offline.
API: http://172.17.0.2:9000 http://127.0.0.1:9000
Console: http://172.17.0.2:9999 http://127.0.0.1:9999
Documentation: https://docs.min.io
 You are running an older version of MinIO released 2 years ago
 Update:
Run `mc admin update`

[root@ubuntu2204 ~]#tree /minio/
/minio/
├── data1
├── data2
├── data3
├── data4
├── data5
├── data6
├── data7
└── data8

8 directories, 0 files

image

image

2.5 基于 Docker Compose 部署

https://github.com/minio/minio/tree/master/docs/orchestration/docker-compose

范例:基于 Docker Compose 部署单节点

[root@ubuntu2404 ~]#cat docker-compose.yaml
version: '3.8'
services:
 minio:
    #image: minio/minio:RELEASE.2024-11-07T00-52-20Z
    #image: registry.cn-beijing.aliyuncs.com/wangxiaochun/minio:RELEASE.2024-11-07T00-52-20Z
    image: registry.cn-beijing.aliyuncs.com/wangxiaochun/minio:RELEASE.2025-04-22T22-12-26Z
    container_name: minio
    restart: always
    environment:
      MINIO_ROOT_USER: 'admin'
      MINIO_ROOT_PASSWORD: '12345678'
      MINIO_ADDRESS: ':9000'
      MINIO_CONSOLE_ADDRESS: ':9001'
    ports:
      - "9000:9000"
      - "9001:9001"
    networks:
      - minio_net
    volumes:
      - ./data:/data
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3
    command: server /data 
    #command: server --console-address ":9001" /data 
networks:
 minio_net:
   driver: bridge

[root@ubuntu2404 ~]#apt update && apt -y install docker-compose python3-pip 
[root@ubuntu2404 ~]#docker-compose version
docker-compose version 1.29.2, build unknown
docker-py version: 5.0.3
CPython version: 3.12.3
OpenSSL version: OpenSSL 3.0.13 30 Jan 2024

[root@ubuntu2404 ~]#docker-compose up -d
[root@ubuntu2404 ~]#docker-compose up -d
[root@ubuntu2404 ~]#docker-compose ps
Name                  Command                         State                                 Ports
---------------------------------------------------------------------------------------------------------------------------------------
minio   /usr/bin/docker-entrypoint ...   Up (healthy)   0.0.0.0:9000->9000/tcp,:::9000->9000/tcp, 0.0.0.0:9001->9001/tcp,:::9001->9001/tcp

[root@ubuntu2404 ~]#docker network ls
NETWORK ID     NAME              DRIVER    SCOPE
dc0d93c3c604   bridge            bridge    local
92408dddfe8e   host              host      local
f26890b252b5   none              null      local
6354afd6686a   root_minio_net    bridge    local

范例: 基于 Docker Compose 部署多服务节点多设备集群并利用 Nginx实现反向代理负载均衡

https://github.com/minio/minio/blob/master/docs/orchestration/docker-compose/docker-compose.yaml

[root@ubuntu2204 minio]#ls
docker-compose.yaml  nginx.conf

[root@ubuntu2204 minio]#cat docker-compose.yaml
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
  image: registry.cn-beijing.aliyuncs.com/wangxiaochun/minio:RELEASE.2025-04-22T22-12-26Z
  command: server --console-address ":9001" http://minio{1...4}/data{1...2}
  expose:
    - "9000"
    - "9001"
  environment:
    MINIO_ROOT_USER: admin
    MINIO_ROOT_PASSWORD: 12345678
  healthcheck:
    test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
    interval: 30s
    timeout: 20s
    retries: 3

# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
  minio1:
    <<: *minio-common
    hostname: minio1
    volumes:
      - data1-1:/data1
      - data1-2:/data2

  minio2:
    <<: *minio-common
    hostname: minio2
    volumes:
      - data2-1:/data1
      - data2-2:/data2

  minio3:
    <<: *minio-common
    hostname: minio3
    volumes:
      - data3-1:/data1
      - data3-2:/data2

  minio4:
    <<: *minio-common
    hostname: minio4
    volumes:
      - data4-1:/data1
      - data4-2:/data2

  nginx:
    image: nginx:1.19.2-alpine
    hostname: nginx
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "9000:9000"
      - "9001:9001"
    depends_on:
      - minio1
      - minio2
      - minio3
      - minio4

## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
  data1-1:
  data1-2:
  data2-1:
  data2-2:
  data3-1:
  data3-2:
  data4-1:
  data4-2:

[root@ubuntu2204 minio]#cat nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  4096;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    keepalive_timeout  65;

    # include /etc/nginx/conf.d/*.conf;

    upstream minio {
        server minio1:9000;
        server minio2:9000;
        server minio3:9000;
        server minio4:9000;
    }

    upstream console {
        ip_hash;
        server minio1:9001;
        server minio2:9001;
        server minio3:9001;
        server minio4:9001;
    }

    server {
        listen       9000;
        listen  [::]:9000;
        server_name  localhost;

        # To allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # To disable buffering
        proxy_buffering off;
        proxy_request_buffering off;

        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_connect_timeout 300;
            # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            chunked_transfer_encoding off;
            proxy_pass http://minio;
        }
    }

    server {
        listen       9001;
        listen  [::]:9001;
        server_name  localhost;

        # To allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # To disable buffering
        proxy_buffering off;
        proxy_request_buffering off;

        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-NginX-Proxy true;
            # This is necessary to pass the correct IP to be hashed
            real_ip_header X-Real-IP;
            proxy_connect_timeout 300;

            # To support websocket
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";

            chunked_transfer_encoding off;
            proxy_pass http://console;
        }
    }
}

#浏览器访问代理:http://minio.wang.org:9001,用户名密码:minioadmin/minioadmin

image-20260406111950930

三、分布式集群部署

3.1 分布式集群机制

分布式Minio 将在不同的机器上的多块硬盘组成一个对象存储服务。

由于硬盘分布在不同的节点上,分布式MInio有更好的性能,同时避免了单点故障。

image

minio server http://host{1...n}/export{1...m}

3.1.1 分布式Minio优势

数据保护

分布式Minio采用纠删码来防范多个节点宕机和位衰减bit rot。

分布式Minio至少需要4个硬盘,使用分布式Minio 会自动引入纠删码功能。

高可用

单机Minio服务存在单点故障,相反,如果是一个有N块硬盘的分布式Minio,只要有N/2硬盘在线,数据就是安全的。不过你需要至少有 $\mathsf { N } / 2 + \mathsf { 1 }$ 个硬盘来创建新的对象。

例如,一个16节点Minio集群,每个节点16块硬盘,就算8台服务器宕机,这个集群仍然可读的,不过需要9台服务器才能写数据。

性能

多个节点负载均衡,提升性能

一致性

Minio在分布式和单机模式下,所有读写操作都严格遵守read-after-write一致性模型。

3.1.2 分布式Minio实现条件

在所有节点运行同样的命令启动一个分布式Minio实例,只需要把硬盘位置做为参数传给minioserver命令即可

分布式Minio中的所有节点需要有同样的环境变量MINIO_ACCESS_KEY和MINIO_SECRET_KEY,才能建立分布式集群

注意:新版本使用环境变量 MINIO_ROOT_USER和MINIO_ROOT_PASSWORD

分布式Minio使用的磁盘里必须是干净的,里面没有数据

分布式Minio里的节点时间差不能超过3秒,可以使用NTP来保证时间一致。

3.1.3 分布式Minio注意事项

minio服务器多块数据磁盘需要使用独立的磁盘,在物理底层要相互独立避免遇到磁盘io竞争,导致minio性能直线下降,如果性能下降严重,数据量大时甚至会导致集群不可用

minio数据磁盘最大不超过2T,如果使用lvm逻辑卷,逻辑卷大小也不要超过2T,过大的磁盘或文件系统会导致后期IO延迟较高导致minio性能降低

minio集群共M节点每个节点N块数据磁盘,磁盘只要存活M*N/2,minio集群数据就是安全的,在节点数剩余M/2+1时节点可以正常读写

如果使用lvm方式扩展集群容量,请在部署阶段minio数据目录就使用lvm。

如果需要备份minio集群数据,请准备存放minio集群所有对象数据的空间容量

比如: 一个2T/盘4块/节点8节点=64T的集群所有容量的一半存储空间即32T服务器,配置内存CPU配置不需要太高

如果网络环境允许请把minio集群节点配置双网卡,节点通信网络与客户端访问网络分开避免网络瓶颈

配置反向代理实现MinIO的负载均衡, 可以使用云服务SLB或者2台haproxy/nginx结合keepalived实现高可用

minio系统中不要安装消耗IO较高的应用,比如:updatedb程序,如安装请排除扫描minio数据目录,否则可以会导致磁盘io延迟过高,会导致cpu负载过高,从而降低minio性能

3.2 分布式集群部署案例

https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html

注意:每个数据目录默认不能使用系统的根文件系统 rootfs,必须为独立的文件系统

3.2.1 二进制部署分布式集群

范例:3个节点,每节点1块盘

#启动分布式Minio实例,3个节点,每节点1块盘,需要在每个节点上都运行下面的命令
#先在每个主机上将磁盘格式化并挂载至/data/minio目录下,
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=12345678
minio server http://10.0.0.{101..103}:9000/data/minio --console-address :50000

范例:3个节点,每节点4块盘

#先每个主机的四块磁盘格式化,并分别挂载到/minio/dirve{1..4}上
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=12345678
minio server https://10.0.0.{101..103}/minio/drive{1..4}  --console-address :50000
#输出信息
Waiting for all MinIO sub-systems to be initialize...
Automatically configured API requests per node based on available memory on the system: 16
All MinIO sub-systems initialized successfully in 3.683865209s
MinIO Object Storage Server
Copyright: 2015-2023 MinIO, Inc.
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Version: RELEASE.2023-08-29T23-07-35Z (go1.19.12 linux/amd64)
Use `mc admin info` to look for latest server/drive info
 Status:         8 Online, 4 Offline.
S3-API: http://10.0.0.101:9000 http://127.0.0.1:9000
RootUser: admin
RootPass: 12345678
Console: http://10.0.0.101:50000 http://127.0.0.1:50000
RootUser: admin
RootPass: 12345678
Command-line: https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart
   $ mc alias set myminio http://10.0.0.102:9000 admin 12345678

在每个节点上创建新的bucket和上传文件,在集群中的其它节点上也可以查看到此bucket和上传的文件

image

image

范例:4个节点每节点4块盘集群

在4个节点都执行下面指令

#在4个节点都执行下面指令
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=12345678
minio server http://10.0.0.101/export1 \
             http://10.0.0.101/export2 \
             http://10.0.0.101/export3 \
             http://10.0.0.101/export4 \
             http://10.0.0.102/export1 \
             http://10.0.0.102/export2 \
             http://10.0.0.102/export3 \
             http://10.0.0.102/export4 \
             http://10.0.0.103/export1 \
             http://10.0.0.103/export2 \
             http://10.0.0.103/export3 \
             http://10.0.0.103/export4 \
             http://10.0.0.104/export1 \
             http://10.0.0.104/export2 \
             http://10.0.0.104/export3 \
             http://10.0.0.104/export4 \
             --console-address :50000
#或者下面用法
minio server http://10.0.0.10{1..4}/export{1..4} --console-address :50000

范例: 基于LVM实现二进制部署MinIO 实现3节点的分布式高可用集群(练习)

image

#准备三台主机,在所有节点上做好名称解析
[root@minio1 ~]#cat /etc/hosts
10.0.0.101 minio1.wang.org
10.0.0.102 minio2.wang.org
10.0.0.103 minio3.wang.org

#在所有节点上准备数据目录和用户
#当前环境中有LV,并且VG中有空闲空间,所以基于当前环境创建新LV实现MiniO存储
[root@ubuntu2204 ~]#lsblk 
NAME                    MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda                       8:0    0 200G  0 disk 
├─sda1                    8:1    0   1M  0 part 
├─sda2                    8:2    0   2G  0 part /boot
└─sda3                    8:3    0 198G  0 part 
 └─ubuntu--vg-ubuntu--lv 253:0    0  99G  0 lvm  /
sr0                      11:0    1 1024M  0 rom  

[root@minio1 ~]#pvs
 PV         VG        Fmt Attr PSize    PFree
/dev/sda3  ubuntu-vg  lvm2 a--  <198.00g 99.00g

[root@minio1 ~]#vgs
 VG        #PV #LV #SN Attr   VSize    VFree 
 ubuntu-vg   1   1   0 wz--n- <198.00g 99.00g

[root@minio1 ~]#lvs
 LV        VG        Attr       LSize    Pool Origin Data% Meta% Move Log Cpy%Sync Convert
 ubuntu-lv ubuntu-vg -wi-ao---- <99.00g  

#基于当前VG创建LV
[root@minio1 ~]#lvcreate -n minio -L 10G ubuntu-vg 
[root@minio1 ~]#lvs
 LV        VG        Attr       LSize    Pool Origin Data% Meta% Move Log Cpy%Sync Convert
 minio     ubuntu-vg -wi-a-----  10.00g                                                    
 ubuntu-lv ubuntu-vg -wi-ao---- <99.00g  

#XFS
[root@minio1 ~]#mkfs.xfs /dev/ubuntu-vg/minio
[root@minio1 ~]#echo /dev/ubuntu-vg/minio /data/ xfs defaults 0 0 >> /etc/fstab

#EXT4
[root@minio1 ~]#mkfs.ext4 /dev/ubuntu-vg/minio
[root@minio1 ~]#echo /dev/ubuntu-vg/minio /data/ ext4 defaults 0 0 >> /etc/fstab

[root@minio1 ~]#mkdir /data/
[root@minio1 ~]#mount -a
[root@minio1 ~]#mkdir -p /data/minio{1..4}
[root@minio1 ~]#useradd -r -s /sbin/nologin minio
[root@minio1 ~]#chown -R minio: /data/minio*

#在所有节点上下载二进制程序
[root@minio1 ~]#wget https://dl.minio.org.cn/server/minio/release/linux-amd64/minio
[root@minio1 ~]#install minio /usr/local/bin/minio

#在所有节点上准备环境变量文件
[root@minio1 ~]#cat > /etc/default/minio 
MINIO_ROOT_USER=admin
MINIO_ROOT_PASSWORD=12345678
MINIO_VOLUMES='http://minio{1...3}.wang.org:9000/data/minio{1...4}'
MINIO_OPTS='--console-address :9001'
MINIO_PROMETHEUS_AUTH_TYPE="public"

#注意事项
#MINIO_ROOT_USER管理员用户名,MINIO_ROOT_PASSWORD管理员密码
#如果minio服务器IP地址连续可以直接写IP地址写法,如果IP地址不连续,则使用前面在本地hosts解析文件配置的连续的主机名
#方法例如:MINIO_VOLUMES="http://10.0.0.10{1...3}:9000/data/minio{1..4}"

#在所有节点上创建service文件
[root@minio1 ~]#cat > /lib/systemd/system/minio.service 
[Unit]
Description=MinIO
Documentation=https://docs.min.io
Wants=network-online.target
After=network-online.target

[Service]
WorkingDirectory=/usr/local
User=minio
Group=minio
EnvironmentFile=-/etc/default/minio
ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /etc/default/minio\"; exit 1; fi"
ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES
Restart=always
LimitNOFILE=1048576
TasksMax=infinity
OOMScoreAdjust=-1000

[Install]
WantedBy=multi-user.target

#在所有节点启动服务
[root@minio1 ~]#systemctl daemon-reload
[root@minio1 ~]#systemctl enable --now minio.service 

[root@minio1 ~]#systemctl status minio.service  minio.service - MinIO
     Loaded: loaded (/lib/systemd/system/minio.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-10-17 16:28:02 CST; 6s ago
       Docs: https://docs.min.io
    Process: 1614 ExecStartPre=/bin/bash -c if [ -z "${MINIO_VOLUMES}" ]; then echo "Variable MINIO_VOLUMES not set i>
   Main PID: 1615 (minio)
     Tasks: 8 (limit: 2193)
     Memory: 145.8M
     CPU: 1.008s
     CGroup: /system.slice/minio.service
             └─1615 /usr/local/bin/minio server --console-address :9001 http://minio{1...3}.wang.org:9000/data/minio{>

Oct 17 16:28:03 minio1.wang.org minio[1615]: All MinIO sub-systems initialized successfully in 10.007151ms
Oct 17 16:28:03 minio1.wang.org minio[1615]: MinIO Object Storage Server
Oct 17 16:28:03 minio1.wang.org minio[1615]: Copyright: 2015-2023 MinIO, Inc.
Oct 17 16:28:03 minio1.wang.org minio[1615]: License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Oct 17 16:28:03 minio1.wang.org minio[1615]: Version: RELEASE.2023-10-16T04-13-43Z (go1.21.3 linux/amd64)
Oct 17 16:28:03 minio1.wang.org minio[1615]: Use `mc admin info` to look for latest server/drive info
Oct 17 16:28:03 minio1.wang.org minio[1615]: Status:         8 Online, 4 Offline.
Oct 17 16:28:03 minio1.wang.org minio[1615]: S3-API: http://10.0.0.103:9000 http://127.0.0.1:9000
Oct 17 16:28:03 minio1.wang.org minio[1615]: Console: http://10.0.0.103:9001 http://127.0.0.1:9001
Oct 17 16:28:03 minio1.wang.org minio[1615]: Documentation: https://min.io/docs/minio/linux/index.html

[root@minio1 ~]#ss -ntlp |grep minio
LISTEN 0      4096       127.0.0.1:9000      0.0.0.0:*   users:(("minio",pid=2685,fd=10))       
LISTEN 0      4096             *:9000        *:*   users:(("minio",pid=2685,fd=11))       
LISTEN 0      4096           [::1]:9000       [::]:*   users:(("minio",pid=2685,fd=12))       
LISTEN 0      4096             *:9001        *:*   users:(("minio",pid=2685,fd=24))  

#浏览器访问 http://minio{1..3}.wang.org:9001/ 用户名密码:admin/12345678

image

image

#配置反向代理,通过Haproxy或者Nginx实现minio的反向代理
#方式1:Haproxy配置
[root@ubuntu2204 ~]#apt update && apt -y install haproxy
[root@ubuntu2204 ~]#cat   /etc/haproxy/haproxy.cfg
......
listen stats
   mode http
   bind 0.0.0.0:9999
   stats enable
   log global
   stats uri /haproxy-status
   stats auth admin:123456
listen minio
   bind 10.0.0.100:9000
   mode http
   log global
   server 10.0.0.101 10.0.0.101:9000 check inter 3000 fall 2 rise 5
   server 10.0.0.102 10.0.0.102:9000 check inter 3000 fall 2 rise 5
   server 10.0.0.103 10.0.0.103:9000 check inter 3000 fall 2 rise 5
listen minio_console
   bind 10.0.0.100:9001
   mode http
   log global
   server 10.0.0.101 10.0.0.101:9001 check inter 3000 fall 2 rise 5
   server 10.0.0.102 10.0.0.102:9001 check inter 3000 fall 2 rise 5
   server 10.0.0.103 10.0.0.103:9001 check inter 3000 fall 2 rise 5

#方式2:nginx配置文件实现反向代理
[root@ubuntu2204 ~]#apt update && apt -y install nginx
[root@ubuntu2204 ~]#vim /etc/nginx/nginx.conf
....
http {
 upstream minio {
 server 10.0.0.101:9000;
 server 10.0.0.102:9000;
 server 10.0.0.103:9000;
 }
 upstream console {
        #ip_hash;
       hash $remote_addr;
       server 10.0.0.101:9001;
       server 10.0.0.102:9001;
       server 10.0.0.103:9001;
 }
 server {
       listen 9000;
       listen [::]:9000;
       server_name localhost;
        # To allow special characters in headers
       ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
       client_max_body_size 0;
        # To disable buffering 
       proxy_buffering off;
       location / {
           proxy_set_header Host $http_host;
           proxy_set_header X-Real-IP $remote_addr;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
           proxy_set_header X-Forwarded-Proto $scheme;
           proxy_connect_timeout 300;
            # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
           proxy_http_version 1.1;
           proxy_set_header Connection "";
           chunked_transfer_encoding off;
           proxy_pass http://minio;
           }
       }
       server {
           listen 9001;
           listen [::]:9001;
           server_name localhost;
            # To allow special characters in headers
           ignore_invalid_headers off;
            # Allow any size file to be uploaded.
            # Set to a value such as 1000m; to restrict file size to a specific value
           client_max_body_size 0;
            # To disable buffering
           proxy_buffering off;
           location / {
               proxy_set_header Host $http_host;
               proxy_set_header X-Real-IP $remote_addr;
               proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
               proxy_set_header X-Forwarded-Proto $scheme;
               proxy_set_header X-NginX-Proxy true;
                # This is necessary to pass the correct IP to be hashed
               real_ip_header X-Real-IP;
               proxy_connect_timeout 300;
                # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
               proxy_http_version 1.1;
               proxy_set_header Connection "";
               chunked_transfer_encoding off;
               proxy_pass http://console;
           }
     }
}
#通过Keepalived服务实现Nginx的高可用#访问代理查看集群状态, 用户名密码:admin/123456
http://minio.wang.org:9999/haproxy-status

image

image

通过代理访问minio的控制台,用户名密码:admin/12345678

http://minio.wang.org:9001/

image

image

3.2.2 容器化部署分布式集群

范例:容器化部署

[root@localhost ~]# docker network create -d macvlan --subnet=192.168.100.0/24 --ip-range=192.168.100.0/24 --gateway=192.168.100.1 -o parent=eth0 minio-net

#启动三个节点的容器
[root@localhost ~]#docker run --name minio1 -d --restart always --network minio-net --ip=192.168.100.101 -v /data/node1/export1:/export1 -v /data/node1/export2:/export2 -e "MINIO_ROOT_USER=admin" -e "MINIO_ROOT_PASSWORD=123456" -p 9000:9000 -p 9001:9001 minio/minio server http://192.168.100.10{1..3}/export{1..2} --console-address ":9001"

[root@localhost ~]#docker run --name minio2 -d --restart always --network minio-net --ip=192.168.100.102 -v /data/node2/export1:/export1 -v /data/node2/export2:/export2 -e "MINIO_ROOT_USER=admin" -e "MINIO_ROOT_PASSWORD=123456" -p 9000:9000 -p 9001:9001 minio/minio server http://192.168.100.10{1..3}/export{1..2} --console-address ":9001"

[root@localhost ~]#docker run --name minio3 -d --restart always --network minio-net --ip=192.168.100.103 -v /data/node3/export1:/export1 -v /data/node3/export2:/export2 -e "MINIO_ROOT_USER=admin" -e "MINIO_ROOT_PASSWORD=123456" -p 9000:9000 -p 9001:9001 minio/minio server http://192.168.100.10{1..3}/export{1..2} --console-address ":9001"

[root@localhost ~]#docker logs minio1
[root@localhost ~]#docker logs minio2
[root@localhost ~]#docker logs minio3

四、基于Kubernetes 部署

https://github.com/minio/minio/tree/master/docs/orchestration/kubernetes

基于Kubernetes 部署 MinIO 有多种方法

  • Manifest YAML 清单文件
  • MinIO Operator
  • Helm

4.1 YAML清单文件

环境要求: 依赖于一个支持PV动态置备的名称为sc-nfs的StorageClass

service 资源

#minio-service.yaml
---
kind: Service
apiVersion: v1
metadata:
  name: minio-headless
  labels:
    app: minio
spec:
  clusterIP: None
  #publishNotReadyAddresses: true
  selector:
    app: minio
  ports:
    - name: http
      port: 9000
      targetPort: 9000
---
apiVersion: v1
kind: Service
metadata:
  name: minio
spec:
  type: LoadBalancer
  selector:
    app: minio
  ports:
    - port: 9000
      targetPort: 9000
      name: http
    - port: 9001
      targetPort: 9001
      name: console

secret 资源

#minio-secret.yaml
apiversion: v1
kind: Secret
metadata:
    name: minio-secret
data:
    MINIO_ROOT_USER: YwRtaw4=
        # username: admin
        MINIO_ROOT_PASSWORD: MTIzNDU2Nzg=
        # root password: 12345678

statefulset 资源

#minio-statefulset.yaml
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: minio
  labels:
    app: minio
spec:
  serviceName: "minio-headless"
  replicas: 4
  podManagementPolicy: "Parallel" #并行启动pod,不配置的话模式是按顺序启动pod,minio、nacos都需要配置并行启动
  selector:
    matchLabels:
      app: minio
  template:
    metadata:
      labels:
        app: minio
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "9000"
        prometheus.io/path: "/minio/v2/metrics/node"
        #prometheus.io/path: "/minio/v2/metrics/cluster"
        #prometheus.io/path: "/minio/v2/metrics/bucket"
    spec:
      containers:
        - name: minio
          #image: minio/minio:RELEASE.2023-12-02T10-51-33Z    
          image: minio/minio:RELEASE.2024-08-03T04-33-23Z
          volumeMounts:
            - name: data
              mountPath: /data
          args:
            - server
            - http://minio-{0...3}.minio-headless.$(MINIO_POD_NAMESPACE).svc.cluster.local/data
            - "--address=:9000"
            - "--console-address=:9001"
          env:
            # MinIO access key and secret key
            - name: MINIO_ROOT_USER
              valueFrom:
                secretKeyRef:
                  name: minio-secret
                  key: MINIO_ROOT_USER
            - name: MINIO_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: minio-secret
                  key: MINIO_ROOT_PASSWORD
            - name: MINIO_POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: MINIO_PROMETHEUS_AUTH_TYPE
              value: "public"
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /minio/health/live
              port: http
              scheme: HTTP
            initialDelaySeconds: 120
            periodSeconds: 15
            successThreshold: 1
            timeoutSeconds: 10
          ports:
            - containerPort: 9000
              name: http
              protocol: TCP
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi        
        storageClassName: "sc-nfs"

ingress 资源

#minio-ingress.yaml
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minio
spec:
  ingressClassName: nginx
  rules:
    - host: minio.wang.org
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: minio
                port:
                  number: 9001

4.2 MinIO Operator

https://github.com/minio/operator/blob/master/README.md

image

4.3 Helm 安装

https://github.com/minio/minio/tree/master/helm/minio

Configure MinIO Helm repo

helm repo add minio https://charts.min.io/

Installing the Chart

Install this chart using:

helm install --namespace minio --set rootUser=rootuser,rootPassword=rootpass123 --generate-name minio/minio

The command deploys MinIO on the Kubernetes cluster in the default configuration. The configuration section lists the parameters that can be configured during installation.

Installing the Chart (toy-setup)

Minimal toy setup for testing purposes can be deployed using:

helm install --set resources.requests.memory=512Mi --set replicas=1 --set 
persistence.enabled=false --set mode=standalone --set 
rootUser=rootuser,rootPassword=rootpass123 --generate-name minio/minio

Upgrading the Chart

You can use Helm to update MinIO version in a live release. Assuming your release is named as my-release , get the values using the command:

helm get values my-release > old_values.yaml

Then change the field image.tag in old_values.yaml file with MinIO image tag you want to use. Now update the chart using

helm upgrade -f old_values.yaml my-release minio/minio

Default upgrade strategies are specified in the values.yaml file. Update these fields if you'd like to use a different strategy.

Configuration

Refer the Values file for all the possible config fields.

You can specify each parameter using the --set key=value[,key=value] argument to helm

install . For example,

helm install --name my-release --set persistence.size=1Ti minio/minio

The above command deploys MinIO server with a 1Ti backing persistent volume.

Alternately, you can provide a YAML file that specifies parameter values while installing the chart.For example,

helm install --name my-release -f values.yaml minio/minio

Persistence

This chart provisions a PersistentVolumeClaim and mounts corresponding persistent volume to

default location /export . You'll need physical storage available in the Kubernetes cluster for this to work. If you'd rather use emptyDir , disable PersistentVolumeClaim by:

helm install --set persistence.enabled=false minio/minio

Existing PersistentVolumeClaim

If a Persistent Volume Claim already exists, specify it during installation.

  • Create the PersistentVolume
  • Create the PersistentVolumeClaim
  • Install the chart
helm install --set persistence.existingClaim=PVC_NAME minio/minio