IP | 主機(jī)名 | 節(jié)點(diǎn) | 操作系統(tǒng) |
---|---|---|---|
192.168.0.10 | test10 | K8S Master | CentOS 8.0.1905 |
192.168.0.11 | test11 | K8S Worker | CentOS 8.0.1905 |
192.168.0.12 | test12 | K8S Worker | CentOS 8.0.1905 |
安裝環(huán)境
軟件名稱 | 軟件版本 | 說明 |
---|---|---|
Docker | 19.03.8 | 提供容器環(huán)境 |
docker-compose | 1.25.5 | 定義和運(yùn)行由多個(gè)容器組成的應(yīng)用 |
K8S | 1.18.2 | 是一個(gè)開源的,用于管理云平臺(tái)中多個(gè)主機(jī)上的容器化的應(yīng)用,Kubernetes的目標(biāo)是讓部署容器化的應(yīng)用簡(jiǎn)單并且高效(powerful),Kubernetes提供了應(yīng)用部署,規(guī)劃,更新,維護(hù)的一種機(jī)制。 |
GitLab | 12.1.6 | 代碼倉庫 |
Harbor | 1.10.2 | 私有鏡像倉庫 |
Jenkins | 2.222.3 | 持續(xù)集成交付 |
Docker 是一個(gè)開源的應(yīng)用容器引擎,基于 Go 語言 并遵從 Apache2.0 協(xié)議開源。
Docker 可以讓開發(fā)者打包他們的應(yīng)用以及依賴包到一個(gè)輕量級(jí)、可移植的容器中,然后發(fā)布到任何流行的 Linux 機(jī)器上,也可以實(shí)現(xiàn)虛擬化。
本文檔基于Docker 19.03.8 版本搭建Docker環(huán)境。
在所有服務(wù)器上創(chuàng)建install_docker.sh腳本,腳本內(nèi)容如下所示。
#使用阿里云鏡像中心 export REGISTRY_MIRROR=https://registry.cn-hangzhou.aliyuncs.com #安裝yum工具 dnf install yum* #安裝docker環(huán)境 yum install -y yum-utils device-mapper-persistent-data lvm2 #配置Docker的yum源 yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo #安裝容器插件 dnf install https://mirrors.aliyun.com/docker-ce/linux/centos/7/x86_64/stable/Packages/containerd.io-1.2.13-3.1.el7.x86_64.rpm #指定安裝docker 19.03.8版本 yum install -y docker-ce-19.03.8 docker-ce-cli-19.03.8 #設(shè)置Docker開機(jī)啟動(dòng) systemctl enable docker.service #啟動(dòng)Docker systemctl start docker.service #查看Docker版本 docker version
在每臺(tái)服務(wù)器上為install_docker.sh腳本賦予可執(zhí)行權(quán)限,并執(zhí)行腳本,如下所示。
# 賦予install_docker.sh腳本可執(zhí)行權(quán)限 chmod a+x ./install_docker.sh # 執(zhí)行install_docker.sh腳本 ./install_docker.sh
Compose 是用于定義和運(yùn)行多容器 Docker 應(yīng)用程序的工具。通過 Compose,您可以使用 YML 文件來配置應(yīng)用程序需要的所有服務(wù)。然后,使用一個(gè)命令,就可以從 YML 文件配置中創(chuàng)建并啟動(dòng)所有服務(wù)。
注意:在每臺(tái)服務(wù)器上安裝docker-compose
#下載并安裝docker-compose curl -L https://github.com/docker/compose/releases/download/1.25.5/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
#賦予docker-compose可執(zhí)行權(quán)限 chmod a+x /usr/local/bin/docker-compose
#查看docker-compose版本 [root@binghe ~]# docker-compose version docker-compose version 1.25.5, build 8a1c60f6 docker-py version: 4.1.0 CPython version: 3.7.5 OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019
Kubernetes是一個(gè)開源的,用于管理云平臺(tái)中多個(gè)主機(jī)上的容器化的應(yīng)用,Kubernetes的目標(biāo)是讓部署容器化的應(yīng)用簡(jiǎn)單并且高效(powerful),Kubernetes提供了應(yīng)用部署,規(guī)劃,更新,維護(hù)的一種機(jī)制。
本文檔基于K8S 1.8.12版本來搭建K8S集群
在所有服務(wù)器上創(chuàng)建install_k8s.sh腳本文件,腳本文件的內(nèi)容如下所示。
#################配置阿里云鏡像加速器開始######################## mkdir -p /etc/docker tee /etc/docker/daemon.json <<-'EOF' { "registry-mirrors": ["https://zz3sblpi.mirror.aliyuncs.com"] } EOF systemctl daemon-reload systemctl restart docker ######################配置阿里云鏡像加速器結(jié)束######################### #安裝nfs-utils yum install -y nfs-utils #安裝wget軟件下載命令 yum install -y wget #啟動(dòng)nfs-server systemctl start nfs-server #配置nfs-server開機(jī)自啟動(dòng) systemctl enable nfs-server #關(guān)閉防火墻 systemctl stop firewalld #取消防火墻開機(jī)自啟動(dòng) systemctl disable firewalld #關(guān)閉SeLinux setenforce 0 sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config # 關(guān)閉 swap swapoff -a yes | cp /etc/fstab /etc/fstab_bak cat /etc/fstab_bak |grep -v swap > /etc/fstab ############################修改 /etc/sysctl.conf開始########################### # 如果有配置,則修改 sed -i "s#^net.ipv4.ip_forward.*#net.ipv4.ip_forward=1#g" /etc/sysctl.conf sed -i "s#^net.bridge.bridge-nf-call-ip6tables.*#net.bridge.bridge-nf-call-ip6tables=1#g" /etc/sysctl.conf sed -i "s#^net.bridge.bridge-nf-call-iptables.*#net.bridge.bridge-nf-call-iptables=1#g" /etc/sysctl.conf sed -i "s#^net.ipv6.conf.all.disable_ipv6.*#net.ipv6.conf.all.disable_ipv6=1#g" /etc/sysctl.conf sed -i "s#^net.ipv6.conf.default.disable_ipv6.*#net.ipv6.conf.default.disable_ipv6=1#g" /etc/sysctl.conf sed -i "s#^net.ipv6.conf.lo.disable_ipv6.*#net.ipv6.conf.lo.disable_ipv6=1#g" /etc/sysctl.conf sed -i "s#^net.ipv6.conf.all.forwarding.*#net.ipv6.conf.all.forwarding=1#g" /etc/sysctl.conf # 可能沒有,追加 echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf echo "net.bridge.bridge-nf-call-ip6tables = 1" >> /etc/sysctl.conf echo "net.bridge.bridge-nf-call-iptables = 1" >> /etc/sysctl.conf echo "net.ipv6.conf.all.disable_ipv6 = 1" >> /etc/sysctl.conf echo "net.ipv6.conf.default.disable_ipv6 = 1" >> /etc/sysctl.conf echo "net.ipv6.conf.lo.disable_ipv6 = 1" >> /etc/sysctl.conf echo "net.ipv6.conf.all.forwarding = 1" >> /etc/sysctl.conf ############################修改 /etc/sysctl.conf結(jié)束########################### # 執(zhí)行命令使修改后的/etc/sysctl.conf文件生效 sysctl -p ################# 配置K8S的yum源開始############################# cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF ################# 配置K8S的yum源結(jié)束############################# # 卸載舊版本K8S yum remove -y kubelet kubeadm kubectl # 安裝kubelet、kubeadm、kubectl,這里我安裝的是1.18.2版本,你也可以安裝1.17.2版本 yum install -y kubelet-1.18.2 kubeadm-1.18.2 kubectl-1.18.2 # 修改docker Cgroup Driver為systemd # # 將/usr/lib/systemd/system/docker.service文件中的這一行 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock # # 修改為 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --exec-opt native.cgroupdriver=systemd # 如果不修改,在添加 worker 節(jié)點(diǎn)時(shí)可能會(huì)碰到如下錯(cuò)誤 # [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". # Please follow the guide at https://kubernetes.io/docs/setup/cri/ sed -i "s#^ExecStart=/usr/bin/dockerd.*#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --exec-opt native.cgroupdriver=systemd#g" /usr/lib/systemd/system/docker.service # 設(shè)置 docker 鏡像,提高 docker 鏡像下載速度和穩(wěn)定性 # 如果訪問 https://hub.docker.io 速度非常穩(wěn)定,也可以跳過這個(gè)步驟,一般不需配置 # curl -sSL https://kuboard.cn/install-script/set_mirror.sh | sh -s ${REGISTRY_MIRROR} # 重新加載配置文件 systemctl daemon-reload #重啟 docker systemctl restart docker # 將kubelet設(shè)置為開機(jī)啟動(dòng)并啟動(dòng)kubelet systemctl enable kubelet && systemctl start kubelet # 查看docker版本 docker version
在每臺(tái)服務(wù)器上為install_k8s.sh腳本賦予可執(zhí)行權(quán)限,并執(zhí)行腳本
# 賦予install_k8s.sh腳本可執(zhí)行權(quán)限 chmod a+x ./install_k8s.sh # 運(yùn)行install_k8s.sh腳本 ./install_k8s.sh
只在test10服務(wù)器上執(zhí)行的操作。
注意:下面的命令需要在命令行手動(dòng)執(zhí)行。
# 只在 master 節(jié)點(diǎn)執(zhí)行 # export 命令只在當(dāng)前 shell 會(huì)話中有效,開啟新的 shell 窗口后,如果要繼續(xù)安裝過程,請(qǐng)重新執(zhí)行此處的 export 命令 export MASTER_IP=192.168.0.10 # 替換 k8s.master 為 您想要的 dnsName export APISERVER_NAME=k8s.master # Kubernetes 容器組所在的網(wǎng)段,該網(wǎng)段安裝完成后,由 kubernetes 創(chuàng)建,事先并不存在于物理網(wǎng)絡(luò)中 export POD_SUBNET=172.18.0.1/16 echo "${MASTER_IP} ${APISERVER_NAME}" >> /etc/hosts
在test10服務(wù)器上創(chuàng)建init_master.sh腳本文件,文件內(nèi)容如下所示。
#!/bin/bash # 腳本出錯(cuò)時(shí)終止執(zhí)行 set -e if [ ${#POD_SUBNET} -eq 0 ] || [ ${#APISERVER_NAME} -eq 0 ]; then echo -e "\033[31;1m請(qǐng)確保您已經(jīng)設(shè)置了環(huán)境變量 POD_SUBNET 和 APISERVER_NAME \033[0m" echo 當(dāng)前POD_SUBNET=$POD_SUBNET echo 當(dāng)前APISERVER_NAME=$APISERVER_NAME exit 1 fi # 查看完整配置選項(xiàng) https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2 rm -f ./kubeadm-config.yaml cat <<EOF > ./kubeadm-config.yaml apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration kubernetesVersion: v1.18.2 imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers controlPlaneEndpoint: "${APISERVER_NAME}:6443" networking: serviceSubnet: "10.96.0.0/16" podSubnet: "${POD_SUBNET}" dnsDomain: "cluster.local" EOF # kubeadm init # 初始化kebeadm kubeadm init --config=kubeadm-config.yaml --upload-certs # 配置 kubectl rm -rf /root/.kube/ mkdir /root/.kube/ cp -i /etc/kubernetes/admin.conf /root/.kube/config # 安裝 calico 網(wǎng)絡(luò)插件 # 參考文檔 https://docs.projectcalico.org/v3.13/getting-started/kubernetes/self-managed-onprem/onpremises echo "安裝calico-3.13.1" rm -f calico-3.13.1.yaml wget https://kuboard.cn/install-script/calico/calico-3.13.1.yaml kubectl apply -f calico-3.13.1.yaml
賦予init_master.sh腳本文件可執(zhí)行權(quán)限并執(zhí)行腳本。
# 賦予init_master.sh文件可執(zhí)行權(quán)限 chmod a+x ./init_master.sh # 運(yùn)行init_master.sh腳本 ./init_master.sh
(1)確保所有容器組處于Running狀態(tài)
# 執(zhí)行如下命令,等待 3-10 分鐘,直到所有的容器組處于 Running 狀態(tài) watch kubectl get pod -n kube-system -o wide
具體執(zhí)行如下所示。
[root@test10 ~]# watch kubectl get pod -n kube-system -o wide Every 2.0s: kubectl get pod -n kube-system -o wide test10: Sun May 10 11:01:32 2020 NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES calico-kube-controllers-5b8b769fcd-5dtlp 1/1 Running 0 118s 172.18.203.66 test10 <none> <none> calico-node-fnv8g 1/1 Running 0 118s 192.168.0.10 test10 <none> <none> coredns-546565776c-27t7h 1/1 Running 0 2m1s 172.18.203.67 test10 <none> <none> coredns-546565776c-hjb8z 1/1 Running 0 2m1s 172.18.203.65 test10 <none> <none> etcd-test10 1/1 Running 0 2m7s 192.168.0.10 test10 <none> <none> kube-apiserver-test10 1/1 Running 0 2m7s 192.168.0.10 test10 <none> <none> kube-controller-manager-test10 1/1 Running 0 2m7s 192.168.0.10 test10 <none> <none> kube-proxy-dvgsr 1/1 Running 0 2m1s 192.168.0.10 test10 <none> <none> kube-scheduler-test10 1/1 Running 0 2m7s 192.168.0.10 test10 <none>
(2) 查看 Master 節(jié)點(diǎn)初始化結(jié)果
# 查看Master節(jié)點(diǎn)的初始化結(jié)果 kubectl get nodes -o wide
具體執(zhí)行如下所示。
[root@test10 ~]# kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME test10 Ready master 3m28s v1.18.2 192.168.0.10 <none> CentOS Linux 8 (Core) 4.18.0-80.el8.x86_64 docker://19.3.8
在Master節(jié)點(diǎn)(test10服務(wù)器)上執(zhí)行如下命令獲取join命令參數(shù)。
kubeadm token create --print-join-command
具體執(zhí)行如下所示。
[root@test10 ~]# kubeadm token create --print-join-command W0510 11:04:34.828126 56132 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io] kubeadm join k8s.master:6443 --token 8nblts.62xytoqufwsqzko2 --discovery-token-ca-cert-hash sha256:1717cc3e34f6a56b642b5751796530e367aa73f4113d09994ac3455e33047c0d
其中,有如下一行輸出。
kubeadm join k8s.master:6443 --token 8nblts.62xytoqufwsqzko2 --discovery-token-ca-cert-hash sha256:1717cc3e34f6a56b642b5751796530e367aa73f4113d09994ac3455e33047c0d
這行代碼就是獲取到的join命令。
注意:join命令中的token的有效時(shí)間為 2 個(gè)小時(shí),2小時(shí)內(nèi),可以使用此 token 初始化任意數(shù)量的 worker 節(jié)點(diǎn)。
針對(duì)所有的 worker 節(jié)點(diǎn)執(zhí)行,在這里,就是在test11服務(wù)器和test12服務(wù)器上執(zhí)行。
在命令分別手動(dòng)執(zhí)行如下命令。
# 只在 worker 節(jié)點(diǎn)執(zhí)行 # 192.168.0.10 為 master 節(jié)點(diǎn)的內(nèi)網(wǎng) IP export MASTER_IP=192.168.0.10 # 替換 k8s.master 為初始化 master 節(jié)點(diǎn)時(shí)所使用的 APISERVER_NAME export APISERVER_NAME=k8s.master echo "${MASTER_IP} ${APISERVER_NAME}" >> /etc/hosts # 替換為 master 節(jié)點(diǎn)上 kubeadm token create 命令輸出的join kubeadm join k8s.master:6443 --token 8nblts.62xytoqufwsqzko2 --discovery-token-ca-cert-hash sha256:1717cc3e34f6a56b642b5751796530e367aa73f4113d09994ac3455e33047c0d
具體執(zhí)行如下所示。
[root@test11 ~]# export MASTER_IP=192.168.0.10 [root@test11 ~]# export APISERVER_NAME=k8s.master [root@test11 ~]# echo "${MASTER_IP} ${APISERVER_NAME}" >> /etc/hosts [root@test11 ~]# kubeadm join k8s.master:6443 --token 8nblts.62xytoqufwsqzko2 --discovery-token-ca-cert-hash sha256:1717cc3e34f6a56b642b5751796530e367aa73f4113d09994ac3455e33047c0d W0510 11:08:27.709263 42795 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set. [preflight] Running pre-flight checks [WARNING FileExisting-tc]: tc not found in system path [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.18" ConfigMap in the kube-system namespace [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
根據(jù)輸出結(jié)果可以看出,Worker節(jié)點(diǎn)加入了K8S集群。
注意:kubeadm join…就是master 節(jié)點(diǎn)上 kubeadm token create 命令輸出的join。
在Master節(jié)點(diǎn)(test10服務(wù)器)執(zhí)行如下命令查看初始化結(jié)果。
kubectl get nodes -o wide
具體執(zhí)行如下所示。
[root@test10 ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION test10 Ready master 20m v1.18.2 test11 Ready <none> 2m46s v1.18.2 test12 Ready <none> 2m46s v1.18.2
注意:kubectl get nodes命令后面加上-o wide參數(shù)可以輸出更多的信息。
Master 節(jié)點(diǎn)的 IP 地址發(fā)生變化,導(dǎo)致 worker 節(jié)點(diǎn)不能啟動(dòng)。需要重新安裝K8S集群,并確保所有節(jié)點(diǎn)都有固定的內(nèi)網(wǎng) IP 地址。
重啟服務(wù)器后使用如下命令查看Pod的運(yùn)行狀態(tài)。
#查看所有pod的運(yùn)行情況 kubectl get pods --all-namespaces
發(fā)現(xiàn)很多 Pod 不在 Running 狀態(tài),此時(shí),需要使用如下命令刪除運(yùn)行不正常的Pod。
kubectl delete pod <pod-name> -n <pod-namespece>
注意:如果Pod 是使用 Deployment、StatefulSet 等控制器創(chuàng)建的,K8S 將創(chuàng)建新的 Pod 作為替代,重新啟動(dòng)的 Pod 通常能夠正常工作。
其中,pod-name表示運(yùn)行在K8S中的pod的名稱,pod-namespece表示命名空間。例如,需要?jiǎng)h除pod名稱為pod-test,命名空間為pod-test-namespace的pod,可以使用下面的命令。
kubectl delete pod pod-test -n pod-test-namespace
作為反向代理將外部流量導(dǎo)入集群內(nèi)部,將 Kubernetes 內(nèi)部的 Service 暴露給外部,在 Ingress 對(duì)象中通過域名匹配 Service,這樣就可以直接通過域名訪問到集群內(nèi)部的服務(wù)了。相對(duì)于 traefik 來說,nginx-ingress 性能更加優(yōu)秀。
注意:在Master節(jié)點(diǎn)(test10服務(wù)器上執(zhí)行)
創(chuàng)建ingress-nginx-namespace.yaml文件,主要的作用是創(chuàng)建ingress-nginx命名空間,文件內(nèi)容如下所示。
apiVersion: v1 kind: Namespace metadata: name: ingress-nginx labels: name: ingress-nginx
執(zhí)行如下命令創(chuàng)建ingress-nginx命名空間。
kubectl apply -f ingress-nginx-namespace.yaml
創(chuàng)建ingress-nginx-mandatory.yaml文件,主要的作用是安裝ingress-nginx。文件內(nèi)容如下所示。
apiVersion: v1 kind: Namespace metadata: name: ingress-nginx --- apiVersion: apps/v1 kind: Deployment metadata: name: default-http-backend labels: app.kubernetes.io/name: default-http-backend app.kubernetes.io/part-of: ingress-nginx namespace: ingress-nginx spec: replicas: 1 selector: matchLabels: app.kubernetes.io/name: default-http-backend app.kubernetes.io/part-of: ingress-nginx template: metadata: labels: app.kubernetes.io/name: default-http-backend app.kubernetes.io/part-of: ingress-nginx spec: terminationGracePeriodSeconds: 60 containers: - name: default-http-backend # Any image is permissible as long as: # 1. It serves a 404 page at / # 2. It serves 200 on a /healthz endpoint image: registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/defaultbackend-amd64:1.5 livenessProbe: httpGet: path: /healthz port: 8080 scheme: HTTP initialDelaySeconds: 30 timeoutSeconds: 5 ports: - containerPort: 8080 resources: limits: cpu: 10m memory: 20Mi requests: cpu: 10m memory: 20Mi --- apiVersion: v1 kind: Service metadata: name: default-http-backend namespace: ingress-nginx labels: app.kubernetes.io/name: default-http-backend app.kubernetes.io/part-of: ingress-nginx spec: ports: - port: 80 targetPort: 8080 selector: app.kubernetes.io/name: default-http-backend app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: nginx-configuration namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: tcp-services namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: udp-services namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- apiVersion: v1 kind: ServiceAccount metadata: name: nginx-ingress-serviceaccount namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: nginx-ingress-clusterrole labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - "" resources: - nodes verbs: - get - apiGroups: - "" resources: - services verbs: - get - list - watch - apiGroups: - "extensions" resources: - ingresses verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - "extensions" resources: - ingresses/status verbs: - update --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: Role metadata: name: nginx-ingress-role namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - pods - secrets - namespaces verbs: - get - apiGroups: - "" resources: - configmaps resourceNames: # Defaults to "<election-id>-<ingress-class>" # Here: "<ingress-controller-leader>-<nginx>" # This has to be adapted if you change either parameter # when launching the nginx-ingress-controller. - "ingress-controller-leader-nginx" verbs: - get - update - apiGroups: - "" resources: - configmaps verbs: - create - apiGroups: - "" resources: - endpoints verbs: - get --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: nginx-ingress-role-nisa-binding namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: nginx-ingress-role subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: nginx-ingress-clusterrole-nisa-binding labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: nginx-ingress-clusterrole subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: apps/v1 kind: Deployment metadata: name: nginx-ingress-controller namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: replicas: 1 selector: matchLabels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx template: metadata: labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx annotations: prometheus.io/port: "10254" prometheus.io/scrape: "true" spec: serviceAccountName: nginx-ingress-serviceaccount containers: - name: nginx-ingress-controller image: registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/nginx-ingress-controller:0.20.0 args: - /nginx-ingress-controller - --default-backend-service=$(POD_NAMESPACE)/default-http-backend - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix=nginx.ingress.kubernetes.io securityContext: capabilities: drop: - ALL add: - NET_BIND_SERVICE # www-data -> 33 runAsUser: 33 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ports: - name: http containerPort: 80 - name: https containerPort: 443 livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 ---
執(zhí)行如下命令安裝ingress controller。
kubectl apply -f ingress-nginx-mandatory.yaml
主要是用來用于暴露pod:nginx-ingress-controller。
創(chuàng)建service-nodeport.yaml文件,文件內(nèi)容如下所示。
apiVersion: v1 kind: Service metadata: name: ingress-nginx namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: type: NodePort ports: - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30080 - name: https port: 443 targetPort: 443 protocol: TCP nodePort: 30443 selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx
執(zhí)行如下命令安裝。
kubectl apply -f service-nodeport.yaml
查看ingress-nginx命名空間的部署情況,如下所示。
[root@test10 k8s]# kubectl get pod -n ingress-nginx NAME READY STATUS RESTARTS AGE default-http-backend-796ddcd9b-vfmgn 1/1 Running 1 10h nginx-ingress-controller-58985cc996-87754 1/1 Running 2 10h
在命令行服務(wù)器命令行輸入如下命令查看ingress-nginx的端口映射情況。
kubectl get svc -n ingress-nginx
具體如下所示。
[root@test10 k8s]# kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default-http-backend ClusterIP 10.96.247.2 <none> 80/TCP 7m3s ingress-nginx NodePort 10.96.40.6 <none> 80:30080/TCP,443:30443/TCP 4m35s
所以,可以通過Master節(jié)點(diǎn)(test10服務(wù)器)的IP地址和30080端口號(hào)來訪問ingress-nginx,如下所示。
[root@test10 k8s]# curl 192.168.0.10:30080 default backend - 404
也可以在瀏覽器打開http://192.168.0.10:30080 來訪問ingress-nginx,如下所示。
GitLab是由GitLabInc.開發(fā),使用MIT許可證的基于網(wǎng)絡(luò)的Git倉庫管理工具,且具有Wiki和issue跟蹤功能。使用Git作為代碼管理工具,并在此基礎(chǔ)上搭建起來的web服務(wù)。
注意:在Master節(jié)點(diǎn)(test10服務(wù)器上執(zhí)行)
創(chuàng)建k8s-ops-namespace.yaml文件,主要作用是創(chuàng)建k8s-ops命名空間。文件內(nèi)容如下所示。
apiVersion: v1 kind: Namespace metadata: name: k8s-ops labels: name: k8s-ops
執(zhí)行如下命令創(chuàng)建命名空間。
kubectl apply -f k8s-ops-namespace.yaml
創(chuàng)建gitlab-redis.yaml文件,文件的內(nèi)容如下所示。
apiVersion: apps/v1 kind: Deployment metadata: name: redis namespace: k8s-ops labels: name: redis spec: selector: matchLabels: name: redis template: metadata: name: redis labels: name: redis spec: containers: - name: redis image: sameersbn/redis imagePullPolicy: IfNotPresent ports: - name: redis containerPort: 6379 volumeMounts: - mountPath: /var/lib/redis name: data livenessProbe: exec: command: - redis-cli - ping initialDelaySeconds: 30 timeoutSeconds: 5 readinessProbe: exec: command: - redis-cli - ping initialDelaySeconds: 10 timeoutSeconds: 5 volumes: - name: data hostPath: path: /data1/docker/xinsrv/redis --- apiVersion: v1 kind: Service metadata: name: redis namespace: k8s-ops labels: name: redis spec: ports: - name: redis port: 6379 targetPort: redis selector: name: redis
首先,在命令行執(zhí)行如下命令創(chuàng)建/data1/docker/xinsrv/redis目錄。
mkdir -p /data1/docker/xinsrv/redis
執(zhí)行如下命令安裝gitlab-redis。
kubectl apply -f gitlab-redis.yaml
創(chuàng)建gitlab-postgresql.yaml,文件內(nèi)容如下所示。
apiVersion: apps/v1 kind: Deployment metadata: name: postgresql namespace: k8s-ops labels: name: postgresql spec: selector: matchLabels: name: postgresql template: metadata: name: postgresql labels: name: postgresql spec: containers: - name: postgresql image: sameersbn/postgresql imagePullPolicy: IfNotPresent env: - name: DB_USER value: gitlab - name: DB_PASS value: passw0rd - name: DB_NAME value: gitlab_production - name: DB_EXTENSION value: pg_trgm ports: - name: postgres containerPort: 5432 volumeMounts: - mountPath: /var/lib/postgresql name: data livenessProbe: exec: command: - pg_isready - -h - localhost - -U - postgres initialDelaySeconds: 30 timeoutSeconds: 5 readinessProbe: exec: command: - pg_isready - -h - localhost - -U - postgres initialDelaySeconds: 5 timeoutSeconds: 1 volumes: - name: data hostPath: path: /data1/docker/xinsrv/postgresql --- apiVersion: v1 kind: Service metadata: name: postgresql namespace: k8s-ops labels: name: postgresql spec: ports: - name: postgres port: 5432 targetPort: postgres selector: name: postgresql
首先,執(zhí)行如下命令創(chuàng)建/data1/docker/xinsrv/postgresql目錄。
mkdir -p /data1/docker/xinsrv/postgresql
接下來,安裝gitlab-postgresql,如下所示。
kubectl apply -f gitlab-postgresql.yaml
(1)配置用戶名和密碼
首先,在命令行使用base64編碼為用戶名和密碼進(jìn)行轉(zhuǎn)碼,本示例中,使用的用戶名為admin,密碼為admin.1231
轉(zhuǎn)碼情況如下所示。
[root@test10 k8s]# echo -n 'admin' | base64 YWRtaW4= [root@test10 k8s]# echo -n 'admin.1231' | base64 YWRtaW4uMTIzMQ==
轉(zhuǎn)碼后的用戶名為:YWRtaW4= 密碼為:YWRtaW4uMTIzMQ==
也可以對(duì)base64編碼后的字符串解碼,例如,對(duì)密碼字符串解碼,如下所示。
[root@test10 k8s]# echo 'YWRtaW4uMTIzMQ==' | base64 --decode admin.1231
接下來,創(chuàng)建secret-gitlab.yaml文件,主要是用戶來配置GitLab的用戶名和密碼,文件內(nèi)容如下所示。
apiVersion: v1 kind: Secret metadata: namespace: k8s-ops name: git-user-pass type: Opaque data: username: YWRtaW4= password: YWRtaW4uMTIzMQ==
執(zhí)行配置文件的內(nèi)容,如下所示。
kubectl create -f ./secret-gitlab.yaml
(2)安裝GitLab
創(chuàng)建gitlab.yaml文件,文件的內(nèi)容如下所示。
apiVersion: apps/v1 kind: Deployment metadata: name: gitlab namespace: k8s-ops labels: name: gitlab spec: selector: matchLabels: name: gitlab template: metadata: name: gitlab labels: name: gitlab spec: containers: - name: gitlab image: sameersbn/gitlab:12.1.6 imagePullPolicy: IfNotPresent env: - name: TZ value: Asia/Shanghai - name: GITLAB_TIMEZONE value: Beijing - name: GITLAB_SECRETS_DB_KEY_BASE value: long-and-random-alpha-numeric-string - name: GITLAB_SECRETS_SECRET_KEY_BASE value: long-and-random-alpha-numeric-string - name: GITLAB_SECRETS_OTP_KEY_BASE value: long-and-random-alpha-numeric-string - name: GITLAB_ROOT_PASSWORD valueFrom: secretKeyRef: name: git-user-pass key: password - name: GITLAB_ROOT_EMAIL value: 12345678@qq.com - name: GITLAB_HOST value: gitlab.binghe.com - name: GITLAB_PORT value: "80" - name: GITLAB_SSH_PORT value: "30022" - name: GITLAB_NOTIFY_ON_BROKEN_BUILDS value: "true" - name: GITLAB_NOTIFY_PUSHER value: "false" - name: GITLAB_BACKUP_SCHEDULE value: daily - name: GITLAB_BACKUP_TIME value: 01:00 - name: DB_TYPE value: postgres - name: DB_HOST value: postgresql - name: DB_PORT value: "5432" - name: DB_USER value: gitlab - name: DB_PASS value: passw0rd - name: DB_NAME value: gitlab_production - name: REDIS_HOST value: redis - name: REDIS_PORT value: "6379" ports: - name: http containerPort: 80 - name: ssh containerPort: 22 volumeMounts: - mountPath: /home/git/data name: data livenessProbe: httpGet: path: / port: 80 initialDelaySeconds: 180 timeoutSeconds: 5 readinessProbe: httpGet: path: / port: 80 initialDelaySeconds: 5 timeoutSeconds: 1 volumes: - name: data hostPath: path: /data1/docker/xinsrv/gitlab --- apiVersion: v1 kind: Service metadata: name: gitlab namespace: k8s-ops labels: name: gitlab spec: ports: - name: http port: 80 nodePort: 30088 - name: ssh port: 22 targetPort: ssh nodePort: 30022 type: NodePort selector: name: gitlab --- apiVersion: extensions/v1beta1 kind: Ingress metadata: name: gitlab namespace: k8s-ops annotations: kubernetes.io/ingress.class: traefik spec: rules: - host: gitlab.binghe.com http: paths: - backend: serviceName: gitlab servicePort: http
注意:在配置GitLab時(shí),監(jiān)聽主機(jī)時(shí),不能使用IP地址,需要使用主機(jī)名或者域名,上述配置中,我使用的是gitlab.binghe.com主機(jī)名。
在命令行執(zhí)行如下命令創(chuàng)建/data1/docker/xinsrv/gitlab目錄。
mkdir -p /data1/docker/xinsrv/gitlab
安裝GitLab,如下所示。
kubectl apply -f gitlab.yaml
查看k8s-ops命名空間部署情況,如下所示。
[root@test10 k8s]# kubectl get pod -n k8s-ops NAME READY STATUS RESTARTS AGE gitlab-7b459db47c-5vk6t 0/1 Running 0 11s postgresql-79567459d7-x52vx 1/1 Running 0 30m redis-67f4cdc96c-h5ckz 1/1 Running 1 10h
也可以使用如下命令查看。
[root@test10 k8s]# kubectl get pod --namespace=k8s-ops NAME READY STATUS RESTARTS AGE gitlab-7b459db47c-5vk6t 0/1 Running 0 36s postgresql-79567459d7-x52vx 1/1 Running 0 30m redis-67f4cdc96c-h5ckz 1/1 Running 1 10h
二者效果一樣。
接下來,查看GitLab的端口映射,如下所示。
[root@test10 k8s]# kubectl get svc -n k8s-ops NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE gitlab NodePort 10.96.153.100 <none> 80:30088/TCP,22:30022/TCP 2m42s postgresql ClusterIP 10.96.203.119 <none> 5432/TCP 32m redis ClusterIP 10.96.107.150 <none> 6379/TCP 10h
此時(shí),可以看到,可以通過Master節(jié)點(diǎn)(test10)的主機(jī)名gitlab.binghe.com和端口30088就能夠訪問GitLab。由于我這里使用的是虛擬機(jī)來搭建相關(guān)的環(huán)境,在本機(jī)訪問虛擬機(jī)映射的gitlab.binghe.com時(shí),需要配置本機(jī)的hosts文件,在本機(jī)的hosts文件中加入如下配置項(xiàng)。
192.168.0.10 gitlab.binghe.com
注意:在Windows操作系統(tǒng)中,hosts文件所在的目錄如下。
C:\Windows\System32\drivers\etc
接下來,就可以在瀏覽器中通過鏈接:http://gitlab.binghe.com:30088 來訪問GitLab了,如下所示。
此時(shí),可以通過用戶名root和密碼admin.1231來登錄GitLab了。
注意:這里的用戶名是root而不是admin,因?yàn)閞oot是GitLab默認(rèn)的超級(jí)用戶。
到此,K8S安裝gitlab完成。
Habor是由VMWare公司開源的容器鏡像倉庫。事實(shí)上,Habor是在Docker Registry上進(jìn)行了相應(yīng)的企業(yè)級(jí)擴(kuò)展,從而獲得了更加廣泛的應(yīng)用,這些新的企業(yè)級(jí)特性包括:管理用戶界面,基于角色的訪問控制 ,AD/LDAP集成以及審計(jì)日志等,足以滿足基本企業(yè)需求。
注意:這里將Harbor私有倉庫安裝在Master節(jié)點(diǎn)(test10服務(wù)器)上,實(shí)際生產(chǎn)環(huán)境中建議安裝在其他服務(wù)器。
wget https://github.com/goharbor/harbor/releases/download/v1.10.2/harbor-offline-installer-v1.10.2.tgz
tar -zxvf harbor-offline-installer-v1.10.2.tgz
解壓成功后,會(huì)在服務(wù)器當(dāng)前目錄生成一個(gè)harbor目錄。
注意:這里,我將Harbor的端口修改成了1180,如果不修改Harbor的端口,默認(rèn)的端口是80。
(1)修改harbor.yml文件
cd harbor vim harbor.yml
修改的配置項(xiàng)如下所示。
hostname: 192.168.0.10 http: port: 1180 harbor_admin_password: binghe123 ###并把https注釋掉,不然在安裝的時(shí)候會(huì)報(bào)錯(cuò):ERROR:root:Error: The protocol is https but attribute ssl_cert is not set #https: #port: 443 #certificate: /your/certificate/path #private_key: /your/private/key/path
(2)修改daemon.json文件
修改/etc/docker/daemon.json文件,沒有的話就創(chuàng)建,在/etc/docker/daemon.json文件中添加如下內(nèi)容。
[root@binghe~]# cat /etc/docker/daemon.json { "registry-mirrors": ["https://zz3sblpi.mirror.aliyuncs.com"], "insecure-registries":["192.168.0.10:1180"] }
也可以在服務(wù)器上使用 ip addr 命令查看本機(jī)所有的IP地址段,將其配置到/etc/docker/daemon.json文件中。這里,我配置后的文件內(nèi)容如下所示。
{ "registry-mirrors": ["https://zz3sblpi.mirror.aliyuncs.com"], "insecure-registries":["192.168.175.0/16","172.17.0.0/16", "172.18.0.0/16", "172.16.29.0/16", "192.168.0.10:1180"] }
配置完成后,輸入如下命令即可安裝并啟動(dòng)Harbor
[root@binghe harbor]# ./install.sh
安裝成功后,在瀏覽器地址欄輸入http://192.168.0.10:1180打開鏈接,輸入用戶名admin和密碼binghe123,登錄系統(tǒng)。
接下來,我們選擇用戶管理,添加一個(gè)管理員賬戶,為后續(xù)打包Docker鏡像和上傳Docker鏡像做準(zhǔn)備。
密碼為Binghe123。點(diǎn)擊確,此時(shí),賬戶binghe還不是管理員,此時(shí)選中binghe賬戶,點(diǎn)擊“設(shè)置為管理員”。
此時(shí),binghe賬戶就被設(shè)置為管理員了。到此,Harbor的安裝就完成了。
如果安裝Harbor后,大家需要修改Harbor的端口,可以按照如下步驟修改Harbor的端口,這里,我以將80端口修改為1180端口為例
(1)修改harbor.yml文件
cd harbor vim harbor.yml
修改的配置項(xiàng)如下所示。
hostname: 192.168.0.10 http: port: 1180 harbor_admin_password: binghe123 ###并把https注釋掉,不然在安裝的時(shí)候會(huì)報(bào)錯(cuò):ERROR:root:Error: The protocol is https but attribute ssl_cert is not set #https: #port: 443 #certificate: /your/certificate/path #private_key: /your/private/key/path
(2)修改docker-compose.yml文件
vim docker-compose.yml
修改的配置項(xiàng)如下所示。
ports: - 1180:80
(3)修改config.yml文件
cd common/config/registry vim config.yml
修改的配置項(xiàng)如下所示。
realm: http://192.168.0.10:1180/service/token
(4)重啟Docker
systemctl daemon-reload systemctl restart docker.service
(5)重啟Harbor
[root@binghe harbor]# docker-compose down Stopping harbor-log ... done Removing nginx ... done Removing harbor-portal ... done Removing harbor-jobservice ... done Removing harbor-core ... done Removing redis ... done Removing registry ... done Removing registryctl ... done Removing harbor-db ... done Removing harbor-log ... done Removing network harbor_harbor [root@binghe harbor]# ./prepare prepare base dir is set to /mnt/harbor Clearing the configuration file: /config/log/logrotate.conf Clearing the configuration file: /config/nginx/nginx.conf Clearing the configuration file: /config/core/env Clearing the configuration file: /config/core/app.conf Clearing the configuration file: /config/registry/root.crt Clearing the configuration file: /config/registry/config.yml Clearing the configuration file: /config/registryctl/env Clearing the configuration file: /config/registryctl/config.yml Clearing the configuration file: /config/db/env Clearing the configuration file: /config/jobservice/env Clearing the configuration file: /config/jobservice/config.yml Generated configuration file: /config/log/logrotate.conf Generated configuration file: /config/nginx/nginx.conf Generated configuration file: /config/core/env Generated configuration file: /config/core/app.conf Generated configuration file: /config/registry/config.yml Generated configuration file: /config/registryctl/env Generated configuration file: /config/db/env Generated configuration file: /config/jobservice/env Generated configuration file: /config/jobservice/config.yml loaded secret from file: /secret/keys/secretkey Generated configuration file: /compose_location/docker-compose.yml Clean up the input dir [root@binghe harbor]# docker-compose up -d Creating network "harbor_harbor" with the default driver Creating harbor-log ... done Creating harbor-db ... done Creating redis ... done Creating registry ... done Creating registryctl ... done Creating harbor-core ... done Creating harbor-jobservice ... done Creating harbor-portal ... done Creating nginx ... done [root@binghe harbor]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS
Jenkins是一個(gè)開源的、提供友好操作界面的持續(xù)集成(CI)工具,起源于Hudson(Hudson是商用的),主要用于持續(xù)、自動(dòng)的構(gòu)建/測(cè)試軟件項(xiàng)目、監(jiān)控外部任務(wù)的運(yùn)行(這個(gè)比較抽象,暫且寫上,不做解釋)。Jenkins用Java語言編寫,可在Tomcat等流行的servlet容器中運(yùn)行,也可獨(dú)立運(yùn)行。通常與版本管理工具(SCM)、構(gòu)建工具結(jié)合使用。常用的版本控制工具有SVN、GIT,構(gòu)建工具有Maven、Ant、Gradle。
使用 nfs 最大的問題就是寫權(quán)限,可以使用 kubernetes 的 securityContext/runAsUser 指定 jenkins 容器中運(yùn)行 jenkins 的用戶 uid,以此來指定 nfs 目錄的權(quán)限,讓 jenkins 容器可寫;也可以不限制,讓所有用戶都可以寫。這里為了簡(jiǎn)單,就讓所有用戶可寫了。
如果之前已經(jīng)安裝過nfs,則這一步可以省略。找一臺(tái)主機(jī),安裝 nfs,這里,我以在Master節(jié)點(diǎn)(test10服務(wù)器)上安裝nfs為例。
在命令行輸入如下命令安裝并啟動(dòng)nfs。
yum install nfs-utils -y systemctl start nfs-server systemctl enable nfs-server
在Master節(jié)點(diǎn)(test10服務(wù)器)上創(chuàng)建 /opt/nfs/jenkins-data
目錄作為nfs的共享目錄,如下所示。
mkdir -p /opt/nfs/jenkins-data
接下來,編輯/etc/exports文件,如下所示。
vim /etc/exports
在/etc/exports文件文件中添加如下一行配置。
/opt/nfs/jenkins-data 192.168.175.0/24(rw,all_squash)
這里的 ip 使用 kubernetes node 節(jié)點(diǎn)的 ip 范圍,后面的 all_squash
選項(xiàng)會(huì)將所有訪問的用戶都映射成 nfsnobody 用戶,不管你是什么用戶訪問,最終都會(huì)壓縮成 nfsnobody,所以只要將 /opt/nfs/jenkins-data
的屬主改為 nfsnobody,那么無論什么用戶來訪問都具有寫權(quán)限。
這個(gè)選項(xiàng)在很多機(jī)器上由于用戶 uid 不規(guī)范導(dǎo)致啟動(dòng)進(jìn)程的用戶不同,但是同時(shí)要對(duì)一個(gè)共享目錄具有寫權(quán)限時(shí)很有效。
接下來,為 /opt/nfs/jenkins-data
目錄授權(quán),并重新加載nfs,如下所示。
#為/opt/nfs/jenkins-data/目錄授權(quán) chown -R 1000 /opt/nfs/jenkins-data/ #重新加載nfs-server systemctl reload nfs-server
在K8S集群中任意一個(gè)節(jié)點(diǎn)上使用如下命令進(jìn)行驗(yàn)證:
#查看nfs系統(tǒng)的目錄權(quán)限 showmount -e NFS_IP
如果能夠看到 /opt/nfs/jenkins-data 就表示 ok 了。
具體如下所示。
[root@test10 ~]# showmount -e 192.168.0.10 Export list for 192.168.0.10: /opt/nfs/jenkins-data 192.168.175.0/24 [root@test11 ~]# showmount -e 192.168.0.10 Export list for 192.168.0.10: /opt/nfs/jenkins-data 192.168.175.0/24
Jenkins 其實(shí)只要加載對(duì)應(yīng)的目錄就可以讀取之前的數(shù)據(jù),但是由于 deployment 無法定義存儲(chǔ)卷,因此我們只能使用 StatefulSet。
首先創(chuàng)建 pv,pv 是給 StatefulSet 使用的,每次 StatefulSet 啟動(dòng)都會(huì)通過 volumeClaimTemplates 這個(gè)模板去創(chuàng)建 pvc,因此必須得有 pv,才能供 pvc 綁定。
創(chuàng)建jenkins-pv.yaml文件,文件內(nèi)容如下所示。
apiVersion: v1 kind: PersistentVolume metadata: name: jenkins spec: nfs: path: /opt/nfs/jenkins-data server: 192.168.0.10 accessModes: ["ReadWriteOnce"] capacity: storage: 1Ti
我這里給了 1T存儲(chǔ)空間,可以根據(jù)實(shí)際配置。
執(zhí)行如下命令創(chuàng)建pv。
kubectl apply -f jenkins-pv.yaml
創(chuàng)建service account,因?yàn)?jenkins 后面需要能夠動(dòng)態(tài)創(chuàng)建 slave,因此它必須具備一些權(quán)限。
創(chuàng)建jenkins-service-account.yaml文件,文件內(nèi)容如下所示。
apiVersion: v1 kind: ServiceAccount metadata: name: jenkins --- kind: Role apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: jenkins rules: - apiGroups: [""] resources: ["pods"] verbs: ["create", "delete", "get", "list", "patch", "update", "watch"] - apiGroups: [""] resources: ["pods/exec"] verbs: ["create", "delete", "get", "list", "patch", "update", "watch"] - apiGroups: [""] resources: ["pods/log"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["secrets"] verbs: ["get"] --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: jenkins roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: jenkins subjects: - kind: ServiceAccount name: jenkins
上述配置中,創(chuàng)建了一個(gè) RoleBinding 和一個(gè) ServiceAccount,并且將 RoleBinding 的權(quán)限綁定到這個(gè)用戶上。所以,jenkins 容器必須使用這個(gè) ServiceAccount 運(yùn)行才行,不然 RoleBinding 的權(quán)限它將不具備。
RoleBinding 的權(quán)限很容易就看懂了,因?yàn)?jenkins 需要?jiǎng)?chuàng)建和刪除 slave,所以才需要上面這些權(quán)限。至于 secrets 權(quán)限,則是 https 證書。
執(zhí)行如下命令創(chuàng)建serviceAccount。
kubectl apply -f jenkins-service-account.yaml
創(chuàng)建jenkins-statefulset.yaml文件,文件內(nèi)容如下所示。
apiVersion: apps/v1 kind: StatefulSet metadata: name: jenkins labels: name: jenkins spec: selector: matchLabels: name: jenkins serviceName: jenkins replicas: 1 updateStrategy: type: RollingUpdate template: metadata: name: jenkins labels: name: jenkins spec: terminationGracePeriodSeconds: 10 serviceAccountName: jenkins containers: - name: jenkins image: docker.io/jenkins/jenkins:lts imagePullPolicy: IfNotPresent ports: - containerPort: 8080 - containerPort: 32100 resources: limits: cpu: 4 memory: 4Gi requests: cpu: 4 memory: 4Gi env: - name: LIMITS_MEMORY valueFrom: resourceFieldRef: resource: limits.memory divisor: 1Mi - name: JAVA_OPTS # value: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 volumeMounts: - name: jenkins-home mountPath: /var/jenkins_home livenessProbe: httpGet: path: /login port: 8080 initialDelaySeconds: 60 timeoutSeconds: 5 failureThreshold: 12 # ~2 minutes readinessProbe: httpGet: path: /login port: 8080 initialDelaySeconds: 60 timeoutSeconds: 5 failureThreshold: 12 # ~2 minutes # pvc 模板,對(duì)應(yīng)之前的 pv volumeClaimTemplates: - metadata: name: jenkins-home spec: accessModes: ["ReadWriteOnce"] resources: requests: storage: 1Ti
jenkins 部署時(shí)需要注意它的副本數(shù),你的副本數(shù)有多少就要有多少個(gè) pv,同樣,存儲(chǔ)會(huì)有多倍消耗。這里我只使用了一個(gè)副本,因此前面也只創(chuàng)建了一個(gè) pv。
使用如下命令安裝Jenkins。
kubectl apply -f jenkins-statefulset.yaml
創(chuàng)建jenkins-service.yaml文件,主要用于后臺(tái)運(yùn)行Jenkins,文件內(nèi)容如下所示。
apiVersion: v1 kind: Service metadata: name: jenkins spec: # type: LoadBalancer selector: name: jenkins # ensure the client ip is propagated to avoid the invalid crumb issue when using LoadBalancer (k8s >=1.7) #externalTrafficPolicy: Local ports: - name: http port: 80 nodePort: 31888 targetPort: 8080 protocol: TCP - name: jenkins-agent port: 32100 nodePort: 32100 targetPort: 32100 protocol: TCP type: NodePort
使用如下命令安裝Service。
kubectl apply -f jenkins-service.yaml
jenkins 的 web 界面需要從集群外訪問,這里我們選擇的是使用 ingress。創(chuàng)建jenkins-ingress.yaml文件,文件內(nèi)容如下所示。
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: jenkins spec: rules: - http: paths: - path: / backend: serviceName: jenkins servicePort: 31888 host: jekins.binghe.com
這里,需要注意的是host必須配置為域名或者主機(jī)名,否則會(huì)報(bào)錯(cuò),如下所示。
The Ingress "jenkins" is invalid: spec.rules[0].host: Invalid value: "192.168.0.10": must be a DNS name, not an IP address
使用如下命令安裝ingress。
kubectl apply -f jenkins-ingress.yaml
最后,由于我這里使用的是虛擬機(jī)來搭建相關(guān)的環(huán)境,在本機(jī)訪問虛擬機(jī)映射的jekins.binghe.com時(shí),需要配置本機(jī)的hosts文件,在本機(jī)的hosts文件中加入如下配置項(xiàng)。
192.168.0.10 jekins.binghe.com
注意:在Windows操作系統(tǒng)中,hosts文件所在的目錄如下。
C:\Windows\System32\drivers\etc
接下來,就可以在瀏覽器中通過鏈接:http://jekins.binghe.com:31888 來訪問Jekins了。
Apache Subversion 通常被縮寫成 SVN,是一個(gè)開放源代碼的版本控制系統(tǒng),Subversion 在 2000 年由 CollabNet Inc 開發(fā),現(xiàn)在發(fā)展成為 Apache 軟件基金會(huì)的一個(gè)項(xiàng)目,同樣是一個(gè)豐富的開發(fā)者和用戶社區(qū)的一部分。
SVN相對(duì)于的RCS、CVS,采用了分支管理系統(tǒng),它的設(shè)計(jì)目標(biāo)就是取代CVS?;ヂ?lián)網(wǎng)上免費(fèi)的版本控制服務(wù)多基于Subversion。
這里,以在Master節(jié)點(diǎn)(binghe101服務(wù)器)上安裝SVN為例。
在命令行執(zhí)行如下命令安裝SVN。
yum -y install subversion
依次執(zhí)行如下命令。
#創(chuàng)建/data/svn mkdir -p /data/svn #初始化svn svnserve -d -r /data/svn #創(chuàng)建代碼倉庫 svnadmin create /data/svn/test
mkdir /data/svn/conf cp /data/svn/test/conf/* /data/svn/conf/ cd /data/svn/conf/ [root@binghe101 conf]# ll 總用量 20 -rw-r--r-- 1 root root 1080 5月 12 02:17 authz -rw-r--r-- 1 root root 885 5月 12 02:17 hooks-env.tmpl -rw-r--r-- 1 root root 309 5月 12 02:17 passwd -rw-r--r-- 1 root root 4375 5月 12 02:17 svnserve.conf
配置authz文件,
vim authz
配置后的內(nèi)容如下所示。
[aliases] # joe = /C=XZ/ST=Dessert/L=Snake City/O=Snake Oil, Ltd./OU=Research Institute/CN=Joe Average [groups] # harry_and_sally = harry,sally # harry_sally_and_joe = harry,sally,&joe SuperAdmin = admin binghe = admin,binghe # [/foo/bar] # harry = rw # &joe = r # * = # [repository:/baz/fuz] # @harry_and_sally = rw # * = r [test:/] @SuperAdmin=rw @binghe=rw
配置passwd文件
vim passwd
配置后的內(nèi)容如下所示。
[users] # harry = harryssecret # sally = sallyssecret admin = admin123 binghe = binghe123
配置 svnserve.conf
vim svnserve.conf
配置后的文件如下所示。
### This file controls the configuration of the svnserve daemon, if you ### use it to allow access to this repository. (If you only allow ### access through http: and/or file: URLs, then this file is ### irrelevant.) ### Visit http://subversion.apache.org/ for more information. [general] ### The anon-access and auth-access options control access to the ### repository for unauthenticated (a.k.a. anonymous) users and ### authenticated users, respectively. ### Valid values are "write", "read", and "none". ### Setting the value to "none" prohibits both reading and writing; ### "read" allows read-only access, and "write" allows complete ### read/write access to the repository. ### The sample settings below are the defaults and specify that anonymous ### users have read-only access to the repository, while authenticated ### users have read and write access to the repository. anon-access = none auth-access = write ### The password-db option controls the location of the password ### database file. Unless you specify a path starting with a /, ### the file's location is relative to the directory containing ### this configuration file. ### If SASL is enabled (see below), this file will NOT be used. ### Uncomment the line below to use the default password file. password-db = /data/svn/conf/passwd ### The authz-db option controls the location of the authorization ### rules for path-based access control. Unless you specify a path ### starting with a /, the file's location is relative to the ### directory containing this file. The specified path may be a ### repository relative URL (^/) or an absolute file:// URL to a text ### file in a Subversion repository. If you don't specify an authz-db, ### no path-based access control is done. ### Uncomment the line below to use the default authorization file. authz-db = /data/svn/conf/authz ### The groups-db option controls the location of the file with the ### group definitions and allows maintaining groups separately from the ### authorization rules. The groups-db file is of the same format as the ### authz-db file and should contain a single [groups] section with the ### group definitions. If the option is enabled, the authz-db file cannot ### contain a [groups] section. Unless you specify a path starting with ### a /, the file's location is relative to the directory containing this ### file. The specified path may be a repository relative URL (^/) or an ### absolute file:// URL to a text file in a Subversion repository. ### This option is not being used by default. # groups-db = groups ### This option specifies the authentication realm of the repository. ### If two repositories have the same authentication realm, they should ### have the same password database, and vice versa. The default realm ### is repository's uuid. realm = svn ### The force-username-case option causes svnserve to case-normalize ### usernames before comparing them against the authorization rules in the ### authz-db file configured above. Valid values are "upper" (to upper- ### case the usernames), "lower" (to lowercase the usernames), and ### "none" (to compare usernames as-is without case conversion, which ### is the default behavior). # force-username-case = none ### The hooks-env options specifies a path to the hook script environment ### configuration file. This option overrides the per-repository default ### and can be used to configure the hook script environment for multiple ### repositories in a single file, if an absolute path is specified. ### Unless you specify an absolute path, the file's location is relative ### to the directory containing this file. # hooks-env = hooks-env [sasl] ### This option specifies whether you want to use the Cyrus SASL ### library for authentication. Default is false. ### Enabling this option requires svnserve to have been built with Cyrus ### SASL support; to check, run 'svnserve --version' and look for a line ### reading 'Cyrus SASL authentication is available.' # use-sasl = true ### These options specify the desired strength of the security layer ### that you want SASL to provide. 0 means no encryption, 1 means ### integrity-checking only, values larger than 1 are correlated ### to the effective key length for encryption (e.g. 128 means 128-bit ### encryption). The values below are the defaults. # min-encryption = 0 # max-encryption = 256
接下來,將/data/svn/conf目錄下的svnserve.conf文件復(fù)制到/data/svn/test/conf/目錄下。如下所示。
[root@binghe101 conf]# cp /data/svn/conf/svnserve.conf /data/svn/test/conf/ cp:是否覆蓋'/data/svn/test/conf/svnserve.conf'? y
(1)創(chuàng)建svnserve.service服務(wù)
創(chuàng)建svnserve.service文件
vim /usr/lib/systemd/system/svnserve.service
文件的內(nèi)容如下所示。
[Unit] Description=Subversion protocol daemon After=syslog.target network.target Documentation=man:svnserve(8) [Service] Type=forking EnvironmentFile=/etc/sysconfig/svnserve #ExecStart=/usr/bin/svnserve --daemon --pid-file=/run/svnserve/svnserve.pid $OPTIONS ExecStart=/usr/bin/svnserve --daemon $OPTIONS PrivateTmp=yes [Install] WantedBy=multi-user.target
接下來執(zhí)行如下命令使配置生效。
systemctl daemon-reload
命令執(zhí)行成功后,修改 /etc/sysconfig/svnserve 文件。
vim /etc/sysconfig/svnserve
修改后的文件內(nèi)容如下所示。
# OPTIONS is used to pass command-line arguments to svnserve. # # Specify the repository location in -r parameter: OPTIONS="-r /data/svn"
(2)啟動(dòng)SVN
首先查看SVN狀態(tài),如下所示。
[root@test10 conf]# systemctl status svnserve.service ● svnserve.service - Subversion protocol daemon Loaded: loaded (/usr/lib/systemd/system/svnserve.service; disabled; vendor preset: disabled) Active: inactive (dead) Docs: man:svnserve(8)
可以看到,此時(shí)SVN并沒有啟動(dòng),接下來,需要啟動(dòng)SVN。
systemctl start svnserve.service
設(shè)置SVN服務(wù)開機(jī)自啟動(dòng)。
systemctl enable svnserve.service
接下來,就可以下載安裝TortoiseSVN,輸入鏈接svn://192.168.0.10/test 并輸入用戶名binghe,密碼binghe123來連接SVN了。
拉取SVN鏡像
docker pull docker.io/elleflorio/svn-server
啟動(dòng)SVN容器
docker run -v /usr/local/svn:/home/svn -v /usr/local/svn/passwd:/etc/subversion/passwd -v /usr/local/apache2:/run/apache2 --name svn_server -p 3380:80 -p 3690:3960 -e SVN_REPONAME=repos -d docker.io/elleflorio/svn-server
進(jìn)入SVN容器內(nèi)部
docker exec -it svn_server bash
進(jìn)入容器后,可以參照物理機(jī)安裝SVN的方式配置SVN倉庫。
注意:安裝Jenkins之前需要安裝JDK和Maven,我這里同樣將Jenkins安裝在Master節(jié)點(diǎn)(binghe101服務(wù)器)。
運(yùn)行以下命令以下載repo文件并導(dǎo)入GPG密鑰:
wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key
執(zhí)行如下命令安裝Jenkis。
yum install jenkins
接下來,修改Jenkins默認(rèn)端口,如下所示。
vim /etc/sysconfig/jenkins
修改后的兩項(xiàng)配置如下所示。
JENKINS_JAVA_CMD="/usr/local/jdk1.8.0_212/bin/java" JENKINS_PORT="18080"
此時(shí),已經(jīng)將Jenkins的端口由8080修改為18080
在命令行輸入如下命令啟動(dòng)Jenkins。
systemctl start jenkins
配置Jenkins開機(jī)自啟動(dòng)。
systemctl enable jenkins
查看Jenkins的運(yùn)行狀態(tài)。
[root@test10 ~]# systemctl status jenkins ● jenkins.service - LSB: Jenkins Automation Server Loaded: loaded (/etc/rc.d/init.d/jenkins; generated) Active: active (running) since Tue 2020-05-12 04:33:40 EDT; 28s ago Docs: man:systemd-sysv-generator(8) Tasks: 71 (limit: 26213) Memory: 550.8M
說明,Jenkins啟動(dòng)成功。
首次安裝后,需要配置Jenkins的運(yùn)行環(huán)境。首先,在瀏覽器地址欄訪問鏈接http://192.168.0.10:18080,打開Jenkins界面。
根據(jù)提示使用如下命令到服務(wù)器上找密碼值,如下所示。
[root@binghe101 ~]# cat /var/lib/jenkins/secrets/initialAdminPassword 71af861c2ab948a1b6efc9f7dde90776
將密碼71af861c2ab948a1b6efc9f7dde90776復(fù)制到文本框,點(diǎn)擊繼續(xù)。會(huì)跳轉(zhuǎn)到自定義Jenkins頁面,如下所示。
這里,可以直接選擇“安裝推薦的插件”。之后會(huì)跳轉(zhuǎn)到一個(gè)安裝插件的頁面,如下所示。
此步驟可能有下載失敗的情況,可直接忽略。
需要安裝的插件
還有更多的插件可供選擇,可點(diǎn)擊 系統(tǒng)管理->管理插件進(jìn)行管理和添加,安裝相應(yīng)的Docker插件、SSH插件、Maven插件。其他的插件可以根據(jù)需要進(jìn)行安裝。如下圖所示。
(1)配置JDK和Maven
在Global Tool Configuration中配置JDK和Maven,如下所示,打開Global Tool Configuration界面。
接下來就開始配置JDK和Maven了。
由于我在服務(wù)器上將Maven安裝在/usr/local/maven-3.6.3目錄下,所以,需要在“Maven 配置”中進(jìn)行配置,如下圖所示。
接下來,配置JDK,如下所示。
注意:不要勾選“Install automatically”
接下來,配置Maven,如下所示。
注意:不要勾選“Install automatically”
(2)配置SSH
進(jìn)入Jenkins的Configure System界面配置SSH,如下所示。
找到 SSH remote hosts 進(jìn)行配置。
配置完成后,點(diǎn)擊Check connection按鈕,會(huì)顯示 Successfull connection。如下所示。
至此,Jenkins的基本配置就完成了。
實(shí)現(xiàn),SpringBoot項(xiàng)目中啟動(dòng)類所在的模塊的pom.xml需要引入打包成Docker鏡像的配置,如下所示。
<properties> <docker.repostory>192.168.0.10:1180</docker.repostory> <docker.registry.name>test</docker.registry.name> <docker.image.tag>1.0.0</docker.image.tag> <docker.maven.plugin.version>1.4.10</docker.maven.plugin.version> </properties> <build> <finalName>test-starter</finalName> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> <!-- docker的maven插件,官網(wǎng):https://github.com/spotify/docker‐maven‐plugin --> <!-- Dockerfile maven plugin --> <plugin> <groupId>com.spotify</groupId> <artifactId>dockerfile-maven-plugin</artifactId> <version>${docker.maven.plugin.version}</version> <executions> <execution> <id>default</id> <goals> <!--如果package時(shí)不想用docker打包,就注釋掉這個(gè)goal--> <goal>build</goal> <goal>push</goal> </goals> </execution> </executions> <configuration> <contextDirectory>${project.basedir}</contextDirectory> <!-- harbor 倉庫用戶名及密碼--> <useMavenSettingsForAuth>useMavenSettingsForAuth>true</useMavenSettingsForAuth> <repository>${docker.repostory}/${docker.registry.name}/${project.artifactId}</repository> <tag>${docker.image.tag}</tag> <buildArgs> <JAR_FILE>target/${project.build.finalName}.jar</JAR_FILE> </buildArgs> </configuration> </plugin> </plugins> <resources> <!-- 指定 src/main/resources下所有文件及文件夾為資源文件 --> <resource> <directory>src/main/resources</directory> <targetPath>${project.build.directory}/classes</targetPath> <includes> <include>**/*</include> </includes> <filtering>true</filtering> </resource> </resources> </build>
接下來,在SpringBoot啟動(dòng)類所在模塊的根目錄創(chuàng)建Dockerfile,內(nèi)容示例如下所示。
#添加依賴環(huán)境,前提是將Java8的Docker鏡像從官方鏡像倉庫pull下來,然后上傳到自己的Harbor私有倉庫中 FROM 192.168.0.10:1180/library/java:8 #指定鏡像制作作者 MAINTAINER binghe #運(yùn)行目錄 VOLUME /tmp #將本地的文件拷貝到容器 ADD target/*jar app.jar #啟動(dòng)容器后自動(dòng)執(zhí)行的命令 ENTRYPOINT [ "java", "-Djava.security.egd=file:/dev/./urandom", "-jar", "/app.jar" ]
根據(jù)實(shí)際情況,自行修改。
注意:FROM 192.168.0.10:1180/library/java:8的前提是執(zhí)行如下命令。
docker pull java:8 docker tag java:8 192.168.0.10:1180/library/java:8 docker login 192.168.0.10:1180 docker push 192.168.0.10:1180/library/java:8
在SpringBoot啟動(dòng)類所在模塊的根目錄創(chuàng)建yaml文件,錄入叫做test.yaml文件,內(nèi)容如下所示。
apiVersion: apps/v1 kind: Deployment metadata: name: test-starter labels: app: test-starter spec: replicas: 1 selector: matchLabels: app: test-starter template: metadata: labels: app: test-starter spec: containers: - name: test-starter image: 192.168.0.10:1180/test/test-starter:1.0.0 ports: - containerPort: 8088 nodeSelector: clustertype: node12 --- apiVersion: v1 kind: Service metadata: name: test-starter labels: app: test-starter spec: ports: - name: http port: 8088 nodePort: 30001 type: NodePort selector: app: test-starter
將項(xiàng)目上傳到SVN代碼庫,例如地址為svn://192.168.0.10/test
接下來,在Jenkins中配置自動(dòng)發(fā)布。步驟如下所示。
點(diǎn)擊新建Item。
在描述文本框中輸入描述信息,如下所示。
接下來,配置SVN信息。
注意:配置GitLab的步驟與SVN相同,不再贅述。
定位到Jenkins的“構(gòu)建模塊”,使用Execute Shell來構(gòu)建發(fā)布項(xiàng)目到K8S集群。
執(zhí)行的命令依次如下所示。
#刪除本地原有的鏡像,不會(huì)影響Harbor倉庫中的鏡像 docker rmi 192.168.0.10:1180/test/test-starter:1.0.0 #使用Maven編譯、構(gòu)建Docker鏡像,執(zhí)行完成后本地Docker容器中會(huì)重新構(gòu)建鏡像文件 /usr/local/maven-3.6.3/bin/mvn -f ./pom.xml clean install -Dmaven.test.skip=true #登錄 Harbor倉庫 docker login 192.168.0.10:1180 -u binghe -p Binghe123 #上傳鏡像到Harbor倉庫 docker push 192.168.0.10:1180/test/test-starter:1.0.0 #停止并刪除K8S集群中運(yùn)行的 /usr/bin/kubectl delete -f test.yaml #將Docker鏡像重新發(fā)布到K8S集群 /usr/bin/kubectl apply -f test.yaml
到此這篇關(guān)于基于Docker+K8S+GitLab/SVN+Jenkins+Harbor搭建持續(xù)集成交付環(huán)境(環(huán)境搭建篇)的文章就介紹到這了,更多相關(guān)docker K8S持續(xù)集成交付環(huán)境內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家!
標(biāo)簽:朝陽 隴南 雞西 運(yùn)城 連云港 遵義 面試通知 荊門
巨人網(wǎng)絡(luò)通訊聲明:本文標(biāo)題《基于Docker+K8S+GitLab/SVN+Jenkins+Harbor搭建持續(xù)集成交付環(huán)境的詳細(xì)教程》,本文關(guān)鍵詞 基于,Docker+K8S+GitLab,SVN+Jenkins+Harbor,;如發(fā)現(xiàn)本文內(nèi)容存在版權(quán)問題,煩請(qǐng)?zhí)峁┫嚓P(guān)信息告之我們,我們將及時(shí)溝通與處理。本站內(nèi)容系統(tǒng)采集于網(wǎng)絡(luò),涉及言論、版權(quán)與本站無關(guān)。