本文主要介绍了使用二进制方式安装kubernetes 1.20版本集群
更新于 2021-04-11
集群规划
初始化配置 下面的初始化操作在所有服务器上进行
部署etcd
根据规划，我们将在三个节点上部署服务，形成一个etcd集群
创建证书
部署etcd集群
启动集群 首先启动第一个节点：
1 2 3 systemctl daemon-reload systemctl start etcd systemctl enable etcd
将上一步中的文件都拷贝到其他节点上：
1 2 3 4 scp -r /opt/etcd/ root@192.168.31.72:/opt/ scp /usr/lib/systemd/system/etcd.service root@192.168.31.72:/usr/lib/systemd/system/ scp -r /opt/etcd/ root@192.168.31.73:/opt/ scp /usr/lib/systemd/system/etcd.service root@192.168.31.73:/usr/lib/systemd/system/
注意，拷贝过去后需要修改一下配置文件的内容，将IP和节点名称修改为当前所在服务器的地址：
1 2 3 4 5 6 7 8 9 10 11 12 13 cat /opt/etcd/cfg/etcd.conf ETCD_NAME="etcd-1" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS="https://192.168.31.71:2380" ETCD_LISTEN_CLIENT_URLS="https://192.168.31.71:2379" ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.31.71:2380" ETCD_ADVERTISE_CLIENT_URLS="https://192.168.31.71:2379" ETCD_INITIAL_CLUSTER="etcd-1=https://192.168.31.71:2380,etcd-2=https://192.168.31.72:2380,etcd-3=https://192.168.31.73:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new"
然后在启动剩下的两个节点，步骤同上。
查看集群状态 1 2 3 4 5 6 7 8 9 10 ETCDCTL_API=3 /opt/etcd/bin/etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.31.71:2379,https://192.168.31.72:2379,https://192.168.31.73:2379" endpoint health --write-out=table +----------------------------+--------+-------------+-------+ | ENDPOINT | HEALTH | TOOK | ERROR | +----------------------------+--------+-------------+-------+ | https://192.168.31.71:2379 | true | 10.301506ms | | | https://192.168.31.73:2379 | true | 12.87467ms | | | https://192.168.31.72:2379 | true | 13.225954ms | | +----------------------------+--------+-------------+-------+
安装docker
在所有的节点都安装docker，也可以换成其他的容器引擎如containerd
下载安装 1 2 3 wget https://download.docker.com/linux/static/stable/x86_64/docker-19.03.9.tgz tar zxvf docker-19.03.9.tgz mv docker/* /usr/bin
创建服务启动文件 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 cat > /usr/lib/systemd/system/docker.service << EOF [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network-online.target firewalld.service Wants=network-online.target [Service] Type=notify ExecStart=/usr/bin/dockerd ExecReload=/bin/kill -s HUP $MAINPID LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity TimeoutStartSec=0 Delegate=yes KillMode=process Restart=on-failure StartLimitBurst=3 StartLimitInterval=60s [Install] WantedBy=multi-user.target EOF
创建配置文件 1 2 3 4 5 6 mkdir /etc/docker cat > /etc/docker/daemon.json << EOF { "registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"] } EOF
启动服务 1 2 3 systemctl daemon-reload systemctl start docker systemctl enable docker
部署master节点 生成kube-apiserver证书
安装kube-apiserver https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.20.md，在这里下载1.20版本的安装包，下载server包就够了，包含了Master和Worker Node二进制文件。
启动kube-apiserver 1 2 3 systemctl daemon-reload systemctl start kube-apiserver systemctl enable kube-apiserver
部署kube-controller-manager
启动kube-controller-manager 1 2 3 systemctl daemon-reload systemctl start kube-controller-manager systemctl enable kube-controller-manager
部署kube-scheduler
启动kube-scheduler 1 2 3 systemctl daemon-reload systemctl start kube-scheduler systemctl enable kube-scheduler
创建kubectl证书文件连接集群 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 cat > admin-csr.json <<EOF { "CN": "admin", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing", "O": "system:masters", "OU": "System" } ] } EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin mkdir /root/.kube KUBE_CONFIG="/root/.kube/config" KUBE_APISERVER="https://192.168.31.71:6443" kubectl config set-cluster kubernetes \ --certificate-authority=/opt/kubernetes/ssl/ca.pem \ --embed-certs=true \ --server=${KUBE_APISERVER} \ --kubeconfig=${KUBE_CONFIG} kubectl config set-credentials cluster-admin \ --client-certificate=./admin.pem \ --client-key=./admin-key.pem \ --embed-certs=true \ --kubeconfig=${KUBE_CONFIG} kubectl config set-context default \ --cluster=kubernetes \ --user=cluster-admin \ --kubeconfig=${KUBE_CONFIG} kubectl config use-context default --kubeconfig=${KUBE_CONFIG}
查看集群状态 1 2 3 4 5 6 7 kubectl get cs NAME STATUS MESSAGE ERROR scheduler Healthy ok controller-manager Healthy ok etcd-2 Healthy {"health" :"true" } etcd-1 Healthy {"health" :"true" } etcd-0 Healthy {"health" :"true" }
授权kubelet-bootstrap用户允许请求证书 1 2 3 kubectl create clusterrolebinding kubelet-bootstrap \ --clusterrole=system:node-bootstrapper \ --user=kubelet-bootstrap
部署node节点 创建工作目录并拷贝二进制文件 1 2 3 mkdir -p /opt/kubernetes/{bin,cfg,ssl,logs} cd kubernetes/server/bincp kubelet kube-proxy /opt/kubernetes/bin
部署kubelet
启动kubelet 1 2 3 systemctl daemon-reload systemctl start kubelet systemctl enable kubelet
审批kubelet证书申请并加入集群 1 2 3 4 5 6 7 8 9 10 11 12 kubectl get csr NAME AGE SIGNERNAME REQUESTOR CONDITION node-csr-uCEGPOIiDdlLODKts8J658HrFq9CZ--K6M4G7bjhk8A 6m3s kubernetes.io/kube-apiserver-client-kubelet kubelet-bootstrap Pending kubectl certificate approve node-csr-uCEGPOIiDdlLODKts8J658HrFq9CZ--K6M4G7bjhk8A kubectl get node NAME STATUS ROLES AGE VERSION k8s-master1 NotReady <none> 7s v1.18.3
由于网络插件还没有部署，节点会没有准备就绪 NotReady
部署kube-proxy
启动kube-proxy 1 2 3 systemctl daemon-reload systemctl start kube-proxy systemctl enable kube-proxy
部署网络插件callico 1 2 kubectl apply -f calico.yaml kubectl get pods -n kube-system
检查集群pod状态 1 2 3 kubectl get node NAME STATUS ROLES AGE VERSION k8s-master Ready <none> 37m v1.20.4
授权apiserver访问kubelet 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 cat > apiserver-to-kubelet-rbac.yaml << EOF apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: annotations: rbac.authorization.kubernetes.io/autoupdate: "true" labels: kubernetes.io/bootstrapping: rbac-defaults name: system:kube-apiserver-to-kubelet rules: - apiGroups: - "" resources: - nodes/proxy - nodes/stats - nodes/log - nodes/spec - nodes/metrics - pods/log verbs: - "*" --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:kube-apiserver namespace: "" roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:kube-apiserver-to-kubelet subjects: - apiGroup: rbac.authorization.k8s.io kind: User name: kubernetes EOF kubectl apply -f apiserver-to-kubelet-rbac.yaml
node节点扩容 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 scp -r /opt/kubernetes root@192.168.31.72:/opt/ scp -r /usr/lib/systemd/system/{kubelet,kube-proxy}.service root@192.168.31.72:/usr/lib/systemd/system scp /opt/kubernetes/ssl/ca.pem root@192.168.31.72:/opt/kubernetes/ssl rm -f /opt/kubernetes/cfg/kubelet.kubeconfig rm -f /opt/kubernetes/ssl/kubelet* vi /opt/kubernetes/cfg/kubelet.conf --hostname-override=k8s-node1 vi /opt/kubernetes/cfg/kube-proxy-config.yml hostnameOverride: k8s-node1 systemctl daemon-reload systemctl start kubelet kube-proxy systemctl enable kubelet kube-proxy systemctl start kubelet kubelet systemctl enable kubelet kubelet kubectl get csr NAME AGE SIGNERNAME REQUESTOR CONDITION node-csr-4zTjsaVSrhuyhIGqsefxzVoZDCNKei-aE2jyTP81Uro 89s kubernetes.io/kube-apiserver-client-kubelet kubelet-bootstrap Pending kubectl certificate approve node-csr-4zTjsaVSrhuyhIGqsefxzVoZDCNKei-aE2jyTP81Uro kubectl get node NAME STATUS ROLES AGE VERSION k8s-master1 Ready <none> 47m v1.20.4 k8s-node1 Ready <none> 6m49s v1.20.4
常用插件部署 部署dashboard 1 2 3 4 5 6 7 kubectl apply -f kubernetes-dashboard.yaml kubectl get pods,svc -n kubernetes-dashboard kubectl create serviceaccount dashboard-admin -n kube-system kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk '/dashboard-admin/{print $1}' )
上一步将输出一个token，访问：https://NodeIP:30001，输入token即可进入页面
部署coredns 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 kubectl apply -f coredns.yaml kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE coredns-5ffbfd976d-j6shb 1/1 Running 0 32s kubectl run -it --rm dns-test --image=busybox:1.28.4 sh If you don't see a command prompt, try pressing enter. / # nslookup kubernetes Server: 10.0.0.2 Address 1: 10.0.0.2 kube-dns.kube-system.svc.cluster.local Name: kubernetes Address 1: 10.0.0.1 kubernetes.default.svc.cluster.local
