基于containerd部署kubernetes v1.20.4

阿里云国内75折 回扣 微信号:monov8
阿里云国际,腾讯云国际,低至75折。AWS 93折 免费开户实名账号 代冲值 优惠多多 微信号:monov8 飞机:@monov6


本次集群部署采用的容器技术是containerd

系统版本:CentOs8.1
k8s版本:v1.20.4
containerd版本:ctr containerd.io 1.4.3

master: 192.168.43.151
node1: 192.168.43.152
node2: 192.168.43.153

1、初始化配置
#下载CentOS 8的repo文件(用阿里的)
 

curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-8.repo

安装 kubelet kubeadm kubectl
master、node节点都需要执行

添加 kubernetes.repo 文件

cat > /etc/yum.repos.d/kubernetes.repo <<EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

# 关闭SElinux

setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
systemctl enable --now kubelet # 开机启动kubelet
systemctl disable --now firewalld # 关闭防火墙

# k8s要求关闭swap

swapoff -a && sysctl -w vm.swappiness=0  # 关闭swap
sed -ri '/^[^#]*swap/s@^@#@' /etc/fstab # 取消开机挂载swap

2,安装containerd
安装前配置

cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter

# 设置必需的 sysctl 参数,这些参数在重新启动后仍然存在。

cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

# Apply sysctl params without reboot

sysctl --system

### 安装所需包
 

dnf install -y yum-utils device-mapper-persistent-data lvm2

### 新增 Docker 仓库
 

dnf config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

## 安装 containerd
 

dnf install -y containerd.io

# 配置 containerd
 

mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml

# 重启 containerd
 

systemctl restart containerd

3、安装kubelet kubeadm kubectl
 

dnf install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

#--disableexcludes=kubernetes  禁掉除了这个之外的别的仓库 

#kubelet配置修改

# config kubelet cgroup
cat > /etc/sysconfig/kubelet <<EOF
KUBELET_EXTRA_ARGS=--cgroup-driver=systemd --container-runtime=remote --container-runtime-endpoint=/run/containerd/containerd.sock
EOF

# config CRI

cat > /etc/crictl.yaml <<EOF
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10
debug: false
EOF

vi /etc/containerd/config.toml
# [plugins."io.containerd.grpc.v1.cri"] 下的 sandbox_image
# 将k8s.gcr.io/pause:3.2修改为如下
sandbox_image="registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2"

# 加上 SystemdCgroup = true
在[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]中加入
  ...
  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
    SystemdCgroup = true

# 重启 containerd
 

sudo systemctl restart containerd

# 重启 kubelet
 

sudo systemctl restart kubelet

4,准备镜像
 

kubeadm config images pull
ctr image pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.20.4
ctr image pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2
ctr image pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.20.4
ctr image pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.20.4
ctr image pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.20.4
ctr image pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.4.13-0
ctr image pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.7.0
ctr image pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2
ctr images list

5,启动集群
master启动

kubeadm init --kubernetes-version=v1.20.4  --apiserver-advertise-address=192.168.43.151 --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers --service-cidr=10.10.0.0/16 --pod-network-cidr=192.168.0.0/16

# --image-repository:指定镜像源
-----执行结果----
[root@ck8s1 ~]# kubeadm init --kubernetes-version=v1.20.4  --apiserver-advertise-address=192.168.43.151 --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers --service-cidr=10.10.0.0/16 --pod-network-cidr=192.168.0.0/16
[init] Using Kubernetes version: v1.20.4
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [ck8s1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.10.0.1 192.168.43.151]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [ck8s1 localhost] and IPs [192.168.43.151 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [ck8s1 localhost] and IPs [192.168.43.151 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[apiclient] All control plane components are healthy after 80.003966 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.20" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node ck8s1 as control-plane by adding the labels "node-role.kubernetes.io/master=''" and "node-role.kubernetes.io/control-plane='' (deprecated)"
[mark-control-plane] Marking the node ck8s1 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: 7pw2tl.x2121e9p3dgr7ktb
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
​​​   https://kubernetes.io/docs/concepts/cluster-administration/addons/​

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.43.151:6443 --token 7pw2tl.x2121e9p3dgr7ktb \
    --discovery-token-ca-cert-hash sha256:694e2924e2cf2211cefba492faf0259d0810d6afb72c5e941120b2911d8d18ab 
----------------------------------------------
#启动之后,需要再执行以下,否则在使用kubectl的时候会报8080端口错误:
# 非root用户
 

mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

# root用户
 

export KUBECONFIG=/etc/kubernetes/admin.conf

6,应用calico网络

#kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
[root@ck8s1 ~]# kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
configmap/calico-config created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created
clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrole.rbac.authorization.k8s.io/calico-node created
clusterrolebinding.rbac.authorization.k8s.io/calico-node created
daemonset.apps/calico-node created
serviceaccount/calico-node created
deployment.apps/calico-kube-controllers created
serviceaccount/calico-kube-controllers created
poddisruptionbudget.policy/calico-kube-controllers created

7,node节点加入:

kubeadm join 192.168.43.151:6443 --token 7pw2tl.x2121e9p3dgr7ktb \
--discovery-token-ca-cert-hash sha256:694e2924e2cf2211cefba492faf0259d0810d6afb72c5e941120b2911d8d18ab --cri-socket /run/containerd/containerd.sock

#前半部分是master输出的语句,最后添加cri的参数
--cri-socket /run/containerd/containerd.sock 的作用是指定cri为containerd

#查看node状态

#kubectl get node
[root@ck8s1 ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
ck8s1 Ready control-plane,master 172m v1.20.4
ck8s2 Ready <none> 84m v1.20.4
ck8s3 Ready <none> 80m v1.20.4

# 查看pod状态

kubectl get pod -A
[root@ck8s1 ~]# kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-6949477b58-zzfnb 1/1 Running 0 98m
kube-system calico-node-7w7p5 1/1 Running 0 83m
kube-system calico-node-9j57n 1/1 Running 0 98m
kube-system calico-node-zdxtq 0/1 Running 0 80m
kube-system coredns-54d67798b7-d426f 1/1 Running 0 172m
kube-system coredns-54d67798b7-pzbgf 1/1 Running 0 172m
kube-system etcd-ck8s1 1/1 Running 0 172m
kube-system kube-apiserver-ck8s1 1/1 Running 0 172m
kube-system kube-controller-manager-ck8s1 1/1 Running 0 172m
kube-system kube-proxy-4pbzn 1/1 Running 0 83m
kube-system kube-proxy-5cbcr 1/1 Running 0 172m
kube-system kube-proxy-75c6f 1/1 Running 0 80m
kube-system kube-scheduler-ck8s1 1/1 Running 0 172m

8,错误排查和解决方法
--报错1
[preflight] Running pre-flight checks
        [WARNING FileExisting-tc]: tc not found in system path
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables does not exist
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
解决方法:
 

dnf install tc -y
modprobe br_netfilter && echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables

--报错2:

Warning  FailedCreatePodSandBox  53m (x9 over 73m)     kubelet  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox image "k8s.gcr.io/pause:3.2": failed to pull image "k8s.gcr.io/pause:3.2": failed to pull and unpack image "k8s.gcr.io/pause:3.2": failed to resolve reference "k8s.gcr.io/pause:3.2": failed to do request: Head https://k8s.gcr.io/v2/pause/manifests/3.2: dial tcp 64.233.187.82:443: i/o timeout
Warning FailedCreatePodSandBox 43m (x3 over 44m) kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox image "k8s.gcr.io/pause:3.2": failed to pull image "k8s.gcr.io/pause:3.2": failed to pull and unpack image "k8s.gcr.io/pause:3.2": failed to resolve reference "k8s.gcr.io/pause:3.2": failed to do request: Head https://k8s.gcr.io/v2/pause/manifests/3.2: dial tcp 74.125.204.82:443: i/o timeout

解决方法:

第一种:
vim /etc/containerd/config.toml
# [plugins."io.containerd.grpc.v1.cri"] 下的 sandbox_image
# 将k8s.gcr.io/pause:3.2修改为如下(替换为可下载的镜像地址)
sandbox_image="registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2"

其实还有其它的方法也能规避掉此问题,不用改config.toml文件,ctr image  pull  把镜像下来,指定namespace 例如:

第二种:

下载镜像

ctr  -n k8s.io image  pull  registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2

将下载的镜像打上k8s.gcr.io的tag

ctr  -n k8s.io image  tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2  k8s.gcr.io/pause:3.2

 

阿里云国内75折 回扣 微信号:monov8
阿里云国际,腾讯云国际,低至75折。AWS 93折 免费开户实名账号 代冲值 优惠多多 微信号:monov8 飞机:@monov6
标签: k8s