Kubernetes集群搭建
阿里云国内75折 回扣 微信号:monov8 |
阿里云国际,腾讯云国际,低至75折。AWS 93折 免费开户实名账号 代冲值 优惠多多 微信号:monov8 飞机:@monov6 |
# Kubernetes集群搭建
[kubeadm官方文档](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/)
https://blog.51cto.com/zhangxueliang/4952945
## 国外镜像下载问题
因谷歌网络限制问题国内的K8ser大多数在学习Kubernetes过程中因为镜像下载失败问题间接地产生些许失落感笔者也因此脑壳疼故翻阅资料得到以下解决方式
在应用yaml文件创建资源时将文件中镜像地址进行内容替换即可
- `k8s.gcr.io` 地址替换为`registry.aliyuncs.com/google_containers`
- `quay.io` 地址替换为`quay.mirrors.ustc.edu.cn`或者`quay-mirror.qiniu.com`
- `gcr.io`地址替换为`registry.aliyuncs.com`
kubeadm依赖的所有镜像可以用如下脚本下载
```
# echo <<EOF | tee kubeadm-images.sh
#!/bin/bash
images=$(kubeadm config images list)
for image in ${images[@]} ; do
imageMirror="registry.aliyuncs.com/google_containers/"$(echo $image | awk -F '/' '{print $NF}')
echo $image "->" $imageMirror
docker pull $imageMirror
docker tag $imageMirror $image
docker rmi $imageMirror
done
EOF
# bash kubeadm-images.sh
```
新版本的kubeadm不再需要上述trick选项`--image-repository`指向国内mirror即可。
## 前置步骤所有节点
- CentOS 7.9 物理机或虚拟机三台CPU 内核数量大于等于 2且内存大于等于 4G
- hostname 不是 localhost且不包含下划线、小数点、大写字母
- 任意节点都有固定的内网 IP 地址(集群机器统一内网)
- 任意节点上 IP 地址 可互通无需 NAT 映射即可相互访问且没有防火墙、安全组隔离
- 任意节点不会直接使用 docker run 或 docker-compose 运行容器和Pod
```
#关闭防火墙 或者阿里云开通安全组端口访问
systemctl stop firewalld
systemctl disable firewalld
#关闭 selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0
#关闭 swap
swapoff -a #临时
sed -ri 's/.*swap.*/#&/' /etc/fstab #永久
#将桥接的 IPv4 流量传递到 iptables 的链
# 修改 /etc/sysctl.conf
# 如果有配置则修改
sed -i "s#^net.ipv4.ip_forward.*#net.ipv4.ip_forward=1#g" /etc/sysctl.conf
sed -i "s#^net.bridge.bridge-nf-call-ip6tables.*#net.bridge.bridge-nf-call-ip6tables=1#g" /etc/sysctl.conf
sed -i "s#^net.bridge.bridge-nf-call-iptables.*#net.bridge.bridge-nf-call-iptables=1#g" /etc/sysctl.conf
sed -i "s#^net.ipv6.conf.all.disable_ipv6.*#net.ipv6.conf.all.disable_ipv6=1#g" /etc/sysctl.conf
sed -i "s#^net.ipv6.conf.default.disable_ipv6.*#net.ipv6.conf.default.disable_ipv6=1#g" /etc/sysctl.conf
sed -i "s#^net.ipv6.conf.lo.disable_ipv6.*#net.ipv6.conf.lo.disable_ipv6=1#g" /etc/sysctl.conf
sed -i "s#^net.ipv6.conf.all.forwarding.*#net.ipv6.conf.all.forwarding=1#g" /etc/sysctl.conf
# 可能没有追加
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
echo "net.bridge.bridge-nf-call-ip6tables = 1" >> /etc/sysctl.conf
echo "net.bridge.bridge-nf-call-iptables = 1" >> /etc/sysctl.conf
echo "net.ipv6.conf.all.disable_ipv6 = 1" >> /etc/sysctl.conf
echo "net.ipv6.conf.default.disable_ipv6 = 1" >> /etc/sysctl.conf
echo "net.ipv6.conf.lo.disable_ipv6 = 1" >> /etc/sysctl.conf
echo "net.ipv6.conf.all.forwarding = 1" >> /etc/sysctl.conf
# 执行命令以应用
sysctl -p
```
## 安装Docker环境所有节点
按照[docker官方文档](https://docs.docker.com/engine/install/centos/) 即可。
为了规避[问题1](https://github.com/containerd/containerd/issues/4581) 以及[问题2](https://blog.csdn.net/Haskei/article/details/128474534) 需要修改所有结点上的containerd配置参考[这里](https://github.com/containerd/containerd/blob/main/docs/PLUGINS.md#version-header)
```
# cat <<EOF | sudo tee /etc/containerd/config.toml
version = 2
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.6"
EOF
# systemctl restart containerd
# containerd config dump|grep sand
sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.6"
```
## 安装k8s、kubelet、kubeadm、kubectl所有节点
```
# 配置K8S的yum源
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
# 卸载旧版本
yum remove -y kubelet kubeadm kubectl
# 安装kubelet、kubeadm、kubectl
yum install -y kubelet-1.17.3 kubeadm-1.17.3 kubectl-1.17.3
#开机启动和重启kubelet
systemctl enable kubelet && systemctl start kubelet
##注意如果此时查看kubelet的状态他会无限重启等待接收集群命令和初始化。这个是正常的。
```
## 初始化master节点单个master节点
```
## 初始化master成功将输出“初始化worker结点”的命令
# kubeadm init --image-repository=registry.aliyuncs.com/google_containers
[init] Using Kubernetes version: v1.26.1
## 配置kubectl
# export KUBECONFIG=/etc/kubernetes/admin.conf
```
## 初始化worker结点多个worker结点
```
## 使用刚才master打印的令牌命令加入
## 如果下面命令失败则执行"kubeadm reset"后重试
# kubeadm join 192.168.110.48:6443 --token dna6ti.7hu4gx3r43rid67w --discovery-token-ca-cert-hash sha256:d68e1e0398dadac6d7196faac56bc9447a499203f30a1e46daf874d59aaf493d
## 如果超过2小时忘记了令牌可以这样做
# kubeadm token create --print-join-command #打印新令牌
# kubeadm token create --ttl 0 --print-join-command #创建个永不过期的令牌
## 在master结点观察worker结点状态
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
vm48 Ready control-plane 85m v1.26.1
vm49 Ready <none> 8m1s v1.26.1
vm50 Ready <none> 41s v1.26.1
```
## 配置pod网络插件calico
注意[官方文档](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network)已指出在安装pod网络插件之前coredns不会启动。
Calico在Cilium之后[添加了基于eBPF的数据面](https://www.tigera.io/blog/introducing-the-calico-ebpf-dataplane/) 用户可在iptables和eBPF之间选择默认是iptables。
```
## 部署calico
# kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
## 查看状态等待calico和coredns相关pod就绪。注意coredns仅在部署pod网络插件后启动
# watch kubectl get pod -n kube-system -o wide
```
冒烟测试部署nginx服务测试dns解析服务名
```
# cat <<EOF > nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
hostPort: 80
protocol: TCP
- containerPort: 443
hostPort: 443
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
name: nginx-srv
spec:
clusterIP: None
ports:
- port: 80
name: http
- port: 443
name: https
selector:
app: nginx
EOF
# kubectl apply -f nginx.yaml
# kubectl run -it busybox --image=busybox --restart=Never
/ # ping nginx-srv
PING nginx-srv (172.16.33.196): 56 data bytes
64 bytes from 172.16.33.196: seq=0 ttl=62 time=1.146 ms
64 bytes from 172.16.33.196: seq=1 ttl=62 time=0.703 ms
^C
```