rancher的k3s证书过期

现象

web上rancher不能访问服务上看443端口没了6443端口仍然在。

rancher报错日志

rancher    | time="2023-01-05T01:56:07.241615176Z" level=info msg="Waiting for master node  startup: resource name may not be empty"
rancher    | 2023/01/05 01:56:07 [INFO] Waiting for server to become available: Get "https://127.0.0.1:6443/version?timeout=15m0s": x509: certificate has expired or is not yet valid: current time 2023-01-05T01:56:07Z is after 2022-12-28T10:04:18Z
rancher    | 2023-01-05 01:56:07.825328 I | http: TLS handshake error from 127.0.0.1:46358: remote error: tls: bad certificate
rancher    | I0105 01:56:08.154170      30 request.go:621] Throttling request took 1.036152126s, request: GET:https://127.0.0.1:6444/apis/scheduling.k8s.io/v1?timeout=32s
rancher    | time="2023-01-05T01:56:08.241957765Z" level=info msg="Waiting for master node  startup: resource name may not be empty"
rancher    | 2023-01-05 01:56:08.313648 I | http: TLS handshake error from 127.0.0.1:46360: remote error: tls: bad certificate
rancher    | 2023-01-05 01:56:08.314903 I | http: TLS handshake error from 127.0.0.1:46362: remote error: tls: bad certificate
rancher    | time="2023-01-05T01:56:08.314951029Z" level=error msg="server https://127.0.0.1:6443/cacerts is not trusted: Get https://127.0.0.1:6443/cacerts: x509: certificate has expired or is not yet valid"

分析

如上可见是rancher 的链接 6443端口因为证书问题超时了 =>
对k3s或者k8s有了解的话我们可以想到6443端口是rancher的k3s服务下面我们解决k3s证书到期的问题。=>
但是进入容器我们可以看到k3s并没有启动起来。我们需要先将它启动起来。
综上我们提出如下解决思路

解决思路

• 修服务器日期到证书过期前让k3s正常启动
• 更新k3s证书
• 改回服务器时间
• 重启容器

解决

• 关闭时间同步如果有的话

[root@rancher rancher-test]# service ntpd stop
Redirecting to /bin/systemctl stop ntpd.service

• 修改服务器时间

[root@rancher rancher]# date -s 20221010

• 重启容器并进入容器

[root@rancher rancher]# docker-compose restart
[root@rancher rancher]# docker ps
CONTAINER ID        IMAGE                                COMMAND                  CREATED                  STATUS                  PORTS               NAMES
d5ba9645f423        harbocto.boe.com.cn/public/rancher   "entrypoint.sh --no-…"   Less than a second ago   Up Less than a second                       rancher
[root@rancher rancher]# docker exec -it d5 bash

• 删除证书

root@rancher:/var/lib/rancher# kubectl --insecure-skip-tls-verify -n kube-system delete secrets k3s-serving
secret "k3s-serving" deleted
root@rancher:/var/lib/rancher# rm -f /var/lib/rancher/k3s/server/tls/dynamic-cert.json

• 退出容器改回正确时间

root@rancher:/var/lib/rancher# exit
exit
[root@rancher rancher]# ntpdate -u 10.10.239.65
 5 Jan 09:55:03 ntpdate[146258]: step time server 10.10.239.65 offset 6400014.715005 sec

• 重启服务重新生成证书

[root@rancher rancher]# docker-compose restart
Restarting rancher ... done

---