Summary
Reference
Root Cause:
The kubelet service prints lots of warning messages in /var/log/messages, making /var/log/messages unreadable.
Impact:
This patch applies to the kubelet service only. When the kubelet service restarts, the node will change to NotReady status for a few seconds.
Impact Versions:
CDF2020.08, CDF2020.11, CDF2021.02
Installation Packages:
BYOK
Solution:
1. Log in to one of master nodes.
2. Check the kubelet version.
${K8S_HOME}/bin/kubelet --version
3. Download the new kubelet server binary.
if your kubelet version is "Kubernetes v1.17.9", download the "kubernetes-server-linux-amd64.tar.gz" file
from: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.17.md#client-binaries
if your kubelet version is "Kubernetes v1.19.0", download the "kubernetes-server-linux-amd64.tar.gz" file
from: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md#client-binaries
4. Upload the downloaded "kubernetes-server-linux-amd64.tar.gz" file to all of the master nodes and worker nodes.
5. Unzip file with command "tar -xvzf kubernetes-server-linux-amd64.tar.gz" on each node.
6. Run following commands on all of the master nodes and worker nodes one after another:
1) Run command "systemctl stop kubelet" to stop kubelet.
2) Run command "mv ${K8S_HOME}/bin/kubelet ${K8S_HOME}/bin/kubelet.bak" to back up the old binary.
3) Copy file "kubernetes/server/bin/kubelet" to path: ${K8S_HOME}/bin/kubelet
cp {Path of tar}/kubernetes/server/bin/kubelet ${K8S_HOME}/bin/
4) Run command "systemctl daemon-reload".
5) Run command "systemctl restart kubelet".
7. After you have performed the above steps on all of the nodes, run kube-status.sh to check your cluster status.
Rollback
1. Run the following commands on all of the master and worker nodes one by one to rollback the change.
1) Run command "systemctl stop kubelet" to stop kubelet.
2) Run command "mv ${K8S_HOME}/bin/kubelet.bak ${K8S_HOME}/bin/kubelet"
3) Run command "systemctl daemon-reload"
4) Run command "systemctl restart kubelet"
Notes
1. If you extend the cluster by adding new nodes, you must perform the steps on each new node also.
2. After upgrading from CDF 2020.08 to 2020.11, you must perform these steps again.