安装

环境 arm平台

 1Kernel
 2
 3Linux rock64 4.4.132-1075-rockchip-ayufan-ga83beded8524 #1 SMP Thu Jul 26 08:22:22 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux
 4
 5System
 6
 7No LSB modules are available.
 8Distributor ID: Ubuntu
 9Description:    Ubuntu 18.04.4 LTS
10Release:        18.04
11Codename:       bionic
12
13Memory
14              total        used        free      shared  buff/cache   available
15Mem:           3924         198         105           3        3619        3692
16Swap:          1962           0        1961
17
18Network
19
20docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
21        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
22        ether 02:42:cf:ca:aa:ce  txqueuelen 0  (Ethernet)
23        RX packets 43147  bytes 2477474 (2.4 MB)
24        RX errors 0  dropped 0  overruns 0  frame 0
25        TX packets 43353  bytes 92327114 (92.3 MB)
26        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
27
28eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
29        inet 192.168.1.58  netmask 255.255.255.0  broadcast 192.168.1.255
30        ether 06:14:d5:2c:82:aa  txqueuelen 1000  (Ethernet)
31        RX packets 2303981  bytes 1891341723 (1.8 GB)
32        RX errors 0  dropped 0  overruns 0  frame 0
33        TX packets 1427379  bytes 369022653 (369.0 MB)
34        TX errors 2  dropped 0 overruns 2  carrier 0  collisions 0
35        device interrupt 40
36
37lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
38        inet 127.0.0.1  netmask 255.0.0.0
39        loop  txqueuelen 1  (Local Loopback)
40        RX packets 1514  bytes 138738 (138.7 KB)
41        RX errors 0  dropped 0  overruns 0  frame 0
42        TX packets 1514  bytes 138738 (138.7 KB)
43        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
44
45Disk
46Filesystem      Size  Used Avail Use% Mounted on
47udev            2.0G     0  2.0G   0% /dev
48tmpfs           393M  664K  392M   1% /run
49/dev/mmcblk1p7  118G   47G   66G  42% /
50.....

禁用swap

1sudo apt install dphys-swapfile \
2sudo dphys-swapfile swapoff && \
3sudo dphys-swapfile uninstall && \
4sudo update-rc.d dphys-swapfile remove

检查交换禁用是否成功

1sudo swapon --summary

docker cgroup使用systemd

见问题一

安装kubelet kubectl kubeadm

增加阿里源 /etc/apt/sources.list

1deb https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial main

执行

1curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -
2apt-get update
3apt-get install -y kubelet kubeadm kubectl

下载对应的镜像

1.查看当前kubeadm对应的镜像

 1kubeadm config images list
 2
 3# version detail:
 4
 5k8s.gcr.io/kube-apiserver:v1.18.3
 6k8s.gcr.io/kube-controller-manager:v1.18.3
 7k8s.gcr.io/kube-scheduler:v1.18.3
 8k8s.gcr.io/kube-proxy:v1.18.3
 9k8s.gcr.io/pause:3.2
10k8s.gcr.io/etcd:3.4.3-0
11k8s.gcr.io/coredns:1.6.7

2.下载 kube-* image

设置对应的版本，里面有这kube-*四个镜像

1wget https://dl.k8s.io/v1.18.3/kubernetes-server-linux-arm64.tar.gz

3.使用tar命令解压

4.快速导入

1ls -al kubernetes/server/bin/*.tar | awk '{print "docker load -i "$NF}' | bash

5.处理被墙的pause、etcd、coredns

登录国外vps, 下载对应平台的image，并推动到hub.docker.com上。绕一圈。

1docker pull k8s.gcr.io/pause-arm64:3.2
2docker pull k8s.gcr.io/etcd-arm64:3.4.3-0 
3
4docker tag k8s.gcr.io/pause-arm64:3.2 clibing/k8s-pause-arm64:3.2 
5docker tag k8s.gcr.io/etcd-arm64:3.4.3-0 clibing/k8s-etcd-arm64:3.4.3-0 
6
7docker push clibing/k8s-pause-arm64:3.2 
8docker push clibing/k8s-etcd-arm64:3.4.3-0

coredns

6.在arm主机上执行

1docker pull coredns/coredns:1.6.7
2docker tag coredns/coredns:1.6.7 k8s.gcr.io/coredns:1.6.7

7.验证是否一致

kubeadm init

1kubeadm init --kubernetes-version=1.18.3  \
2--apiserver-advertise-address=192.168.1.58   \
3--service-cidr=10.10.0.0/16 --pod-network-cidr=10.122.0.0/16

问题一

  • [WARNING IsDockerSystemdCheck]: detected “cgroupfs” as the Docker cgroup driver. The recommended driver is “systemd”. Please follow the guide at https://kubernetes.io/docs/setup/cri/

vi /etc/docker/daemon.json，加入下述内容：

1{
2  "exec-opts": ["native.cgroupdriver=systemd"]
3}

重启docker：

1systemctl restart docker

查看修改后的状态：

1docker info | grep Cgroup
  • [ERROR Swap]: running with swap on is not supported. Please disable swap
1sync; echo 3 > /proc/sys/vm/drop_caches
2swapoff -a

已经执行swap了， 不知道为什么没有生效，估计是没有重启rock64

问题二

  • join node token expire

默认token的有效期为24小时，当过期之后，该token就不可用了。解决方法如下：

  • 重新生成新的token
1kubeadm token create
2kubeadm token list

TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS aa78f6.8b4cafc8ed26c34f 23h 2017-12-26T16:36:29+08:00 authentication,signing system:bootstrappers:kubeadm:default-node-token

  • 获取ca证书sha256编码hash值
1openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
  • 节点加入集群
1kubeadm join 192.168.1.58:6443 --token qz7h0p.bn867v3gbsksg2p9 \
2    --discovery-token-ca-cert-hash sha256:计算出来的值

参考