SundayHK
背景
在国内使用 kubespray 安装 Kubernetes 集群，下载依赖的文件和镜像时，往往会遇到下载失败，这时我们可以利用 kubespray 离线安装配置的能力来部署集群。
离线准备
- 一个不受 GFW 限制的机器，用于下载Kubernetes 所依赖的文件和镜像。
- 一个Web服务，用于提供二进制静态文件下载。可以用nginx或python
- 一个镜像仓库，用于存放Kubernetes镜像
生成依赖文件和镜像的列表
$ cd contrib/offline
$ bash generate_list.sh
$ tree temp/
temp/
├── files.list
├── files.list.template
├── images.list
└── images.list.template- flies.list 是依赖文件的列表。
- images.list 是依赖镜像的列表。
-
搬运文件
执行以下命令将依赖的静态文件全部下载到 temp/files 目录下:
# 可使用代理
# export https_proxy="192.168.10.250:7890"
wget -x -P temp/files -i temp/files.list将静态文件通过静态服务器暴露出来，比如使用 nginx，根据情况修改 nginx 配置，比如:
user root;
server {
listen 80 default_server;
listen [::]:80 default_server;
location /k8s/ {
alias /root/kubespray/contrib/offline/temp/files/;
}
}搬运镜像
使用 skopeo 将kubernetes依赖的镜像同步到内部harbor镜像仓库
提前创建仓库，如harbor提前创建名为 "k8s" 的项目，以便将所有镜像都同步到 "k8s" 这个项目路径下
访问级别：公开
skopeo 官方安装文档
推荐使用apt源安装，若使用二进制安装 skopeo，需提前创建好配置文件 /etc/containers/policy.json，内容可以用默认的，参考 default-policy.json
apt-get install skopeo安装好后，登录下自己的镜像仓库:
$ skopeo login harbor.sundayhk.com
Username: admin
Password:
Login Succeeded!将依赖镜像同步到内部harbor镜像仓库:
# 可使用代理
# export https_proxy="192.168.10.250:7890"
my_registry="harbor.sundayhk.com"
for image in $(cat temp/images.list); do skopeo copy docker://${image} docker://${my_registry}/k8s/${image#*/}; done修改 offline.yml
搬运好文件和镜像，修改 /root/kubespray/inventory/mycluster/group_vars/all/offline.yml
- registry_host: harbor.sundayhk.com/k8s
- files_repo: http://192.168.10.220/k8s
registry_host: "harbor.sundayhk.com/k8s"
files_repo: "http://192.168.10.220/k8s"
## Container Registry overrides
kube_image_repo: "{{ registry_host }}"
gcr_image_repo: "{{ registry_host }}"
github_image_repo: "{{ registry_host }}"
docker_image_repo: "{{ registry_host }}"
quay_image_repo: "{{ registry_host }}"
## Kubernetes components
kubeadm_download_url: "{{ files_repo }}/dl.k8s.io/release/v{{ kube_version }}/bin/linux/{{ image_arch }}/kubeadm"
kubectl_download_url: "{{ files_repo }}/dl.k8s.io/release/v{{ kube_version }}/bin/linux/{{ image_arch }}/kubectl"
kubelet_download_url: "{{ files_repo }}/dl.k8s.io/release/v{{ kube_version }}/bin/linux/{{ image_arch }}/kubelet"
## Two options - Override entire repository or override only a single binary.
## [Optional] 1 - Override entire binary repository
# github_url: "https://my_github_proxy"
# dl_k8s_io_url: "https://my_dl_k8s_io_proxy"
# storage_googleapis_url: "https://my_storage_googleapi_proxy"
# get_helm_url: "https://my_helm_sh_proxy"
## [Optional] 2 - Override a specific binary
## CNI Plugins
cni_download_url: "{{ files_repo }}/github.com/containernetworking/plugins/releases/download/v{{ cni_version }}/cni-plugins-linux-{{ image_arch }}-v{{ cni_version }}.tgz"
## cri-tools
crictl_download_url: "{{ files_repo }}/github.com/kubernetes-sigs/cri-tools/releases/download/v{{ crictl_version }}/crictl-v{{ crictl_version }}-{{ ansible_system | lower }}-{{ image_arch }}.tar.gz"
## [Optional] etcd: only if you use etcd_deployment=host
etcd_download_url: "{{ files_repo }}/github.com/etcd-io/etcd/releases/download/v{{ etcd_version }}/etcd-v{{ etcd_version }}-linux-{{ image_arch }}.tar.gz"
# [Optional] Calico: If using Calico network plugin
calicoctl_download_url: "{{ files_repo }}/github.com/projectcalico/calico/releases/download/v{{ calico_ctl_version }}/calicoctl-linux-{{ image_arch }}"
# [Optional] Calico with kdd: If using Calico network plugin with kdd datastore
calico_crds_download_url: "{{ files_repo }}/github.com/projectcalico/calico/archive/v{{ calico_version }}.tar.gz"
# [Optional] Cilium: If using Cilium network plugin
ciliumcli_download_url: "{{ files_repo }}/github.com/cilium/cilium-cli/releases/download/v{{ cilium_cli_version }}/cilium-linux-{{ image_arch }}.tar.gz"
# [Optional] helm: only if you set helm_enabled: true
helm_download_url: "{{ files_repo }}/get.helm.sh/helm-v{{ helm_version }}-linux-{{ image_arch }}.tar.gz"
# [Optional] crun: only if you set crun_enabled: true
crun_download_url: "{{ files_repo }}/github.com/containers/crun/releases/download/{{ crun_version }}/crun-{{ crun_version }}-linux-{{ image_arch }}"
# [Optional] kata: only if you set kata_containers_enabled: true
kata_containers_download_url: "{{ files_repo }}/github.com/kata-containers/kata-containers/releases/download/{{ kata_containers_version }}/kata-static-{{ kata_containers_version }}-{{ image_arch }}.tar.xz"
# [Optional] cri-dockerd: only if you set container_manager: docker
cri_dockerd_download_url: "{{ files_repo }}/github.com/Mirantis/cri-dockerd/releases/download/v{{ cri_dockerd_version }}/cri-dockerd-{{ cri_dockerd_version }}.{{ image_arch }}.tgz"
# [Optional] runc: if you set container_manager to containerd or crio
runc_download_url: "{{ files_repo }}/github.com/opencontainers/runc/releases/download/v{{ runc_version }}/runc.{{ image_arch }}"
# [Optional] cri-o: only if you set container_manager: crio
crio_download_base: "download.opensuse.org/repositories/devel:kubic:libcontainers:stable"
crio_download_crio: "http://{{ crio_download_base }}:/cri-o:/"
crio_download_url: "{{ files_repo }}/storage.googleapis.com/cri-o/artifacts/cri-o.{{ image_arch }}.v{{ crio_version }}.tar.gz"
skopeo_download_url: "{{ files_repo }}/github.com/lework/skopeo-binary/releases/download/v{{ skopeo_version }}/skopeo-linux-{{ image_arch }}"
# [Optional] containerd: only if you set container_runtime: containerd
containerd_download_url: "{{ files_repo }}/github.com/containerd/containerd/releases/download/v{{ containerd_version }}/containerd-{{ containerd_version }}-linux-{{ image_arch }}.tar.gz"
nerdctl_download_url: "{{ files_repo }}/github.com/containerd/nerdctl/releases/download/v{{ nerdctl_version }}/nerdctl-{{ nerdctl_version }}-{{ ansible_system | lower }}-{{ image_arch }}.tar.gz"
# [Optional] runsc,containerd-shim-runsc: only if you set gvisor_enabled: true
gvisor_runsc_download_url: "{{ files_repo }}/storage.googleapis.com/gvisor/releases/release/{{ gvisor_version }}/{{ ansible_architecture }}/runsc"
gvisor_containerd_shim_runsc_download_url: "{{ files_repo }}/storage.googleapis.com/gvisor/releases/release/{{ gvisor_version }}/{{ ansible_architecture }}/containerd-shim-runsc-v1"
