2021-06-16 /posts/6da98871/ 99+ 12 分钟 1.9 k 0次访问

华为鲲鹏+银河麒麟v10 安装 docker-ce 踩坑

在 arm64(aarch64) 架构服务器上基于国产化操作系统安装 docker 服务

shell

# cat /etc/os-release
NAME="Kylin Linux Advanced Server"
VERSION="V10 (Tercel)"
ID="kylin"
VERSION_ID="V10"
PRETTY_NAME="Kylin Linux Advanced Server V10 (Tercel)"
ANSI_COLOR="0;31"

吐槽

所谓的国产操作系统在我看来即换皮改名操作系统，不可否认他们在权限审计方面做的比原版开源的操作系统更复杂更细腻（但是这些应该都可以自己通过 PAM 之类的配置吧）。

由于工作原因需要接触当前主流的大部分 GNU/Linux、*BSD、国产操作系统，在目前已接触的多款所谓的基于 Debian 或 Fedora 二次开发的操作系统中感触最深的不是他们上面加的各种权限审计限制，而是他们改了包名导致在安装 deb 或者 rpm 包时出现各种依赖问题。例如 CentOS7 的 rpm 包标识为 el7 麒麟上面则改成了 ky10，在安装一些软件时由于依赖问题导致同名包安装不上，如果卸载系统上已有包可能会出现系统某些软件服务出现问题，如果不卸载则只能带上痛苦面具去解决冲突。真就自主研发靠改名了。

国产的各种麒麟操作系统由于使用者多为政府单位，运行环境又是隔离内网，导致一般情况下只有安装光盘没有完整的软件源（光盘自带的完全不够用）。ε=(´ο｀*)))唉

寻找软件源

据说银河麒麟基于 CentOS7，但是通过测试最终添加 CentOS8 的源才可以用，因为他喵的 CentOS7 只有 x86_64，而 CentOS8 才有 aarch64，厂商的话都信不得哦。手动配置了 CentOS8 的源后，yum makecache 可以正常缓存，但是 yum -y update 会出现多个依赖错误问题，通过 yum -y install <package-name> 可以安装软件，但是依赖问题依然很难受。

最终在配置好 CentOS8 与 Docker-ce 官方源后由于依赖问题放弃了通过 yum 在线安装，直接下载如下 rpm 包安装依然不行。

通过二进制安装 docker

通过在线软件源和 rpm 包不能直接安装，那么只能选择通过编译安装了，去官网找了下发现提供有编译好的 docker 二进制包，直接下载二进制包安装吧，感谢 golang 的跨平台性。

安装条件

64位的操作系统
shell
1
2
# uname -p
aarch64
Linux 内核版本 ≥ 3.10
shell
1
2
# uname -r
4.19.90-17.ky10.aarch64

iptables 版本 ≥ 1.4

shell

1 2	# iptables --version iptables v1.8.1 (legacy)

一个 ps 可执行文件，通常由 procps 或类似的包提供。

安装 Docker-ce

选择并下载 docker-ce 二进制包文件

官网下载地址：https://download.docker.com/linux/static/stable/aarch64/
shell
1
wget https://download.docker.com/linux/static/stable/aarch64/docker-20.10.7.tgz
解压下载好的压缩包
shell
1
tar -zxvf docker-20.10.7.tgz
移动解压出来的二进制文件到 /usr/bin 目录中
shell
1
mv docker/* /usr/bin/
测试启动
shell
1
dockerd

添加 systemd

添加 docker 的 systemd 服务脚本至 /usr/lib/systemd/system/

脚本参考自 https://github.com/docker/docker-ce

/usr/lib/systemd/system/docker.service

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target docker.socket firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket containerd.service

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutStartSec=0
RestartSec=2
Restart=always

# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
# to make them work for either version of systemd.
StartLimitBurst=3

# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
# this option work for either version of systemd.
StartLimitInterval=60s

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Comment TasksMax if your systemd version does not support it.
# Only systemd 226 and above support this option.
TasksMax=infinity

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process
OOMScoreAdjust=-500

[Install]
WantedBy=multi-user.target

根据 docker.service 中 Unit.After 需求添加 docker.socket 脚本至 /usr/lib/systemd/system/

脚本参考自 https://github.com/docker/docker-ce

/usr/lib/systemd/system/docker.socket

[Unit]
Description=Docker Socket for the API

[Socket]
# If /var/run is not implemented as a symlink to /run, you may need to
# specify ListenStream=/var/run/docker.sock instead.
ListenStream=/run/docker.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker

[Install]
WantedBy=sockets.target

注意：如果缺少该文件，启动 docker 时会报如下错误：

shell

1 2	# systemctl start docker Failed to start docker.service: Unit docker.socket not found.

根据 docker.service 中 Unit.After 需求添加 containerd.service 脚本至 /usr/lib/systemd/system/

脚本参考自 https://github.com/containerd/containerd

/usr/lib/systemd/system/containerd.service

# Copyright The containerd Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target

[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/containerd

Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999

[Install]
WantedBy=multi-user.target

注意：如果缺少该文件，启动 docker 时会报如下错误：

shell

1 2	# systemctl restart docker Failed to restart docker.service: Unit containerd.service not found.

重载 systemd 配置文件
shell
1
systemctl daemon-reload

创建 docker 组

shell

1	groupadd docker

如不创建 docker 组在通过 systemctl 启动时会报错如下

systemctl status docker

1 2	Dependency failed for Docker Application Container Engine. Job docker.service/start failed with result 'dependency'.

启动 docker 服务

shell

1 2	systemctl start docker systemctl enable docker

修改 docker 配置文件并查看安装好的 docker 基本信息

在 /etc/docker/daemon.json 中添加如下内容：

/etc/docker/daemon.json

{
    "graph": "/data/docker",
    "storage-driver": "overlay2",
    "exec-opts": [
        "native.cgroupdriver=systemd"
    ],
    "registry-mirrors": [
        "https://t5t8q6wn.mirror.aliyuncs.com"
    ],
    "bip": "172.8.94.1/24"
}

重启 docker 服务
shell
1
systemctl restart docker

查看 docker info

docker info

# docker info
Client:
Context:    default
Debug Mode: false

Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 20.10.7
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 4.19.90-17.ky10.aarch64
Operating System: Kylin Linux Advanced Server V10 (Tercel)
OSType: linux
Architecture: aarch64
CPUs: 96
Total Memory: 510.6GiB
Name: ip-173-7-8-94.hstntx.spcsdns.net
ID: ZWMP:FDMN:5F3O:6VLV:JGB4:A2U7:W7VA:I3WP:RPHT:GRHV:TNZS:TGSG
Docker Root Dir: /data/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
https://t5t8q6wn.mirror.aliyuncs.com/
Live Restore Enabled: false
Product License: Community Engine