用Harbor实现容器镜像仓库的管理和运维<一>

发表于 | 分类于 | 人
| 字数统计: 2,916 | 阅读时长 ≈ 16

Harbor介绍

Harbor是由VMware中国研发团队负责开发的开源企业级Registry，可帮助用户迅速搭建企业级的Registry服务。该项目发布5多个月以来，深受用户喜爱，在GitHub获得了近1000个点赞星星和200多个Forks。 Harbor的GitHub地址： https://github.com/vmware/harbor
其目标是帮助用户迅速搭建一个企业级的Dockerregistry服务。它以Docker公司开源的registry为基础，提供了管理UI，基于角色的访问控制(Role Based Access Control)，AD/LDAP集成、以及审计日志(Auditlogging) 等企业用户需求的功能，同时还原生支持中文。Harbor的每个组件都是以Docker容器的形式构建的，使用Docker Compose来对它进行部署。用于部署Harbor的Docker Compose模板位于 /$harbor/docker-compose.yml，由5个容器组成：
Proxy：由Nginx 服务器构成的反向代理。
Registry：由Docker官方的开源registry镜像构成的容器实例。
UI：即架构中的core services， 构成此容器的代码是Harbor项目的主体。
MySQL：由官方MySQL镜像构成的数据库容器。
Log：运行着rsyslogd的容器，通过log-driver的形式收集其他容器的日志。
这几个容器通过Docker link的形式连接在一起，在容器之间通过容器名字互相访问。对终端用户而言，只需要暴露proxy （即Nginx）的服务端口。

环境准备

harbor离线下载包地址:harbor离线安装包harbor-offline-installer-v1.2.2.tgz
Harbor使用几个Docker容器来部署的，因此部署Harbor的节点上要求安装好Docker服务，同时要求安装好Python，DockerCompose，相关组件的版本如下：
python >= 2.7

1
2
[root@registry01 harbor]# python --version
Python 2.7.5

Docker engine >= 1.10

1
2
[root@registry01 harbor]# docker --version
Docker version 1.12.6, build 0fdc778/1.12.6

Docker Compose >= 1.6.0

1
2
[root@registry01 harbor]# docker-compose --version
docker-compose version 1.17.0, build ac53b73

说明：在openshift中，默认的registry上没有安装docker-compose，因此需要手动安装docker-compose。

docker-compose安装方法

1
2
3
4
5
6
7
8
[root@registry01 bin]# curl -L https://github.com/docker/compose/releases/download/1.17.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   617    0   617    0     0    429      0 --:--:--  0:00:01 --:--:--   429
  0     0    0     0    0     0      0      0 --:--:--  0:00:21 --:--:--     0
[root@registry01 bin]#chmod +x usr/local/bin/docker-compose
[root@registry01 bin]# docker-compose --version
docker-compose version 1.17.0, build ac53b73

Harbor安装（HTTP模式）

harbor的安装包下载好之后，上传到registry服务器中。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
[root@registry01 ~]# mkdir /data/
[root@registry01 ~]# tar -xzvf harbor.v1.2.2.tar.gz -C /data
[root@registry01 harbor]# vi harbor.cfg  修改hostname
[root@registry01 harbor]# ./prepare 
Generated and saved secret to file: /data/secretkey
Generated configuration file: ./common/config/nginx/nginx.conf
Generated configuration file: ./common/config/adminserver/env
Generated configuration file: ./common/config/ui/env
Generated configuration file: ./common/config/registry/config.yml
Generated configuration file: ./common/config/db/env
Generated configuration file: ./common/config/jobservice/env
Generated configuration file: ./common/config/jobservice/app.conf
Generated configuration file: ./common/config/ui/app.conf
Generated certificate, key file: ./common/config/ui/private_key.pem, cert file: ./common/config/registry/root.crt
The configuration files are ready, please use docker-compose to start the service.
[root@registry01 harbor]# ./install.sh
[Step 0]: checking installation environment ...
Note: docker version: 1.12.6
Note: docker-compose version: 1.17.0
[Step 1]: loading Harbor images ...
dd60b611baaa: Loading layer [==================================================>] 133.2 MB/133.2 MB
abf0579c40fd: Loading layer [==================================================>] 1.536 kB/1.536 kB
ea1fc7bed9c5: Loading layer [==================================================>] 22.48 MB/22.48 MB
1d6671367c69: Loading layer [==================================================>] 7.168 kB/7.168 kB
b322bb3e4765: Loading layer [==================================================>] 5.339 MB/5.339 MB
0cf512d418ac: Loading layer [==================================================>] 9.728 kB/9.728 kB
4a7cdc0b1a2b: Loading layer [==================================================>]  2.56 kB/2.56 kB
ef1130526636: Loading layer [==================================================>] 22.48 MB/22.48 MB
Loaded image: vmware/harbor-ui:v1.2.2                                           ] 229.4 kB/22.48 MB
4a050fccec52: Loading layer [==================================================>] 12.16 MB/12.16 MB
d918d73369ec: Loading layer [==================================================>]  17.3 MB/17.3 MB
22898836924e: Loading layer [==================================================>] 15.87 kB/15.87 kB
Loaded image: vmware/notary-photon:server-0.5.0                                 ]    512 B/15.87 kB
76c156eab077: Loading layer [==================================================>]   134 MB/134 MB
1eae6563289a: Loading layer [==================================================>] 16.42 MB/16.42 MB
Loaded image: vmware/nginx-photon:1.11.13                                       ] 171.1 kB/16.42 MB
2e814f7ef645: Loading layer [==================================================>] 2.048 kB/2.048 kB
bc5742b580db: Loading layer [==================================================>] 2.048 kB/2.048 kB
5413bcdb81b0: Loading layer [==================================================>]  2.56 kB/2.56 kB
c4e2be066795: Loading layer [==================================================>] 3.584 kB/3.584 kB
a4ea62be60b0: Loading layer [==================================================>]  22.8 MB/22.8 MB
800a351ae5da: Loading layer [==================================================>]  22.8 MB/22.8 MB
Loaded image: vmware/registry:2.6.2-photon                                      ] 229.4 kB/22.8 MB
Loaded image: photon:1.0
a39bd6a7f897: Loading layer [==================================================>] 10.95 MB/10.95 MB
6f79b8337a1f: Loading layer [==================================================>]  17.3 MB/17.3 MB
74bbd0e81dd0: Loading layer [==================================================>] 15.87 kB/15.87 kB
Loaded image: vmware/notary-photon:signer-0.5.0                                 ]    512 B/15.87 kB
2202528221a2: Loading layer [==================================================>]  7.07 MB/7.07 MB
4fe250d3c912: Loading layer [==================================================>]  7.07 MB/7.07 MB
Loaded image: vmware/harbor-adminserver:v1.2.2                                  ]  98.3 kB/7.07 MB
9463fb852970: Loading layer [==================================================>] 75.37 MB/75.37 MB
d2c9a2a395d9: Loading layer [==================================================>] 3.584 kB/3.584 kB
b08aea2a8a82: Loading layer [==================================================>] 3.072 kB/3.072 kB
103e65a1013b: Loading layer [==================================================>] 3.072 kB/3.072 kB
Loaded image: vmware/harbor-log:v1.2.2                                          ]    512 B/3.072 kB
5d6cbe0dbcf9: Loading layer [==================================================>] 129.2 MB/129.2 MB
435f2dfbd884: Loading layer [==================================================>] 344.6 kB/344.6 kB
814d7b59f0cc: Loading layer [==================================================>] 4.657 MB/4.657 MB
aae399245bd0: Loading layer [==================================================>] 1.536 kB/1.536 kB
21e2ae955f72: Loading layer [==================================================>] 33.84 MB/33.84 MB
a2d0f7b84059: Loading layer [==================================================>] 25.09 kB/25.09 kB
819fa6af55b8: Loading layer [==================================================>] 3.584 kB/3.584 kB
78914c99a468: Loading layer [==================================================>] 167.7 MB/167.7 MB
36e79c658afb: Loading layer [==================================================>] 6.144 kB/6.144 kB
f73503aca003: Loading layer [==================================================>] 9.216 kB/9.216 kB
a21b39f6da59: Loading layer [==================================================>] 1.536 kB/1.536 kB
ef81eb7c77b3: Loading layer [==================================================>] 8.704 kB/8.704 kB
08d0cfe60b0d: Loading layer [==================================================>] 4.608 kB/4.608 kB
0864dda8f611: Loading layer [==================================================>] 4.608 kB/4.608 kB
Loaded image: vmware/harbor-db:v1.2.2                                           ]    512 B/4.608 kB
29d1f4ae97dd: Loading layer [==================================================>] 18.31 MB/18.31 MB
7caf936e1402: Loading layer [==================================================>] 18.31 MB/18.31 MB
Loaded image: vmware/harbor-jobservice:v1.2.2                                   ] 196.6 kB/18.31 MB
78dbfa5b7cbc: Loading layer [==================================================>] 130.9 MB/130.9 MB
5f70bf18a086: Loading layer [==================================================>] 1.024 kB/1.024 kB
8deec01122be: Loading layer [==================================================>] 344.6 kB/344.6 kB
574ab36807f2: Loading layer [==================================================>] 1.536 kB/1.536 kB
d8f2cde2eef8: Loading layer [==================================================>] 20.48 kB/20.48 kB
eaa3924b054e: Loading layer [==================================================>]  5.12 kB/5.12 kB
8aa2c772121c: Loading layer [==================================================>] 184.3 MB/184.3 MB
c3014bbccb0b: Loading layer [==================================================>] 8.704 kB/8.704 kB
978a35efaa8c: Loading layer [==================================================>] 4.608 kB/4.608 kB
c2385ae7d6e5: Loading layer [==================================================>]  16.6 MB/16.6 MB
Loaded image: vmware/harbor-notary-db:mariadb-10.1.10                           ] 196.6 kB/16.6 MB
c192a34d4ff4: Loading layer [==================================================>] 155.2 MB/155.2 MB
d012a9276a83: Loading layer [==================================================>] 10.75 MB/10.75 MB
b8befd881cb5: Loading layer [==================================================>] 10.75 MB/10.75 MB
Loaded image: vmware/clair:v2.0.1-photon                                        ] 131.1 kB/10.75 MB
bbda1562018e: Loading layer [==================================================>] 101.6 MB/101.6 MB
1171ab08cc04: Loading layer [==================================================>] 6.656 kB/6.656 kB
6df81d3a0683: Loading layer [==================================================>] 6.656 kB/6.656 kB
Loaded image: vmware/postgresql:9.6.4-photon                                    ]    512 B/6.656 kB
[Step 2]: preparing environment ...
Clearing the configuration file: ./common/config/adminserver/env
Clearing the configuration file: ./common/config/ui/env
Clearing the configuration file: ./common/config/ui/app.conf
Clearing the configuration file: ./common/config/ui/private_key.pem
Clearing the configuration file: ./common/config/db/env
Clearing the configuration file: ./common/config/jobservice/env
Clearing the configuration file: ./common/config/jobservice/app.conf
Clearing the configuration file: ./common/config/registry/config.yml
Clearing the configuration file: ./common/config/registry/root.crt
Clearing the configuration file: ./common/config/nginx/nginx.conf
loaded secret from file: /data/secretkey
Generated configuration file: ./common/config/nginx/nginx.conf
Generated configuration file: ./common/config/adminserver/env
Generated configuration file: ./common/config/ui/env
Generated configuration file: ./common/config/registry/config.yml
Generated configuration file: ./common/config/db/env
Generated configuration file: ./common/config/jobservice/env
Generated configuration file: ./common/config/jobservice/app.conf
Generated configuration file: ./common/config/ui/app.conf
Generated certificate, key file: ./common/config/ui/private_key.pem, cert file: ./common/config/registry/root.crt
The configuration files are ready, please use docker-compose to start the service.
[Step 3]: checking existing instance of Harbor ...
[Step 4]: starting Harbor ...
Creating network "harbor_harbor" with the default driver
Creating harbor-log ... 
Creating harbor-log ... done
Creating harbor-db ... 
Creating registry ... 
Creating harbor-adminserver ... 
Creating registry
Creating harbor-adminserver
Creating registry ... done
Creating harbor-db ... done
Creating harbor-ui ... done
Creating nginx ... 
Creating harbor-jobservice ... 
Creating nginx
Creating nginx ... done
✔ ----Harbor has been installed and started successfully.----
Now you should be able to visit the admin portal at http://registry01.demo.com. 
For more details, please visit https://github.com/vmware/harbor .
[root@registry01 harbor]# systemctl daemon-reload
[root@registry01 harbor]# systemctl restart docker.service
[root@registry01 harbor]# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2017-12-04 10:44:55 CST; 30min ago
     Docs: http://docs.docker.com
 Main PID: 13161 (dockerd-current)
   Memory: 50.9M
   CGroup: /system.slice/docker.service
           ├─13161 /usr/bin/dockerd-current --insecure-registry 192.168.10.16:5000 --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current --default-runtime=d...
           ├─13167 /usr/bin/docker-containerd-current -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim docker-containerd-shim --metrics-interval=0 ...
           ├─13333 /usr/libexec/docker/docker-proxy-current -proto tcp -host-ip 127.0.0.1 -host-port 1514 -container-ip 172.18.0.2 -container-port 514
           ├─13341 /usr/bin/docker-containerd-shim-current 985eeeaa997ba93fe546b13910d31e0f1f4a78891e09855443b48be9c00ce063 /var/run/docker/libcontainerd/985eeeaa997ba9...
           ├─13412 /usr/bin/docker-containerd-shim-current 42fa92850b638e76618cb99810ddbf03f54ab1535c5d2e5514214467a1287718 /var/run/docker/libcontainerd/42fa92850b638e...
           ├─13495 /usr/bin/docker-containerd-shim-current 9cde9a9c34678f1f05c00081c96acc3f70a0dbc82862a365f8292b9b55e49a57 /var/run/docker/libcontainerd/9cde9a9c34678f...
           ├─13575 /usr/bin/docker-containerd-shim-current fe86c99f103293578fa0a91b2589761fbee017af51bf6c6fe1465000ea3d8c39 /var/run/docker/libcontainerd/fe86c99f103293...
           ├─13664 /usr/libexec/docker/docker-proxy-current -proto tcp -host-ip 0.0.0.0 -host-port 4443 -container-ip 172.18.0.6 -container-port 4443
           ├─13672 /usr/libexec/docker/docker-proxy-current -proto tcp -host-ip 0.0.0.0 -host-port 443 -container-ip 172.18.0.6 -container-port 443
           ├─13680 /usr/libexec/docker/docker-proxy-current -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 172.18.0.6 -container-port 80
           ├─13769 /usr/bin/docker-containerd-shim-current 7051a590f014bb795b98c772e4fe6c1285f098f8ec389f53e8f54a09bce8ad30 /var/run/docker/libcontainerd/7051a590f014bb...
           ├─16083 /usr/bin/docker-containerd-shim-current 9c91a1b08ca13f7182363c99a70ffa7dc8545e4913cac41090a241ea4dcf5e52 /var/run/docker/libcontainerd/9c91a1b08ca13f...
           └─16156 /usr/bin/docker-containerd-shim-current ee4f90c39d99bb24aa94654fb519673f01e2ae8cc9bcd81f8b6ae7071ff5d27a /var/run/docker/libcontainerd/ee4f90c39d99bb...
[root@registry01 harbor]# docker ps
CONTAINER ID        IMAGE                              COMMAND                  CREATED             STATUS              PORTS                                                              NAMES
fe86c99f1032        vmware/harbor-jobservice:v1.2.2    "/harbor/harbor_jobse"   About an hour ago   Up 38 minutes                                                                          harbor-jobservice
9c91a1b08ca1        vmware/nginx-photon:1.11.13        "nginx -g 'daemon off"   About an hour ago   Up 38 minutes       0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp   nginx
ee4f90c39d99        vmware/harbor-ui:v1.2.2            "/harbor/harbor_ui"      About an hour ago   Up 38 minutes                                                                          harbor-ui
7051a590f014        vmware/harbor-db:v1.2.2            "docker-entrypoint.sh"   About an hour ago   Up 38 minutes       3306/tcp                                                           harbor-db
42fa92850b63        vmware/registry:2.6.2-photon       "/entrypoint.sh serve"   About an hour ago   Up 38 minutes       5000/tcp                                                           registry
9cde9a9c3467        vmware/harbor-adminserver:v1.2.2   "/harbor/harbor_admin"   About an hour ago   Up 38 minutes                                                                          harbor-adminserver
985eeeaa997b        vmware/harbor-log:v1.2.2           "/bin/sh -c 'crond &&"   About an hour ago   Up 38 minutes       127.0.0.1:1514->514/tcp                                            harbor-log

问题1：在执行./install.sh时报错

1
2
3
4
5
6
7
[root@registry01 harbor]# ./install.sh 
➜ Please set hostname and other necessary attributes in harbor.cfg first. DO NOT use localhost or 127.0.0.1 for hostname, because Harbor needs to be accessed by external clients.
Please set --with-notary if needs enable Notary in Harbor, and set ui_url_protocol/ssl_cert/ssl_cert_key in harbor.cfg bacause notary must run under https. 
Please set --with-clair if needs enable Clair in Harbor
[root@registry01 harbor]#vim harbor.cfg
****解决方法****：
修改harbor.cfg，将hostname=127.0.0.1改为hostname=registry01.demo.com

安装完成之后，启动docker-compose

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
[root@registry01 harbor]# docker-compose start
Starting log         ... done
Starting adminserver ... done
Starting registry    ... done
Starting ui          ... done
Starting mysql       ... done
Starting jobservice  ... done
Starting proxy       ... done
[root@registry01 harbor]# docker-compose ps
       Name                     Command               State                                Ports                              
------------------------------------------------------------------------------------------------------------------------------
harbor-adminserver   /harbor/harbor_adminserver       Up                                                                      
harbor-db            docker-entrypoint.sh mysqld      Up      3306/tcp                                                        
harbor-jobservice    /harbor/harbor_jobservice        Up                                                                      
harbor-log           /bin/sh -c crond && rm -f  ...   Up      127.0.0.1:1514->514/tcp                                         
harbor-ui            /harbor/harbor_ui                Up                                                                      
nginx                nginx -g daemon off;             Up      0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp, 0.0.0.0:80->80/tcp
registry             /entrypoint.sh serve /etc/ ...   Up      5000/tcp

问题2：执行docker-compose start时报错

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[root@registry01 ~]# docker-compose start
ERROR: 
        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?
        Supported filenames: docker-compose.yml, docker-compose.yaml
****解决方法****
启动docker-compose需要进入到harbor的目录下启动，
[root@registry01 ~]# cd /data/harbor/
[root@registry01 harbor]# docker-compose start
Starting log         ... done
Starting adminserver ... done
Starting registry    ... done
Starting ui          ... done
Starting mysql       ... done
Starting jobservice  ... done
Starting proxy       ... done

登陆及镜像上传

命令行登陆 （默认密码为Harbor12345，可在harbor.cfg中修改）

1
2
3
4
[root@registry01 harbor]# docker login registry01.demo.com:80
Username (admin): admin
Password: 
Login Succeeded

网页登陆

登录并创建自己的镜像仓库(初始用户名/密码初始化为：admin/Harbor12345)。
这里我们使用HTTP方式，那么访问地址为：
http://192.168.10.16/
harbor——login

镜像上传

上传镜像需要先设置insecure-registry

1
2
3
4
5
6
7
8
9
10
11
12
13
[root@registry01 ~]# vi /usr/lib/systemd/system/docker.service 
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=network.target
Wants=docker-storage-setup.service
Requires=rhel-push-plugin.socket registries.service
Requires=docker-cleanup.timer
...省略中间部分
ExecStart=/usr/bin/dockerd-current \
          --insecure-registry registry01.demo.com \ <!--将insecure-registry设置为registry.demo.com-->
          --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current \

修改好之后，然后修改docker-compose.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
[root@registry01 harbor]# cp docker-compose.yml docker-compose.yml-`date +%F`
[root@registry01 harbor]# vi docker-compose.yml
version: '2'
services:
  log:
    image: vmware/harbor-log:v1.2.2
    container_name: harbor-log
    restart: always
    volumes:
      - /var/log/harbor/:/var/log/docker/:z
    ports:
      - 127.0.0.1:1514:514
...
proxy:
    image: vmware/nginx-photon:1.11.13
    container_name: nginx
    restart: always
    volumes:
      - ./common/config/nginx:/etc/nginx:z
    networks:
      - harbor
    ports:
      - 80:80
      - 443:443
      - 4443:4443
      - 5000:5000  <!--添加registry的端口-->

修改好之后，从命令行登陆

1
2
3
4
5
6
7
8
9
10
11
12
13
[root@registry01 harbor]# docker login registry01.demo.com
Username: admin
Password: 
Login Succeeded
[root@registry01 harbor]# docker push registry01.demo.com/library/photon:1.0
The push refers to a repository [registry01.demo.com/library/photon]
dd60b611baaa: Pushed 
1.0: digest: sha256:6d8fda39c2eb8fdc7b18c27f53fb6c01ac7721e7d55e7d6ae4cf6b1f3f0109fb size: 529
[root@registry01 harbor]# docker images
REPOSITORY                                                                 TAG                 IMAGE ID            CREATED             SIZE
...
registry01.demo.com/library/photon                                         1.0                 e6e4e4a2ba1b        17 months ago       127.5 MB
photon                                                                     1.0                 e6e4e4a2ba1b        17 months ago       127.5 MB

上传成功
harbor会告诉使用者相关上传镜像的命令（以项目名为library为例）

1
2
docker tag SOURCE_IMAGE[:TAG] registry01.demo.com/library/IMAGE[:TAG]
docker push registry01.demo.com/library/IMAGE[:TAG]

登陆到网页上查看

PS：
1、如果需要用harbor管理registry上已有的镜像时，需要先将已有的镜像导入到harbor管理的镜像仓库中。
2、往registry中上传镜像之前需要先从命令行登陆，登陆之后才能push镜像

----本文结束感谢您的阅读----
0%