Harbor介绍及我们的改造

说明：我们是基于Harbor V0.4.1进行分析和改造的。 ##为什么不是直接使用Registry V2，而选用Harbor？ ###可以用Harbor做以下事情：

• Manage your projects.
• Manage members of a project.
• Replicate projects to a remote registry.
• Search projects and repositories.
• Manage Harbor system if you are the system administrator:
• Manage users.
• Manage destinations.
• Manage replication policies.
• Pull and push images using Docker client.
• Delete repositories.

###Harbor提供RBAC (Role Based Access Control)能力

• Guest: Guest has read-only privilege for a specified project.
• Developer: Developer has read and write privileges for a project.
• ProjectAdmin: When creating a new project, you will be assigned the "ProjectAdmin" role to the project. Besides read-write privileges, the "ProjectAdmin" also has some management privileges, such as adding and removing members.
• SysAdmin: "SysAdmin" has the most privileges. In addition to the privileges mentioned above, "SysAdmin" can also list all projects, set an ordinary user as administrator and delete users. The public project "library" is also owned by the administrator.
• Anonymous: When a user is not logged in, the user is considered as an "anonymous" user. An anonymous user has no access to private projects and has read-only access to public projects.

在我们的生产环境中，不会启动Anonymous用户。

###Harbor支持2种Project Types

• Public: All users have the read privilege to a public project, it‘s convenient for you to share some repositories with others in this way.
• Private: A private project can only be accessed by users with proper privileges.

##Harbor的架构

Harbor架构

##Harbor Workflow

the workflow of docker login to harbor

the workflow of docker push to harbor

具体的架构说明和workflow说明请移步https://github.com/vmware/harbor/wiki/Architecture-Overview-of-Harbor

###Harbor replicate功能源码分析 Harbor中一个非常重要的功能就是同步复制，下面是我基于Harbor v0.4.1版本的代码做的源码分析，标明了其中我认为比较重要的逻辑。

code analysis for harbor replicate

注意，目前只支持从一个Harbor Instance同步到另一个或者多个Harbor Instance，不支持Target为Registry。

##Harbor HA 目前，Harbor官方还不提供HA的能力和方案（Harbor HA feature design proposal/discussion #327）。因此需要自己根据公司实际的场景进行HA方案的设计。 上面的PR中提到HA需具备的3个关键条件：

• Harbor's registry should have multiple instances to provide service. Failure of any individual instance doesn't interrupt the service.
• Image storage should have redundancy and shareable by multiple registry instances.
• MySQL database should have high availability. ###Harbor HA One 你可能很快会想到如下的HA部署：

当然，实际上的DB HA方案不会是这样，你或许采用主备，读写分离等等，在这里只是表示简单表示要做DB的HA.

Harbor HA One

上面的部署架构需要你具备把Harbor的6个服务进行拆分的能力，丢弃官方方便的compose部署，实现自己的自动部署(ansible is good)。 多机房的镜像同步通过配置Harbor的replicated功能来完成。

###Harbor HA Two 如果你不想破坏Harbor官方部署的完整性，那么你可能会想到如下方案：

Harbor HA Two

这个方案中，通过HAProxy代理后端多个完整的Harbor Instances，这样构建的Harbor Cluster还需要你配置多个mysql instance之间的数据同步。 同一机房的Harbor Cluster使用Ceph作为后端共享存储。 多机房的镜像同步仍然可以通过配置Harbor的replicated功能来完成。

###Our Harbor HA 家家有本难念的经！ 由于各种原因，生产环境无法提供分布式存储的能力，只有使用local disk作为registry的存储。因此方案开始变得复杂，甚至可以说是恶心。

Our Harbor HA

由于这样的部署模型，带来很多原本Harbor好好的功能，变得不可用或者很多Bug，还好最终这些Bug都解决了,一言难尽。后面会提到一些。

##我们基于Harbor做了哪些定制开发 先看看我们用gitlab issues跟踪的任务清单： ![Task List]

这里写图片描述

###对接CAS

• Harbor UI login/logout Redirect to CAS
• code a golang cas client, to be a beego controller.
• Detach CAS User with Harbor User
• Bug fix
• docker login 401
• Modify harbor user password error
• Portal modify slightly
• …

###对接CMDB

integration with cmdb 1

integration with cmdb 2

###对接OA

integration with oa

###Harbor组件拆分 在我们的部署模型中，需要将mysql和registry单独部署在其他Host上，其他四个组件继续在一个Host上部署，这其中涉及到很多配置的更改。 我们的所有Deploy，都是基于Ansible进行自动化部署的。

harbor split

###其他工作

• Build audit log file and adapt dragonfly
• Format Harbor log to adapt dragonfly
• Ansible scripts to deploy HA cluster
• Integration with OA user validate api
• HAProxy session affinity Config
• Auto config target, auto create & enable policies for all domains
• Bugs fix
• Replicate Function cannot be used in our deploy scene.
• Mistaken delete repositories when syncing from registry to DB .
• Trigger endless loop to call “DELETE /api/repository” when Delete repository in Portal.
• …

###下一步做什么

• New Portal
• Integerate with Clair
• Integerate with PaaS CI/CD
• New Deploy Model (urges using ceph)
• Separate token service from harbor
• Performance Test
• Fault simulation
• GC scheme
• Known bugs fix
• ......