部署日期 | 参与 | 部署目标 | 版本 |
---|---|---|---|
Mon 19 Sep 2022 11:00:37 AM CST | @devops | 部署一套本地化的分布式持久存储的Clickhouse集群 | v0.1 |
「集群节点信息」
主机名 | IP地址 | Clickhouse节点信息 |
---|---|---|
bigdata1 | 10.0.10.195 | clickhouse-node1 |
bigdata2 | 10.0.10.194 | clickhouse-node2 |
Bigdata3 | 10.0.10.196 | clickhouse-node3 |
「节点基础信息」
操作系统 | 内核版本 | Docker版本 | Clickhouse版本 | 安装方式 |
---|---|---|---|---|
Ubuntu 20.04.3 LTS | 5.4.0-91-generic | 20.10.12 | v22.8.5.29-lts[1] | Apt包管理器 |
集群端口预留信息
端口 | 说明 | 备注 |
---|---|---|
8123 | http端口 | |
9000 | tcp端口 |
「集群配置信息」
配置项 | 配置信息 |
---|---|
服务端口 | - 8123: http端口- 9000: tcp端口 |
数据目录 | /data/clickhouse_bigdata |
集群名称 | bigdata_3shards_1replicas |
配置文件路径 | 集群配置:- 默认配置文件:/etc/clickhouse-server/config.xml- zookeeper配置文件: /etc/clickhouse-server/config.d/zookeeper.xml- 宏配置文件: /etc/clickhouse-server/config.d/macros.xml- 集群配置切片副本配置文件: /etc/clickhouse-server/config.d/remote-servers.xml用户授权认证配置:- 管理员clickhouse_manager:/etc/clickhouse-server/users.d/clickhouse_manager.xml- 普通用户clickhouse_operator: /etc/clickhouse-server/users.d/clickhouse_operator.xml |
日志目录 | /var/log/clickhouse-server/ |
分布式集群模式 | 3shards_1replicas |
服务管理 | sudo clickhouse [start/stop] |
负载均衡地址 | - HTTP: clickhouse.devopsman.cn- TCP: clickhouse-tcp.devopsman.cn |
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates dirmngr
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 8919F6BD2B48D754
echo "deb https://packages.clickhouse.com/deb stable main" | sudo tee \
/etc/apt/sources.list.d/clickhouse.list
sudo apt-get update
apt-cache madison clickhouse-server | head -n5
sudo apt-get install -y clickhouse-server=22.8.5.29 clickhouse-client=22.8.5.29
# 修改clickhouse的家目录以及相关属主组
mkdir /home/clickhouse
chown -R clickhouse.clickhouse /home/clickhouse
usermod -d /home/clickhouse clickhouse
# 准备clickhouse的数据存储目录
mkdir /data/clickhouse_bigdata
chown -R clickhouse.clickhouse /data/clickhouse_bigdata
# 在clickhouse的默认配置文件中修改clickhouse的数据目录
sed -e 's#/var/lib/clickhouse#/data/clickhouse_bigdata#g' /etc/clickhouse-server/config.xml |grep -i '/data/clickhouse_bigdata'
sed -i 's#/var/lib/clickhouse#/data/clickhouse_bigdata#g' /etc/clickhouse-server/config.xml |grep -i '/data/clickhouse_bigdata'
# sudo service clickhouse-server start
# clickhouse-client # or "clickhouse-client --password" if you've set up a password.
⚠️ 每个节点都需要配置
cat >> /etc/hosts << EOF
10.0.10.195 cknode1.devopsman.cn bigdata1
10.0.10.194 cknode2.devopsman.cn bigdata2
10.0.10.196 cknode3.devopsman.cn bigdata3
EOF
<clickhouse>
<default_on_cluster_name>bigdata</default_on_cluster_name>
<default_replica_path>/clickhouse/tables/{cluster}/{shard}/{database}/{table}</default_replica_path>
<default_replica_name>{replica}</default_replica_name>
<remote_servers>
<!-- User-specified clusters -->
<bigdata_3shards_1replicas>
<shard>
<replica>
<host>cknode1.devopsman.cn</host>
<port>9000</port>
</replica>
</shard>
<shard>
<replica>
<host>cknode2.devopsman.cn</host>
<port>9000</port>
</replica>
</shard>
<shard>
<replica>
<host>cknode3.devopsman.cn</host>
<port>9000</port>
</replica>
</shard>
</bigdata_3shards_1replicas>
</remote_servers>
</clickhouse>
<clickhouse>
<zookeeper>
<node>
<host>zookeeper-1.devopsman.cn</host>
<port>2181</port>
</node>
<node>
<host>zookeeper-2.devopsman.cn</host>
<port>2181</port>
</node>
<node>
<host>zookeeper-3.devopsman.cn</host>
<port>2181</port>
</node>
<session_timeout_ms>30000</session_timeout_ms>
<operation_timeout_ms>10000</operation_timeout_ms>
</zookeeper>
<distributed_ddl>
<path>/clickhouse/bigdata/task_queue/ddl</path>
</distributed_ddl>
</clickhouse>
<clickhouse>
<macros>
<installation>bigdata</installation>
<cluster>bigdata</cluster>
<shard>2</shard>
<replica>1</replica>
</macros>
</clickhouse>
<clickhouse>
<listen_try>1</listen_try>
<listen_host>0.0.0.0</listen_host>
<listen_host>127.0.0.1</listen_host>
</clickhouse>
<clickhouse>
<users>
<clickhouse_manager>
<access_management>1</access_management>
<networks>
<ip>10.0.10.0/24</ip>
<ip>127.0.0.1</ip>
<ip>::/0</ip>
</networks>
<password_sha256_hex>ff3ebe110b384ba70a48e23344f866d0e4e8bfcc3104709f3858d95e51111111110e2e683cbb3da4</password_sha256_hex>
<profile>clickhouse_manager</profile>
<quota>default</quota>
</clickhouse_manager>
</users>
<profiles>
<clickhouse_manager>
<log_queries>1</log_queries>
<skip_unavailable_shards>1</skip_unavailable_shards>
<http_connection_timeout>10</http_connection_timeout>
<parallel_view_processing>1</parallel_view_processing>
</clickhouse_manager>
</profiles>
</clickhouse>
<clickhouse>
<users>
<clickhouse_operator>
<networks>
<ip>10.0.10.0/24</ip>
<ip>127.0.0.1</ip>
<ip>::/0</ip>
</networks>
<password_sha256_hex>716b36073a923230c6fe1d445ac431af85f4777c5b7a155cea35911111111111111</password_sha256_hex>
<profile>clickhouse_operator</profile>
<quota>default</quota>
</clickhouse_operator>
</users>
<profiles>
<clickhouse_operator>
<log_queries>1</log_queries>
<skip_unavailable_shards>1</skip_unavailable_shards>
<http_connection_timeout>10</http_connection_timeout>
</clickhouse_operator>
</profiles>
</clickhouse>
在所有的配置文件配置完成后,切记要修改xml文件的所属属性
chown clickhouse.clickhouse /etc/clickhouse-server/config.d/*.xml /etc/clickhouse-server/users.d/*.xml
Creating clickhouse group if it does not exist.
groupadd -r clickhouse
Creating clickhouse user if it does not exist.
useradd -r --shell /bin/false --home-dir /nonexistent -g clickhouse clickhouse
Will set ulimits for clickhouse user in /etc/security/limits.d/clickhouse.conf.
Creating config directory /etc/clickhouse-server/config.d that is used for tweaks of main server configuration.
Creating config directory /etc/clickhouse-server/users.d that is used for tweaks of users configuration.
Config file /etc/clickhouse-server/config.xml already exists, will keep it and extract path info from it.
/etc/clickhouse-server/config.xml has /var/lib/clickhouse/ as data path.
/etc/clickhouse-server/config.xml has /var/log/clickhouse-server/ as log path.
Users config file /etc/clickhouse-server/users.xml already exists, will keep it and extract users info from it.
Creating log directory /var/log/clickhouse-server/.
Creating data directory /var/lib/clickhouse/.
Creating pid directory /var/run/clickhouse-server.
chown -R clickhouse:clickhouse '/var/log/clickhouse-server/'
chown -R clickhouse:clickhouse '/var/run/clickhouse-server'
chown clickhouse:clickhouse '/var/lib/clickhouse/'
groupadd -r clickhouse-bridge
useradd -r --shell /bin/false --home-dir /nonexistent -g clickhouse-bridge clickhouse-bridge
chown -R clickhouse-bridge:clickhouse-bridge '/usr/bin/clickhouse-odbc-bridge'
chown -R clickhouse-bridge:clickhouse-bridge '/usr/bin/clickhouse-library-bridge'
Enter password for default user: # 》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》此处是配置default的密码
Password for default user is saved in file /etc/clickhouse-server/users.d/default-password.xml.
Setting capabilities for clickhouse binary. This is optional.
chown -R clickhouse:clickhouse '/etc/clickhouse-server'
ClickHouse has been successfully installed.
Start clickhouse-server with:
sudo clickhouse start
Start clickhouse-client with:
clickhouse-client --password
Synchronizing state of clickhouse-server.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable clickhouse-server
Created symlink /etc/systemd/system/multi-user.target.wants/clickhouse-server.service → /lib/systemd/system/clickhouse-server.service.
Setting up clickhouse-client (22.8.5.29) ...
Processing triggers for systemd (245.4-4ubuntu3.13) ...
sudo -u clickhouse clickhouse-server --config-file=/etc/clickhouse-server/config.xml
[1]
clickhouse-v22.8.5.29-lts: https://github.com/ClickHouse/ClickHouse/releases/tag/v22.8.5.29-lts
[2]
ClickHouse分布式集群部署: https://clickhouse.com/docs/zh/getting-started/tutorial#cluster-deployment
[3]
单机部署: https://clickhouse.com/docs/zh/getting-started/install/
[4]
ClickHouse配置文件: https://clickhouse.com/docs/zh/operations/configuration-files/
[5]
大数据ClickHouse进阶: https://bbs.huaweicloud.com/blogs/375043#:~:text=%E4%BE%8B%E5%A6%82%E4%B8%80%E4%B8%AAClic,%E4%B8%AA%E5%88%86%E7%89%87%E8%AE%BE%E7%BD%AE%E5%89%AF%E6%9C%AC%E3%80%82