MySQL Group Replication MGR 启动报错，Error in `/xxx/mysql/bin/mysqld‘: malloc(): memory corruption:

hugo_lei

发布于 2021-08-16 17:28:28

5920

发布于 2021-08-16 17:28:28

文章被收录于专栏：Cloud Native 云原生自习室Cloud Native 云原生自习室

1 背景

在物理机上装了k8s
在物理机上继续装MySQL
启动MySQL MGR
报错如下：

*** Error in `/data/uncle/mysql/bin/mysqld': malloc(): memory corruption: 0x00007fd300026390 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x82aa6)[0x7fd46dac9aa6]
/lib64/libc.so.6(__libc_malloc+0x4c)[0x7fd46dacc6fc]
/lib64/libstdc++.so.6(_Znwm+0x1d)[0x7fd46e38c18d]
/lib64/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59)[0x7fd46e3eace9]
/lib64/libstdc++.so.6(_ZNSs12_S_constructIPKcEEPcT_S3_RKSaIcESt20forward_iterator_tag+0x21)[0x7fd46e3ec571]
/lib64/libstdc++.so.6(_ZNSsC2EPKcRKSaIcE+0x38)[0x7fd46e3ec9a8]
/data/timatrix/mysql/lib/plugin/group_replication.so(_Z24get_ipv4_local_addressesRSt3mapISsiSt4lessISsESaISt4pairIKSsiEEEb+0xa6a)[0x7fd320ab586a]
/data/timatrix/mysql/lib/plugin/group_replication.so(_Z28is_parameters_syntax_correctRK24Gcs_interface_parameters+0x13c3)[0x7fd320acab93]
/data/timatrix/mysql/lib/plugin/group_replication.so(_ZN18Gcs_xcom_interface10initializeERK24Gcs_interface_parameters+0x2c8)[0x7fd320ae04f8]
/data/timatrix/mysql/lib/plugin/group_replication.so(_ZN14Gcs_operations9configureERK24Gcs_interface_parameters+0x8c)[0x7fd320af711c]
/data/timatrix/mysql/lib/plugin/group_replication.so(_Z29configure_group_communicationP23st_server_ssl_variables+0xcf6)[0x7fd320b07c76]
/data/timatrix/mysql/lib/plugin/group_replication.so(_Z26initialize_plugin_and_join25enum_plugin_con_isolationP29Delayed_initialization_thread+0x1bb)[0x7fd320b08c1b]
/data/timatrix/mysql/lib/plugin/group_replication.so(_Z30plugin_group_replication_startv+0x585)[0x7fd320b09355]
/data/timatrix/mysql/bin/mysqld(_Z23group_replication_startv+0x84)[0xde5a24]
/data/timatrix/mysql/bin/mysqld(_Z21mysql_execute_commandP3THDb+0x2ec5)[0xc870f5]
/data/timatrix/mysql/bin/mysqld(_Z11mysql_parseP3THDP12Parser_state+0x3dd)[0xc8abed]
/data/timatrix/mysql/bin/mysqld(_Z16dispatch_commandP3THDPK8COM_DATA19enum_server_command+0xb3a)[0xc8b7da]
/data/timatrix/mysql/bin/mysqld(_Z10do_commandP3THD+0x177)[0xc8d1f7]
/data/timatrix/mysql/bin/mysqld(handle_connection+0x278)[0xd4baa8]
/data/timatrix/mysql/bin/mysqld(pfs_spawn_thread+0x1b1)[0x11a51e1]
/lib64/libpthread.so.0(+0x7ea5)[0x7fd46f556ea5]
/lib64/libc.so.6(clone+0x6d)[0x7fd46db458dd]

2 问题原因

MGR和k8s CNI同时安装时，会出bug，bug详情见GR fail to start,conflict with k8s CNI(flannel)

bug信息摘要：

Solved the problem when uninstalling k8s and flannel

ifconfig
--------------------------------------------------------------------------------
eth0: flags=4163  mtu 1500
        inet 172.24.0.61  netmask 255.255.255.0  broadcast 172.24.0.255
        ether 00:16:3e:11:9c:10  txqueuelen 1000  (Ethernet)
        RX packets 362096646  bytes 482421352506 (449.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 67049446  bytes 11734902488 (10.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 1475791  bytes 3737124512 (3.4 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1475791  bytes 3737124512 (3.4 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
--------------------------------------------------------------------------------

How to repeat:
1 Install k8s and flannel on aliyun ECS CentOS 7.4

2 Deploy cni0(Container Network Interface）with k8s and flannel

3 Initialize and start group_replication

Suggested fix:
The problem may be group_replication.so get_ipv4_local_addresses