mydumper安装及安装故障汇总

   mydumper是针对mysql数据库备份的一个轻量级第三方的开源工具,备份方式术语逻辑备份。它支持多线程,备份速度远高于原生态的mysqldump以及众多优异特性。因此该工具是DBA们的不二选择。本文主要描述该工具的安装及其出现异常的解决方法。

1、mydumper特性(直接参考readme)   * Parallelism (hence, speed) and performance (avoids expensive character set conversion routines, efficient code overall)   * Easier to manage output (separate files for tables, dump metadata, etc, easy to view/parse data)   * Consistency - maintains snapshot across all threads, provides accurate master and slave log positions, etc   * Manageability - supports PCRE for specifying database and tables inclusions and exclusions   It does not support schema dumping and leaves that to 'mysqldump --no-data'

2、获取mydumper及安装方法 下载地址:https://launchpad.net/mydumper == How to build it? == Run:  cmake .  make

One needs to install development versions of required libaries (MySQL, GLib, ZLib, PCRE): NOTE: you must use the correspondent mysql devel package.

* Ubuntu or Debian: apt-get install libglib2.0-dev libmysqlclient15-dev zlib1g-dev libpcre3-dev libssl-dev * Fedora, RedHat and CentOS: yum install glib2-devel mysql-devel zlib-devel pcre-devel openssl-devel * openSUSE: zypper install glib2-devel libmysqlclient-devel pcre-devel zlib-devel * MacOSX: port install glib2 mysql5 pcre pkgconfig cmake  (You may want to run 'port select mysql mysql5' afterwards)

One has to make sure, that pkg-config, mysql_config, pcre-config are all in $PATH

Binlog dump is disabled by default to compile with it you need to add -DWITH_BINLOG=ON to cmake options ###如果要使用binlog dump在编译的时候需要使用-DWITH_BINLOG=ON

3、快照工作原理 == How does consistent snapshot work? ==

This is all done following best MySQL practices and traditions:

* As a precaution, slow running queries on the server either abort the dump, or get killed * Global write lock is acquired ("FLUSH TABLES WITH READ LOCK") * Various metadata is read ("SHOW SLAVE STATUS","SHOW MASTER STATUS") * Other threads connect and establish snapshots ("START TRANSACTION WITH CONSISTENT SNAPSHOT") ** On pre-4.1.8 it creates dummy InnoDB table, and reads from it. * Once all worker threads announce the snapshot establishment, master executes "UNLOCK TABLES" and starts queueing jobs.

This for now does not provide consistent snapshots for non-transactional engines - support for that is expected in 0.2 :)

4、如何过滤数据库 == How to exclude (or include) databases? ==

Once can use --regex functionality, for example not to dump mysql and test databases:

 mydumper --regex '^(?!(mysql|test))'

Of course, regex functionality can be used to describe pretty much any list of tables.

== How to exclude MERGE or Federated tables ==

Use same --regex exclusion syntax. Again, engine-specific behaviors are targetted for 0.2

5、实际安装过程 # ls mydump* mydumper-0.6.2.tar.gz # tar -xvf mydumper-0.6.2.tar.gz # cd mydumper-0.6.2 [root@GZ-APP-BAK01 mydumper-0.6.2]# cmake . [root@GZ-APP-BAK01 mydumper-0.6.2]# make && make install

6、安装过程中的异常 错误1: CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: PCRE_INCLUDE_DIR (ADVANCED)    used as include directory in directory /root/mydumper-0.6.2 PCRE_PCRE_LIBRARY (ADVANCED)     linked by target "mydumper" in directory /root/mydumper-0.6.2     linked by target "myloader" in directory /root/mydumper-0.6.2 ###需要安装pcre-devel包,yum install pcre-devel  

错误2: CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: MYSQL_INCLUDE_DIR (ADVANCED)    used as include directory in directory /home/robin/mydumper-0.6.2    used as include directory in directory /home/robin/mydumper-0.6.2    used as include directory in directory /home/robin/mydumper-0.6.2 ###出现上述错误,是由于没有配置MYSQL_INCLUDE_DIR目录。 ###如果是编译安装到非缺省路径,可以将mysql安装路径添加到/etc/profile或者家目录~/.bash_profile后source生效

错误3: [root@GZ-APP-BAK01 ~]# mydumper --help|more mydumper: error while loading shared libraries: libmysqlclient.so.18: cannot open shared object file: No such file or directory [root@GZ-APP-BAK01 ~]# mydumper mydumper: error while loading shared libraries: libmysqlclient.so.18: cannot open shared object file: No such file or directory ###出现上述错误,应考虑建立软链 # which libmysqlclient.so.18 /app/soft/mysql/lib/libmysqlclient.so.18 # ln -s  /app/soft/mysql/lib/libmysqlclient.so.18 /usr/lib/libmysqlclient.so.18

错误4: # mydumper -uusr1 -ppwd -B blos -o /tmp/bak option parsing failed: Error parsing option -r, try --help ###参数和字符串不能连着写,笔者刚开始是也是丈二和尚摸不着头脑。

# mydumper --version mydumper 0.6.2, built against MySQL 5.6.22

7、获取帮助 # mydumper --help|more Usage:   mydumper [OPTION...] multi-threaded MySQL dumping

Help Options:   -?, --help                  Show help options

Application Options:   -B, --database              Database to dump   -T, --tables-list           Comma delimited table list to dump (does not exclude regex option)   -o, --outputdir             Directory to output files to   -s, --statement-size        Attempted size of INSERT statement in bytes, default 1000000   -r, --rows                  Try to split tables into chunks of this many rows. This option turns off --chunk-filesize   -F, --chunk-filesize        Split tables into chunks of this output file size. This value is in MB   -c, --compress              Compress output files   -e, --build-empty-files     Build dump files even if no data available from table   -x, --regex                 Regular expression for 'db.table' matching   -i, --ignore-engines        Comma delimited list of storage engines to ignore   -m, --no-schemas            Do not dump table schemas with the data   -k, --no-locks              Do not execute the temporary shared read lock.  WARNING: This will cause inconsistent backups   --less-locking              Minimize locking time on InnoDB tables.   -l, --long-query-guard      Set long query timer in seconds, default 60   -K, --kill-long-queries     Kill long running queries (instead of aborting)   -D, --daemon                Enable daemon mode   -I, --snapshot-interval     Interval between each dump snapshot (in minutes), requires --daemon, default 60   -L, --logfile               Log file name to use, by default stdout is used   --tz-utc                    SET TIME_ZONE='+00:00' at top of dump to allow dumping of TIMESTAMP data when a server has                               data in different time zones or data is being moved between servers with different time zones,                               defaults to on use --skip-tz-utc to disable.   --skip-tz-utc                 --use-savepoints            Use savepoints to reduce metadata locking issues, needs SUPER privilege   --success-on-1146           Not increment error count and Warning instead of Critical in case of table doesn't exist   --lock-all-tables           Use LOCK TABLE for all, instead of FTWRL   -h, --host                  The host to connect to   -u, --user                  Username with privileges to run the dump   -p, --password              User password   -P, --port                  TCP/IP port to connect to   -S, --socket                UNIX domain socket file to use for connection   -t, --threads               Number of threads to use, default 4   -C, --compress-protocol     Use compression on the MySQL connection   -V, --version               Show the program version and exit   -v, --verbose               Verbosity of output, 0 = silent, 1 = errors, 2 = warnings, 3 = info, default 2

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏乐沙弥的世界

Failed to upgrade Oracle Cluster Registry configuration(root.sh)

    最近在给客户基于Suse 11 sp3安装Oracle 10g RAC,在安装完clusterware执行/u01/app/crs/root.sh时收...

10910
来自专栏JavaNew

Spring Boot实战:数据库操作

442150
来自专栏杨建荣的学习笔记

oracle工具集初探(r4笔记第8天)

今天无意中看了下ORACLE_HOME/bin下面的东西,发现里面还是存在不少的东西。除了常用的sqlplus,tnsping,rman,exp/expdp,i...

31040
来自专栏杨建荣的学习笔记

11g Dataguard中的snapshot standby特性(r8笔记第49天)

11g中的ADG特性本身已经非常有特色,促使很多对于10g中不太灵便的备库升级到11g,对于DBA是一大福利,那么还有一个福利就是snapshot standb...

31850
来自专栏用户2442861的专栏

Intellij IDEA 14中使用MyBatis-generator 自动生成MyBatis代码

  Intellij IDEA 14 作为Java IDE 神器,接触后发现,非常好用,对它爱不释手,打算离开eclipse和myeclipse,投入Inte...

68210
来自专栏aoho求索

Spring Boot集成MyBatis实现通用Mapper

关于MyBatis,大部分人都很熟悉。MyBatis 是一款优秀的持久层框架,它支持定制化 SQL、存储过程以及高级映射。MyBatis 避免了几乎所有的 JD...

59510
来自专栏Albert陈凯

2018-11-17 面试必问问题TransactionalJava事务之一——Java事务的基本问题

关于加@Transactional注解的方法之间调用,事务是否生效的问题 https://blog.csdn.net/blacktal/article/det...

35420
来自专栏醉生梦死

shell脚本--练习1(爬虫)

39740
来自专栏杨建荣的学习笔记

10g升级至11g后数据库无法启动的问题解决

今天升级数据库碰到一个很郁闷的问题,把10g的数据库升级到11g以后,结果有一个改动,需要重启数据库,就敲了shutdown immediate,结果再star...

42260
来自专栏编程

PHP中用PDO查询Mysql来避免SQL注入风险的方法

当我们使用传统的 mysql_connect 、mysql_query方法来连接查询数据库时,如果过滤不严,就有SQL注入风险,导致网站被攻击,失去控制。虽然可...

28480

扫码关注云+社区

领取腾讯云代金券