GDB多线程多进程调试

原创

chain

发布于 2018-06-05 13:10:55

12.2K3

发布于 2018-06-05 13:10:55

文章被收录于专栏：开发 & 算法杂谈开发 & 算法杂谈

gdb多线程调试

gdb提供的多线程调试工具

新线程创建自动提醒
thread thread-id实现不同线程之间的切换
info threads查询存在的线程
thread apply [thread-id-list] [all] args在一系列线程上执行命令
线程中设置指定的断点
set print thread-events控制打印线程启动或结束是的信息
set scheduler-locking off|on|step在使用step或是continue进行调试的时候，其他可能也会并行的执行，如何才能够只让被调试的线程执行呢？该命令工具可以达到这个效果。

off：不锁定任何线程，也就是所有的线程都执行，这是默认值。

on：只有当前被调试的线程能够执行。

step：阻止其他线程在当前线程单步调试时，抢占当前线程。只有当next、continue、util以及finish的时候，其他线程才会获得重新运行的机会。

示例代码

#include <iostream>
#include <pthread.h>
#include <iostream>

void* threadPrintHello(void* arg)
{
    while(1)
    {
        sleep(5);
        std::cout << "hello" << std::endl;
    }
}

void* threadPrintWorld(void* arg)
{
    while(1)
    {
        sleep(5);
        std::cout << "world" << std::endl;
    }
}

int main( int argc , char* argv[])
{
    pthread_t pid_hello , pid_world;

    int ret = 0;

    ret = pthread_create(&pid_hello , NULL , threadPrintHello , NULL);

    if( ret != 0 )
    {
        std::cout << "Create threadHello error" << std::endl;
        return -1;
    }

    ret = pthread_create(&pid_world , NULL , threadPrintWorld , NULL);

    if( ret != 0 )
    {
        std::cout << "Create threadWorld error" << std::endl;
        return -1;
    }

    while(1)
    {
        sleep(10);
        std::cout << "In main thread"  << std::endl;
    }

    pthread_join(pid_hello , NULL);
    pthread_join(pid_world , NULL);

    return 0;
}

线程创建提醒

在GNU/Linux上，如果gdb检测一个新的线程，会给出如下通知

[New Thread 0x7ffff708b700 (LWP 20567)]
[New Thread 0x7ffff688a700 (LWP 20568)]

查询已经存在的线程

使用info threads可以看到程序中所有线程的信息

(gdb) info threads
  3 Thread 0x7ffff688a700 (LWP 20568)  0x00007ffff70be8e0 in sigprocmask () from /lib64/libc.so.6
  2 Thread 0x7ffff708b700 (LWP 20567)  0x00007ffff7138a3d in nanosleep () from /lib64/libc.so.6
* 1 Thread 0x7ffff7fe5720 (LWP 20564)  main (argc=1, argv=0x7fffffffe628) at multithreads.cpp:39

主要包括gdb分配的线程id号（例如1，2，3），操作系统分配的线程id（例如20568），线程的名字以及线程相关的调用栈信息。

切换线程

thread threadno可以切换到指定的线程，threadno就是上面gdb分配的线程id号。

(gdb) thread 2
[Switching to thread 2 (Thread 0x7ffff708b700 (LWP 20567))]#0  0x00007ffff7138a3d in nanosleep () from /lib64/libc.so.6

锁定一个线程

默认情况下gdb不锁定任何线程

(gdb) thread 2
[Switching to thread 2 (Thread 0x7ffff708b700 (LWP 20567))]#0  threadPrintHello (arg=0x0) at multithreads.cpp:10
10              std::cout << "hello" << std::endl;
(gdb) n
helloworld

In main thread
7           while(1)
(gdb) n
9               sleep(5);
(gdb) n
world
In main thread
10              std::cout << "hello" << std::endl;

当我们切换到线程2的时候，使用next执行的时候，发现打印的结果中包含有其他线程的打印信息。

可以使用set scheduler-locking on来锁定只有当前的线程能够执行。

(gdb) thread 2
[Switching to thread 2 (Thread 0x7ffff708b700 (LWP 20567))]#0  threadPrintHello (arg=0x0) at multithreads.cpp:10
10              std::cout << "hello" << std::endl;
(gdb) set scheduler-locking on
(gdb) n
hello
7           while(1)
(gdb) n
9               sleep(5);
(gdb) n
10              std::cout << "hello" << std::endl;
(gdb) n
hello
7           while(1)
(gdb) n
9               sleep(5);
(gdb) n
10              std::cout << "hello" << std::endl;

可以发现锁定线程之后，使用next执行变不会有其它线程的打印结果。

执行命令

使用thread apply来让一个或是多个线程执行指定的命令。例如让所有的线程打印调用栈信息。

(gdb) thread apply all bt

Thread 3 (Thread 0x7ffff688a700 (LWP 20568)):
#0  0x00007ffff7138a3d in nanosleep () from /lib64/libc.so.6
#1  0x00007ffff71388b0 in sleep () from /lib64/libc.so.6
#2  0x000000000040091e in threadPrintWorld (arg=0x0) at multithreads.cpp:18
#3  0x00007ffff74279d1 in start_thread () from /lib64/libpthread.so.0
#4  0x00007ffff71748fd in clone () from /lib64/libc.so.6

Thread 2 (Thread 0x7ffff708b700 (LWP 20567)):
#0  threadPrintHello (arg=0x0) at multithreads.cpp:10
#1  0x00007ffff74279d1 in start_thread () from /lib64/libpthread.so.0
#2  0x00007ffff71748fd in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7ffff7fe5720 (LWP 20564)):
#0  0x00007ffff7138a3d in nanosleep () from /lib64/libc.so.6
#1  0x00007ffff71388b0 in sleep () from /lib64/libc.so.6
#2  0x00000000004009ea in main (argc=1, argv=0x7fffffffe628) at multithreads.cpp:47

gdb多进程调试

gdb进行多进程调试主要有以下几种方法，分别是follow-fork-mode 方法，attach 子进程方法。

示例程序

该示例程序中子进程会有除0异常

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/wait.h>
int wib(int no1, int no2)
{
    int result, diff;
    diff = no1 - no2;
    result = no1 / diff;
    return result;
}

int main()
{
    pid_t pid;

    pid = fork();
    if (pid < 0)
    {
        printf("fork err\n");
        exit(-1);
    }
    else if (pid == 0)
    {
        /* in child process */
        sleep(6);
        int value = 10;
        int div = 6;
        int total = 0;
        int i = 0;
        int result = 0;

        for (i = 0; i < 10; i++)
        {
            result = wib(value, div);
            total += result;
            div++;
            value--;
        }

        printf("%d wibed by %d equals %d\n", value, div, total);
        exit(0);
    }
    else
    {
        /* in parent process */
        sleep(4);
        waitpid(-1, NULL, 0);
        exit(0);
    }
}

follow-fork-mode

follow-fork-mode：set follow-fork-mode parent|child

parent：fork之后继续调试父进程，子进程不受影响。

child：fork之后调试子进程，父进程不受影响。

如果需要调试子进程，那么gdb启动之后，执行`(gdb) set follow-fork-mode child`并在子进程处设置断点。

detach-on-fork：set detach-on-fork on|off

on：断开调试follow-fork-mode指定的进程。

off：gdb将控制父进程和子进程。follow-fork-mode指定的进程将被调试，另一个进程置于暂停（suspended）状态。

inferior ID：切换到指定的inferior（gdb称任何执行中的进程为inferior）
info inferiors：列出当前被gdb调试的每个inferior信息
detach inferior ID：detach指定的inferior，允许其正常运行

follow-fork-mode调试多进程

设置断点，并且设置detach-on-fork为关闭来终止fork之后子进程的运行（默认情况下follow-fork-mode都是从父进程开始）

(gdb) b 18                 
Breakpoint 1 at 0x400740: file multiprocess.cpp, line 18.
(gdb) set detach-on-fork off
(gdb) run
Starting program: /data/home/chainyang/small_program/multiprocess 
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000
[New process 3689]
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.149.tl1.6.x86_64 libgcc-4.4.6-4.tl1.x86_64 libstdc++-4.4.6-4.tl1.x86_64

Breakpoint 1, main () at multiprocess.cpp:18
18          if (pid < 0)
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.149.tl1.6.x86_64 libgcc-4.4.6-4.tl1.x86_64 libstdc++-4.4.6-4.tl1.x86_64

当有一个新的子进程通过fork被创建出来，gdb会给出提示信息，如上面的[New process 3689]

通过info inferiors列出当前被gdb调试的进程，其中*表示当前所在的进程

(gdb) info inferiors 
  Num  Description       Executable        
  2    process 3689      /data/home/chainyang/small_program/multiprocess 
* 1    process 3686      /data/home/chainyang/small_program/multiprocess

进程相关的信息主要包括inferior ID（gdb用来标识的进程id），操作系统标识的进程id以及可执行程序的名字。

通过inferior ID切换到指定的进程运行

(gdb) b 31
Breakpoint 2 at 0x40079a: file multiprocess.cpp, line 31. (2 locations)
(gdb) inferior 2
[Switching to inferior 2 [process 6251] (/data/home/chainyang/small_program/multiprocess)]
[Switching to thread 2 (process 6251)] 
#0  0x00007ffff7355b1d in fork () from /lib64/libc.so.6
(gdb) c
Continuing.

Breakpoint 1, main () at multiprocess.cpp:18
warning: Source file is more recent than executable.
18          if (pid < 0)
(gdb) c
Continuing.

Breakpoint 2, main () at multiprocess.cpp:31
31              int result = 0;

示例程序在子进程中有算数异常（除0）

Program received signal SIGFPE, Arithmetic exception.
0x0000000000400725 in wib (no1=8, no2=8) at multiprocess.cpp:9
9           result = no1 / diff;

通过detach inferior ID来detach指定的进程，让它自由执行完

(gdb) detach inferior 2
Detaching from program: /data/home/chainyang/small_program/multiprocess, process 6948
(gdb) info inferiors 
  Num  Description       Executable        
* 2    <null>            /data/home/chainyang/small_program/multiprocess 
  1    process 6945      /data/home/chainyang/small_program/multiprocess

被detach的进程会显示null

此时切换到主进程中继续运行，由于子进程异常终止，父进程收到异常信号之后就退出程序。

(gdb) inferior 1
[Switching to inferior 1 [process 6945] (/data/home/chainyang/small_program/multiprocess)]
[Switching to thread 1 (process 6945)] 
#0  main () at multiprocess.cpp:18
18          if (pid < 0)
(gdb) c
Continuing.

Program exited normally.
Segmentation fault

attach进程

gdb可以通过attach对正在执行的程序进行调度，它允许开发人员中断程序并查看其状态，之后还能让这个程序正常地继续执行。

attach调试子进程

更改上述示例程序子进程睡眠时间为60秒，然后让子进程在后台运行，然后通过ps命令查询到子进程ID

[chainyang@DSNO_DP_PD_2 ~/small_program]$ ./multiprocess &
[1] 9448
[chainyang@DSNO_DP_PD_2 ~/small_program]$ ps aux | grep multiprocess
511       9448  0.0  0.0  11788   720 pts/4    S    11:38   0:00 ./multiprocess
511       9449  0.0  0.0  11788   136 pts/4    S    11:38   0:00 ./multiprocess
511       9451  0.0  0.0   6428   592 pts/4    S+   11:38   0:00 grep multiprocess

这里子进程id为9449

然后gdb attach到该子进程，通过stop中止当前的进程，并且设置断点

<http://www.gnu.org/software/gdb/bugs/>.
(gdb) attach 9449
Attaching to process 9449
Reading symbols from /data/home/chainyang/small_program/multiprocess...done.
Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libstdc++.so.6
Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgcc_s.so.1
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffd145be000
0x00007fb9d69f7a20 in __nanosleep_nocancel () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.149.tl1.6.x86_64 libgcc-4.4.6-4.tl1.x86_64 libstdc++-4.4.6-4.tl1.x86_64
(gdb) stop
(gdb) b 37
Breakpoint 1 at 0x4007b2: file multiprocess.cpp, line 37.

然后通过断点调试找到除0异常

(gdb) c
Continuing.

Program received signal SIGFPE, Arithmetic exception.
0x0000000000400725 in wib (no1=8, no2=8) at multiprocess.cpp:9
9           result = no1 / diff;
(gdb) p diff
$2 = 0