很早之前我有写过有关 defer 的博客,现在看来起标题的时候有点蠢,有点标题党,(https://cloud.tencent.com/developer/article/2090789) 其中主要是注重与 defer 的使用,避免使用上的问题,对于 defer 具体实现其实只是点了一下,而今天就让我们详细看看 defer 究竟是如何实现的。
- defer 的基本使用规则
- 逃逸分析:[https://www.linkinstars.com/post/1ceb1a77.html](https://www.linkinstars.com/post/1ceb1a77.html)
- 函数调用规约:[https://www.linkinstars.com/post/fecd400.html](https://www.linkinstars.com/post/fecd400.html)
因为在 1.14 之后 defer 是有优化过的(https://golang.org/doc/go1.14#runtime),所以需要注意,本文使用的 go 版本是 1.17
首先我们来看看 defer 究竟是长什么样子
// A _defer holds an entry on the list of deferred calls.
// If you add a field here, add code to clear it in freedefer and deferProcStack
// This struct must match the code in cmd/compile/internal/reflectdata/reflect.go:deferstruct
// and cmd/compile/internal/gc/ssa.go:(*state).call.
// Some defers will be allocated on the stack and some on the heap.
// All defers are logically part of the stack, so write barriers to
// initialize them are not required. All defers must be manually scanned,
// and for heap defers, marked.
type _defer struct {
siz int32 // includes both arguments and results
started bool
heap bool
// openDefer indicates that this _defer is for a frame with open-coded
// defers. We have only one defer record for the entire frame (which may
// currently have 0, 1, or more defers active).
openDefer bool
sp uintptr // sp at time of defer
pc uintptr // pc at time of defer
fn *funcval // can be nil for open-coded defers
_panic *_panic // panic that is running defer
link *_defer
// If openDefer is true, the fields below record values about the stack
// frame and associated function that has the open-coded defer(s). sp
// above will be the sp for the frame, and pc will be address of the
// deferreturn call in the function.
fd unsafe.Pointer // funcdata for the function associated with the frame
varp uintptr // value of varp for the stack frame
// framepc is the current pc associated with the stack frame. Together,
// with sp above (which is the sp associated with the stack frame),
// framepc/sp can be used as pc/sp pair to continue a stack trace via
// gentraceback().
framepc uintptr
}
defer 结构本身不复杂,其中的字段看名字就很好理解,这里说几个要点,后面会用到的
heap
专门用于标识是分配在栈上还是堆上siz
字段后面的注释说的很清楚,这里表示的 size 是包含参数和结果的,也就是说参数和返回值的空间一开始就分配好了link
字段就是串联了整个 defer 链表,因为我们的 defer 是可以使用多次的,而 defer 的顺序就是通过这个字段串起来的现在我们知道了 defer 的样子,就来看看它编译之后是怎么样实现的吧
package main
func main() {
println(a())
}
func a() int {
x := 123
defer defFun(x)
return x
}
func defFun(a int) {
a++
}
go tool compile -S -N -l main.go>> main.md
main 函数就是直接调用了 a 函数,故 main 里面的我们就不看了,直接看 a 函数里面究竟做了什么事情
"".a STEXT size=213 args=0x0 locals=0x90 funcid=0x0
......
0x0026 00038 (main.go:8) MOVQ $0, "".~r0+8(SP)
0x002f 00047 (main.go:9) MOVQ $123, "".x+16(SP)
0x0038 00056 (main.go:10) MOVQ $123, ""..autotmp_2+24(SP)
0x0041 00065 (main.go:10) MOVUPS X15, ""..autotmp_4+32(SP)
0x0047 00071 (main.go:10) LEAQ ""..autotmp_4+32(SP), CX
0x004c 00076 (main.go:10) MOVQ CX, ""..autotmp_3+128(SP)
0x0054 00084 (main.go:10) TESTB AL, (CX)
0x0056 00086 (main.go:10) LEAQ "".a·dwrap·1(SB), DX
0x005d 00093 (main.go:10) MOVQ DX, ""..autotmp_4+32(SP)
0x0062 00098 (main.go:10) TESTB AL, (CX)
0x0064 00100 (main.go:10) MOVQ ""..autotmp_2+24(SP), DX
0x0069 00105 (main.go:10) MOVQ DX, ""..autotmp_4+40(SP)
0x006e 00110 (main.go:10) MOVL $0, ""..autotmp_5+48(SP)
0x0076 00118 (main.go:10) MOVQ CX, ""..autotmp_5+72(SP)
0x007b 00123 (main.go:10) LEAQ ""..autotmp_5+48(SP), AX
0x0080 00128 (main.go:10) PCDATA $1, $0
0x0080 00128 (main.go:10) CALL runtime.deferprocStack(SB) // 这里调用了 deferprocStack 方法
0x0085 00133 (main.go:10) TESTL AX, AX
0x0087 00135 (main.go:10) JNE 176
0x0089 00137 (main.go:10) JMP 139
0x008b 00139 (main.go:11) MOVQ "".x+16(SP), AX
0x0090 00144 (main.go:11) MOVQ AX, "".~r0+8(SP)
0x0095 00149 (main.go:11) XCHGL AX, AX
0x0096 00150 (main.go:11) CALL runtime.deferreturn(SB) // 这里调用了 deferreturn 方法
0x009b 00155 (main.go:11) MOVQ "".~r0+8(SP), AX
0x00a0 00160 (main.go:11) MOVQ 136(SP), BP
0x00a8 00168 (main.go:11) ADDQ $144, SP
0x00af 00175 (main.go:11) RET
0x00b0 00176 (main.go:10) XCHGL AX, AX
0x00b1 00177 (main.go:10) CALL runtime.deferreturn(SB) // 这里调用了 deferreturn 方法
0x00b6 00182 (main.go:10) MOVQ "".~r0+8(SP), AX
0x00bb 00187 (main.go:10) MOVQ 136(SP), BP
0x00c3 00195 (main.go:10) ADDQ $144, SP
0x00ca 00202 (main.go:10) RET
0x00cb 00203 (main.go:10) NOP
0x00cb 00203 (main.go:8) PCDATA $1, $-1
0x00cb 00203 (main.go:8) PCDATA $0, $-2
0x00cb 00203 (main.go:8) CALL runtime.morestack_noctxt(SB)
0x00d0 00208 (main.go:8) PCDATA $0, $-1
0x00d0 00208 (main.go:8) JMP 0
从这里我们可以看到和 defer 有关的两个方法是:
那么我们写在 defer 里面的方法去哪里了呢?其实被 dwarp 包住了,让我们来看看这个生成的 dwarp 方法里面做了什么
"".a·dwrap·1 STEXT size=77 args=0x0 locals=0x18 funcid=0x16
..................
0x001d 00029 (main.go:10) MOVQ 8(DX), AX
0x0021 00033 (main.go:10) MOVQ AX, ""..autotmp_2+8(SP)
0x0026 00038 (main.go:10) PCDATA $1, $0
0x0026 00038 (main.go:10) CALL "".defFun(SB) // 这里就是我们写的 defFun 方法,原来在这里才调用
0x002b 00043 (main.go:10) MOVQ 16(SP), BP
0x0030 00048 (main.go:10) ADDQ $24, SP
0x0034 00052 (main.go:10) RET
0x0035 00053 (main.go:10) NOP
0x0035 00053 (main.go:10) PCDATA $1, $-1
0x0035 00053 (main.go:10) PCDATA $0, $-2
0x0035 00053 (main.go:10) CALL runtime.morestack(SB)
......................
最后看看 defFun 可以看到基本没有什么特别的,就是一个普通的函数
"".defFun STEXT nosplit size=14 args=0x8 locals=0x0 funcid=0x0
0x0000 00000 (main.go:14) TEXT "".defFun(SB), NOSPLIT|ABIInternal, $0-8
...............
0x0000 00000 (main.go:14) MOVQ AX, "".a+8(SP)
0x0005 00005 (main.go:15) INCQ AX
0x0008 00008 (main.go:15) MOVQ AX, "".a+8(SP)
0x000d 00013 (main.go:16) RET
然后让我们看看 deferprocStack
方法究竟在做什么
func deferprocStack(d *_defer) {
gp := getg()
if gp.m.curg != gp {
// go code on the system stack can't defer
throw("defer on system stack")
}
if goexperiment.RegabiDefer && d.siz != 0 {
throw("defer with non-empty frame")
}
// siz and fn are already set.
// The other fields are junk on entry to deferprocStack and
// are initialized here.
d.started = false
d.heap = false
d.openDefer = false
d.sp = getcallersp()
d.pc = getcallerpc()
d.framepc = 0
d.varp = 0
// The lines below implement:
// d.panic = nil
// d.fd = nil
// d.link = gp._defer
// gp._defer = d
// But without write barriers. The first three are writes to
// the stack so they don't need a write barrier, and furthermore
// are to uninitialized memory, so they must not use a write barrier.
// The fourth write does not require a write barrier because we
// explicitly mark all the defer structures, so we don't need to
// keep track of pointers to them with a write barrier.
*(*uintptr)(unsafe.Pointer(&d._panic)) = 0
*(*uintptr)(unsafe.Pointer(&d.fd)) = 0
// 这里就是 defer 串联的地方,首先将自己的 link 字段赋值为当前 g 的 _defer,然后将当前 g 的 _defer 赋值为自己
// 假设当前为 g.defer = a ; a.link = b; 现在的是 c
// 则 c.link = a ; g.defer = c ; a.link = b; 从而串起来了。
// 其实这也就是为什么 defer 是先进后出的原因
*(*uintptr)(unsafe.Pointer(&d.link)) = uintptr(unsafe.Pointer(gp._defer))
*(*uintptr)(unsafe.Pointer(&gp._defer)) = uintptr(unsafe.Pointer(d))
return0()
// No code can go here - the C return register has
// been set and must not be clobbered.
}
其实主要是一个初始化的操作,要点:
func deferreturn() {
gp := getg()
d := gp._defer
// 如果当前没有 defer 了,直接返回
if d == nil {
return
}
// 获取 sp 进行比较
sp := getcallersp()
// 如果与当前的 sp 不一致,那么证明这个 defer 的调用函数并不是这个,直接返回
if d.sp != sp {
return
}
// openDefer 的问题后面我们再详细说
if d.openDefer {
done := runOpenDeferFrame(gp, d)
if !done {
throw("unfinished open-coded defers in deferreturn")
}
gp._defer = d.link
freedefer(d)
return
}
// 获取执行 fn 所需要的参数
// Moving arguments around.
//
// Everything called after this point must be recursively
// nosplit because the garbage collector won't know the form
// of the arguments until the jmpdefer can flip the PC over to
// fn.
argp := getcallersp() + sys.MinFrameSize
switch d.siz {
case 0:
// Do nothing.
case sys.PtrSize:
*(*uintptr)(unsafe.Pointer(argp)) = *(*uintptr)(deferArgs(d))
default:
memmove(unsafe.Pointer(argp), deferArgs(d), uintptr(d.siz))
}
fn := d.fn
d.fn = nil
// 相当于出栈了一个 defer
gp._defer = d.link
// 释放对应的空间
freedefer(d)
// If the defer function pointer is nil, force the seg fault to happen
// here rather than in jmpdefer. gentraceback() throws an error if it is
// called with a callback on an LR architecture and jmpdefer is on the
// stack, because the stack trace can be incorrect in that case - see
// issue #8153).
_ = fn.fn
// 执行 defer 需要执行的函数
jmpdefer(fn, argp)
}
最重要的就是最后一个方法 jmpdefer
TEXT runtime·jmpdefer(SB), NOSPLIT, $0-8
MOVL fv+0(FP), DX // fn
MOVL argp+4(FP), BX // caller sp
LEAL -4(BX), SP // caller sp after CALL
#ifdef GOBUILDMODE_shared
SUBL $16, (SP) // return to CALL again
#else
SUBL $5, (SP) // return to CALL again
#endif
MOVL 0(DX), BX
JMP BX // but first run the deferred function
runtime.jmpdefer
是一个用汇编语言实现的运行时函数,它的主要工作是跳转到 defer
所在的代码段并在执行结束之后跳转回 runtime.deferreturn
这就解释了为什么在运行完成了一个 defer 里面的函数之后能运行下一个,因为又回去运行了 deferreturn
方法
让我们先来小结一下到目前为止 defer 的实现:
这样看完之后能解释两个问题,一个问题是 defer 执行之前入参就已经被确认了,另一个问题是 defer 为什么能串起来最终倒序执行
但是 go 之所以在 1.14 优化也是有原因的,因为不是所有的 defer 都能使用栈,有的只能分配到堆上,当然还有更加厉害的优化,我们继续往下看
我们修改原来的 demo 函数
package main
func main() {
println(a())
}
func a() int {
x := 123
for i := 0; i < 2; i++ {
defer defFun(x)
}
return x
}
func defFun(a int) {
a++
}
我们使用一个 for 嵌套 defer (注意!在实际业务代码中不要这样写,会导致资源泄露)
go tool compile -S -N -l main.go>> main.md
"".a STEXT size=222 args=0x0 locals=0x40 funcid=0x0
.............
0x0018 00024 (main.go:8) MOVQ $0, "".~r0+16(SP)
0x0021 00033 (main.go:9) MOVQ $123, "".x+24(SP)
0x002a 00042 (main.go:10) MOVQ $0, "".i+32(SP)
0x0033 00051 (main.go:10) JMP 53
0x0035 00053 (main.go:10) CMPQ "".i+32(SP), $2
0x003b 00059 (main.go:10) JLT 63
0x003d 00061 (main.go:10) JMP 180
0x003f 00063 (main.go:11) MOVQ "".x+24(SP), CX
0x0044 00068 (main.go:11) MOVQ CX, ""..autotmp_3+40(SP)
0x0049 00073 (main.go:11) LEAQ type.noalg.struct { F uintptr; ""..autotmp_3 int }(SB), AX
0x0050 00080 (main.go:11) PCDATA $1, $0
0x0050 00080 (main.go:11) CALL runtime.newobject(SB)
0x0055 00085 (main.go:11) MOVQ AX, ""..autotmp_4+48(SP)
0x005a 00090 (main.go:11) LEAQ "".a·dwrap·1(SB), CX
0x0061 00097 (main.go:11) MOVQ CX, (AX)
0x0064 00100 (main.go:11) MOVQ ""..autotmp_4+48(SP), CX
0x0069 00105 (main.go:11) TESTB AL, (CX)
0x006b 00107 (main.go:11) MOVQ ""..autotmp_3+40(SP), DX
0x0070 00112 (main.go:11) MOVQ DX, 8(CX)
0x0074 00116 (main.go:11) MOVQ ""..autotmp_4+48(SP), BX
0x0079 00121 (main.go:11) XORL AX, AX
0x007b 00123 (main.go:11) NOP
0x0080 00128 (main.go:11) CALL runtime.deferproc(SB) // 这里调用的是 deferproc 方法
0x0085 00133 (main.go:11) TESTL AX, AX
0x0087 00135 (main.go:11) JNE 156
0x0089 00137 (main.go:11) JMP 139
0x008b 00139 (main.go:10) PCDATA $1, $-1
0x008b 00139 (main.go:10) JMP 141
0x008d 00141 (main.go:10) MOVQ "".i+32(SP), CX
0x0092 00146 (main.go:10) INCQ CX
0x0095 00149 (main.go:10) MOVQ CX, "".i+32(SP)
0x009a 00154 (main.go:10) JMP 53
0x009c 00156 (main.go:11) PCDATA $1, $0
0x009c 00156 (main.go:11) XCHGL AX, AX
0x009d 00157 (main.go:11) NOP
0x00a0 00160 (main.go:11) CALL runtime.deferreturn(SB) // 这里调用的还是 deferreturn 方法
0x00a5 00165 (main.go:11) MOVQ "".~r0+16(SP), AX
0x00aa 00170 (main.go:11) MOVQ 56(SP), BP
0x00af 00175 (main.go:11) ADDQ $64, SP
0x00b3 00179 (main.go:11) RET
0x00b4 00180 (main.go:13) MOVQ "".x+24(SP), AX
0x00b9 00185 (main.go:13) MOVQ AX, "".~r0+16(SP)
0x00be 00190 (main.go:13) XCHGL AX, AX
0x00bf 00191 (main.go:13) NOP
0x00c0 00192 (main.go:13) CALL runtime.deferreturn(SB) // 这里调用的还是 deferreturn 方法
0x00c5 00197 (main.go:13) MOVQ "".~r0+16(SP), AX
0x00ca 00202 (main.go:13) MOVQ 56(SP), BP
0x00cf 00207 (main.go:13) ADDQ $64, SP
0x00d3 00211 (main.go:13) RET
0x00d4 00212 (main.go:13) NOP
0x00d4 00212 (main.go:8) PCDATA $1, $-1
0x00d4 00212 (main.go:8) PCDATA $0, $-2
0x00d4 00212 (main.go:8) CALL runtime.morestack_noctxt(SB)
0x00d9 00217 (main.go:8) PCDATA $0, $-1
0x00d9 00217 (main.go:8) JMP 0
我们发现了这里使用了 deferproc 方法
func deferproc(siz int32, fn *funcval) { // arguments of fn follow fn
gp := getg()
if gp.m.curg != gp {
// go code on the system stack can't defer
throw("defer on system stack")
}
if goexperiment.RegabiDefer && siz != 0 {
// TODO: Make deferproc just take a func().
throw("defer with non-empty frame")
}
// the arguments of fn are in a perilous state. The stack map
// for deferproc does not describe them. So we can't let garbage
// collection or stack copying trigger until we've copied them out
// to somewhere safe. The memmove below does that.
// Until the copy completes, we can only call nosplit routines.
sp := getcallersp()
argp := uintptr(unsafe.Pointer(&fn)) + unsafe.Sizeof(fn)
callerpc := getcallerpc()
// 这里分配了 defer 在堆上
d := newdefer(siz)
if d._panic != nil {
throw("deferproc: d.panic != nil after newdefer")
}
d.link = gp._defer
gp._defer = d
d.fn = fn
d.pc = callerpc
d.sp = sp
switch siz {
case 0:
// Do nothing.
case sys.PtrSize:
*(*uintptr)(deferArgs(d)) = *(*uintptr)(unsafe.Pointer(argp))
default:
memmove(deferArgs(d), unsafe.Pointer(argp), uintptr(siz))
}
// deferproc returns 0 normally.
// a deferred func that stops a panic
// makes the deferproc return 1.
// the code the compiler generates always
// checks the return value and jumps to the
// end of the function if deferproc returns != 0.
return0()
// No code can go here - the C return register has
// been set and must not be clobbered.
}
这里基本和 deferprocStack 一样,只是分配的时候到了堆上
func newdefer(siz int32) *_defer {
var d *_defer
sc := deferclass(uintptr(siz))
gp := getg()
if sc < uintptr(len(p{}.deferpool)) {
pp := gp.m.p.ptr()
if len(pp.deferpool[sc]) == 0 && sched.deferpool[sc] != nil {
// Take the slow path on the system stack so
// we don't grow newdefer's stack.
systemstack(func() {
lock(&sched.deferlock)
for len(pp.deferpool[sc]) < cap(pp.deferpool[sc])/2 && sched.deferpool[sc] != nil {
d := sched.deferpool[sc]
sched.deferpool[sc] = d.link
d.link = nil
pp.deferpool[sc] = append(pp.deferpool[sc], d)
}
unlock(&sched.deferlock)
})
}
if n := len(pp.deferpool[sc]); n > 0 {
d = pp.deferpool[sc][n-1]
pp.deferpool[sc][n-1] = nil
pp.deferpool[sc] = pp.deferpool[sc][:n-1]
}
}
if d == nil {
// Allocate new defer+args.
systemstack(func() {
total := roundupsize(totaldefersize(uintptr(siz)))
d = (*_defer)(mallocgc(total, deferType, true))
})
}
d.siz = siz
d.heap = true
return d
}
这里的 newdefer 看起来很长,其实关键就是先去 deferpool 中拿,看看能不能拿到,它是调度器的延迟调用缓存池,如果拿不到就只能 Allocate 了
然后 deferreturn 和之前还是一样的
原函数还是使用最一开始的,而这次我们将优化打开使用命令 go tool compile -S main.go>> main.md
再来看下
"".a STEXT size=165 args=0x0 locals=0x30 funcid=0x0
.............
0x0024 00036 (main.go:8) FUNCDATA $4, "".a.opendefer(SB) // 这里调用的是 opendefer 方法
0x0024 00036 (main.go:8) MOVB $0, ""..autotmp_5+7(SP)
0x0029 00041 (main.go:8) MOVQ $0, "".~r0+8(SP)
0x0032 00050 (main.go:10) MOVUPS X15, ""..autotmp_4+16(SP)
0x0038 00056 (main.go:10) LEAQ "".a·dwrap·1(SB), AX
0x003f 00063 (main.go:10) MOVQ AX, ""..autotmp_4+16(SP)
0x0044 00068 (main.go:10) MOVQ $123, ""..autotmp_4+24(SP)
0x004d 00077 (main.go:10) LEAQ ""..autotmp_4+16(SP), AX
0x0052 00082 (main.go:10) MOVQ AX, ""..autotmp_6+32(SP)
0x0057 00087 (main.go:10) MOVB $1, ""..autotmp_5+7(SP)
0x005c 00092 (main.go:11) MOVQ $123, "".~r0+8(SP)
0x0065 00101 (main.go:11) MOVB $0, ""..autotmp_5+7(SP)
0x006a 00106 (main.go:11) MOVQ ""..autotmp_6+32(SP), DX
0x006f 00111 (main.go:11) MOVQ (DX), AX
0x0072 00114 (main.go:11) PCDATA $1, $1
0x0072 00114 (main.go:11) CALL AX
0x0074 00116 (main.go:11) MOVQ "".~r0+8(SP), AX
0x0079 00121 (main.go:11) MOVQ 40(SP), BP
0x007e 00126 (main.go:11) ADDQ $48, SP
0x0082 00130 (main.go:11) RET
0x0083 00131 (main.go:11) CALL runtime.deferreturn(SB) // 这里还是 deferreturn 方法
....
if d.openDefer {
done := runOpenDeferFrame(gp, d)
if !done {
throw("unfinished open-coded defers in deferreturn")
}
gp._defer = d.link
freedefer(d)
return
}
还记得我们之前在 deferreturn 方法中过掉的细节吗?没错就是它,当使用功能 opendefer 的时候是不会触发 jmpdefer 的,而是使用了 runOpenDeferFrame 方法
由于 openDefer 比较复杂,细节很多,而且是属于编译期间的优化我就简单想着总结一下:
简单的说 open-coded 模式下把被延迟的方法和 deferreturn 直接插入到函数尾部
通过 8 个比特位去标识 defer 是否需要被执行,所以如果需要优化为 opendefer 的话 defer 的数量不能超过 8 个(当然还有比的条件)
详细内容可参考:https://draveness.me/golang/docs/part2-foundation/ch05-keyword/golang-defer/
我一开始是认为 defer 就是放到一个类似栈的数据结构里面了,然后运行完成函数之后就依次出栈执行,没想到其实 defer 一共有三种模式
不同的模式就是为了优化 defer 的性能,没想到一个小小的 defer 就有那么大大的学问哦,那么作为平常使用的时候我们能从今天学到什么呢?
一个是不要在循环中嵌套 defer,一个是注意 defer 使用的时候已经确定了传入的参数(这里要注意,虽然是值传递,但是如果值是地址,地址对应的数据发生改变,自然也就改变了)
最后给出一些参考链接,供你继续深入寻找 defer 的答案
https://blog.csdn.net/love666666shen/article/details/113845493
http://xiaorui.cc/archives/6579
https://my.oschina.net/u/5011810/blog/4968645
https://draveness.me/golang/docs/part2-foundation/ch05-keyword/golang-defer/