torch.nn.init.kaiming_normal_(tensor, a=0, mode='fan_in', nonlinearity='leaky_relu')[source]
Fills the input Tensor with values according to the method described in Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification - He, K. et al. (2015), using a normal distribution. The resulting tensor will have values sampled from
where
Also known as He initialization.
Parameters
'fan_in'
(default) or 'fan_out'
. Choosing 'fan_in'
preserves the magnitude of the variance of the weights in the forward pass. Choosing 'fan_out'
preserves the magnitudes in the backwards pass.'relu'
or 'leaky_relu'
(default).Examples:
>>> w = torch.empty(3, 5)
>>> nn.init.kaiming_normal_(w, mode='fan_out', nonlinearity='relu')