我能够恢复一个模型,并从检查点文件中提取权重、偏差和batch_norm层的参数。然而,对于多个检查点文件(初始模型等),我找不到BN层的缩放/伽马因子。
例如,在公共inceptionV3检查点中,我可以找到:InceptionV3/Mixed_5d/Branch_2/Conv2d_0a_1x1/BatchNorm/moving_mean (DT_FLOAT) [64] InceptionV3/Mixed_5d/Branch_2/Conv2d_0a_1x1/BatchNorm/moving_variance (DT_FLOAT) [64] InceptionV3/Mixed_5d/Branch_2/Conv2d_0a_1x1/BatchNorm/beta (DT_FLOAT) [64]
然而,没有像InceptionV3/Mixed_5d/Branch_2/Conv2d_0a_1x1/BatchNorm/gamma这样的东西。
如何获取伽马值或默认情况下将其重新缩放为1?
非常感谢!
发布于 2017-05-07 13:33:47
因此,大多数网络使用来自SLIM的batch_norm,默认情况下没有缩放/gamma参数。
scale:如果为真,则乘以gamma。如果为False,则不使用gamma。当下一层是线性的(也是nn.relu)时,这可以被禁用,因为缩放可以由下一层完成。
发布于 2018-01-09 00:36:19
我也遇到了与slim库中预先训练好的inceptionV2相同的问题。
一开始我使用这个arg_scope,我遇到了这个问题:
def _batch_norm_arg_scope(list_ops,
use_batch_norm=True,
batch_norm_decay=0.9997,
batch_norm_epsilon=0.001,
batch_norm_scale=False,
train_batch_norm=False):
"""Slim arg scope for InceptionV2 batch norm."""
if use_batch_norm:
batch_norm_params = {
'is_training': train_batch_norm,
'scale': batch_norm_scale,
'decay': batch_norm_decay,
'epsilon': batch_norm_epsilon
}
normalizer_fn = slim.batch_norm
else:
normalizer_fn = None
batch_norm_params = None
return slim.arg_scope(list_ops,
normalizer_fn=normalizer_fn,
normalizer_params=batch_norm_params)我使用slim库中的arg_scope解决了这个问题。
with slim.arg_scope(inception_v2.inception_v2_arg_scope()):这很简单:
def inception_arg_scope(weight_decay=0.00004,
use_batch_norm=True,
batch_norm_decay=0.9997,
batch_norm_epsilon=0.001,
activation_fn=tf.nn.relu):
"""Defines the default arg scope for inception models.
Args:
weight_decay: The weight decay to use for regularizing the model.
use_batch_norm: "If `True`, batch_norm is applied after each convolution.
batch_norm_decay: Decay for batch norm moving average.
batch_norm_epsilon: Small float added to variance to avoid dividing by zero
in batch norm.
activation_fn: Activation function for conv2d.
Returns:
An `arg_scope` to use for the inception models.
"""
batch_norm_params = {
# Decay for the moving averages.
'decay': batch_norm_decay,
# epsilon to prevent 0s in variance.
'epsilon': batch_norm_epsilon,
# collection containing update_ops.
'updates_collections': tf.GraphKeys.UPDATE_OPS,
# use fused batch norm if possible.
'fused': None,
}
if use_batch_norm:
normalizer_fn = slim.batch_norm
normalizer_params = batch_norm_params
else:
normalizer_fn = None
normalizer_params = {}
# Set weight_decay for weights in Conv and FC layers.
with slim.arg_scope([slim.conv2d, slim.fully_connected],
weights_regularizer=slim.l2_regularizer(weight_decay)):
with slim.arg_scope(
[slim.conv2d],
weights_initializer=slim.variance_scaling_initializer(),
activation_fn=activation_fn,
normalizer_fn=normalizer_fn,
normalizer_params=normalizer_params) as sc:
return schttps://stackoverflow.com/questions/43813549
复制相似问题