blocks|key|4761804|text|因此，大多数网络使用来自SLIM的batch_norm，默认情况下没有缩放/gamma参数。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4761805|scale:如果为真，则乘以gamma。如果为False，则不使用gamma。当下一层是线性的(也是nn.relu)时，这可以被禁用，因为缩放可以由下一层完成。|offset|length|style|CODE|4761806|https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/layers/python/layers/layers.py#L365-L386|4761807|entityMap|0|LINK|mutability|MUTABLE|url^0|0|E|5|X|5|1E|7|0|0|34|0|0^^$0|@$1|2|3|4|5|6|7|Q|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|R|8|@$D|S|E|T|F|G]|$D|U|E|V|F|G]|$D|W|E|X|F|G]]|9|@]|A|$]]|$1|H|3|I|5|6|7|Y|8|@]|9|@$D|Z|E|10|1|11]]|A|$]]|$1|J|3|-4|5|6|7|12|8|@]|9|@]|A|$]]]|K|$L|$5|M|N|O|A|$P|I]]]]

So most of the network uses the batch_norm from SLIM which by default without scaling/gamma parameter.

scale: If True, multiply by <code>gamma</code>. If False, <code>gamma</code> is
 not used. When the next layer is linear (also e.g. <code>nn.relu</code>), this can be
 disabled since the scaling can be done by the next layer.

<a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/layers/python/layers/layers.py#L365-L386" rel="nofollow noreferrer">https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/layers/python/layers/layers.py#L365-L386</a>

blocks|key|2377333|text|我也遇到了与slim库中预先训练好的inceptionV2相同的问题。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2377334|一开始我使用这个arg_scope，我遇到了这个问题：|2377335|def+_batch_norm_arg_scope(list_ops,
++++++++++++++++++++++++++use_batch_norm=True,
++++++++++++++++++++++++++batch_norm_decay=0.9997,
++++++++++++++++++++++++++batch_norm_epsilon=0.001,
++++++++++++++++++++++++++batch_norm_scale=False,
++++++++++++++++++++++++++train_batch_norm=False):
++++"""Slim+arg+scope+for+InceptionV2+batch+norm."""
++++if+use_batch_norm:
++++++++batch_norm_params+=+{
++++++++++++'is_training':+train_batch_norm,
++++++++++++'scale':+batch_norm_scale,
++++++++++++'decay':+batch_norm_decay,
++++++++++++'epsilon':+batch_norm_epsilon
++++++++}
++++++++normalizer_fn+=+slim.batch_norm
++++else:
++++++++normalizer_fn+=+None
++++++++batch_norm_params+=+None

++++return+slim.arg_scope(list_ops,
++++++++++++++++++++++++++normalizer_fn=normalizer_fn,
++++++++++++++++++++++++++normalizer_params=batch_norm_params)|code-block|syntax|javascript|2377336|我使用slim库中的arg_scope解决了这个问题。|2377337|with+slim.arg_scope(inception_v2.inception_v2_arg_scope()):|2377338|这很简单：|2377339|def+inception_arg_scope(weight_decay=0.00004,
++++++++++++++++++++++++use_batch_norm=True,
++++++++++++++++++++++++batch_norm_decay=0.9997,
++++++++++++++++++++++++batch_norm_epsilon=0.001,
++++++++++++++++++++++++activation_fn=tf.nn.relu):
++"""Defines+the+default+arg+scope+for+inception+models.

++Args:
++++weight_decay:+The+weight+decay+to+use+for+regularizing+the+model.
++++use_batch_norm:+"If+`True`,+batch_norm+is+applied+after+each+convolution.
++++batch_norm_decay:+Decay+for+batch+norm+moving+average.
++++batch_norm_epsilon:+Small+float+added+to+variance+to+avoid+dividing+by+zero
++++++in+batch+norm.
++++activation_fn:+Activation+function+for+conv2d.

++Returns:
++++An+`arg_scope`+to+use+for+the+inception+models.
++"""
++batch_norm_params+=+{
++++++#+Decay+for+the+moving+averages.
++++++'decay':+batch_norm_decay,
++++++#+epsilon+to+prevent+0s+in+variance.
++++++'epsilon':+batch_norm_epsilon,
++++++#+collection+containing+update_ops.
++++++'updates_collections':+tf.GraphKeys.UPDATE_OPS,
++++++#+use+fused+batch+norm+if+possible.
++++++'fused':+None,
++}
++if+use_batch_norm:
++++normalizer_fn+=+slim.batch_norm
++++normalizer_params+=+batch_norm_params
++else:
++++normalizer_fn+=+None
++++normalizer_params+=+{}
++#+Set+weight_decay+for+weights+in+Conv+and+FC+layers.
++with+slim.arg_scope([slim.conv2d,+slim.fully_connected],
++++++++++++++++++++++weights_regularizer=slim.l2_regularizer(weight_decay)):
++++with+slim.arg_scope(
++++++++[slim.conv2d],
++++++++weights_initializer=slim.variance_scaling_initializer(),
++++++++activation_fn=activation_fn,
++++++++normalizer_fn=normalizer_fn,
++++++++normalizer_params=normalizer_params)+as+sc:
++++++return+sc|2377340|entityMap^0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|S|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|T|8|@]|9|@]|A|$]]|$1|D|3|E|5|F|7|U|8|@]|9|@]|A|$G|H]]|$1|I|3|J|5|6|7|V|8|@]|9|@]|A|$]]|$1|K|3|L|5|F|7|W|8|@]|9|@]|A|$G|H]]|$1|M|3|N|5|6|7|X|8|@]|9|@]|A|$]]|$1|O|3|P|5|F|7|Y|8|@]|9|@]|A|$G|H]]|$1|Q|3|-4|5|6|7|Z|8|@]|9|@]|A|$]]]|R|$]]

I have the same problem with the pretrained inceptionV2 of the slim library. 

First I was using this arg_scope, i got that problem:

<pre><code>def _batch_norm_arg_scope(list_ops,
 use_batch_norm=True,
 batch_norm_decay=0.9997,
 batch_norm_epsilon=0.001,
 batch_norm_scale=False,
 train_batch_norm=False):
 """Slim arg scope for InceptionV2 batch norm."""
 if use_batch_norm:
 batch_norm_params = {
 'is_training': train_batch_norm,
 'scale': batch_norm_scale,
 'decay': batch_norm_decay,
 'epsilon': batch_norm_epsilon
 }
 normalizer_fn = slim.batch_norm
 else:
 normalizer_fn = None
 batch_norm_params = None

 return slim.arg_scope(list_ops,
 normalizer_fn=normalizer_fn,
 normalizer_params=batch_norm_params)
</code></pre>

I solved using the arg_scope in the slim library.

<pre><code>with slim.arg_scope(inception_v2.inception_v2_arg_scope()):
</code></pre>

that simply is:

<pre><code>def inception_arg_scope(weight_decay=0.00004,
 use_batch_norm=True,
 batch_norm_decay=0.9997,
 batch_norm_epsilon=0.001,
 activation_fn=tf.nn.relu):
 """Defines the default arg scope for inception models.

 Args:
 weight_decay: The weight decay to use for regularizing the model.
 use_batch_norm: "If `True`, batch_norm is applied after each convolution.
 batch_norm_decay: Decay for batch norm moving average.
 batch_norm_epsilon: Small float added to variance to avoid dividing by zero
 in batch norm.
 activation_fn: Activation function for conv2d.

 Returns:
 An `arg_scope` to use for the inception models.
 """
 batch_norm_params = {
 # Decay for the moving averages.
 'decay': batch_norm_decay,
 # epsilon to prevent 0s in variance.
 'epsilon': batch_norm_epsilon,
 # collection containing update_ops.
 'updates_collections': tf.GraphKeys.UPDATE_OPS,
 # use fused batch norm if possible.
 'fused': None,
 }
 if use_batch_norm:
 normalizer_fn = slim.batch_norm
 normalizer_params = batch_norm_params
 else:
 normalizer_fn = None
 normalizer_params = {}
 # Set weight_decay for weights in Conv and FC layers.
 with slim.arg_scope([slim.conv2d, slim.fully_connected],
 weights_regularizer=slim.l2_regularizer(weight_decay)):
 with slim.arg_scope(
 [slim.conv2d],
 weights_initializer=slim.variance_scaling_initializer(),
 activation_fn=activation_fn,
 normalizer_fn=normalizer_fn,
 normalizer_params=normalizer_params) as sc:
 return sc
</code></pre>

I am able to restore a model and extract parameters for weights, biases and batch_norm layers from checkpoint files.
However for multiple checkpoint files (inception models etc.), I cannot find the scaling/gamma factor of the BN layer. 

For example, in the public inceptionV3 checkpoint, I can locate:
<code>InceptionV3/Mixed_5d/Branch_2/Conv2d_0a_1x1/BatchNorm/moving_mean (DT_FLOAT) [64]</code>
<code>InceptionV3/Mixed_5d/Branch_2/Conv2d_0a_1x1/BatchNorm/moving_variance (DT_FLOAT) [64]</code>
<code>InceptionV3/Mixed_5d/Branch_2/Conv2d_0a_1x1/BatchNorm/beta (DT_FLOAT) [64]</code>

However, there is nothing such as <code>InceptionV3/Mixed_5d/Branch_2/Conv2d_0a_1x1/BatchNorm/gamma</code>.

How can I get the gamma value or it is rescaled to 1 by default?

Thanks a lot!

restoring Tensorflow model: cannot find gamma/scale for batch_norm layers in the checkpoint file

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

我能够恢复一个模型，并从检查点文件中提取权重、偏差和batch_norm层的参数。然而，对于多个检查点文件(初始模型等)，我找不到BN层的缩放/伽马因子。例如，在公共inceptionV3检查点中，我可以找到：InceptionV3/Mixed_5d/Branch_2/Conv2d_0a_1x1/BatchNorm/m...

问正在还原Tensorflow模型:在检查点文件中找不到batch_norm图层的gamma/scale
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问正在还原Tensorflow模型:在检查点文件中找不到batch_norm图层的gamma/scaleEN