专栏首页机器学习实践二三事What's the difference of name scope and a variable scope in tensorflow?

What's the difference of name scope and a variable scope in tensorflow?

Let’s begin by a short introduction to variable sharing. It is a mechanism in TensorFlow that allows for sharing variables accessed in different parts of the code without passing references to the variable around. The method tf.get_variable can be used with the name of the variable as argument to either create a new variable with such name or retrieve the one that was created before. This is different from using the tf.Variable constructor which will create a new variable every time it is called (and potentially add a suffix to the variable name if a variable with such name already exists). It is for the purpose of the variable sharing mechanism that a separate type of scope (variable scope) was introduced.

As a result, we end up having two different types of scopes:

name scope, created using tf.name_scope or tf.op_scope variable scope, created using tf.variable_scope or tf.variable_op_scope Both scopes have the same effect on all operations as well as variables created using tf.Variable, i.e. the scope will be added as a prefix to the operation or variable name.

1. tf.name_scope creates namespace for operators in the default graph. 2. tf.variable_scope creates namespace for both variables and operators in the default graph.

However, name scope is ignored by tf.get_variable. We can see that in the following example:

with tf.name_scope("my_scope"):
        v1 = tf.get_variable("var1", [1], dtype=tf.float32)
        v2 = tf.Variable(1, name="var2", dtype=tf.float32)
        a = tf.add(v1, v2)
print(v1.name)  # var1:0
print(v2.name)  # my_scope/var2:0
print(a.name)   # my_scope/Add:0

The only way to place a variable accessed using tf.get_variable in a scope is to use variable scope, as in the following example:

with tf.variable_scope("my_scope"):
    v1 = tf.get_variable("var1", [1], dtype=tf.float32)
    v2 = tf.Variable(1, name="var2", dtype=tf.float32)
    a = tf.add(v1, v2)

print(v1.name)  # my_scope/var1:0
print(v2.name)  # my_scope/var2:0
print(a.name)   # my_scope/Add:0

Finally, let’s look at the difference between the different methods for creating scopes. We can group them in two categories:

  • tf.name_scope(name) (for name scope) and tf.variable_scope(name_or_scope, …) (for variable scope) create a scope with the name specified as argument
  • tf.op_scope(values, name, default_name=None) (for name scope) and tf.variable_op_scope(values, name_or_scope, default_name=None, …) (for variable scope) create a scope, just like the functions above, but besides the scope name, they accept an argument default_name which is used instead of name when it is set to None. Moreover, they accept a list of tensors (values) in order to check if all the tensors are from the same, default graph. This is useful when creating new operations, for example, see the implementation of tf.histogram_summary.

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

我来说两句

0 条评论
登录 后参与评论

相关文章

  • gcc命令

    在Linux底下搞开发,不可避免的要使用到gcc,gcc选项众多,下面记录下常见的一些选项,网上好多博客也说这个但是很多的都是不对的,我的博客记录参见man g...

    GavinZhou
  • Out of bag error in Random Forest

    sklearn中的RandomForestClassifier有一个参数: oob_score : bool (default=False) Whethe...

    GavinZhou
  • Python-OpenCV(4)

    我想把 NBA 的标志放到另一幅图像(LBJ传球)上。但是现在有个问题: 如果我使用加法,颜色会改变,如果使用混合,会得到透明效果,但是我不想要透明的 ...

    GavinZhou
  • task8 GAN text-to-image

    https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization https:...

    平凡的学生族
  • tensorflow学习笔记(十七):name&variable scope

    在tensorflow中,有两个scope, 一个是name_scope一个是variable_scope,这两个scope到底有什么区别呢? 先看第一个程序...

    ke1th
  • TensorFlow 使用变量共享

    TensorFlow中的变量一般就是模型的参数。当模型复杂的时候共享变量会无比复杂。

    郭耀华
  • 【tensorflow】tf.name_scope与tf.variable_scope区别

    但是,tf.Variable() 每次都会新建变量。 如果希望重用(共享)一些变量,必须用到get_variable(),它会去搜索变量名,有就直接用,没有再...

    JNingWei
  • tensorflow学习笔记(十五): variable scope

    tensorflow 为了更好的管理变量,提供了variable scope机制 官方解释: Variable scope object to carr...

    ke1th
  • 初识Hibernate之关联映射(二)

         上篇我们介绍了关联映射的几种形式,有单向多对一,单向一对多,还有双向一对多。本篇接着介绍有关关联映射的其他几种映射方式,主要有以下几种: 基于外键的单...

    Single
  • 技术无关 自己写的项目 一些数据库的想法

    +----------+--------------+------+-----+---------+----------------+ | Field    |...

    ydymz

扫码关注云+社区

领取腾讯云代金券