blocks|key|838770|text|正如您在第二点中所说的，使用regularizer参数是推荐的方法。您可以在get_variable中使用它，也可以在variable_scope中设置一次，并将所有变量规范化。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|838771|损失收集在图表中，您需要手动将它们添加到成本函数中，如下所示。|838772|++reg_losses+=+tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)
++reg_constant+=+0.01++#+Choose+an+appropriate+one.
++loss+=+my_normal_loss+%2B+reg_constant+*+sum(reg_losses)|code-block|syntax|javascript|838773|希望这能帮上忙！|838774|entityMap^0|E|B|12|C|1N|E|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Q|8|@$9|R|A|S|B|C]|$9|T|A|U|B|C]|$9|V|A|W|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|X|8|@]|D|@]|E|$]]|$1|H|3|I|5|J|7|Y|8|@]|D|@]|E|$K|L]]|$1|M|3|N|5|6|7|Z|8|@]|D|@]|E|$]]|$1|O|3|-4|5|6|7|10|8|@]|D|@]|E|$]]]|P|$]]

As you say in the second point, using the <code>regularizer</code> argument is the recommended way. You can use it in <code>get_variable</code>, or set it once in your <code>variable_scope</code> and have all your variables regularized.

The losses are collected in the graph, and you need to manually add them to your cost function like this.

<pre><code> reg_losses = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)
 reg_constant = 0.01 # Choose an appropriate one.
 loss = my_normal_loss + reg_constant * sum(reg_losses)
</code></pre>

Hope that helps!

blocks|key|838786|text|对我来说，现有答案的几个方面还不清楚，下面是一个逐步的指南：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|838787|定义一个正则化者。这就是可以设置正则化常数的地方，例如：
正则化=+tf.contrib.layers.l2_regularizer(scale=0.1)|ordered-list-item|838788|通过以下方式创建变量：
权重=+tf.get_variable(+name=“权重”，regularizer=regularizer，.)
同样，可以通过常规的weights+=+tf.Variable(...)构造函数创建变量，然后使用tf.add_to_collection(tf.GraphKeys.REGULARIZATION_LOSSES,+weights)创建变量。|offset|length|style|CODE|838789|定义一些loss术语并添加正则化术语：
(+tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)+reg_term+=+tf.contrib.layers.apply_regularization(regularizer，reg_variables)损失%2B=+reg_term
注意:看起来tf.contrib.layers.apply_regularization是作为一个AddN实现的，因此或多或少相当于sum(reg_variables)。|838790|entityMap^0|0|0|28|Q|3B|1T|0|4|4|4S|12|5Z|4|6G|I|0^^$0|@$1|2|3|4|5|6|7|O|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|P|8|@]|9|@]|A|$]]|$1|E|3|F|5|D|7|Q|8|@$G|R|H|S|I|J]|$G|T|H|U|I|J]]|9|@]|A|$]]|$1|K|3|L|5|D|7|V|8|@$G|W|H|X|I|J]|$G|Y|H|Z|I|J]|$G|10|H|11|I|J]|$G|12|H|13|I|J]]|9|@]|A|$]]|$1|M|3|-4|5|6|7|14|8|@]|9|@]|A|$]]]|N|$]]

A few aspects of the existing answer were not immediately clear to me, so here is a step-by-step guide:

<ol>
<li>Define a regularizer. This is where the regularization constant can be set, e.g.:

<pre><code>regularizer = tf.contrib.layers.l2_regularizer(scale=0.1)
</code></pre></li>
<li>Create variables via:

<pre><code> weights = tf.get_variable(
 name="weights",
 regularizer=regularizer,
 ...
 )
</code></pre>

Equivalently, variables can be created via the regular <code>weights = tf.Variable(...)</code> constructor, followed by <code>tf.add_to_collection(tf.GraphKeys.REGULARIZATION_LOSSES, weights)</code>.</li>
<li>Define some <code>loss</code> term and add the regularization term:

<pre><code>reg_variables = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)
reg_term = tf.contrib.layers.apply_regularization(regularizer, reg_variables)
loss += reg_term
</code></pre>

Note: It looks like <code>tf.contrib.layers.apply_regularization</code> is implemented as an <code>AddN</code>, so more or less equivalent to <code>sum(reg_variables)</code>.</li>
</ol>

blocks|key|868004|text|使用contrib.learn库的另一个选项是基于Tensorflow网站上的深度MNIST教程，如下所示。首先，假设您导入了相关的库(如import+tensorflow.contrib.layers+as+layers)，您可以在一个单独的方法中定义一个网络：|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|868005|def+easier_network(x,+reg):
++++"""+A+network+based+on+tf.contrib.learn,+with+input+`x`.+"""
++++with+tf.variable_scope('EasyNet'):
++++++++out+=+layers.flatten(x)
++++++++out+=+layers.fully_connected(out,+
++++++++++++++++num_outputs=200,
++++++++++++++++weights_initializer+=+layers.xavier_initializer(uniform=True),
++++++++++++++++weights_regularizer+=+layers.l2_regularizer(scale=reg),
++++++++++++++++activation_fn+=+tf.nn.tanh)
++++++++out+=+layers.fully_connected(out,+
++++++++++++++++num_outputs=200,
++++++++++++++++weights_initializer+=+layers.xavier_initializer(uniform=True),
++++++++++++++++weights_regularizer+=+layers.l2_regularizer(scale=reg),
++++++++++++++++activation_fn+=+tf.nn.tanh)
++++++++out+=+layers.fully_connected(out,+
++++++++++++++++num_outputs=10,+#+Because+there+are+ten+digits!
++++++++++++++++weights_initializer+=+layers.xavier_initializer(uniform=True),
++++++++++++++++weights_regularizer+=+layers.l2_regularizer(scale=reg),
++++++++++++++++activation_fn+=+None)
++++++++return+out+|code-block|syntax|javascript|868006|然后，在main方法中，可以使用以下代码片段：|868007|def+main(_):
++++mnist+=+input_data.read_data_sets(FLAGS.data_dir,+one_hot=True)
++++x+=+tf.placeholder(tf.float32,+[None,+784])
++++y_+=+tf.placeholder(tf.float32,+[None,+10])

++++#+Make+a+network+with+regularization
++++y_conv+=+easier_network(x,+FLAGS.regu)
++++weights+=+tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,+'EasyNet')+
++++print("")
++++for+w+in+weights:
++++++++shp+=+w.get_shape().as_list()
++++++++print("-+{}+shape:{}+size:{}".format(w.name,+shp,+np.prod(shp)))
++++print("")
++++reg_ws+=+tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES,+'EasyNet')
++++for+w+in+reg_ws:
++++++++shp+=+w.get_shape().as_list()
++++++++print("-+{}+shape:{}+size:{}".format(w.name,+shp,+np.prod(shp)))
++++print("")

++++#+Make+the+loss+function+`loss_fn`+with+regularization.
++++cross_entropy+=+tf.reduce_mean(
++++++++tf.nn.softmax_cross_entropy_with_logits(labels=y_,+logits=y_conv))
++++loss_fn+=+cross_entropy+%2B+tf.reduce_sum(reg_ws)
++++train_step+=+tf.train.AdamOptimizer(1e-4).minimize(loss_fn)|868008|要想让它正常工作，您需要遵循我之前链接到的MNIST教程并导入相关的库，但是学习TensorFlow是一个很好的练习，很容易看出正则化是如何影响输出的。如果将正则化作为参数应用，可以看到以下内容：|868009|-+EasyNet/fully_connected/weights:0+shape:[784,+200]+size:156800
-+EasyNet/fully_connected/biases:0+shape:[200]+size:200
-+EasyNet/fully_connected_1/weights:0+shape:[200,+200]+size:40000
-+EasyNet/fully_connected_1/biases:0+shape:[200]+size:200
-+EasyNet/fully_connected_2/weights:0+shape:[200,+10]+size:2000
-+EasyNet/fully_connected_2/biases:0+shape:[10]+size:10

-+EasyNet/fully_connected/kernel/Regularizer/l2_regularizer:0+shape:[]+size:1.0
-+EasyNet/fully_connected_1/kernel/Regularizer/l2_regularizer:0+shape:[]+size:1.0
-+EasyNet/fully_connected_2/kernel/Regularizer/l2_regularizer:0+shape:[]+size:1.0|868010|注意，正则化部分根据可用项给出了三个项。|868011|当正则化为0、0.0001、0.01和1.0时，我得到了分别为0.9468、0.9476、0.9183和0.1135的测试精度值，显示了高正则化术语的危险性。|868012|entityMap|0|LINK|mutability|MUTABLE|url|https://www.tensorflow.org/get_started/mnist/pros^0|2|D|1X|16|13|9|0|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|14|8|@$9|15|A|16|B|C]|$9|17|A|18|B|C]]|D|@$9|19|A|1A|1|1B]]|E|$]]|$1|F|3|G|5|H|7|1C|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|1D|8|@]|D|@]|E|$]]|$1|M|3|N|5|H|7|1E|8|@]|D|@]|E|$I|J]]|$1|O|3|P|5|6|7|1F|8|@]|D|@]|E|$]]|$1|Q|3|R|5|H|7|1G|8|@]|D|@]|E|$I|J]]|$1|S|3|T|5|6|7|1H|8|@]|D|@]|E|$]]|$1|U|3|V|5|6|7|1I|8|@]|D|@]|E|$]]|$1|W|3|-4|5|6|7|1J|8|@]|D|@]|E|$]]]|X|$Y|$5|Z|10|11|E|$12|13]]]]

Another option to do this with the <code>contrib.learn</code> library is as follows, based on the <a href="https://www.tensorflow.org/get_started/mnist/pros" rel="noreferrer">Deep MNIST tutorial</a> on the Tensorflow website. First, assuming you've imported the relevant libraries (such as <code>import tensorflow.contrib.layers as layers</code>), you can define a network in a separate method:

<pre><code>def easier_network(x, reg):
 """ A network based on tf.contrib.learn, with input `x`. """
 with tf.variable_scope('EasyNet'):
 out = layers.flatten(x)
 out = layers.fully_connected(out, 
 num_outputs=200,
 weights_initializer = layers.xavier_initializer(uniform=True),
 weights_regularizer = layers.l2_regularizer(scale=reg),
 activation_fn = tf.nn.tanh)
 out = layers.fully_connected(out, 
 num_outputs=200,
 weights_initializer = layers.xavier_initializer(uniform=True),
 weights_regularizer = layers.l2_regularizer(scale=reg),
 activation_fn = tf.nn.tanh)
 out = layers.fully_connected(out, 
 num_outputs=10, # Because there are ten digits!
 weights_initializer = layers.xavier_initializer(uniform=True),
 weights_regularizer = layers.l2_regularizer(scale=reg),
 activation_fn = None)
 return out 
</code></pre>

Then, in a main method, you can use the following code snippet:

<pre><code>def main(_):
 mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)
 x = tf.placeholder(tf.float32, [None, 784])
 y_ = tf.placeholder(tf.float32, [None, 10])

 # Make a network with regularization
 y_conv = easier_network(x, FLAGS.regu)
 weights = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, 'EasyNet') 
 print("")
 for w in weights:
 shp = w.get_shape().as_list()
 print("- {} shape:{} size:{}".format(w.name, shp, np.prod(shp)))
 print("")
 reg_ws = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES, 'EasyNet')
 for w in reg_ws:
 shp = w.get_shape().as_list()
 print("- {} shape:{} size:{}".format(w.name, shp, np.prod(shp)))
 print("")

 # Make the loss function `loss_fn` with regularization.
 cross_entropy = tf.reduce_mean(
 tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y_conv))
 loss_fn = cross_entropy + tf.reduce_sum(reg_ws)
 train_step = tf.train.AdamOptimizer(1e-4).minimize(loss_fn)
</code></pre>

To get this to work you need to follow the MNIST tutorial I linked to earlier and import the relevant libraries, but it's a nice exercise to learn TensorFlow and it's easy to see how the regularization affects the output. If you apply a regularization as an argument, you can see the following:

<pre><code>- EasyNet/fully_connected/weights:0 shape:[784, 200] size:156800
- EasyNet/fully_connected/biases:0 shape:[200] size:200
- EasyNet/fully_connected_1/weights:0 shape:[200, 200] size:40000
- EasyNet/fully_connected_1/biases:0 shape:[200] size:200
- EasyNet/fully_connected_2/weights:0 shape:[200, 10] size:2000
- EasyNet/fully_connected_2/biases:0 shape:[10] size:10

- EasyNet/fully_connected/kernel/Regularizer/l2_regularizer:0 shape:[] size:1.0
- EasyNet/fully_connected_1/kernel/Regularizer/l2_regularizer:0 shape:[] size:1.0
- EasyNet/fully_connected_2/kernel/Regularizer/l2_regularizer:0 shape:[] size:1.0
</code></pre>

Notice that the regularization portion gives you three items, based on the items available.

With regularizations of 0, 0.0001, 0.01, and 1.0, I get test accuracy values of 0.9468, 0.9476, 0.9183, and 0.1135, respectively, showing the dangers of high regularization terms.

blocks|key|868038|text|我会提供一个简单的正确答案，因为我没有找到一个。您需要两个简单的步骤，其余步骤由tensorflow魔术完成：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|868039|在创建变量或层时添加正则化程序：
(x，kernel_regularizer=tf.contrib.layers.l2_regularizer(0.001))+#或tf.get_variable('a'，regularizer=tf.contrib.layers.l2_regularizer(0.001))+)|ordered-list-item|868040|在定义损失时添加正规术语：
损失=+ordinary_loss+%2B+tf.losses.get_regularization_loss()|868041|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$]]|$1|E|3|F|5|D|7|K|8|@]|9|@]|A|$]]|$1|G|3|-4|5|6|7|L|8|@]|9|@]|A|$]]]|H|$]]

I'll provide a simple correct answer since I didn't find one. You need two simple steps, the rest is done by tensorflow magic:

<ol>
<li>Add regularizers when creating variables or layers:

<pre><code>tf.layers.dense(x, kernel_regularizer=tf.contrib.layers.l2_regularizer(0.001))
# or
tf.get_variable('a', regularizer=tf.contrib.layers.l2_regularizer(0.001))
</code></pre></li>
<li>Add the regularization term when defining loss:

<pre><code>loss = ordinary_loss + tf.losses.get_regularization_loss()
</code></pre></li>
</ol>

blocks|key|1622534|text|一些答案使我更加confused.Here，我给出了两种方法来使它清楚。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1622535|#1.adding+all+regs+by+hand
var1+=+tf.get_variable(name='v1',shape=[1],dtype=tf.float32)
var2+=+tf.Variable(name='v2',initial_value=1.0,dtype=tf.float32)
regularizer+=+tf.contrib.layers.l1_regularizer(0.1)
reg_term+=+tf.contrib.layers.apply_regularization(regularizer,[var1,var2])
#here+reg_term+is+a+scalar

#2.auto+added+and+read,but+using+get_variable
with+tf.variable_scope('x',
++++++++regularizer=tf.contrib.layers.l2_regularizer(0.1)):
++++var1+=+tf.get_variable(name='v1',shape=[1],dtype=tf.float32)
++++var2+=+tf.get_variable(name='v2',shape=[1],dtype=tf.float32)
reg_losses+=+tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)
#here+reg_losses+is+a+list,should+be+summed+|code-block|syntax|javascript|1622536|然后，它可以被添加到总损失中。|1622537|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|K|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|L|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|M|8|@]|9|@]|A|$]]|$1|I|3|-4|5|6|7|N|8|@]|9|@]|A|$]]]|J|$]]

Some answers make me more confused.Here I give two methods to make it clearly.

<pre><code>#1.adding all regs by hand
var1 = tf.get_variable(name='v1',shape=[1],dtype=tf.float32)
var2 = tf.Variable(name='v2',initial_value=1.0,dtype=tf.float32)
regularizer = tf.contrib.layers.l1_regularizer(0.1)
reg_term = tf.contrib.layers.apply_regularization(regularizer,[var1,var2])
#here reg_term is a scalar

#2.auto added and read,but using get_variable
with tf.variable_scope('x',
 regularizer=tf.contrib.layers.l2_regularizer(0.1)):
 var1 = tf.get_variable(name='v1',shape=[1],dtype=tf.float32)
 var2 = tf.get_variable(name='v2',shape=[1],dtype=tf.float32)
reg_losses = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)
#here reg_losses is a list,should be summed 
</code></pre>

Then,it can be added into the total loss

blocks|key|838810|text|cross_entropy+=+tf.losses.softmax_cross_entropy(
++logits=logits,+onehot_labels=labels)

l2_loss+=+weight_decay+*+tf.add_n(
+++++[tf.nn.l2_loss(tf.cast(v,+tf.float32))+for+v+in+tf.trainable_variables()])

loss+=+cross_entropy+%2B+l2_loss|type|code-block|depth|inlineStyleRanges|entityRanges|data|syntax|javascript|838811|unstyled|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|G|8|@]|9|@]|A|$B|C]]|$1|D|3|-4|5|E|7|H|8|@]|9|@]|A|$]]]|F|$]]

<pre><code>cross_entropy = tf.losses.softmax_cross_entropy(
 logits=logits, onehot_labels=labels)

l2_loss = weight_decay * tf.add_n(
 [tf.nn.l2_loss(tf.cast(v, tf.float32)) for v in tf.trainable_variables()])

loss = cross_entropy + l2_loss
</code></pre>

blocks|key|1622605|text|我用图中的一个tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)和tf.losses.get_regularization_loss()测试了它们，发现它们返回相同的值。通过观察值的数量，我猜reg_constant已经通过设置tf.contrib.layers.l2_regularizer的参数来理解这个值了。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1622606|entityMap^0|7|1H|1P|Z|3Y|W|0^^$0|@$1|2|3|4|5|6|7|H|8|@$9|I|A|J|B|C]|$9|K|A|L|B|C]|$9|M|A|N|B|C]]|D|@]|E|$]]|$1|F|3|-4|5|6|7|O|8|@]|D|@]|E|$]]]|G|$]]

I tested <code>tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)</code> and <code>tf.losses.get_regularization_loss()</code> with one <code>l2_regularizer</code> in the graph, and found that they return the same value. By observing the value's quantity, I guess reg_constant has already make sense on the value by setting the parameter of <code>tf.contrib.layers.l2_regularizer</code>.

blocks|key|1622657|text|如果你有CNN，你可以做以下几件事：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1622658|在您的模型功能中：|1622659|conv+=+tf.layers.conv2d(inputs=input_layer,
++++++++++++++++++++++++filters=32,
++++++++++++++++++++++++kernel_size=[3,+3],
++++++++++++++++++++++++kernel_initializer='xavier',
++++++++++++++++++++++++kernel_regularizer=tf.contrib.layers.l2_regularizer(1e-5),
++++++++++++++++++++++++padding="same",
++++++++++++++++++++++++activation=None)+
...|code-block|syntax|javascript|1622660|在你的损失函数中：|1622661|onehot_labels+=+tf.one_hot(indices=tf.cast(labels,+tf.int32),+depth=num_classes)
loss+=+tf.losses.softmax_cross_entropy(onehot_labels=onehot_labels,+logits=logits)
regularization_losses+=+tf.losses.get_regularization_losses()
loss+=+tf.add_n([loss]+%2B+regularization_losses)|1622662|entityMap^0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|O|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|P|8|@]|9|@]|A|$]]|$1|D|3|E|5|F|7|Q|8|@]|9|@]|A|$G|H]]|$1|I|3|J|5|6|7|R|8|@]|9|@]|A|$]]|$1|K|3|L|5|F|7|S|8|@]|9|@]|A|$G|H]]|$1|M|3|-4|5|6|7|T|8|@]|9|@]|A|$]]]|N|$]]

If you have CNN you may do the following:

In your model function:

<pre><code>conv = tf.layers.conv2d(inputs=input_layer,
 filters=32,
 kernel_size=[3, 3],
 kernel_initializer='xavier',
 kernel_regularizer=tf.contrib.layers.l2_regularizer(1e-5),
 padding="same",
 activation=None) 
...
</code></pre>

In your loss function:

<pre><code>onehot_labels = tf.one_hot(indices=tf.cast(labels, tf.int32), depth=num_classes)
loss = tf.losses.softmax_cross_entropy(onehot_labels=onehot_labels, logits=logits)
regularization_losses = tf.losses.get_regularization_losses()
loss = tf.add_n([loss] + regularization_losses)
</code></pre>

blocks|key|840037|text|如果有人还在查看，我只想在tf.keras中添加，您可以通过将它们作为参数传递到层中来添加权重正则化。从Tensorflow+Keras教程站点大量添加L2正则化的示例：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|840038|model+=+keras.models.Sequential([
++++keras.layers.Dense(16,+kernel_regularizer=keras.regularizers.l2(0.001),
+++++++++++++++++++++++activation=tf.nn.relu,+input_shape=(NUM_WORDS,)),
++++keras.layers.Dense(16,+kernel_regularizer=keras.regularizers.l2(0.001),
+++++++++++++++++++++++activation=tf.nn.relu),
++++keras.layers.Dense(1,+activation=tf.nn.sigmoid)
])|code-block|syntax|javascript|840039|据我所知，没有必要用这种方法手动添加正则化损失。|840040|参考资料：正则化|offset|length|840041|entityMap|0|LINK|mutability|MUTABLE|url|https://www.tensorflow.org/tutorials/keras/overfit_and_underfit#add_weight_regularization^0|0|0|0|5|3|0|0^^$0|@$1|2|3|4|5|6|7|U|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|V|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|W|8|@]|9|@]|A|$]]|$1|I|3|J|5|6|7|X|8|@]|9|@$K|Y|L|Z|1|10]]|A|$]]|$1|M|3|-4|5|6|7|11|8|@]|9|@]|A|$]]]|N|$O|$5|P|Q|R|A|$S|T]]]]

If anyone's still looking, I'd just like to add on that in tf.keras you may add weight regularization by passing them as arguments in your layers. An example of adding L2 regularization taken wholesale from the Tensorflow Keras Tutorials site:

<pre><code>model = keras.models.Sequential([
 keras.layers.Dense(16, kernel_regularizer=keras.regularizers.l2(0.001),
 activation=tf.nn.relu, input_shape=(NUM_WORDS,)),
 keras.layers.Dense(16, kernel_regularizer=keras.regularizers.l2(0.001),
 activation=tf.nn.relu),
 keras.layers.Dense(1, activation=tf.nn.sigmoid)
])
</code></pre>

There's no need to manually add in the regularization losses with this method as far as I know.

Reference: <a href="https://www.tensorflow.org/tutorials/keras/overfit_and_underfit#add_weight_regularization" rel="noreferrer">https://www.tensorflow.org/tutorials/keras/overfit_and_underfit#add_weight_regularization</a>

blocks|key|840049|text|tf.GraphKeys.REGULARIZATION_LOSSES不会自动添加，但是有一种简单的方法可以添加它们：|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|840050|reg_loss+=+tf.losses.get_regularization_loss()
total_loss+=+loss+%2B+reg_loss|code-block|syntax|javascript|840051|tf.losses.get_regularization_loss()使用tf.add_n对tf.GraphKeys.REGULARIZATION_LOSSES元素的条目进行求和。tf.GraphKeys.REGULARIZATION_LOSSES通常是使用正则化函数计算的标量列表。它从指定了tf.get_variable参数的regularizer调用中获取条目。您还可以手动添加到该集合中。在使用tf.Variable时，以及在指定活动正则化器或其他自定义正则化器时，这将是非常有用的。例如：|840052|#This+will+add+an+activity+regularizer+on+y+to+the+regloss+collection
regularizer+=+tf.contrib.layers.l2_regularizer(0.1)
y+=+tf.nn.sigmoid(x)
act_reg+=+regularizer(y)
tf.add_to_collection(tf.GraphKeys.REGULARIZATION_LOSSES,+act_reg)|840053|(在本例中，将x正则化可能更有效，因为y对于大型x确实是平坦的。)|840054|entityMap^0|0|Y|0|0|0|Z|11|8|1A|Y|2I|Y|43|F|4L|B|5L|B|0|0|0^^$0|@$1|2|3|4|5|6|7|S|8|@$9|T|A|U|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|V|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|W|8|@$9|X|A|Y|B|C]|$9|Z|A|10|B|C]|$9|11|A|12|B|C]|$9|13|A|14|B|C]|$9|15|A|16|B|C]|$9|17|A|18|B|C]|$9|19|A|1A|B|C]]|D|@]|E|$]]|$1|M|3|N|5|H|7|1B|8|@]|D|@]|E|$I|J]]|$1|O|3|P|5|6|7|1C|8|@]|D|@]|E|$]]|$1|Q|3|-4|5|6|7|1D|8|@]|D|@]|E|$]]]|R|$]]

<code>tf.GraphKeys.REGULARIZATION_LOSSES</code> will not be added automatically, but there is a simple way to add them:

<pre><code>reg_loss = tf.losses.get_regularization_loss()
total_loss = loss + reg_loss
</code></pre>

<code>tf.losses.get_regularization_loss()</code> uses <code>tf.add_n</code> to sum the entries of <code>tf.GraphKeys.REGULARIZATION_LOSSES</code> element-wise. <code>tf.GraphKeys.REGULARIZATION_LOSSES</code> will typically be a list of scalars, calculated using regularizer functions. It gets entries from calls to <code>tf.get_variable</code> that have the <code>regularizer</code> parameter specified. You can also add to that collection manually. That would be useful when using <code>tf.Variable</code> and also when specifying activity regularizers or other custom regularizers. For instance:

<pre><code>#This will add an activity regularizer on y to the regloss collection
regularizer = tf.contrib.layers.l2_regularizer(0.1)
y = tf.nn.sigmoid(x)
act_reg = regularizer(y)
tf.add_to_collection(tf.GraphKeys.REGULARIZATION_LOSSES, act_reg)
</code></pre>

(In this example it would presumably be more effective to regularize x, as y really flattens out for large x.)

I found in many available neural network code implemented using TensorFlow that regularization terms are often implemented by manually adding an additional term to loss value.

My questions are:

<ol>
<li>Is there a more elegant or recommended way of regularization than doing it manually?</li>
<li>I also find that <code>get_variable</code> has an argument <code>regularizer</code>. How should it be used? According to my observation, if we pass a regularizer to it (such as <code>tf.contrib.layers.l2_regularizer</code>, a tensor representing regularized term will be computed and added to a graph collection named <code>tf.GraphKeys.REGULARIZATOIN_LOSSES</code>. Will that collection be automatically used by TensorFlow (e.g. used by optimizers when training)? Or is it expected that I should use that collection by myself?</li>
</ol>

How to add regularizations in TensorFlow?

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

我在许多可用的使用TensorFlow实现的神经网络代码中发现，正则化项通常是通过在损失值中添加一个附加项来实现的。我的问题是：是否有一种比手动更优雅或推荐的正则化方法？我还发现get_variable有一个论点，regularizer。它应该如何使用？根据我的观察，如果我们将正则化项传递给它(例如tf.contrib...

问如何在TensorFlow中添加正则化？
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何在TensorFlow中添加正则化？EN