tensorflow: 损失函数(Losses Functions) 探究

损失函数定义

From Tensorflow - Losses：

Losses The loss ops measure error between two tensors, or between a tensor and zero. These can be used for measuring accuracy of a network in a regression task or for regularization purposes (weight decay). tf.nn.l2_loss tf.nn.log_poisson_loss

即：

Losses 损失运算 用于测量两个张量之间或张量与0之间的误差。 这些可以用于测量回归任务中的网络的精确度，或用于正则化的目的（权重衰减）。 tf.nn.l2_loss tf.nn.log_poisson_loss

l2_loss

From tf.nn.l2_loss：

tf.nn.l2_loss l2_loss( t, name=None ) Defined in tensorflow/python/ops/gen_nn_ops.py. See the guide: Neural Network > Losses L2 Loss. Computes half the L2 norm of a tensor without the sqrt: output = sum(t ** 2) / 2 Args: t: A Tensor. Must be one of the following types: half, float32, float64. Typically 2-D, but may have any dimensions. name: A name for the operation (optional). Returns: A Tensor. Has the same type as t. 0-D.

易得 l2_loss( t, name=None ) 等同于 output = sum(t ** 2) / 2

实验思路

1. 新建三个shape为 [10, 5, 1] 的张量，一个全0，一个全1，还有一个全2；
2. 用 tf.nn.l2_loss 分别计算出 tf.nn.l2_loss(a-b)、tf.nn.l2_loss(a-c)；
3. 用 sum(t ** 2) / 2 分别计算出 (((0-1) ** 2) * 50) / 2、l_2 = (((0-2) ** 2) * 50) / 2；
4. 如果对应结果相同，则验证了该公式。

实验源码

自己编写代码进行验证：

import tensorflow as tf
import numpy as np


a = np.zeros(shape=[10, 5, 1], dtype=np.float32)
b = np.ones(shape=[10, 5, 1], dtype=np.float32)
c = b * 2

# tf.nn.l2_loss(a-b) = sum((a-b)**2) / 2
loss_1 = tf.Session().run(tf.nn.l2_loss(a-b))
loss_2 = tf.Session().run(tf.nn.l2_loss(a-c))

l_1 = (((0-1) ** 2) * 50) / 2
l_2 = (((0-2) ** 2) * 50) / 2

print 'tf.nn.l2_loss(a-b) = ', loss_1
print 'tf.nn.l2_loss(a-c) = ', loss_2

assert loss_1 == l_1 and loss_2 == l_2

成功验证了公式：

tf.nn.l2_loss(a-b) =  25.0
tf.nn.l2_loss(a-c) =  100.0