blocks|key|1597554|text|PyTorch解决方案|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1597555|好吧，实际上我已经看过文档了，你可以简单地使用pos_weight。|offset|length|style|CODE|1597556|此参数为每个类的正样本赋予权重，因此，如果您有270类，则应该使用shape+(270,)传递torch.Tensor，为每个类定义权重。|1597557|下面是来自documentation的略微修改过的代码片段|1597558|#+270+classes,+batch+size+=+64++++
target+=+torch.ones([64,+270],+dtype=torch.float32)++
#+Logits+outputted+from+your+network,+no+activation
output+=+torch.full([64,+270],+0.9)
#+Weights,+each+being+equal+to+one.+You+can+input+your+own+here.
pos_weight+=+torch.ones([270])
criterion+=+torch.nn.BCEWithLogitsLoss(pos_weight=pos_weight)
criterion(output,+target)++#+-log(sigmoid(0.9))|code-block|syntax|javascript|1597559|自制解决方案|1597560|说到权重，没有内置的解决方案，但你可以很容易地自己编写一个：|1597561|import+torch

class+WeightedMultilabel(torch.nn.Module):
++++def+__init__(self,+weights:+torch.Tensor):
++++++++self.loss+=+torch.nn.BCEWithLogitsLoss()
++++++++self.weights+=+weights.unsqueeze()

++++def+forward(outputs,+targets):
++++++++return+self.loss(outputs,+targets)+*+self.weights|1597562|Tensor的长度必须与多标签分类中的类别数量(270)相同，每个类别都为您的特定示例赋予权重。|1597563|计算权重|1597564|您只需在数据集中添加每个样本的标签，除以最小值并在末尾倒数。|1597565|一种代码片段：|1597566|weights+=+torch.zeros_like(dataset[0])
for+element+in+dataset:
++++weights+%2B=+element

weights+=+1+/+(weights+/+torch.min(weights))|1597567|使用这种方法，出现最少的类将给出正常的损失，而其他类的权重将小于1。|1597568|但是，它可能会在训练过程中造成一些不稳定，所以您可能希望试验一下这些值(也许可以使用log变换而不是线性变换？)|1597569|其他方法|1597570|你可能会考虑上采样/下采样(虽然这个操作很复杂，因为你还会添加/删除其他类，所以我认为需要高级启发式)。|1597571|entityMap|0|LINK|mutability|MUTABLE|url|https://pytorch.org/docs/stable/nn.html#torch.nn.BCEWithLogitsLoss^0|0|N|A|0|N|3|13|6|1B|C|0|5|D|0|0|0|0|0|0|0|6|0|0|0|0|0|W|1|0|16|3|0|0|0^^$0|@$1|2|3|4|5|6|7|1M|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|1N|8|@$D|1O|E|1P|F|G]]|9|@]|A|$]]|$1|H|3|I|5|6|7|1Q|8|@$D|1R|E|1S|F|G]|$D|1T|E|1U|F|G]|$D|1V|E|1W|F|G]]|9|@]|A|$]]|$1|J|3|K|5|6|7|1X|8|@]|9|@$D|1Y|E|1Z|1|20]]|A|$]]|$1|L|3|M|5|N|7|21|8|@]|9|@]|A|$O|P]]|$1|Q|3|R|5|6|7|22|8|@]|9|@]|A|$]]|$1|S|3|T|5|6|7|23|8|@]|9|@]|A|$]]|$1|U|3|V|5|N|7|24|8|@]|9|@]|A|$O|P]]|$1|W|3|X|5|6|7|25|8|@$D|26|E|27|F|G]]|9|@]|A|$]]|$1|Y|3|Z|5|6|7|28|8|@]|9|@]|A|$]]|$1|10|3|11|5|6|7|29|8|@]|9|@]|A|$]]|$1|12|3|13|5|6|7|2A|8|@]|9|@]|A|$]]|$1|14|3|15|5|N|7|2B|8|@]|9|@]|A|$O|P]]|$1|16|3|17|5|6|7|2C|8|@$D|2D|E|2E|F|G]]|9|@]|A|$]]|$1|18|3|19|5|6|7|2F|8|@$D|2G|E|2H|F|G]]|9|@]|A|$]]|$1|1A|3|1B|5|6|7|2I|8|@]|9|@]|A|$]]|$1|1C|3|1D|5|6|7|2J|8|@]|9|@]|A|$]]|$1|1E|3|-4|5|6|7|2K|8|@]|9|@]|A|$]]]|1F|$1G|$5|1H|1I|1J|A|$1K|1L]]]]

<h1>PyTorch solution</h1>

Well, actually I have gone through docs and you can simply use <code>pos_weight</code> indeed.

This argument gives weight to positive sample for each class, hence if you have <code>270</code> classes you should pass <code>torch.Tensor</code> with shape <code>(270,)</code> defining weight for each class.

Here is marginally modified snippet from <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.BCEWithLogitsLoss" rel="nofollow noreferrer">documentation</a>:

<pre><code># 270 classes, batch size = 64 
target = torch.ones([64, 270], dtype=torch.float32) 
# Logits outputted from your network, no activation
output = torch.full([64, 270], 0.9)
# Weights, each being equal to one. You can input your own here.
pos_weight = torch.ones([270])
criterion = torch.nn.BCEWithLogitsLoss(pos_weight=pos_weight)
criterion(output, target) # -log(sigmoid(0.9))
</code></pre>

<h1>Self-made solution</h1>

When it comes to weighting, there is no built-in solution, but you may code one yourself really easily:

<pre><code>import torch

class WeightedMultilabel(torch.nn.Module):
 def __init__(self, weights: torch.Tensor):
 self.loss = torch.nn.BCEWithLogitsLoss()
 self.weights = weights.unsqueeze()

 def forward(outputs, targets):
 return self.loss(outputs, targets) * self.weights
</code></pre>

<code>Tensor</code> has to be of the same length as the number of classes in your multilabel classification (270), each giving weight for your specific example.

<h2>Calculating weights</h2>

You just add labels of every sample in your dataset, divide by the minimum value and inverse at the end.

Sort of snippet:

<pre><code>weights = torch.zeros_like(dataset[0])
for element in dataset:
 weights += element

weights = 1 / (weights / torch.min(weights))
</code></pre>

Using this approach class occurring the least will give normal loss, while others will have weights smaller than <code>1</code>.

It might cause some instability during training though, so you might want to experiment with those values a little (maybe <code>log</code> transform instead of linear?)

<h1>Other approach</h1>

You may think about upsampling/downsampling (though this operation is complicated as you would add/delete other classes as well, so advanced heuristics would be needed I think).

blocks|key|1597813|text|PyTorch+documentation+for+BCEWithLogitsLoss建议pos_weight为每个类别的负数和正数之间的比率。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1597814|因此，如果len(dataset)为1000，则多热编码的元素0有100个正计数，那么pos_weights_vector的元素0应该为900/100+=+9。这意味着二进制交叉发送损失将表现为数据集包含900个正样本而不是100个。|style|CODE|1597815|下面是我的实现：|1597816|++def+calculate_pos_weights(class_counts):
++++pos_weights+=+np.ones_like(class_counts)
++++neg_counts+=+[len(data)-pos_count+for+pos_count+in+class_counts]
++++for+cdx,+pos_count,+neg_count+in+enumerate(zip(class_counts,++neg_counts)):
++++++pos_weights[cdx]+=+neg_count+/+(pos_count+%2B+1e-5)

++++return+torch.as_tensor(pos_weights,+dtype=torch.float)|code-block|syntax|javascript|1597817|其中class_counts只是正样本的按列求和。我在PyTorch论坛上进行了posted+it，其中一位PyTorch开发人员对此表示支持。|1597818|entityMap|0|LINK|mutability|MUTABLE|url|https://pytorch.org/docs/stable/nn.html#torch.nn.BCEWithLogitsLoss|1|https://discuss.pytorch.org/t/weights-in-bcewithlogitsloss/27452/12?u=crypdick^0|8|Z|0|0|5|C|17|I|1W|B|0|0|0|2|C|14|9|1|0^^$0|@$1|2|3|4|5|6|7|10|8|@]|9|@$A|11|B|12|1|13]]|C|$]]|$1|D|3|E|5|6|7|14|8|@$A|15|B|16|F|G]|$A|17|B|18|F|G]|$A|19|B|1A|F|G]]|9|@]|C|$]]|$1|H|3|I|5|6|7|1B|8|@]|9|@]|C|$]]|$1|J|3|K|5|L|7|1C|8|@]|9|@]|C|$M|N]]|$1|O|3|P|5|6|7|1D|8|@$A|1E|B|1F|F|G]]|9|@$A|1G|B|1H|1|1I]]|C|$]]|$1|Q|3|-4|5|6|7|1J|8|@]|9|@]|C|$]]]|R|$S|$5|T|U|V|C|$W|X]]|Y|$5|T|U|V|C|$W|Z]]]]

The PyTorch <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.BCEWithLogitsLoss" rel="nofollow noreferrer">documentation for BCEWithLogitsLoss</a> recommends the pos_weight to be a ratio between the negative counts and the positive counts for each class.
So, if <code>len(dataset)</code> is 1000, element 0 of your multihot encoding has 100 positive counts, then element 0 of the <code>pos_weights_vector</code> should be <code>900/100 = 9</code>. That means that the binary crossent loss will behave as if the dataset contains 900 positive examples instead of 100.
Here is my implementation:
<pre><code> def calculate_pos_weights(class_counts):
 pos_weights = np.ones_like(class_counts)
 neg_counts = [len(data)-pos_count for pos_count in class_counts]
 for cdx, pos_count, neg_count in enumerate(zip(class_counts, neg_counts)):
 pos_weights[cdx] = neg_count / (pos_count + 1e-5)

 return torch.as_tensor(pos_weights, dtype=torch.float)
</code></pre>
Where <code>class_counts</code> is just a column-wise sum of the positive samples. I <a href="https://discuss.pytorch.org/t/weights-in-bcewithlogitsloss/27452/12?u=crypdick" rel="nofollow noreferrer">posted it</a> on the PyTorch forum and one of the PyTorch devs gave it his blessing.

blocks|key|1812103|text|只是为了快速修改@crypdick的答案，这个函数的实现对我来说是有效的：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1812104|def+calculate_pos_weights(class_counts,data):
++++pos_weights+=+np.ones_like(class_counts)
++++neg_counts+=+[len(data)-pos_count+for+pos_count+in+class_counts]
++++for+cdx,+(pos_count,+neg_count)+in+enumerate(zip(class_counts,++neg_counts)):
++++++++pos_weights[cdx]+=+neg_count+/+(pos_count+%2B+1e-5)

++++return+torch.as_tensor(pos_weights,+dtype=torch.float)|code-block|syntax|javascript|1812105|其中data是您尝试应用权重的数据集。|offset|length|style|CODE|1812106|entityMap^0|0|0|2|4|0^^$0|@$1|2|3|4|5|6|7|O|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|P|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|Q|8|@$I|R|J|S|K|L]]|9|@]|A|$]]|$1|M|3|-4|5|6|7|T|8|@]|9|@]|A|$]]]|N|$]]

Just to provide a quick revision on @crypdick's answer, this implementation of the function worked for me:
<pre><code>def calculate_pos_weights(class_counts,data):
 pos_weights = np.ones_like(class_counts)
 neg_counts = [len(data)-pos_count for pos_count in class_counts]
 for cdx, (pos_count, neg_count) in enumerate(zip(class_counts, neg_counts)):
 pos_weights[cdx] = neg_count / (pos_count + 1e-5)

 return torch.as_tensor(pos_weights, dtype=torch.float)
</code></pre>
Where <code>data</code> is the dataset you're trying to apply weights to.

blocks|key|1812283|text|可能有点晚了，但我是这样计算的：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1812284|num_positives+=+torch.sum(train_dataset.data.y,+dim=0)
num_negatives+=+len(train_dataset.data.y)+-+num_positives
pos_weight++=+num_negatives+/+num_positives|code-block|syntax|javascript|1812285|然后可以轻松地使用权重，如下所示：|1812286|criterion+=+torch.nn.BCEWithLogitsLoss(pos_weight+=+pos_weight)|1812287|entityMap^0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|N|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|O|8|@]|9|@]|A|$]]|$1|I|3|J|5|D|7|P|8|@]|9|@]|A|$E|F]]|$1|K|3|-4|5|6|7|Q|8|@]|9|@]|A|$]]]|L|$]]

maybe is a little late, but here is how I calculate the same:
<pre><code>num_positives = torch.sum(train_dataset.data.y, dim=0)
num_negatives = len(train_dataset.data.y) - num_positives
pos_weight = num_negatives / num_positives
</code></pre>
Then the weights can be used easily as:
<pre><code>criterion = torch.nn.BCEWithLogitsLoss(pos_weight = pos_weight)
</code></pre>

I am trying to solve one multilabel problem with <code>270</code> labels and i have converted target labels into one hot encoded form. I am using <code>BCEWithLogitsLoss()</code>. Since training data is unbalanced, I am using <code>pos_weight</code> argument but i am bit confused.

<code>pos_weight</code> (Tensor, optional) – a weight of positive examples. Must be a vector with length equal to the number of classes.

Do i need to give total count of positive values of each label as a tensor or they mean something else by weights?

How to calculate unbalanced weights for BCEWithLogitsLoss in pytorch

我正在尝试用270标签来解决一个多标签问题，并且我已经将目标标签转换成了一个热编码形式。我正在使用BCEWithLogitsLoss()。由于训练数据不平衡，我使用了pos_weight参数，但我有点困惑。pos_weight (张量，可选)-正例的权重。必须是长度等于类数的向量。我是否需要给出每个标签的正值的总计数作...

问如何计算pytorch中BCEWithLogitsLoss的不平衡权重
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何计算pytorch中BCEWithLogitsLoss的不平衡权重EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何计算pytorch中BCEWithLogitsLoss的不平衡权重
EN